index

package
v0.3.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 6, 2026 License: MIT Imports: 10 Imported by: 0

Documentation

Overview

Package index provides a global registry and search layer for tools.

This package implements tool registration, storage, retrieval, and search capabilities. It supports multiple index backends and pluggable search strategies.

Index Types

The package provides a built-in index implementation:

  • InMemoryIndex: Fast, in-memory storage with optional custom searcher

Usage

Create and populate an index: 

idx := index.NewInMemoryIndex()

tool := model.Tool{
    Tool: mcp.Tool{
        Name:        "calculator",
        Description: "Performs arithmetic operations",
    },
    Namespace: "math",
    Tags:      []string{"arithmetic", "math"},
}
backend := model.ToolBackend{
    Kind: model.BackendKindMCP,
    MCP:  &model.MCPBackend{ServerName: "math-server"},
}
err := idx.RegisterTool(tool, backend)

Search for tools: 

results, err := idx.Search("arithmetic", 10)

The index accepts a custom Searcher for advanced search capabilities: 

type MySearcher struct{}
func (s *MySearcher) Search(query string, limit int, docs []index.SearchDoc) ([]index.Summary, error) {
    // Custom search implementation
}

idx := index.NewInMemoryIndex(index.WithSearcher(&MySearcher{}))

Progressive Disclosure

Tools support progressive disclosure through Summary objects that contain only essential information for display and discovery:

  • ID: Canonical tool identifier (namespace:name)
  • Name: Tool name
  • Namespace: Optional namespace for grouping
  • ShortDescription: Truncated description (max 120 chars)
  • Summary: Short summary (mirrors ShortDescription)
  • Category: Optional category label
  • InputModes: Supported input media types
  • OutputModes: Supported output media types
  • SecuritySummary: Short auth summary
  • Tags: Associated tags for filtering

Pagination

Search and list operations support cursor-based pagination: 

results, nextCursor, err := idx.SearchPage("query", 10, "")
if nextCursor != "" {
    moreResults, nextCursor, err = idx.SearchPage("query", 10, nextCursor)
}

Change Notifications

The index supports change notifications for reactive updates: 

unsub := idx.OnChange(func(event index.ChangeEvent) {
    // Handle tool added/removed/updated
})
defer unsub()

Migration Note

This package was migrated from github.com/jonwraymond/toolindex as part of the ApertureStack consolidation (PRD-130).

Package index provides a global registry and search layer for tools. It ingests model.Tool and model.ToolBackend and provides progressive discovery (summaries + namespaces) and canonical lookup by tool ID.

Index

Examples

Constants

View Source 
const MaxShortDescriptionLen = 120

MaxShortDescriptionLen is the maximum length of the ShortDescription field in Summary.

Variables

View Source 
var (
	ErrNotFound                 = errors.New("tool not found")
	ErrInvalidTool              = errors.New("invalid tool")
	ErrInvalidBackend           = errors.New("invalid backend")
	ErrInvalidCursor            = errors.New("invalid cursor")
	ErrNonDeterministicSearcher = errors.New("searcher is non-deterministic")
)

Error values for consistent error handling by callers.

Functions

func DefaultBackendSelector 

func DefaultBackendSelector(backends []model.ToolBackend) model.ToolBackend

DefaultBackendSelector implements the default priority: local > provider > mcp. Exported so other modules (for example, toolrun) can match the same policy.

Types

type BackendSelector 

type BackendSelector func([]model.ToolBackend) model.ToolBackend

BackendSelector is a function that selects the default backend from a list.

type ChangeEvent 

type ChangeEvent struct {
	Type    ChangeType
	ToolID  string
	Backend model.ToolBackend
	Version uint64
}

ChangeEvent captures a mutation in the index for reactive integration.

type ChangeListener 

type ChangeListener func(ChangeEvent)

ChangeListener receives change events from an Index implementation.

type ChangeNotifier 

type ChangeNotifier interface {
	OnChange(listener ChangeListener) (unsubscribe func())
}

ChangeNotifier is an optional interface for receiving change events.

Contract: - OnChange must be safe for concurrent use. - It must return a non-nil unsubscribe function that is safe to call multiple times. - Passing a nil listener must return a no-op unsubscribe function.

type ChangeType 

type ChangeType string

ChangeType describes a mutation event in the index. 

const (
	ChangeRegistered     ChangeType = "registered"
	ChangeUpdated        ChangeType = "updated"
	ChangeBackendRemoved ChangeType = "backend_removed"
	ChangeToolRemoved    ChangeType = "tool_removed"
	ChangeRefreshed      ChangeType = "refreshed"
)

type DeterministicSearcher 

type DeterministicSearcher interface {
	Searcher
	Deterministic() bool
}

DeterministicSearcher indicates whether a searcher provides deterministic ordering.

Contract: - Deterministic reports whether Search ordering is stable for identical inputs. - Implementations must not return true if ordering is non-deterministic.

type InMemoryIndex 

type InMemoryIndex struct {
	// contains filtered or unexported fields
}

InMemoryIndex is the default in-memory implementation of Index.

func NewInMemoryIndex 

func NewInMemoryIndex(opts ...IndexOptions) *InMemoryIndex

NewInMemoryIndex creates a new in-memory tool index.

Example 
package main

import (
	"fmt"

	"github.com/modelcontextprotocol/go-sdk/mcp"

	"github.com/jonwraymond/tooldiscovery/index"
	"github.com/jonwraymond/toolfoundation/model"
)

func main() {
	idx := index.NewInMemoryIndex()

	tool := model.Tool{
		Tool: mcp.Tool{
			Name:        "search",
			Description: "Search for files in the filesystem",
			InputSchema: map[string]any{
				"type": "object",
				"properties": map[string]any{
					"query": map[string]any{"type": "string"},
				},
			},
		},
		Namespace: "files",
		Tags:      []string{"search", "filesystem"},
	}
	backend := model.NewMCPBackend("files-server")

	_ = idx.RegisterTool(tool, backend)

	results, _ := idx.Search("search", 10)
	fmt.Println("Found:", len(results))
}

Output:

Found: 1

func (*InMemoryIndex) GetAllBackends 

func (idx *InMemoryIndex) GetAllBackends(id string) ([]model.ToolBackend, error)

GetAllBackends returns all backends for a tool.

func (*InMemoryIndex) GetTool 

func (idx *InMemoryIndex) GetTool(id string) (model.Tool, model.ToolBackend, error)

GetTool returns the full tool and its default backend.

Example 
package main

import (
	"fmt"

	"github.com/modelcontextprotocol/go-sdk/mcp"

	"github.com/jonwraymond/tooldiscovery/index"
	"github.com/jonwraymond/toolfoundation/model"
)

func main() {
	idx := index.NewInMemoryIndex()

	tool := model.Tool{
		Tool: mcp.Tool{
			Name:        "read_file",
			Description: "Read contents of a file",
			InputSchema: map[string]any{"type": "object"},
		},
		Namespace: "files",
	}
	backend := model.NewMCPBackend("files-server")
	_ = idx.RegisterTool(tool, backend)

	// Retrieve the tool by its canonical ID
	retrieved, retrievedBackend, err := idx.GetTool("files:read_file")
	if err != nil {
		fmt.Println("Error:", err)
		return
	}

	fmt.Println("Name:", retrieved.Name)
	fmt.Println("Backend:", retrievedBackend.MCP.ServerName)
}

Output:

Name: read_file
Backend: files-server

func (*InMemoryIndex) ListNamespaces 

func (idx *InMemoryIndex) ListNamespaces() ([]string, error)

ListNamespaces returns all namespaces in alphabetical order.

Example 
package main

import (
	"fmt"

	"github.com/modelcontextprotocol/go-sdk/mcp"

	"github.com/jonwraymond/tooldiscovery/index"
	"github.com/jonwraymond/toolfoundation/model"
)

func main() {
	idx := index.NewInMemoryIndex()

	// Register tools in different namespaces
	namespaces := []string{"git", "docker", "kubectl"}
	for _, ns := range namespaces {
		tool := model.Tool{
			Tool: mcp.Tool{
				Name:        "tool",
				Description: "A tool",
				InputSchema: map[string]any{"type": "object"},
			},
			Namespace: ns,
		}
		_ = idx.RegisterTool(tool, model.NewMCPBackend("server"))
	}

	// List all namespaces (alphabetically sorted)
	nsList, _ := idx.ListNamespaces()
	for _, ns := range nsList {
		fmt.Println(ns)
	}
}

Output:

docker
git
kubectl

func (*InMemoryIndex) ListNamespacesPage 

func (idx *InMemoryIndex) ListNamespacesPage(limit int, cursor string) ([]string, string, error)

ListNamespacesPage returns namespaces with cursor pagination.

func (*InMemoryIndex) OnChange 

func (idx *InMemoryIndex) OnChange(listener ChangeListener) func()

OnChange registers a listener for index mutations. Returns an unsubscribe function.

Example 
package main

import (
	"fmt"

	"github.com/modelcontextprotocol/go-sdk/mcp"

	"github.com/jonwraymond/tooldiscovery/index"
	"github.com/jonwraymond/toolfoundation/model"
)

func main() {
	idx := index.NewInMemoryIndex()

	// Subscribe to changes
	unsubscribe := idx.OnChange(func(event index.ChangeEvent) {
		fmt.Printf("Event: %s for %s\n", event.Type, event.ToolID)
	})
	defer unsubscribe()

	tool := model.Tool{
		Tool: mcp.Tool{
			Name:        "my_tool",
			Description: "A tool",
			InputSchema: map[string]any{"type": "object"},
		},
	}

	// This triggers a change event
	_ = idx.RegisterTool(tool, model.NewMCPBackend("server"))
}

Output:

Event: registered for my_tool

func (*InMemoryIndex) Refresh 

func (idx *InMemoryIndex) Refresh() uint64

Refresh rebuilds the search docs cache and emits a refresh event.

func (*InMemoryIndex) RegisterTool 

func (idx *InMemoryIndex) RegisterTool(tool model.Tool, backend model.ToolBackend) error

RegisterTool registers a single tool with its backend.

func (*InMemoryIndex) RegisterTools 

func (idx *InMemoryIndex) RegisterTools(regs []ToolRegistration) error

RegisterTools registers multiple tools in batch.

func (*InMemoryIndex) RegisterToolsFromMCP 

func (idx *InMemoryIndex) RegisterToolsFromMCP(serverName string, tools []model.Tool) error

RegisterToolsFromMCP is a convenience method for registering tools from an MCP server.

Example 
package main

import (
	"fmt"

	"github.com/modelcontextprotocol/go-sdk/mcp"

	"github.com/jonwraymond/tooldiscovery/index"
	"github.com/jonwraymond/toolfoundation/model"
)

func main() {
	idx := index.NewInMemoryIndex()

	// Simulate receiving tools from an MCP server
	mcpTools := []model.Tool{
		{
			Tool: mcp.Tool{
				Name:        "list_files",
				Description: "List files in directory",
				InputSchema: map[string]any{"type": "object"},
			},
		},
		{
			Tool: mcp.Tool{
				Name:        "read_file",
				Description: "Read file contents",
				InputSchema: map[string]any{"type": "object"},
			},
		},
	}

	// Register all tools from the MCP server
	_ = idx.RegisterToolsFromMCP("filesystem-server", mcpTools)

	results, _ := idx.Search("file", 10)
	fmt.Println("File tools:", len(results))
}

Output:

File tools: 2

func (*InMemoryIndex) Search 

func (idx *InMemoryIndex) Search(query string, limit int) ([]Summary, error)

Search performs a search over the indexed tools.

Example 
package main

import (
	"fmt"

	"github.com/modelcontextprotocol/go-sdk/mcp"

	"github.com/jonwraymond/tooldiscovery/index"
	"github.com/jonwraymond/toolfoundation/model"
)

func main() {
	idx := index.NewInMemoryIndex()

	// Register multiple tools
	tools := []model.Tool{
		{
			Tool: mcp.Tool{
				Name:        "git_status",
				Description: "Show the working tree status",
				InputSchema: map[string]any{"type": "object"},
			},
			Namespace: "git",
			Tags:      []string{"vcs"},
		},
		{
			Tool: mcp.Tool{
				Name:        "git_commit",
				Description: "Record changes to the repository",
				InputSchema: map[string]any{"type": "object"},
			},
			Namespace: "git",
			Tags:      []string{"vcs"},
		},
		{
			Tool: mcp.Tool{
				Name:        "docker_ps",
				Description: "List running containers",
				InputSchema: map[string]any{"type": "object"},
			},
			Namespace: "docker",
			Tags:      []string{"containers"},
		},
	}

	backend := model.NewMCPBackend("dev-tools")
	for _, tool := range tools {
		_ = idx.RegisterTool(tool, backend)
	}

	// Search for git-related tools
	results, _ := idx.Search("git", 10)
	fmt.Println("Git tools found:", len(results))

	// Search for containers
	results, _ = idx.Search("containers", 10)
	fmt.Println("Container tools found:", len(results))
}

Output:

Git tools found: 2
Container tools found: 1

func (*InMemoryIndex) SearchPage 

func (idx *InMemoryIndex) SearchPage(query string, limit int, cursor string) ([]Summary, string, error)

SearchPage performs a search over the indexed tools with cursor pagination.

Example 
package main

import (
	"fmt"

	"github.com/modelcontextprotocol/go-sdk/mcp"

	"github.com/jonwraymond/tooldiscovery/index"
	"github.com/jonwraymond/toolfoundation/model"
)

func main() {
	idx := index.NewInMemoryIndex()

	// Register several tools
	for i := 0; i < 5; i++ {
		tool := model.Tool{
			Tool: mcp.Tool{
				Name:        fmt.Sprintf("tool_%d", i),
				Description: "A sample tool",
				InputSchema: map[string]any{"type": "object"},
			},
		}
		_ = idx.RegisterTool(tool, model.NewMCPBackend("server"))
	}

	// Paginate through results
	var cursor string
	page := 1
	for {
		results, nextCursor, _ := idx.SearchPage("", 2, cursor)
		fmt.Printf("Page %d: %d results\n", page, len(results))

		if nextCursor == "" {
			break
		}
		cursor = nextCursor
		page++
	}
}

Output:

Page 1: 2 results
Page 2: 2 results
Page 3: 1 results

func (*InMemoryIndex) UnregisterBackend 

func (idx *InMemoryIndex) UnregisterBackend(toolID string, kind model.BackendKind, backendID string) error

UnregisterBackend removes a specific backend from a tool. If the last backend is removed, the tool is also removed.

For provider backends, backendID must be in the format "providerID:toolID". For MCP backends, backendID is the server name. For local backends, backendID is the handler name.

func (*InMemoryIndex) Version added in v0.2.2 

func (idx *InMemoryIndex) Version() uint64

Version returns the current index version.

type Index 

type Index interface {
	// Registration
	RegisterTool(tool model.Tool, backend model.ToolBackend) error
	RegisterTools(regs []ToolRegistration) error
	RegisterToolsFromMCP(serverName string, tools []model.Tool) error

	// Unregistration
	UnregisterBackend(toolID string, kind model.BackendKind, backendID string) error

	// Lookup
	GetTool(id string) (model.Tool, model.ToolBackend, error)
	GetAllBackends(id string) ([]model.ToolBackend, error)

	// Discovery
	Search(query string, limit int) ([]Summary, error)
	SearchPage(query string, limit int, cursor string) ([]Summary, string, error)
	ListNamespaces() ([]string, error)
	ListNamespacesPage(limit int, cursor string) ([]string, string, error)
}

Index defines the interface for a tool registry.

Contract:

  • Concurrency: implementations must be safe for concurrent use.
  • Errors: validation failures should return ErrInvalidTool/ErrInvalidBackend; missing tools/backends should return ErrNotFound; cursor issues should return ErrInvalidCursor; pagination with non-deterministic searchers should return ErrNonDeterministicSearcher. Callers must use errors.Is.
  • Ownership: returned slices are caller-owned; elements are read-only and may be shared.
  • Determinism: Search/List methods must return stable ordering for identical inputs.
  • Nil/zero: empty inputs are treated as no-ops; SearchPage requires limit > 0.
  • Atomicity: batch registration is not guaranteed to be atomic on error.

type IndexOptions 

type IndexOptions struct {
	BackendSelector BackendSelector
	Searcher        Searcher
	// RequireDeterministicSearcher enforces deterministic ordering for pagination.
	// When true, SearchPage returns ErrNonDeterministicSearcher if the configured
	// searcher does not declare deterministic ordering.
	RequireDeterministicSearcher *bool
}

IndexOptions configures the behavior of an Index implementation.

type Refresher 

type Refresher interface {
	Refresh() uint64
}

Refresher is an optional interface for forcing a refresh of cached search docs.

Contract: - Refresh returns a monotonic version for the search doc cache. - Refresh must be safe for concurrent use.

type SearchDoc 

type SearchDoc struct {
	ID      string  // Canonical tool ID
	DocText string  // Lowercased concatenation of name/namespace/description/tags
	Summary Summary // Prebuilt summary for fast return
}

SearchDoc is the internal/exported struct used by Searcher implementations. It contains precomputed search data for efficient querying.

type Searcher 

type Searcher interface {
	// Search returns summaries ordered by relevance.
	// When used with SearchPage, the ordering must be deterministic for the same
	// docs/query input to guarantee cursor stability.
	Search(query string, limit int, docs []SearchDoc) ([]Summary, error)
}

Searcher is the interface for search implementations.

Contract: - Concurrency: implementations should be safe for concurrent use or document otherwise. - Ownership: docs and summaries are read-only and must not be mutated. - Determinism: identical inputs must yield stable ordering; tie-break deterministically. - Nil/zero: limit <= 0 must return an empty result set with nil error.

type Summary 

type Summary struct {
	ID               string   `json:"id"`
	Name             string   `json:"name"`
	Namespace        string   `json:"namespace,omitempty"`
	ShortDescription string   `json:"shortDescription,omitempty"`
	Summary          string   `json:"summary,omitempty"`
	Category         string   `json:"category,omitempty"`
	InputModes       []string `json:"inputModes,omitempty"`
	OutputModes      []string `json:"outputModes,omitempty"`
	SecuritySummary  string   `json:"securitySummary,omitempty"`
	Tags             []string `json:"tags,omitempty"`
}

Summary represents a lightweight view of a tool for search results. It contains only the essential information for display and discovery, without the full schema payloads.

type ToolRegistration 

type ToolRegistration struct {
	Tool    model.Tool
	Backend model.ToolBackend
}

ToolRegistration pairs a tool with its backend for batch registration.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.