optimizer

package
v0.9.4 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Feb 16, 2026 License: Apache-2.0 Imports: 9 Imported by: 0

Documentation

Overview

Package optimizer provides the Optimizer interface for intelligent tool discovery and invocation in the Virtual MCP Server.

When the optimizer is enabled, vMCP exposes only two tools to clients:

  • find_tool: Semantic search over available tools
  • call_tool: Dynamic invocation of any backend tool

This reduces token usage by avoiding the need to send all tool definitions to the LLM, instead allowing it to discover relevant tools on demand.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func NewDummyOptimizerFactory added in v0.9.4

func NewDummyOptimizerFactory() func(context.Context, []server.ServerTool) (Optimizer, error)

NewDummyOptimizerFactory returns an OptimizerFactory that creates DummyOptimizer instances backed by a shared InMemoryToolStore. All optimizers created by the returned factory share the same underlying storage, enabling cross-session search.

func NewDummyOptimizerFactoryWithStore added in v0.9.4

func NewDummyOptimizerFactoryWithStore(
	store ToolStore, counter TokenCounter,
) func(context.Context, []server.ServerTool) (Optimizer, error)

NewDummyOptimizerFactoryWithStore returns an OptimizerFactory that creates DummyOptimizer instances backed by the given ToolStore. All optimizers created by the returned factory share the same store, enabling cross-session search.

Use this when you need to provide a specific store implementation (e.g., SQLiteToolStore for FTS5-based search) instead of the default InMemoryToolStore.

Types

type CallToolInput

type CallToolInput struct {
	// ToolName is the name of the tool to invoke.
	ToolName string `json:"tool_name" description:"Name of the tool to call"`

	// Parameters are the arguments to pass to the tool.
	Parameters map[string]any `json:"parameters" description:"Parameters to pass to the tool"`
}

CallToolInput contains the parameters for calling a tool.

type CharDivTokenCounter added in v0.9.4

type CharDivTokenCounter struct {
	Divisor int
}

CharDivTokenCounter estimates token count by serialising the full mcp.Tool to JSON and dividing the byte length by a configurable divisor.

func (CharDivTokenCounter) CountTokens added in v0.9.4

func (c CharDivTokenCounter) CountTokens(tool mcp.Tool) int

CountTokens returns len(json(tool)) / divisor. Returns 0 if the divisor is zero or serialisation fails.

type DummyOptimizer

type DummyOptimizer struct {
	// contains filtered or unexported fields
}

DummyOptimizer implements the Optimizer interface using a shared ToolStore for search and a local handler map for tool invocation.

This implementation is intended for testing and development. It delegates search to the ToolStore (which performs case-insensitive substring matching for InMemoryToolStore) and scopes results to only the tools this instance was created with.

For production use, see the EmbeddingOptimizer which uses semantic similarity.

func (*DummyOptimizer) CallTool

func (d *DummyOptimizer) CallTool(ctx context.Context, input CallToolInput) (*mcp.CallToolResult, error)

CallTool invokes a tool by name using its registered handler.

The tool is looked up by exact name match. If found, the handler is invoked directly with the given parameters.

func (*DummyOptimizer) FindTool

func (d *DummyOptimizer) FindTool(ctx context.Context, input FindToolInput) (*FindToolOutput, error)

FindTool searches for tools using the shared ToolStore, scoped to this instance's tools.

Returns all matching tools with a score of 1.0 (exact match semantics). TokenMetrics quantify the token savings from returning only matching tools instead of the full set of available tools.

type FindToolInput

type FindToolInput struct {
	// ToolDescription is a natural language description of the tool to find.
	ToolDescription string `json:"tool_description" description:"Natural language description of the tool to find"`

	// ToolKeywords is an optional list of keywords to narrow the search.
	ToolKeywords []string `json:"tool_keywords,omitempty" description:"Optional keywords to narrow search"`
}

FindToolInput contains the parameters for finding tools.

type FindToolOutput

type FindToolOutput struct {
	// Tools contains the matching tools, ranked by relevance.
	Tools []ToolMatch `json:"tools"`

	// TokenMetrics provides information about token savings from using the optimizer.
	TokenMetrics TokenMetrics `json:"token_metrics"`
}

FindToolOutput contains the results of a tool search.

type InMemoryToolStore added in v0.9.4

type InMemoryToolStore struct {
	// contains filtered or unexported fields
}

InMemoryToolStore implements ToolStore using an in-memory map with case-insensitive substring matching. Thread-safe via sync.RWMutex.

func NewInMemoryToolStore added in v0.9.4

func NewInMemoryToolStore() *InMemoryToolStore

NewInMemoryToolStore creates a new InMemoryToolStore.

func (*InMemoryToolStore) Close added in v0.9.4

func (*InMemoryToolStore) Close() error

Close is a no-op for InMemoryToolStore since there are no external resources to release. It is safe to call Close multiple times.

func (*InMemoryToolStore) Search added in v0.9.4

func (s *InMemoryToolStore) Search(_ context.Context, query string, allowedTools []string) ([]ToolMatch, error)

Search finds tools matching the query string using case-insensitive substring matching on tool name and description. The allowedTools parameter limits results to only tools with names in the given set. If allowedTools is empty, no results are returned (empty = no access).

func (*InMemoryToolStore) UpsertTools added in v0.9.4

func (s *InMemoryToolStore) UpsertTools(_ context.Context, tools []server.ServerTool) error

UpsertTools adds or updates tools in the store. Tools are identified by name; duplicate names are overwritten.

type Optimizer

type Optimizer interface {
	// FindTool searches for tools matching the given description and keywords.
	// Returns matching tools ranked by relevance score.
	FindTool(ctx context.Context, input FindToolInput) (*FindToolOutput, error)

	// CallTool invokes a tool by name with the given parameters.
	// Returns the tool's result or an error if the tool is not found or execution fails.
	// Returns the MCP CallToolResult directly from the underlying tool handler.
	CallTool(ctx context.Context, input CallToolInput) (*mcp.CallToolResult, error)
}

Optimizer defines the interface for intelligent tool discovery and invocation.

Implementations may use various strategies for tool matching:

  • DummyOptimizer: Exact string matching (for testing)
  • EmbeddingOptimizer: Semantic similarity via embeddings (production)

func NewDummyOptimizer

func NewDummyOptimizer(ctx context.Context, store ToolStore, counter TokenCounter, tools []server.ServerTool) (Optimizer, error)

NewDummyOptimizer creates a new DummyOptimizer backed by the given ToolStore.

The tools slice should contain all backend tools (as ServerTool with handlers). Tools are upserted into the shared store and scoped for this optimizer instance. Token counts are precomputed using the provided counter for metrics calculation.

type TokenCounter added in v0.9.4

type TokenCounter interface {
	CountTokens(tool mcp.Tool) int
}

TokenCounter estimates the number of tokens a tool definition would consume when sent to an LLM. Implementations may use character-based heuristics or real tokenizers.

func DefaultTokenCounter added in v0.9.4

func DefaultTokenCounter() TokenCounter

DefaultTokenCounter returns a CharDivTokenCounter with a divisor of 4, which is a reasonable approximation for most LLM tokenizers.

type TokenMetrics

type TokenMetrics struct {
	// BaselineTokens is the estimated tokens if all tools were sent.
	BaselineTokens int `json:"baseline_tokens"`

	// ReturnedTokens is the actual tokens for the returned tools.
	ReturnedTokens int `json:"returned_tokens"`

	// SavingsPercent is the percentage of tokens saved.
	SavingsPercent float64 `json:"savings_percent"`
}

TokenMetrics provides information about token usage optimization.

type ToolMatch

type ToolMatch = types.ToolMatch

ToolMatch represents a tool that matched the search criteria. It is defined in the internal/types package and aliased here so that external consumers continue to use optimizer.ToolMatch.

type ToolStore added in v0.9.4

type ToolStore interface {
	// UpsertTools adds or updates tools in the store.
	// Tools are identified by name; duplicate names are overwritten.
	UpsertTools(ctx context.Context, tools []server.ServerTool) error

	// Search finds tools matching the query string.
	// The allowedTools parameter limits results to only tools with names in the given set.
	// If allowedTools is empty, no results are returned (empty = no access).
	// Returns matches ranked by relevance.
	Search(ctx context.Context, query string, allowedTools []string) ([]ToolMatch, error)

	// Close releases any resources held by the store (e.g., database connections).
	// For in-memory stores this is a no-op.
	// It is safe to call Close multiple times.
	Close() error
}

ToolStore defines the interface for storing and searching tools. Implementations may use in-memory maps, SQLite FTS5, or other backends.

A ToolStore is shared across multiple optimizer instances (one per session) and is accessed concurrently. Implementations must be thread-safe.

func NewSQLiteToolStore added in v0.9.4

func NewSQLiteToolStore() (ToolStore, error)

NewSQLiteToolStore creates a new ToolStore backed by SQLite for search. The store uses an in-memory SQLite database with shared cache for concurrent access.

Directories

Path Synopsis
internal
sqlite_store
Package sqlitestore implements a SQLite-based ToolStore for search over MCP tool metadata.
Package sqlitestore implements a SQLite-based ToolStore for search over MCP tool metadata.
types
Package types defines shared types used across optimizer sub-packages.
Package types defines shared types used across optimizer sub-packages.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL