Documentation
¶
Overview ¶
Package optimizer provides the Optimizer interface for intelligent tool discovery and invocation in the Virtual MCP Server.
When the optimizer is enabled, vMCP exposes only two tools to clients:
- find_tool: Semantic search over available tools
- call_tool: Dynamic invocation of any backend tool
This reduces token usage by avoiding the need to send all tool definitions to the LLM, instead allowing it to discover relevant tools on demand.
Index ¶
- func NewDummyOptimizerFactory() func(context.Context, []server.ServerTool) (Optimizer, error)
- func NewDummyOptimizerFactoryWithStore(store ToolStore, counter TokenCounter) func(context.Context, []server.ServerTool) (Optimizer, error)
- type CallToolInput
- type CharDivTokenCounter
- type DummyOptimizer
- type FindToolInput
- type FindToolOutput
- type InMemoryToolStore
- type Optimizer
- type TokenCounter
- type TokenMetrics
- type ToolMatch
- type ToolStore
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func NewDummyOptimizerFactory ¶ added in v0.9.4
NewDummyOptimizerFactory returns an OptimizerFactory that creates DummyOptimizer instances backed by a shared InMemoryToolStore. All optimizers created by the returned factory share the same underlying storage, enabling cross-session search.
func NewDummyOptimizerFactoryWithStore ¶ added in v0.9.4
func NewDummyOptimizerFactoryWithStore( store ToolStore, counter TokenCounter, ) func(context.Context, []server.ServerTool) (Optimizer, error)
NewDummyOptimizerFactoryWithStore returns an OptimizerFactory that creates DummyOptimizer instances backed by the given ToolStore. All optimizers created by the returned factory share the same store, enabling cross-session search.
Use this when you need to provide a specific store implementation (e.g., SQLiteToolStore for FTS5-based search) instead of the default InMemoryToolStore.
Types ¶
type CallToolInput ¶
type CallToolInput struct {
// ToolName is the name of the tool to invoke.
ToolName string `json:"tool_name" description:"Name of the tool to call"`
// Parameters are the arguments to pass to the tool.
Parameters map[string]any `json:"parameters" description:"Parameters to pass to the tool"`
}
CallToolInput contains the parameters for calling a tool.
type CharDivTokenCounter ¶ added in v0.9.4
type CharDivTokenCounter struct {
Divisor int
}
CharDivTokenCounter estimates token count by serialising the full mcp.Tool to JSON and dividing the byte length by a configurable divisor.
func (CharDivTokenCounter) CountTokens ¶ added in v0.9.4
func (c CharDivTokenCounter) CountTokens(tool mcp.Tool) int
CountTokens returns len(json(tool)) / divisor. Returns 0 if the divisor is zero or serialisation fails.
type DummyOptimizer ¶
type DummyOptimizer struct {
// contains filtered or unexported fields
}
DummyOptimizer implements the Optimizer interface using a shared ToolStore for search and a local handler map for tool invocation.
This implementation is intended for testing and development. It delegates search to the ToolStore (which performs case-insensitive substring matching for InMemoryToolStore) and scopes results to only the tools this instance was created with.
For production use, see the EmbeddingOptimizer which uses semantic similarity.
func (*DummyOptimizer) CallTool ¶
func (d *DummyOptimizer) CallTool(ctx context.Context, input CallToolInput) (*mcp.CallToolResult, error)
CallTool invokes a tool by name using its registered handler.
The tool is looked up by exact name match. If found, the handler is invoked directly with the given parameters.
func (*DummyOptimizer) FindTool ¶
func (d *DummyOptimizer) FindTool(ctx context.Context, input FindToolInput) (*FindToolOutput, error)
FindTool searches for tools using the shared ToolStore, scoped to this instance's tools.
Returns all matching tools with a score of 1.0 (exact match semantics). TokenMetrics quantify the token savings from returning only matching tools instead of the full set of available tools.
type FindToolInput ¶
type FindToolInput struct {
// ToolDescription is a natural language description of the tool to find.
ToolDescription string `json:"tool_description" description:"Natural language description of the tool to find"`
// ToolKeywords is an optional list of keywords to narrow the search.
ToolKeywords []string `json:"tool_keywords,omitempty" description:"Optional keywords to narrow search"`
}
FindToolInput contains the parameters for finding tools.
type FindToolOutput ¶
type FindToolOutput struct {
// Tools contains the matching tools, ranked by relevance.
Tools []ToolMatch `json:"tools"`
// TokenMetrics provides information about token savings from using the optimizer.
TokenMetrics TokenMetrics `json:"token_metrics"`
}
FindToolOutput contains the results of a tool search.
type InMemoryToolStore ¶ added in v0.9.4
type InMemoryToolStore struct {
// contains filtered or unexported fields
}
InMemoryToolStore implements ToolStore using an in-memory map with case-insensitive substring matching. Thread-safe via sync.RWMutex.
func NewInMemoryToolStore ¶ added in v0.9.4
func NewInMemoryToolStore() *InMemoryToolStore
NewInMemoryToolStore creates a new InMemoryToolStore.
func (*InMemoryToolStore) Close ¶ added in v0.9.4
func (*InMemoryToolStore) Close() error
Close is a no-op for InMemoryToolStore since there are no external resources to release. It is safe to call Close multiple times.
func (*InMemoryToolStore) Search ¶ added in v0.9.4
func (s *InMemoryToolStore) Search(_ context.Context, query string, allowedTools []string) ([]ToolMatch, error)
Search finds tools matching the query string using case-insensitive substring matching on tool name and description. The allowedTools parameter limits results to only tools with names in the given set. If allowedTools is empty, no results are returned (empty = no access).
func (*InMemoryToolStore) UpsertTools ¶ added in v0.9.4
func (s *InMemoryToolStore) UpsertTools(_ context.Context, tools []server.ServerTool) error
UpsertTools adds or updates tools in the store. Tools are identified by name; duplicate names are overwritten.
type Optimizer ¶
type Optimizer interface {
// FindTool searches for tools matching the given description and keywords.
// Returns matching tools ranked by relevance score.
FindTool(ctx context.Context, input FindToolInput) (*FindToolOutput, error)
// CallTool invokes a tool by name with the given parameters.
// Returns the tool's result or an error if the tool is not found or execution fails.
// Returns the MCP CallToolResult directly from the underlying tool handler.
CallTool(ctx context.Context, input CallToolInput) (*mcp.CallToolResult, error)
}
Optimizer defines the interface for intelligent tool discovery and invocation.
Implementations may use various strategies for tool matching:
- DummyOptimizer: Exact string matching (for testing)
- EmbeddingOptimizer: Semantic similarity via embeddings (production)
func NewDummyOptimizer ¶
func NewDummyOptimizer(ctx context.Context, store ToolStore, counter TokenCounter, tools []server.ServerTool) (Optimizer, error)
NewDummyOptimizer creates a new DummyOptimizer backed by the given ToolStore.
The tools slice should contain all backend tools (as ServerTool with handlers). Tools are upserted into the shared store and scoped for this optimizer instance. Token counts are precomputed using the provided counter for metrics calculation.
type TokenCounter ¶ added in v0.9.4
TokenCounter estimates the number of tokens a tool definition would consume when sent to an LLM. Implementations may use character-based heuristics or real tokenizers.
func DefaultTokenCounter ¶ added in v0.9.4
func DefaultTokenCounter() TokenCounter
DefaultTokenCounter returns a CharDivTokenCounter with a divisor of 4, which is a reasonable approximation for most LLM tokenizers.
type TokenMetrics ¶
type TokenMetrics struct {
// BaselineTokens is the estimated tokens if all tools were sent.
BaselineTokens int `json:"baseline_tokens"`
// ReturnedTokens is the actual tokens for the returned tools.
ReturnedTokens int `json:"returned_tokens"`
// SavingsPercent is the percentage of tokens saved.
SavingsPercent float64 `json:"savings_percent"`
}
TokenMetrics provides information about token usage optimization.
type ToolMatch ¶
ToolMatch represents a tool that matched the search criteria. It is defined in the internal/types package and aliased here so that external consumers continue to use optimizer.ToolMatch.
type ToolStore ¶ added in v0.9.4
type ToolStore interface {
// UpsertTools adds or updates tools in the store.
// Tools are identified by name; duplicate names are overwritten.
UpsertTools(ctx context.Context, tools []server.ServerTool) error
// Search finds tools matching the query string.
// The allowedTools parameter limits results to only tools with names in the given set.
// If allowedTools is empty, no results are returned (empty = no access).
// Returns matches ranked by relevance.
Search(ctx context.Context, query string, allowedTools []string) ([]ToolMatch, error)
// Close releases any resources held by the store (e.g., database connections).
// For in-memory stores this is a no-op.
// It is safe to call Close multiple times.
Close() error
}
ToolStore defines the interface for storing and searching tools. Implementations may use in-memory maps, SQLite FTS5, or other backends.
A ToolStore is shared across multiple optimizer instances (one per session) and is accessed concurrently. Implementations must be thread-safe.
func NewSQLiteToolStore ¶ added in v0.9.4
NewSQLiteToolStore creates a new ToolStore backed by SQLite for search. The store uses an in-memory SQLite database with shared cache for concurrent access.
Directories
¶
| Path | Synopsis |
|---|---|
|
internal
|
|
|
sqlite_store
Package sqlitestore implements a SQLite-based ToolStore for search over MCP tool metadata.
|
Package sqlitestore implements a SQLite-based ToolStore for search over MCP tool metadata. |
|
types
Package types defines shared types used across optimizer sub-packages.
|
Package types defines shared types used across optimizer sub-packages. |