routing

package
v0.11.0-beta.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 23, 2026 License: Apache-2.0, MIT Imports: 9 Imported by: 0

Documentation

Overview

Package routing — adapter that lets a SmartRouter satisfy the agent.ModelRouter contract without the agent package importing routing.

Package routing — TOML catalog integration (master plan §4.2 + §5.2).

The router was originally constructed from a network catalog (providers.FetchCatalog → models.dev mirror). The §4.2 plan calls for a LOCAL TOML catalog at `~/.overkill/models/` as the canonical source. This adapter bridges the local catalog into the existing SmartRouter API and adds family/capability operations the network catalog couldn't express.

Two integration points:

  1. Construction-time: ProviderModelsFromCatalog flattens local TOML into the existing `[]ProviderModels` shape so NewSmartRouter keeps the same signature.
  2. Runtime: SmartRouter.WithCatalog attaches the catalog so ModelInFamily / ModelWithCapabilities / FailoverInFamily can consult it for family-aware decisions.

Index

Constants

View Source 
const DefaultEstimatedOutputTokens = 1024

DefaultEstimatedOutputTokens is the completion assumption for routing estimates.

Variables

This section is empty.

Functions

This section is empty.

Types

type AgentAdapter 

type AgentAdapter struct {
	// contains filtered or unexported fields
}

AgentAdapter wraps SmartRouter so its Route() output is consumable by the agent's ModelRouter interface.

func NewAgentAdapter 

func NewAgentAdapter(r *SmartRouter) *AgentAdapter

NewAgentAdapter wraps r. Returns nil when r is nil so the agent can short-circuit cleanly.

func (*AgentAdapter) PickModel 

func (a *AgentAdapter) PickModel(snap agent.RouteSnapshot) (string, string, bool)

PickModel runs the underlying classifier+router and returns the chosen model ID. Failures return ok=false so the caller falls back to its static model.

type Classifier 

type Classifier struct {
	// contains filtered or unexported fields
}

func NewClassifier 

func NewClassifier(thresholds ClassifierThresholds) *Classifier

func (*Classifier) Classify 

func (c *Classifier) Classify(req RouteRequest) ComplexityScore

type ClassifierThresholds 

type ClassifierThresholds struct {
	SimpleModerate  float64 `json:"simple_moderate"`
	ModerateComplex float64 `json:"moderate_complex"`
	ComplexCritical float64 `json:"complex_critical"`
}

func DefaultThresholds 

func DefaultThresholds() ClassifierThresholds

type ComplexityLevel 

type ComplexityLevel int
const (
	ComplexitySimple ComplexityLevel = iota
	ComplexityModerate
	ComplexityComplex
	ComplexityCritical
)

func (ComplexityLevel) String 

func (l ComplexityLevel) String() string

type ComplexityScore 

type ComplexityScore struct {
	Level       ComplexityLevel    `json:"level"`
	Score       float64            `json:"score"`
	Factors     map[string]float64 `json:"factors"`
	Explanation string             `json:"explanation"`
}

type ProviderModels 

type ProviderModels struct {
	ProviderName string            `json:"provider_name"`
	Models       []providers.Model `json:"models"`
	// PricingKnown preserves explicit zero pricing versus absent metadata.
	// An absent entry infers known only from finite nonzero rates. A false
	// entry marks pricing unknown even when scalar rates contain stale values.
	PricingKnown map[string]bool `json:"pricing_known,omitempty"`
}

func ProviderModelsFromCatalog 

func ProviderModelsFromCatalog(cat *models.Catalog) []ProviderModels

ProviderModelsFromCatalog converts a TOML model catalog into the []ProviderModels slice the SmartRouter constructor expects. One ProviderModels per provider, derived from the ID prefix before `/` ("openai/gpt-5" → provider "openai"). Returns nil when the catalog is empty so callers can fall back to a network catalog cleanly.

type RouteRequest 

type RouteRequest struct {
	UserInput      string `json:"user_input"`
	HistoryLength  int    `json:"history_length"`
	ToolCallCount  int    `json:"tool_call_count"`
	HasAttachments bool   `json:"has_attachments"`
	CodeBlockCount int    `json:"code_block_count"`
	// EstimatedTokens is the estimated input token count (legacy field name).
	// Zero infers UserInput text only, without hidden history/media tokens;
	// callers with a full prompt estimate should supply it. Negative values are invalid.
	EstimatedTokens int `json:"estimated_tokens"`
	// EstimatedOutputTokens is planned completion size, not a provider limit.
	// Zero uses DefaultEstimatedOutputTokens; negative estimates are invalid.
	EstimatedOutputTokens int      `json:"estimated_output_tokens,omitempty"`
	RequiredCapabilities  []string `json:"required_capabilities"`
}

type RouteResult 

type RouteResult struct {
	ModelID    string          `json:"model_id"`
	ModelName  string          `json:"model_name"`
	Provider   string          `json:"provider"`
	Complexity ComplexityScore `json:"complexity"`
	// CostEstimate is USD for this request, not a per-million-token rate.
	// It is meaningful only when CostEstimateKnown is true. Cache, media,
	// tiered pricing, retries and subsequent tool turns are not included.
	CostEstimate          float64 `json:"cost_estimate"`
	CostEstimateKnown     bool    `json:"cost_estimate_known"`
	EstimatedInputTokens  int     `json:"estimated_input_tokens"`
	EstimatedOutputTokens int     `json:"estimated_output_tokens"`
	Reason                string  `json:"reason"`
}

type Router 

type Router interface {
	Route(ctx context.Context, req RouteRequest) (*RouteResult, error)
}

type SmartRouter 

type SmartRouter struct {
	// contains filtered or unexported fields
}

func NewSmartRouter 

func NewSmartRouter(classifier *Classifier, providers []ProviderModels, defaultModel string) *SmartRouter

func (*SmartRouter) CheapestModel 

func (r *SmartRouter) CheapestModel(needsVision, needsTools bool) (modelID, provider string)

CheapestModel returns the single cheapest model matching the given capability requirements. Used by the built-in agent system to assign cost-appropriate models: Explore/Plan get the cheapest reader, Verify gets the cheapest tool-capable model, General-purpose inherits. With no request mix this compares equal input/output volumes and requires known prices.

func (*SmartRouter) FailoverInFamily 

func (r *SmartRouter) FailoverInFamily(family string) []string

FailoverInFamily returns the family's members ordered by cost, non-deprecated only. Caller iterates and picks the first reachable provider — e.g. when claude-opus's primary endpoint is rate-limited, retry the next-cheapest opus before leaving the family.

func (*SmartRouter) ModelForComplexity 

func (r *SmartRouter) ModelForComplexity(level ComplexityLevel) (string, string, error)

ModelForComplexity preserves the legacy signature for callers that don't know their capability requirements. Internally delegates to modelForComplexityWithCaps with no-caps; new code should call the capability-aware form directly. Without a request token mix, this method ranks equal input/output volumes; Route uses its explicit token assumptions.

func (*SmartRouter) ModelInFamily 

func (r *SmartRouter) ModelInFamily(family string) (string, string, error)

ModelInFamily returns the cheapest non-deprecated model in the family, falling back to the registered providers when the catalog isn't attached. Empty family returns ErrNotFound.

Returns (modelID, providerID, error). Provider derived from the model ID's prefix.

func (*SmartRouter) ModelWithCapabilities 

func (r *SmartRouter) ModelWithCapabilities(req models.Capabilities) (string, string, error)

ModelWithCapabilities returns the cheapest model whose capability flags satisfy every requested flag in req. When the catalog is attached we use its rich Capabilities struct; otherwise fall back to the legacy providers.Model bool fields.

func (*SmartRouter) Route 

func (r *SmartRouter) Route(ctx context.Context, req RouteRequest) (*RouteResult, error)

func (*SmartRouter) SetCostPriority 

func (r *SmartRouter) SetCostPriority(enabled bool)

SetCostPriority switches from legacy complexity tiers to the lowest known request estimate, after capability and token-budget filtering. It is safe to toggle while routing. This is a cost preference, not a measured quality score.

func (*SmartRouter) WithCatalog 

func (r *SmartRouter) WithCatalog(cat *models.Catalog) *SmartRouter

WithCatalog attaches a TOML catalog so the router can answer family + capability queries that the flat []ProviderModels view can't represent cheaply. Idempotent — replaces any previously attached catalog. Pass nil to detach.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.