router

package
v0.8.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 10, 2026 License: Apache-2.0 Imports: 6 Imported by: 0

Documentation

Overview

Package router handles intelligent model selection.

Index

Constants

View Source
const (
	// HintChannel identifies the request source: "ollama", "homeassistant", "voice", "api"
	HintChannel = "channel"
	// HintQualityFloor is the minimum quality rating (1-10) the caller requires.
	HintQualityFloor = "quality_floor"
	// HintModelPreference suggests a specific model (soft preference, not override).
	HintModelPreference = "model_preference"
	// HintMission describes the task context: "conversation", "device_control", "background", "anticipation", "automation"
	HintMission = "mission"
	// HintLocalOnly restricts routing to free/local models when set to "true".
	HintLocalOnly = "local_only"
	// HintDelegationGating controls whether delegation-first tool gating is
	// active. Set to "disabled" to give the model direct access to all tools
	// on every iteration (used by thane:ops).
	HintDelegationGating = "delegation_gating"
	// HintPreferSpeed indicates the caller benefits from faster response
	// times over higher quality. When "true", any model with Speed >= 7
	// receives a scoring bonus regardless of cost tier or provider. Can be
	// decisive among similarly priced options. Use for background/delegation
	// tasks where latency and resource efficiency matter more than maximum
	// output quality.
	HintPreferSpeed = "prefer_speed"
)

Hint keys for routing decisions. Callers set these to influence model selection.

Variables

This section is empty.

Functions

This section is empty.

Types

type Complexity

type Complexity int

Complexity categorizes query difficulty.

const (
	ComplexitySimple   Complexity = iota // Direct command, single action
	ComplexityModerate                   // Multi-step or needs context
	ComplexityComplex                    // Reasoning, analysis, explanation
)

func (Complexity) String

func (c Complexity) String() string

String returns the human-readable name of a complexity level.

type Config

type Config struct {
	Models       []Model // Available models
	DefaultModel string  // Fallback if no rules match
	LocalFirst   bool    // Prefer local models when possible
	MaxAuditLog  int     // How many decisions to keep in memory
}

Config holds router configuration.

type Decision

type Decision struct {
	RequestID string    `json:"request_id"`
	Timestamp time.Time `json:"timestamp"`

	// Input analysis
	QueryLength    int        `json:"query_length"`
	ContextSize    int        `json:"context_size"`
	NeedsTools     bool       `json:"needs_tools"`
	Priority       string     `json:"priority"`
	DetectedIntent string     `json:"detected_intent,omitempty"`
	Complexity     Complexity `json:"complexity"`

	// Decision process
	RulesEvaluated []string       `json:"rules_evaluated"`
	RulesMatched   []string       `json:"rules_matched"`
	Scores         map[string]int `json:"scores,omitempty"`

	// Outcome
	ModelSelected string `json:"model_selected"`
	Reasoning     string `json:"reasoning"`

	// Post-execution (filled in later)
	LatencyMs  int64 `json:"latency_ms,omitempty"`
	TokensUsed int   `json:"tokens_used,omitempty"`
	Success    *bool `json:"success,omitempty"`
}

Decision records why a model was selected.

type Model

type Model struct {
	Name          string     // Model identifier (e.g., "qwen3:4b")
	Provider      string     // "ollama" or "anthropic" etc
	SupportsTools bool       // Can do tool calling
	ContextWindow int        // Max tokens
	Speed         int        // Relative speed (1-10, 10=fastest)
	Quality       int        // Relative quality (1-10, 10=best)
	CostTier      int        // 0=free/local, 1=cheap, 2=moderate, 3=expensive
	MinComplexity Complexity // Don't use for simpler than this
}

Model represents an available model with its capabilities.

type Priority

type Priority int

Priority indicates latency requirements.

const (
	PriorityInteractive Priority = iota // User waiting, needs fast response
	PriorityBackground                  // Can take longer for better quality
)

type Request

type Request struct {
	Query       string            // The user's input
	ContextSize int               // Estimated tokens of context (talents, history)
	NeedsTools  bool              // Whether tool calling is required
	ToolCount   int               // Number of tools available
	Priority    Priority          // Latency requirements
	Hints       map[string]string // Caller-supplied routing hints (see HintXxx constants)
}

Request contains the information needed for routing decisions.

type Router

type Router struct {
	// contains filtered or unexported fields
}

Router selects models based on request characteristics.

func NewRouter

func NewRouter(logger *slog.Logger, config Config) *Router

NewRouter creates a router with the given configuration.

func (*Router) Explain

func (r *Router) Explain(requestID string) *Decision

Explain returns details about why a specific decision was made.

func (*Router) GetAuditLog

func (r *Router) GetAuditLog(limit int) []Decision

GetAuditLog returns recent routing decisions.

func (*Router) GetModels added in v0.8.0

func (r *Router) GetModels() []Model

GetModels returns a copy of the configured model list. The returned slice is safe to mutate without affecting the router.

func (*Router) GetStats

func (r *Router) GetStats() Stats

GetStats returns routing statistics.

func (*Router) MaxQuality added in v0.6.0

func (r *Router) MaxQuality() int

MaxQuality returns the highest quality rating among configured models. If no models are configured it returns 10 as a safe default that selects the best available model at runtime.

func (*Router) RecordOutcome

func (r *Router) RecordOutcome(requestID string, latencyMs int64, tokensUsed int, success bool)

RecordOutcome updates a decision with execution results.

func (*Router) Route

func (r *Router) Route(ctx context.Context, req Request) (string, *Decision)

Route selects a model for the given request.

type Stats

type Stats struct {
	TotalRequests    int64            `json:"total_requests"`
	ModelCounts      map[string]int64 `json:"model_counts"`
	AvgLatencyMs     map[string]int64 `json:"avg_latency_ms"`
	ComplexityCounts map[string]int64 `json:"complexity_counts"`
}

Stats tracks routing statistics.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL