router

type Client ¶

type Client struct {
	// Endpoint is the zen-router base URL (e.g. http://engine.hanzo.svc:3000).
	// Empty disables the engine strategy — heuristic only.
	Endpoint string
	// Timeout hard-caps the engine round-trip (default DefaultTimeout).
	Timeout time.Duration
	// Policy is the heuristic task→model table (and the map for an engine reply
	// that returns only a task).
	Policy Policy
	// HTTP is the client used for the engine call (default http.DefaultClient).
	HTTP *http.Client
	// Known, if set, restricts choices to models the caller can serve; a choice
	// it rejects is skipped. nil accepts all.
	Known func(string) bool
}

Client resolves a Request to a concrete model id. Strategy order:

1. engine — POST {Endpoint}/route and use its choice (a learned zen-router served OpenAI-compatibly by hanzo-engine); hard-capped by Timeout.
2. heuristic — Classify + Policy, a pure-Go port of the Rust rule router.

Any engine error (unreachable, timeout, non-200, bad body, unservable choice) falls through to the heuristic, so `auto` always resolves with zero extra infrastructure.

func (c Client) Observe(ctx context.Context, obs ObserveRequest)

func (c Client) Route(ctx context.Context, req Request, slo Slo) (model string, task Task)

func (c Client) RouteDecision(ctx context.Context, req Request, slo Slo) Decision

type Decision ¶

type Decision struct {
	Model      string
	Task       Task
	Confidence float64
	Source     string
	Features   []float64
}

Decision is the full outcome of resolving a request: the chosen model, the task it was classified as, the engine confidence (0 for the heuristic), the Source that produced it, and the engine's optional feature vector. It carries exactly what the routing ledger records — no prompt text.

type ObserveRequest ¶ added in v1.813.1

type ObserveRequest struct {
	User     string    `json:"user"`
	Org      string    `json:"org"`
	Features []float64 `json:"features"`
	Model    string    `json:"model"`
	Reward   float64   `json:"reward"`
}

ObserveRequest is the online-learning feedback the gateway posts to the engine after a reward is joined: the request's feature vector `x` (the same vector the engine returned at decision time), the arm that served, the caller (keys the per-user LinUCB bandit), the caller org (selects the scope), and the outcome reward in [0,1]. No prompt text.

type Policy ¶

type Policy struct {
	// Prefer maps a task tag (Task values, plus "default") to an ordered list of
	// model ids. The first entry accepted by the `known` predicate wins.
	Prefer map[string][]string
	// CostCeiling is an optional advisory cost cap in USD PER 1K TOKENS (per-1k — NOT
	// the $/1M unit of the pricing table) forwarded verbatim to the engine SLO as
	// Slo.MaxCost; it does not filter the heuristic table (the table is already
	// curated). See the PerMillionToPerThousand note above for the unit relationship.
	CostCeiling float64
}

Policy is the per-task model preference table plus a cost knob. It mirrors the declarative shape of hanzo-router's router_policy.yaml: ordered model-id preference per task, first usable wins, with a General/`default` catch-all.

func (p Policy) ForTask(task Task, known func(string) bool) string

type Request ¶

type Request struct {
	// Text is the concatenated user text (the last user turn is enough).
	Text string
	// ApproxTokens is the approximate prompt length in tokens (drives
	// long-context routing).
	ApproxTokens int
	// HasMedia is true when the request carries image/video input.
	HasMedia bool
	// TaskHint is an optional explicit task override; "" skips it.
	TaskHint Task
}

Request is the value the router reasons about — extracted from the chat request by the caller so classification stays pure. Mirrors classify.rs's Request.

type Slo ¶

type Slo struct {
	MaxCost      float64 `json:"max_cost"`       // cost cap, USD PER 1K TOKENS (per-1k); 0 disables
	MaxLatencyMs int     `json:"max_latency_ms"` // latency cap, milliseconds; 0 disables
}

Slo is the per-request service-level objective forwarded to the engine. `0` fields disable the corresponding ceiling. It mirrors hanzo-router's Slo (the operator's budget for this request), distinct from a user's learned taste.

type Task ¶

type Task string

Task is the coarse task bucket a request is classified into. Values mirror the snake_case tags of hanzo-router's registry::Task so a learned classifier and this heuristic produce interchangeable labels.

const (
	TaskCode        Task = "code"
	TaskReasoning   Task = "reasoning"
	TaskMath        Task = "math"
	TaskCreative    Task = "creative"
	TaskVision      Task = "vision"
	TaskLongContext Task = "long_context"
	TaskCheapChat   Task = "cheap_chat"
	TaskGeneral     Task = "general"
)

func Classify(req Request) Task