router

type Client ¶

type Client struct {
	// Endpoint is the zen-router base URL (e.g. http://engine.hanzo.svc:3000).
	// Empty disables the engine strategy — heuristic only.
	Endpoint string
	// Timeout hard-caps the engine round-trip (default DefaultTimeout).
	Timeout time.Duration
	// Policy is the heuristic task→model table (and the map for an engine reply
	// that returns only a task).
	Policy Policy
	// HTTP is the client used for the engine call (default http.DefaultClient).
	HTTP *http.Client
	// Known, if set, restricts choices to models the caller can serve; a choice
	// it rejects is skipped. nil accepts all.
	Known func(string) bool
	// Allow, if set, is the HARD allowlist (the org's enabled-models set) — a
	// constraint the heuristic's last-resort fallback must NOT relax, unlike Known
	// (servability), which it does relax to avoid dead-ending `auto`. So a model the
	// org has DISABLED is never routed to, even when no servable preferred model
	// remains; only servability is relaxed as a last resort, never the allowlist.
	// nil = no allowlist (the last resort relaxes everything, the historical behavior).
	Allow func(string) bool
	// Explore is the epsilon exploration floor in [0,1]: the probability the
	// heuristic samples a RANDOM servable NON-champion arm instead of the champion,
	// so the bandit keeps collecting reward on the whole pool (and reacts to newly
	// added models) rather than collapsing onto today's champion — the reward from
	// an explore request trains it exactly like an exploit request. 0 = pure
	// exploit. The engine strategy does its own UCB exploration, so this governs the
	// heuristic path.
	Explore float64
	// Rand, if set, sources the exploration coin flip (injectable for deterministic
	// tests); nil uses the auto-seeded package generator.
	Rand *rand.Rand
}

Client resolves a Request to a concrete model id. Strategy order:

1. engine — POST {Endpoint}/route and use its choice (a learned zen-router served OpenAI-compatibly by hanzo-engine); hard-capped by Timeout.
2. heuristic — Classify + Policy, a pure-Go port of the Rust rule router.

Any engine error (unreachable, timeout, non-200, bad body, unservable choice) falls through to the heuristic, so `auto` always resolves with zero extra infrastructure.

func (c Client) Observe(ctx context.Context, obs ObserveRequest)

func (c Client) Route(ctx context.Context, req Request, slo Slo) (model string, task Task)

func (c Client) RouteDecision(ctx context.Context, req Request, slo Slo) Decision

func (c Client) RouteDecisionFor(ctx context.Context, req Request, slo Slo, rp RoutingPolicy) Decision

override → strategy (Enso engine, when selected and reachable) → heuristic

type Decision ¶

type Decision struct {
	Model      string
	Task       Task
	Confidence float64
	Source     string
	Features   []float64
}

Decision is the full outcome of resolving a request: the chosen model, the task it was classified as, the engine confidence (0 for the heuristic), the Source that produced it, and the engine's optional feature vector. It carries exactly what the routing ledger records — no prompt text.

type ObserveRequest ¶ added in v1.813.1

type ObserveRequest struct {
	User     string    `json:"user"`
	Org      string    `json:"org"`
	Features []float64 `json:"features"`
	Model    string    `json:"model"`
	Reward   float64   `json:"reward"`
}

ObserveRequest is the online-learning feedback the gateway posts to the engine after a reward is joined: the request's feature vector `x` (the same vector the engine returned at decision time), the arm that served, the caller (keys the per-user LinUCB bandit), the caller org (selects the scope), and the outcome reward in [0,1]. No prompt text.

type Policy ¶

type Policy struct {
	// Prefer maps a task tag (Task values, plus "default") to an ordered list of
	// model ids. The first entry accepted by the `known` predicate wins.
	Prefer map[string][]string
	// CostCeiling is an optional advisory cost cap in USD PER 1K TOKENS (per-1k — NOT
	// the $/1M unit of the pricing table) forwarded verbatim to the engine SLO as
	// Slo.MaxCost; it does not filter the heuristic table (the table is already
	// curated). See the PerMillionToPerThousand note above for the unit relationship.
	CostCeiling float64
}

Policy is the per-task model preference table plus a cost knob. It mirrors the declarative shape of hanzo-router's router_policy.yaml: ordered model-id preference per task, first usable wins, with a General/`default` catch-all.

func (p Policy) Candidates(task Task, known func(string) bool) []string

func (p Policy) ForTask(task Task, known func(string) bool) string

type Request ¶

type Request struct {
	// Text is the concatenated user text (the last user turn is enough).
	Text string
	// ApproxTokens is the approximate prompt length in tokens (drives
	// long-context routing).
	ApproxTokens int
	// HasMedia is true when the request carries image/video input.
	HasMedia bool
	// TaskHint is an optional explicit task override; "" skips it.
	TaskHint Task
}

Request is the value the router reasons about — extracted from the chat request by the caller so classification stays pure. Mirrors classify.rs's Request.

type RoutingPolicy ¶ added in v1.827.0

type RoutingPolicy struct {
	Strategy  Strategy
	Overrides map[string]string
	Enabled   func(string) bool
	Prefer    Policy
}

RoutingPolicy is the per-org (or per-surface) configuration the Hanzo Router resolves a request against — a VALUE, not a place. It composes four orthogonal knobs, each independently complete:

• Strategy — which strategy leads: StrategyEnso (learned) or StrategyHeuristic (rules only). Overrides always precede it; the heuristic is always the floor.
• Overrides — deterministic task→model pins the org sets by hand: Overrides["code"] forces a model for coding tasks, Overrides[DefaultTaskKey] is the catch-all. An override naming a model the org cannot serve (per Enabled) is skipped, never honored blindly.
• Enabled — the org's servable model set (its connected providers + toggled models). This is exactly the Client.Known predicate, sourced per-org rather than per-process. nil means "all models allowed".
• Prefer — the org's heuristic task→model table (+ cost ceiling). An empty table means "use the Client's process-default Policy".

The zero RoutingPolicy is inert — nil Enabled, no Overrides, empty Prefer, empty Strategy (enso-leaning) — so it reproduces the historical Client behavior exactly. An org with no configured policy routes as it always did.

type Slo ¶

type Slo struct {
	MaxCost      float64 `json:"max_cost"`       // cost cap, USD PER 1K TOKENS (per-1k); 0 disables
	MaxLatencyMs int     `json:"max_latency_ms"` // latency cap, milliseconds; 0 disables
}

Slo is the per-request service-level objective forwarded to the engine. `0` fields disable the corresponding ceiling. It mirrors hanzo-router's Slo (the operator's budget for this request), distinct from a user's learned taste.

type Strategy ¶ added in v1.827.0

type Strategy string

Strategy selects which routing strategy leads for an org or surface. A deterministic Override always precedes the strategy, and the heuristic is always the final fallback, so a Decision never dead-ends.

const (
	// StrategyEnso delegates the decision to the learned Enso model — the engine
	// reached at Client.Endpoint. It is the default: the zero Strategy behaves as
	// enso-leaning (try the engine, fall back to the heuristic).
	StrategyEnso Strategy = "enso"
	// StrategyHeuristic uses the rule router only (Classify + Policy) and never
	// calls the engine — the choice for an org that wants deterministic, auditable
	// routing with no learned component.
	StrategyHeuristic Strategy = "heuristic"
)

type Task ¶

type Task string

Task is the coarse task bucket a request is classified into. Values mirror the snake_case tags of hanzo-router's registry::Task so a learned classifier and this heuristic produce interchangeable labels.

const (
	TaskCode        Task = "code"
	TaskReasoning   Task = "reasoning"
	TaskMath        Task = "math"
	TaskCreative    Task = "creative"
	TaskVision      Task = "vision"
	TaskLongContext Task = "long_context"
	TaskCheapChat   Task = "cheap_chat"
	TaskGeneral     Task = "general"
)

func Classify(req Request) Task