Documentation
¶
Overview ¶
Package models puts every model provider behind two interfaces: Decider (which rule does this email match?) and Generator (free-form structured output). Nothing outside this package imports a provider SDK.
Index ¶
- Constants
- Variables
- func FallbackSkipped(cfg *config.Config, spec string) string
- func IsWorkspaceNeeded(err error) bool
- func RenderEmail(e message.Summary) string
- func WithPurpose(ctx context.Context, purpose string) context.Context
- type Anthropic
- type AnthropicAuth
- type AnthropicWorkspaces
- type Call
- type Caller
- type Candidate
- type DecideRequest
- type Decider
- type Decision
- type Deps
- type Fake
- func (f *Fake) Decide(_ context.Context, req DecideRequest) (Decision, Usage, error)
- func (f *Fake) DecideSpread(ctx context.Context, req DecideRequest) (Decision, map[int64]float64, Usage, error)
- func (f *Fake) Generate(_ context.Context, system, user string, schema json.RawMessage, out any) (Usage, error)
- func (f *Fake) Name() string
- func (f *Fake) Requests() []DecideRequest
- type Generator
- type LabeledEmail
- type LabeledRule
- type Labels
- type Ollama
- type OpenAI
- type Price
- type Prices
- type Report
- type Result
- type Router
- func (r *Router) Decide(ctx context.Context, req DecideRequest) (Decision, Usage, error)
- func (r *Router) For(purpose string) *Router
- func (r *Router) MeteredResult(res Result) bool
- func (r *Router) Name() string
- func (r *Router) OnOperatorKeys() bool
- func (r *Router) OwnKeysOnly() *Router
- func (r *Router) Route(ctx context.Context, req DecideRequest) (Result, error)
- type Spreader
- type StatusError
- type Usage
- type UsageStore
- type Workspace
- type WorkspaceAnswer
Constants ¶
const ( FallbackNotSkipped = "" FallbackNeedsKey = "needs_key" // the fallback is a Claude model and there is no Anthropic key FallbackIsPrimary = "is_primary" // the decision model already is that Claude model )
Why a configured fallback model is not asked. FallbackNotSkipped ("") means it is, or none is configured.
const ( // DefaultJevURL and DefaultClefURL are the providers' public hosts. DefaultJevURL = "https://openrouter.ai" DefaultClefURL = "https://api.cloudflare.com" // DefaultJevModel is pinned so tuned thresholds do not move under us. DefaultJevModel = "typesafe/jev-1.13" DefaultClefModel = "clef" )
const (
// DefaultAnthropicModel is the plan's fallback and composer model.
DefaultAnthropicModel = "claude-haiku-4-5"
)
const ReadTries = 3
ReadTries is how many times one model is asked about one email while its answer cannot be read: the first call and two retries (PRD R5).
Variables ¶
var ErrAuth = errors.New("model provider rejected the credentials")
ErrAuth means the provider rejected the API key or token.
var ErrBadOutput = errors.New("model output is not usable")
ErrBadOutput means the provider answered, but not with something usable.
Functions ¶
func FallbackSkipped ¶ added in v0.1.2
FallbackSkipped says why NewRouter leaves out the configured fallback for a decider spec: FallbackNeedsKey, FallbackIsPrimary, or FallbackNotSkipped when the fallback is built or none is set (FallbackModel empty). It is the one place that decides, so the startup log and the Settings screen cannot disagree with the router.
func IsWorkspaceNeeded ¶
IsWorkspaceNeeded reports whether err is Anthropic refusing a request because its key covers a whole organisation and the request named no workspace: a 400 invalid_request_error that says so. The words are Anthropic's ("This API key is not scoped to a workspace, so this request must include the anthropic-workspace-id header").
func RenderEmail ¶
RenderEmail is the email summary shown to models. It is identical for every provider, so accuracy numbers compare models and not prompts.
Types ¶
type Anthropic ¶
type Anthropic struct {
// contains filtered or unexported fields
}
Anthropic talks to the Messages API through the official SDK.
func NewAnthropic ¶
func NewAnthropic(auth AnthropicAuth, model string, deps Deps) *Anthropic
NewAnthropic builds the adapter. deps.AnthropicURL empty means the public API.
func (*Anthropic) Generate ¶
func (a *Anthropic) Generate(ctx context.Context, system, user string, schema json.RawMessage, out any) (Usage, error)
Generate caches the system prompt and forces one call of a tool whose input schema is schema, at temperature 0. Forced tool choice and temperature are accepted by Haiku 4.5; newer Opus and Sonnet models reject both.
type AnthropicAuth ¶
type AnthropicAuth struct {
APIKey string
WorkspaceID string // sent as anthropic-workspace-id; empty = not sent
}
AnthropicAuth is what every request to Anthropic carries: the key and, for a key that covers a whole organisation rather than one workspace, the workspace to run in.
func AnthropicAuthOf ¶
func AnthropicAuthOf(cfg *config.Config) AnthropicAuth
AnthropicAuthOf is the Anthropic credentials of a configuration: every Claude adapter is built from it, so none runs without the workspace the settings name.
type AnthropicWorkspaces ¶
type AnthropicWorkspaces struct {
Deps Deps // its HTTP client, Caller and AnthropicURL
}
AnthropicWorkspaces finds out which workspace a Claude key needs, with List Workspaces (https://platform.claude.com/docs/en/api/beta/organization/workspaces/list, read 2026-10-07), an Admin API call. The Admin API takes a personal or service-account key only when it covers a whole organisation, so a key made for one workspace is refused there; so is a key whose role may not list workspaces. A refusal is told apart by asking Anthropic to count the tokens of one word without naming a workspace (free of charge): that works only for a key that needs none.
func (AnthropicWorkspaces) Find ¶
func (f AnthropicWorkspaces) Find(ctx context.Context, apiKey string) (WorkspaceAnswer, error)
Find answers which workspace apiKey needs. An error means it could not be told: the listing was refused and the key cannot run without a workspace either, or Anthropic could not be reached. Its text is a *StatusError's, scrubbed, fit for the log.
type Call ¶
type Call struct {
Provider, Model string
}
Call names what a model call is for, so its log line says which provider and model it went to. It carries no prompt, no response and no credentials.
type Caller ¶
type Caller struct {
Timeout time.Duration // per attempt
Retries int // extra attempts after the first
Backoff time.Duration // wait before the first retry; doubles each time
Sleep func(ctx context.Context, d time.Duration) error
// contains filtered or unexported fields
}
Caller runs model calls under one global concurrency cap, with a per-attempt timeout and retries on 429 and 5xx. Share one Caller between all adapters.
func NewCaller ¶
NewCaller allows at most concurrency calls in flight (MAILRULES_MODEL_CONCURRENCY).
type Candidate ¶
type Candidate struct {
RuleID int64
Name string // the rule's name, shown to models and in the synthesized reason
Intent string // plain-English intent
Exceptions string // rendered "unless" text, may be empty
}
Candidate is one intent rule the email might match.
type DecideRequest ¶
type DecideRequest struct {
Email message.Summary // headers, signals, text <= BODY_CHARS
Candidates []Candidate // eligible intent rules in priority order
Examples []learn.Example // few-shot corrections (fallback model only)
}
DecideRequest is one email and the rules it may match.
type Decider ¶
type Decider interface {
Name() string
Decide(ctx context.Context, req DecideRequest) (Decision, Usage, error)
}
Decider picks the rule an email matches.
func NewDecider ¶
NewDecider builds one decider from config. spec is a MAILRULES_DECIDER name, optionally followed by ":model" (for example "clef:clef-flash" or "ollama:llama3.2"). jev, clef and anthropic have default models; openai and ollama need one named (MAILRULES_DECIDER_MODEL, see config.DeciderSpec).
func NewLLMDecider ¶
NewLLMDecider makes a generative model decide through a forced structured answer.
type Decision ¶
type Decision struct {
RuleID int64 // 0 = none of the rules
Confidence float64 // calibrated 0..1
Reason string // one sentence; synthesized when the model gives none
}
Decision is a model's answer.
type Deps ¶
type Deps struct {
HTTP *http.Client // nil = http.DefaultClient
Caller *Caller // the global concurrency cap, timeout and retries
Prices Prices
// AnthropicURL is where the Claude adapters and the workspace lookup send their
// requests; empty = the public API. Only tests point it elsewhere.
AnthropicURL string
// WorkspaceNeeded is asked when Anthropic refuses a Claude request because its key covers
// a whole organisation and no workspace was named. It answers the workspace to retry the
// request in, or "" when there is none to use. nil = the refusal stands.
WorkspaceNeeded func(ctx context.Context, apiKey string) string
}
Deps are what every adapter shares.
type Fake ¶
type Fake struct {
NameValue string
// DecideFunc answers Decide; nil answers "none" with confidence 1.
DecideFunc func(req DecideRequest) (Decision, Usage, error)
// Spread, when set, is what DecideSpread reports as the probability of each option.
Spread map[int64]float64
// GenerateFunc answers Generate, usually by unmarshalling canned JSON into out.
GenerateFunc func(system, user string, schema json.RawMessage, out any) (Usage, error)
// contains filtered or unexported fields
}
Fake is a scripted Decider and Generator for tests. It is safe for concurrent use.
func (*Fake) DecideSpread ¶
func (f *Fake) DecideSpread(ctx context.Context, req DecideRequest) (Decision, map[int64]float64, Usage, error)
DecideSpread implements Spreader with Spread.
func (*Fake) Generate ¶
func (f *Fake) Generate(_ context.Context, system, user string, schema json.RawMessage, out any) (Usage, error)
Generate implements Generator.
func (*Fake) Requests ¶
func (f *Fake) Requests() []DecideRequest
Requests returns every request Decide has seen, in order.
type Generator ¶
type Generator interface {
Generate(ctx context.Context, system, user string, schema json.RawMessage, out any) (Usage, error)
}
Generator is for free-form structured output (composer, AI rule builder).
func NewGenerator ¶
NewGenerator builds the generative model a composer_model spec names (see config.SplitComposerModel): a bare name is a Claude model on Anthropic, otherwise anthropic:, openai: or ollama: and the model. It does not check that the provider's key or URL is set (config.ComposerReady does); a call without them fails at the provider.
type LabeledEmail ¶
LabeledEmail is one email summary and the rule a person says it matches.
type LabeledRule ¶
type LabeledRule struct {
ID int64 `json:"id"`
Name string `json:"name"`
Intent string `json:"intent"`
Exceptions string `json:"exceptions"`
}
LabeledRule is one rule of an eval file's rule set.
type Labels ¶
type Labels struct {
Rules []LabeledRule
Emails []LabeledEmail
}
Labels is a parsed labeled.jsonl.
type Ollama ¶
type Ollama struct {
// contains filtered or unexported fields
}
Ollama talks to a local Ollama server's /api/chat with plain net/http (https://docs.ollama.com/api/chat, read 2026-10-06).
func NewOllama ¶
NewOllama builds the adapter; baseURL is OLLAMA_URL, e.g. http://localhost:11434.
type OpenAI ¶
type OpenAI struct {
// contains filtered or unexported fields
}
OpenAI talks to any OpenAI-compatible chat completions endpoint (OpenAI, OpenRouter, Groq, DeepSeek) through the official SDK.
type Price ¶
type Price struct {
Input float64 `json:"input"`
Output float64 `json:"output"`
CacheRead float64 `json:"cache_read"` // input tokens served from a prompt cache
CacheWrite float64 `json:"cache_write"` // input tokens written to a prompt cache
}
Price is what a model costs, in USD per million tokens.
type Prices ¶
Prices maps provider, then model id, to a price. A model that is missing costs 0, which is also the right answer for local Ollama models.
func DefaultPrices ¶
func DefaultPrices() Prices
DefaultPrices is the built-in table. Sources, read on 2026-10-06: Jev from the worked example in the OpenRouter tutorial (476 input tokens cost $0.000019992; output is free), Clef from the Workers AI model pages, Haiku 4.5 from Anthropic's model table ($1 in, $5 out) with the cache rates OpenRouter lists for the same model. OpenAI-compatible endpoints differ per host, so they have no built-in entry: add them in MAILRULES_PRICES_FILE.
func LoadPrices ¶
LoadPrices returns the built-in table with the entries of the JSON file at path (MAILRULES_PRICES_FILE) laid over it. An empty path means built-in only. The file looks like {"openai": {"gpt-x": {"input": 0.25, "output": 2}}}.
type Report ¶
type Report struct {
Decider string
Fallback string // "" = no fallback
Total int
Correct int
Errors int // emails the decider could not answer; they count as wrong
Escalated int
CostUSD float64
Latencies []time.Duration // per answered email, both calls added, sorted
Confusion map[[2]string]int
}
Report is the outcome of one eval run.
type Result ¶
type Result struct {
Decision
Escalated bool // the fallback was asked
Primary Usage // always set; summed over the primary's calls
Fallback *Usage // set when the fallback answered; summed over its calls
FallbackErr error // set when the fallback was asked and failed; Decision is then the primary's
Calls int // model calls made, counting each retry of an unreadable answer
// Probabilities is what the primary gave each candidate, by rule id (0 = none of
// them), when it is a Spreader; nil otherwise. It stays the primary's when the
// fallback answers: it is why the fallback was asked.
Probabilities map[int64]float64
}
Result is a routed decision with what each model call cost.
type Router ¶
type Router struct {
Primary Decider
Fallback Decider // nil = never escalate
EscalateBelow float64
Usage UsageStore // nil = do not record
Now func() time.Time // nil = time.Now
// Purpose, when set, is the ledger purpose of every call ("test" for the rule tester);
// empty records the primary's calls as "decide" and the fallback's as "escalate".
Purpose string
// Metered says, by provider (Usage.Provider), whether calls to it go out on a key the
// operator holds (MAILRULES_MODE=cloud, the tenant has no key of its own), so that the
// emails they decide count against the tenant's monthly allowance. nil = none do.
Metered func(provider string) bool
}
Router is the Decider the pipeline uses. It asks the primary decider and, when that answer's confidence is below EscalateBelow, asks the fallback with the few-shot examples and uses the fallback's answer. A model whose answer cannot be read (ErrBadOutput) is asked again, up to ReadTries calls in all.
func NewRouter ¶
NewRouter builds the Router for a decider spec. The fallback is Anthropic's MAILRULES_FALLBACK_MODEL; it is left out when that model is empty, when there is no Anthropic key, or when the primary already is that model (see FallbackSkipped).
func (*Router) Decide ¶
Decide implements Decider. The usage it returns names the model whose answer was used and adds up tokens, cost and latency over both calls.
func (*Router) MeteredResult ¶ added in v0.3.0
MeteredResult reports whether a routed decision was made on a key the operator holds: the primary's, or the fallback's when it answered.
func (*Router) OnOperatorKeys ¶ added in v0.3.0
OnOperatorKeys reports whether a model the router may ask runs on a key the operator holds: an email it decides may count against the tenant's allowance.
func (*Router) OwnKeysOnly ¶ added in v0.3.0
OwnKeysOnly is the router without the models that run on a key the operator holds: nil when the primary does, else a copy without a fallback that does (an unsure answer is then not double-checked). It is how a tenant whose allowance is used up still decides on its own keys.
type Spreader ¶
type Spreader interface {
Decider
DecideSpread(ctx context.Context, req DecideRequest) (Decision, map[int64]float64, Usage, error)
}
Spreader is a Decider that can also say how sure it was of every option: the probability it gave each candidate, by rule id, with 0 for "none of these". The decision models (Jev, Clef) are; a generative model asked for one answer is not.
type StatusError ¶
type StatusError struct {
Provider string
Code int
Type string // the provider's error type, such as "invalid_request_error"; may be empty
Detail string // the provider's message, scrubbed; may be empty
RequestID string // the provider's id for the request, for its support; may be empty
}
StatusError is a non-2xx answer from a provider. Type, Detail and RequestID come from the provider's error reply so the log can say why a request was refused (MAI-56). Provider error bodies can echo parts of the request, so Detail is scrubbed: everything quoted and every email address is cut out, and it is capped (see scrubDetail).
func (*StatusError) Error ¶
func (e *StatusError) Error() string
func (*StatusError) Is ¶
func (e *StatusError) Is(target error) bool
Is makes errors.Is(err, ErrAuth) true for 401 and 403.
type Usage ¶
type Usage struct {
Provider, Model string
TokensIn, TokensOut int
CostUSD float64
Latency time.Duration
}
Usage is what one model call cost.
type UsageStore ¶
type UsageStore interface {
AddUsage(ctx context.Context, day, provider, model, purpose string, calls, tokensIn, tokensOut int, costUSD float64) error
}
UsageStore is the cost ledger (store.Store implements it): it adds to the usage_daily row for that day, provider, model and purpose.
type WorkspaceAnswer ¶
type WorkspaceAnswer struct {
NoneNeeded bool
Workspaces []Workspace // the Default Workspace included; never archived ones
}
WorkspaceAnswer is what Anthropic says about the workspace a key needs: none (the key was made for one workspace), or the organisation's workspaces to choose from.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package anthropictest is a fake Anthropic API for tests: the Messages API, token counting and List Workspaces, answering as the test sets it up and recording what it was sent.
|
Package anthropictest is a fake Anthropic API for tests: the Messages API, token counting and List Workspaces, answering as the test sets it up and recording what it was sent. |