models

package
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 9, 2026 License: AGPL-3.0 Imports: 27 Imported by: 0

Documentation

Overview

Package models puts every model provider behind two interfaces: Decider (which rule does this email match?) and Generator (free-form structured output). Nothing outside this package imports a provider SDK.

Index

Constants

View Source
const (
	FallbackNotSkipped = ""
	FallbackNeedsKey   = "needs_key"  // the fallback is a Claude model and there is no Anthropic key
	FallbackIsPrimary  = "is_primary" // the decision model already is that Claude model
)

Why a configured fallback model is not asked. FallbackNotSkipped ("") means it is, or none is configured.

View Source
const (
	// DefaultJevURL and DefaultClefURL are the providers' public hosts.
	DefaultJevURL  = "https://openrouter.ai"
	DefaultClefURL = "https://api.cloudflare.com"
	// DefaultJevModel is pinned so tuned thresholds do not move under us.
	DefaultJevModel  = "typesafe/jev-1.13"
	DefaultClefModel = "clef"
)
View Source
const (
	// DefaultAnthropicModel is the plan's fallback and composer model.
	DefaultAnthropicModel = "claude-haiku-4-5"
)
View Source
const ReadTries = 3

ReadTries is how many times one model is asked about one email while its answer cannot be read: the first call and two retries (PRD R5).

Variables

View Source
var ErrAuth = errors.New("model provider rejected the credentials")

ErrAuth means the provider rejected the API key or token.

View Source
var ErrBadOutput = errors.New("model output is not usable")

ErrBadOutput means the provider answered, but not with something usable.

Functions

func FallbackSkipped added in v0.1.2

func FallbackSkipped(cfg *config.Config, spec string) string

FallbackSkipped says why NewRouter leaves out the configured fallback for a decider spec: FallbackNeedsKey, FallbackIsPrimary, or FallbackNotSkipped when the fallback is built or none is set (FallbackModel empty). It is the one place that decides, so the startup log and the Settings screen cannot disagree with the router.

func IsWorkspaceNeeded

func IsWorkspaceNeeded(err error) bool

IsWorkspaceNeeded reports whether err is Anthropic refusing a request because its key covers a whole organisation and the request named no workspace: a 400 invalid_request_error that says so. The words are Anthropic's ("This API key is not scoped to a workspace, so this request must include the anthropic-workspace-id header").

func RenderEmail

func RenderEmail(e message.Summary) string

RenderEmail is the email summary shown to models. It is identical for every provider, so accuracy numbers compare models and not prompts.

func WithPurpose

func WithPurpose(ctx context.Context, purpose string) context.Context

WithPurpose says why the calls made under ctx are made ("decide", "escalate", "test", "compose", "cleanup", "suggest"): the cost ledger's purposes, which the debug line of every call repeats.

Types

type Anthropic

type Anthropic struct {
	// contains filtered or unexported fields
}

Anthropic talks to the Messages API through the official SDK.

func NewAnthropic

func NewAnthropic(auth AnthropicAuth, model string, deps Deps) *Anthropic

NewAnthropic builds the adapter. deps.AnthropicURL empty means the public API.

func (*Anthropic) Generate

func (a *Anthropic) Generate(ctx context.Context, system, user string, schema json.RawMessage, out any) (Usage, error)

Generate caches the system prompt and forces one call of a tool whose input schema is schema, at temperature 0. Forced tool choice and temperature are accepted by Haiku 4.5; newer Opus and Sonnet models reject both.

type AnthropicAuth

type AnthropicAuth struct {
	APIKey      string
	WorkspaceID string // sent as anthropic-workspace-id; empty = not sent
}

AnthropicAuth is what every request to Anthropic carries: the key and, for a key that covers a whole organisation rather than one workspace, the workspace to run in.

func AnthropicAuthOf

func AnthropicAuthOf(cfg *config.Config) AnthropicAuth

AnthropicAuthOf is the Anthropic credentials of a configuration: every Claude adapter is built from it, so none runs without the workspace the settings name.

type AnthropicWorkspaces

type AnthropicWorkspaces struct {
	Deps Deps // its HTTP client, Caller and AnthropicURL
}

AnthropicWorkspaces finds out which workspace a Claude key needs, with List Workspaces (https://platform.claude.com/docs/en/api/beta/organization/workspaces/list, read 2026-10-07), an Admin API call. The Admin API takes a personal or service-account key only when it covers a whole organisation, so a key made for one workspace is refused there; so is a key whose role may not list workspaces. A refusal is told apart by asking Anthropic to count the tokens of one word without naming a workspace (free of charge): that works only for a key that needs none.

func (AnthropicWorkspaces) Find

Find answers which workspace apiKey needs. An error means it could not be told: the listing was refused and the key cannot run without a workspace either, or Anthropic could not be reached. Its text is a *StatusError's, scrubbed, fit for the log.

type Call

type Call struct {
	Provider, Model string
}

Call names what a model call is for, so its log line says which provider and model it went to. It carries no prompt, no response and no credentials.

type Caller

type Caller struct {
	Timeout time.Duration // per attempt
	Retries int           // extra attempts after the first
	Backoff time.Duration // wait before the first retry; doubles each time
	Sleep   func(ctx context.Context, d time.Duration) error
	// contains filtered or unexported fields
}

Caller runs model calls under one global concurrency cap, with a per-attempt timeout and retries on 429 and 5xx. Share one Caller between all adapters.

func NewCaller

func NewCaller(concurrency int) *Caller

NewCaller allows at most concurrency calls in flight (MAILRULES_MODEL_CONCURRENCY).

func (*Caller) Do

func (c *Caller) Do(ctx context.Context, call Call, fn func(ctx context.Context) error) error

Do runs fn, retrying it while it returns a StatusError with 429 or 5xx. Every attempt writes one debug line: provider, model, purpose, attempt, duration and outcome class, never what was asked or answered.

type Candidate

type Candidate struct {
	RuleID     int64
	Name       string // the rule's name, shown to models and in the synthesized reason
	Intent     string // plain-English intent
	Exceptions string // rendered "unless" text, may be empty
}

Candidate is one intent rule the email might match.

type DecideRequest

type DecideRequest struct {
	Email      message.Summary // headers, signals, text <= BODY_CHARS
	Candidates []Candidate     // eligible intent rules in priority order
	Examples   []learn.Example // few-shot corrections (fallback model only)
}

DecideRequest is one email and the rules it may match.

type Decider

type Decider interface {
	Name() string
	Decide(ctx context.Context, req DecideRequest) (Decision, Usage, error)
}

Decider picks the rule an email matches.

func NewClef

func NewClef(baseURL, accountID, apiToken, model string, deps Deps) Decider

NewClef decides with Clef on Workers AI. model is "clef" or "clef-flash".

func NewDecider

func NewDecider(cfg *config.Config, spec string, deps Deps) (Decider, error)

NewDecider builds one decider from config. spec is a MAILRULES_DECIDER name, optionally followed by ":model" (for example "clef:clef-flash" or "ollama:llama3.2"). jev, clef and anthropic have default models; openai and ollama need one named (MAILRULES_DECIDER_MODEL, see config.DeciderSpec).

func NewJev

func NewJev(baseURL, apiKey, model string, deps Deps) Decider

NewJev decides with Jev through OpenRouter's Decisions API.

func NewLLMDecider

func NewLLMDecider(name string, gen Generator) Decider

NewLLMDecider makes a generative model decide through a forced structured answer.

type Decision

type Decision struct {
	RuleID     int64   // 0 = none of the rules
	Confidence float64 // calibrated 0..1
	Reason     string  // one sentence; synthesized when the model gives none
}

Decision is a model's answer.

type Deps

type Deps struct {
	HTTP   *http.Client // nil = http.DefaultClient
	Caller *Caller      // the global concurrency cap, timeout and retries
	Prices Prices
	// AnthropicURL is where the Claude adapters and the workspace lookup send their
	// requests; empty = the public API. Only tests point it elsewhere.
	AnthropicURL string
	// WorkspaceNeeded is asked when Anthropic refuses a Claude request because its key covers
	// a whole organisation and no workspace was named. It answers the workspace to retry the
	// request in, or "" when there is none to use. nil = the refusal stands.
	WorkspaceNeeded func(ctx context.Context, apiKey string) string
}

Deps are what every adapter shares.

type Fake

type Fake struct {
	NameValue string
	// DecideFunc answers Decide; nil answers "none" with confidence 1.
	DecideFunc func(req DecideRequest) (Decision, Usage, error)
	// Spread, when set, is what DecideSpread reports as the probability of each option.
	Spread map[int64]float64
	// GenerateFunc answers Generate, usually by unmarshalling canned JSON into out.
	GenerateFunc func(system, user string, schema json.RawMessage, out any) (Usage, error)
	// contains filtered or unexported fields
}

Fake is a scripted Decider and Generator for tests. It is safe for concurrent use.

func (*Fake) Decide

func (f *Fake) Decide(_ context.Context, req DecideRequest) (Decision, Usage, error)

Decide implements Decider and remembers the request.

func (*Fake) DecideSpread

func (f *Fake) DecideSpread(ctx context.Context, req DecideRequest) (Decision, map[int64]float64, Usage, error)

DecideSpread implements Spreader with Spread.

func (*Fake) Generate

func (f *Fake) Generate(_ context.Context, system, user string, schema json.RawMessage, out any) (Usage, error)

Generate implements Generator.

func (*Fake) Name

func (f *Fake) Name() string

Name implements Decider.

func (*Fake) Requests

func (f *Fake) Requests() []DecideRequest

Requests returns every request Decide has seen, in order.

type Generator

type Generator interface {
	Generate(ctx context.Context, system, user string, schema json.RawMessage, out any) (Usage, error)
}

Generator is for free-form structured output (composer, AI rule builder).

func NewGenerator

func NewGenerator(cfg *config.Config, spec string, deps Deps) (Generator, error)

NewGenerator builds the generative model a composer_model spec names (see config.SplitComposerModel): a bare name is a Claude model on Anthropic, otherwise anthropic:, openai: or ollama: and the model. It does not check that the provider's key or URL is set (config.ComposerReady does); a call without them fails at the provider.

type LabeledEmail

type LabeledEmail struct {
	Email    message.Summary
	Expected int64 // 0 = none of the rules
}

LabeledEmail is one email summary and the rule a person says it matches.

type LabeledRule

type LabeledRule struct {
	ID         int64  `json:"id"`
	Name       string `json:"name"`
	Intent     string `json:"intent"`
	Exceptions string `json:"exceptions"`
}

LabeledRule is one rule of an eval file's rule set.

type Labels

type Labels struct {
	Rules  []LabeledRule
	Emails []LabeledEmail
}

Labels is a parsed labeled.jsonl.

func LoadLabels

func LoadLabels(r io.Reader) (*Labels, error)

LoadLabels parses labeled.jsonl and checks every expected rule exists.

type Ollama

type Ollama struct {
	// contains filtered or unexported fields
}

Ollama talks to a local Ollama server's /api/chat with plain net/http (https://docs.ollama.com/api/chat, read 2026-10-06).

func NewOllama

func NewOllama(baseURL, model string, deps Deps) *Ollama

NewOllama builds the adapter; baseURL is OLLAMA_URL, e.g. http://localhost:11434.

func (*Ollama) Generate

func (o *Ollama) Generate(ctx context.Context, system, user string, schema json.RawMessage, out any) (Usage, error)

Generate asks for a non-streamed answer constrained to schema ("format"), at temperature 0.

type OpenAI

type OpenAI struct {
	// contains filtered or unexported fields
}

OpenAI talks to any OpenAI-compatible chat completions endpoint (OpenAI, OpenRouter, Groq, DeepSeek) through the official SDK.

func NewOpenAI

func NewOpenAI(baseURL, apiKey, model string, deps Deps) *OpenAI

NewOpenAI builds the adapter. An empty baseURL means api.openai.com.

func (*OpenAI) Generate

func (o *OpenAI) Generate(ctx context.Context, system, user string, schema json.RawMessage, out any) (Usage, error)

Generate forces one function call whose parameters are schema, at temperature 0.

type Price

type Price struct {
	Input      float64 `json:"input"`
	Output     float64 `json:"output"`
	CacheRead  float64 `json:"cache_read"`  // input tokens served from a prompt cache
	CacheWrite float64 `json:"cache_write"` // input tokens written to a prompt cache
}

Price is what a model costs, in USD per million tokens.

type Prices

type Prices map[string]map[string]Price

Prices maps provider, then model id, to a price. A model that is missing costs 0, which is also the right answer for local Ollama models.

func DefaultPrices

func DefaultPrices() Prices

DefaultPrices is the built-in table. Sources, read on 2026-10-06: Jev from the worked example in the OpenRouter tutorial (476 input tokens cost $0.000019992; output is free), Clef from the Workers AI model pages, Haiku 4.5 from Anthropic's model table ($1 in, $5 out) with the cache rates OpenRouter lists for the same model. OpenAI-compatible endpoints differ per host, so they have no built-in entry: add them in MAILRULES_PRICES_FILE.

func LoadPrices

func LoadPrices(path string) (Prices, error)

LoadPrices returns the built-in table with the entries of the JSON file at path (MAILRULES_PRICES_FILE) laid over it. An empty path means built-in only. The file looks like {"openai": {"gpt-x": {"input": 0.25, "output": 2}}}.

func (Prices) Cost

func (p Prices) Cost(provider, model string, in, out, cacheRead, cacheWrite int) float64

Cost prices one call. in is uncached input tokens; cacheRead and cacheWrite are counted separately because providers bill them at different rates.

type Report

type Report struct {
	Decider   string
	Fallback  string // "" = no fallback
	Total     int
	Correct   int
	Errors    int // emails the decider could not answer; they count as wrong
	Escalated int
	CostUSD   float64
	Latencies []time.Duration // per answered email, both calls added, sorted
	Confusion map[[2]string]int
}

Report is the outcome of one eval run.

func Evaluate

func Evaluate(ctx context.Context, r *Router, l *Labels) Report

Evaluate runs every labeled email through the router, one at a time so the latency numbers are not skewed by our own concurrency. ponytail: sequential; fan out under the Caller's cap if 500 lines get slow.

func (Report) Write

func (rep Report) Write(w io.Writer)

Write prints the report: accuracy, share escalated, latency, cost and the confused rule pairs.

type Result

type Result struct {
	Decision
	Escalated   bool   // the fallback was asked
	Primary     Usage  // always set; summed over the primary's calls
	Fallback    *Usage // set when the fallback answered; summed over its calls
	FallbackErr error  // set when the fallback was asked and failed; Decision is then the primary's
	Calls       int    // model calls made, counting each retry of an unreadable answer
	// Probabilities is what the primary gave each candidate, by rule id (0 = none of
	// them), when it is a Spreader; nil otherwise. It stays the primary's when the
	// fallback answers: it is why the fallback was asked.
	Probabilities map[int64]float64
}

Result is a routed decision with what each model call cost.

type Router

type Router struct {
	Primary       Decider
	Fallback      Decider // nil = never escalate
	EscalateBelow float64
	Usage         UsageStore       // nil = do not record
	Now           func() time.Time // nil = time.Now
	// Purpose, when set, is the ledger purpose of every call ("test" for the rule tester);
	// empty records the primary's calls as "decide" and the fallback's as "escalate".
	Purpose string
}

Router is the Decider the pipeline uses. It asks the primary decider and, when that answer's confidence is below EscalateBelow, asks the fallback with the few-shot examples and uses the fallback's answer. A model whose answer cannot be read (ErrBadOutput) is asked again, up to ReadTries calls in all.

func NewRouter

func NewRouter(cfg *config.Config, spec string, deps Deps, usage UsageStore) (*Router, error)

NewRouter builds the Router for a decider spec. The fallback is Anthropic's MAILRULES_FALLBACK_MODEL; it is left out when that model is empty, when there is no Anthropic key, or when the primary already is that model (see FallbackSkipped).

func (*Router) Decide

func (r *Router) Decide(ctx context.Context, req DecideRequest) (Decision, Usage, error)

Decide implements Decider. The usage it returns names the model whose answer was used and adds up tokens, cost and latency over both calls.

func (*Router) For

func (r *Router) For(purpose string) *Router

For returns a copy of the router whose calls are recorded under purpose.

func (*Router) Name

func (r *Router) Name() string

Name reports the primary decider's name.

func (*Router) Route

func (r *Router) Route(ctx context.Context, req DecideRequest) (Result, error)

Route is Decide with the escalation detail kept apart. When the primary's answer still cannot be read after ReadTries calls, the error wraps ErrBadOutput and the Result carries what those calls cost.

type Spreader

type Spreader interface {
	Decider
	DecideSpread(ctx context.Context, req DecideRequest) (Decision, map[int64]float64, Usage, error)
}

Spreader is a Decider that can also say how sure it was of every option: the probability it gave each candidate, by rule id, with 0 for "none of these". The decision models (Jev, Clef) are; a generative model asked for one answer is not.

type StatusError

type StatusError struct {
	Provider  string
	Code      int
	Type      string // the provider's error type, such as "invalid_request_error"; may be empty
	Detail    string // the provider's message, scrubbed; may be empty
	RequestID string // the provider's id for the request, for its support; may be empty
}

StatusError is a non-2xx answer from a provider. Type, Detail and RequestID come from the provider's error reply so the log can say why a request was refused (MAI-56). Provider error bodies can echo parts of the request, so Detail is scrubbed: everything quoted and every email address is cut out, and it is capped (see scrubDetail).

func (*StatusError) Error

func (e *StatusError) Error() string

func (*StatusError) Is

func (e *StatusError) Is(target error) bool

Is makes errors.Is(err, ErrAuth) true for 401 and 403.

type Usage

type Usage struct {
	Provider, Model     string
	TokensIn, TokensOut int
	CostUSD             float64
	Latency             time.Duration
}

Usage is what one model call cost.

type UsageStore

type UsageStore interface {
	AddUsage(ctx context.Context, day, provider, model, purpose string, calls, tokensIn, tokensOut int, costUSD float64) error
}

UsageStore is the cost ledger (store.Store implements it): it adds to the usage_daily row for that day, provider, model and purpose.

type Workspace

type Workspace struct {
	ID   string `json:"id"`
	Name string `json:"name"`
}

Workspace is one of an Anthropic organisation's workspaces.

type WorkspaceAnswer

type WorkspaceAnswer struct {
	NoneNeeded bool
	Workspaces []Workspace // the Default Workspace included; never archived ones
}

WorkspaceAnswer is what Anthropic says about the workspace a key needs: none (the key was made for one workspace), or the organisation's workspaces to choose from.

Directories

Path Synopsis
Package anthropictest is a fake Anthropic API for tests: the Messages API, token counting and List Workspaces, answering as the test sets it up and recording what it was sent.
Package anthropictest is a fake Anthropic API for tests: the Messages API, token counting and List Workspaces, answering as the test sets it up and recording what it was sent.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL