provider

type Chunk ¶

type Chunk struct {
	Kind  ChunkKind
	Index int64  // content block index (text/thinking deltas)
	Text  string // text/thinking fragment

	ToolUse *ToolUse // KindToolUse only

	StopReason string // KindDone only: end_turn | tool_use | max_tokens | …
	// Usage is KindDone only, and nil means the endpoint reported none —
	// not that it reported zeroes. An adapter must send nil when no usage
	// object arrived on the wire: an OpenAI-compatible gateway that ignores
	// stream_options.include_usage would otherwise be indistinguishable from
	// a model that ran for free, and the token metric would record it as one
	// (#90).
	Usage *domain.ModelUsage
}

Chunk is one streaming increment.

type ChunkKind ¶

type ChunkKind string

ChunkKind discriminates streaming increments of a turn.

const (
	// KindTextDelta appends text to the content block at Index.
	KindTextDelta ChunkKind = "text_delta"
	// KindThinkingDelta appends thinking text to the block at Index.
	KindThinkingDelta ChunkKind = "thinking_delta"
	// KindToolUse is one complete tool invocation (input fully accumulated).
	KindToolUse ChunkKind = "tool_use"
	// KindDone closes the turn with stop reason and usage.
	KindDone ChunkKind = "done"
)

type Config ¶

type Config struct {
	Protocol string            // "anthropic" | "openai"
	Model    string            // model id sent to the upstream endpoint
	BaseURL  string            // endpoint base URL; required (no implicit default host)
	APIKey   string            // credential for the endpoint
	Headers  map[string]string // optional extra headers (gateway routing etc.)
	// StallTimeout is how long this endpoint may send nothing at all before its
	// turn is abandoned (#121); zero takes DefaultStallTimeout. It is per route
	// because the worst legitimate silence is a property of the endpoint — a
	// hosted gateway answers in seconds, a queued self-hosted model may take
	// minutes to send its first byte. See StallGuard for what it bounds and why
	// it is not an HTTP client timeout.
	StallTimeout time.Duration
	// MaxTokens is the default output cap for turns that set none themselves;
	// a Request.MaxTokens always wins. Per route because the right cap is a
	// property of the endpoint and the workload behind it: an agent that
	// writes whole files through tool calls dies mid-input on a cap sized for
	// chat (the outcomes acceptance hit exactly that). Zero keeps the
	// adapter's own default — anthropic sends its required-field fallback,
	// openai omits the field so the endpoint's default applies.
	MaxTokens int64
}

Config constructs one provider instance.

type Descriptor ¶ added in v0.2.0

type Descriptor struct {
	Protocol string
	Model    string
}

Descriptor is what may be said out loud about the backend a model routes to: the protocol its endpoint speaks and the model id sent upstream. It exists so telemetry can name the backend without being handed a Config, which carries the credential — the redaction is the type's shape, not a caller's discipline.

type Factory ¶

type Factory func(Config) (Provider, error)

Factory constructs a Provider for one protocol. It must be cheap and must acquire no per-instance resource — no private http.Transport, no connection pool, no goroutine — because the registry calls it once per turn rather than retaining what it builds (see Registry). Both adapters satisfy this by sharing http.DefaultClient.

It must also be safe to call concurrently: the registry holds no lock, so turns in flight on different sessions enter the factory at the same time.

An adapter that genuinely cannot be cheap must cache the shared resource itself, and must key that cache by the endpoint alone — protocol, base URL, credential, headers. NOT by the whole Config: under a pass-through route Config.Model is the agent's model string, so a Config-keyed cache would be keyed by client input and would rebuild issue #88 inside the adapter.

None of this forbids a *turn* from acquiring something: Generate starts the StallGuard's watcher goroutine, which the stream releases on Close. What the factory must not acquire is anything that outlives the instance it builds.

type Message ¶

type Message struct {
	Role    string          // "user" | "assistant"
	Content json.RawMessage // Anthropic content: a string or an array of blocks
}

Message is one conversational turn.

type Provider ¶

type Provider interface {
	Generate(ctx context.Context, req Request) (Stream, error)
}

Provider runs single model turns against one configured backend.

type Redactor ¶ added in v0.2.0

type Redactor struct {
	// contains filtered or unexported fields
}

Redactor removes the credentials one provider was configured with from text that quotes an endpoint.

Adapters quote what the endpoint said about a failure, because a status alone rarely explains a gateway misconfiguration. But an endpoint that echoes the request's auth header into its own diagnostic body — some gateways do on a 401 — would otherwise put the credential into an error that becomes a session.error event: append-only in Postgres, and re-served to API clients on every read.

Matching is by exact value, not by token shape. The adapter holds the secret, so it need not guess what a credential looks like — a base_url may point at any gateway, proxy, or self-hosted model, whose token format is unknowable. Shape-matching "Bearer …" would also have missed the very leak this was written for: the Anthropic protocol sends x-api-key, and the observed echo was a bare value with no scheme prefix and no header name beside it.

func NewRedactor(cfg Config) Redactor

func (r Redactor) Error(err error) error

func (r Redactor) Longest() int

func (r Redactor) String(s string) string

type Registry ¶

type Registry struct {
	// contains filtered or unexported fields
}

Registry resolves agent model strings to constructed providers. Routing is exact-match with an optional "*" default, so an enterprise maps its model names onto endpoints purely in configuration.

Nothing is retained, deliberately. Providers used to be cached per model string, and under a "*" route that string is whatever a client put on its agent — so the map grew without bound in the long-running brain (issue #88). Keying it by the route instead would bound it, but the cache was buying too little to pay for the branch: a constructed provider is a value struct over the shared http.DefaultClient, the connection pool that matters lives in the process-global http.DefaultTransport, and Provider is called once per turn immediately before a model round trip that dwarfs the construction. With nothing to insert into, the bound is a property of the type rather than of a policy — and the registry owns copies of everything it was given, so it is immutable after NewRegistry and needs no lock.

func NewRegistry(routes []Route, factories map[string]Factory) (*Registry, error)

func (r *Registry) Describe(model string) (Descriptor, bool)

func (r *Registry) Provider(model string) (Provider, error)

type Request ¶

type Request struct {
	System    string
	Messages  []Message
	Tools     []json.RawMessage // Anthropic tool definitions, verbatim
	MaxTokens int64
}

Request is one model turn in Anthropic Messages semantics. Content and tool definitions stay as raw Anthropic wire JSON so the anthropic-protocol adapter is near-zero-conversion; lossy mappings are confined to the non-Anthropic adapters (see provider/openai).

type Route ¶

type Route struct {
	Model  string
	Config Config
}

Route binds an agent-facing model string to a provider config. Model "*" is the default route.

func LoadRoutes(path string) ([]Route, error)

type StallGuard ¶ added in v0.2.0

type StallGuard struct {
	// contains filtered or unexported fields
}

StallGuard bounds a turn by the endpoint's silence rather than by its duration: it cancels the request context once nothing has arrived for the budget, and every sign of life on the wire buys another budget. Duration alone cannot be the bound — a model streaming a large answer legitimately holds one request open for many minutes, while an endpoint that completes the handshake and then never speaks holds it open forever (#121). Both halves of that hang are one silence: the wait for response headers, which the anthropic SDK's own timeout covers only when the caller lets it install its HTTP client, and a stream that dies mid-SSE, which nothing covered at all.

It guards the request context rather than the HTTP client, deliberately. Registered on a client instead, a header timeout surfaces as a transport error the SDK retries — three wedged attempts, three budgets — while a per-route client would give every provider instance its own connection pool, which the registry's per-turn construction cannot afford (see Registry).

The shape mirrors queue.KeepLease: the constructor returns the context to run the work under, the work reports progress, and Stop releases the watcher.

Three limits are worth naming, because what this measures is strictly "no response byte was read for the budget". A consumer that stops reading charges its own slowness to the endpoint's budget (between chunks the brain does one event append, so only a database stall of a whole budget could reach it — at the default it cannot, at a tightened per-route one it could, and the turn would then blame the model for Postgres). A request body still being uploaded counts as silence too, which is the right reading: a peer too slow to receive the request is the wedged peer this bounds. And a peer that dribbles one byte per budget is never bounded at all, by construction — every byte buys another budget. Bounding that one needs a total deadline, which would have to be sized for the longest healthy turn and so would not bound anything worth bounding.

func NewStallGuard(ctx context.Context, d time.Duration) (context.Context, *StallGuard)

func (g *StallGuard) Cause(err error) error

func (g *StallGuard) Progress()

func (g *StallGuard) Stop()

type Stream ¶

type Stream interface {
	Next() bool
	Chunk() Chunk
	Err() error
	Close() error
}

Stream yields one turn's chunks in order.

for stream.Next() { chunk := stream.Chunk(); … }
if err := stream.Err(); err != nil { … }

type ToolUse ¶

type ToolUse struct {
	ID    string
	Name  string
	Input json.RawMessage
}

ToolUse is a complete tool call emitted by the model.