Documentation
¶
Overview ¶
Package provider is part of the GoFastr harness.
See docs/harness-architecture.md for the architecture this package implements.
Package provider defines the Provider abstraction the engine uses to talk to language models. Provider adapters live in subpackages (openrouter, zai, copilot, routing) and translate to/from the canonical Anthropic-shape message form used internally.
See docs/harness-architecture.md § Providers.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func USDForUsage ¶
USDForUsage computes the dollar cost of a single Usage record at the given Pricing. Returns 0 if Pricing is zeroed (unknown model).
Math: per-MTok rates divided by 1,000,000 ⇒ per-token, multiplied by the token counts.
Types ¶
type CacheAttribution ¶
type CacheAttribution struct {
// ReadTokens is the count of tokens the provider says were
// served from prompt cache. 0 if the provider doesn't report it.
ReadTokens int
// WriteTokens is the count of tokens the provider says were
// freshly added to its cache. Non-zero only on Anthropic-shape.
WriteTokens int
// Quality categorizes how reliable the attribution is:
// - "explicit" — provider returned a structured cache breakdown
// - "midstream" — markers parsed mid-stream (Anthropic-shape on partial)
// - "estimate" — heuristic when the provider doesn't surface cache info
// - "none" — the provider doesn't support prompt caching
Quality string
}
CacheAttribution describes the prompt-cache breakdown for one provider call. Surfaced alongside Usage so the cost ledger can price cache reads at a different rate than fresh tokens.
Per § Future extensions → PROV-CACHE-ATTRIB: this works on every provider but the "quality" field is honest about what each one reports.
type CacheBreakpoint ¶
type CacheBreakpoint struct {
MessageIndex int // 0-based index into Request.Messages
Marker string // optional opaque marker
}
CacheBreakpoint marks a position in the history where prompt cache should be anchored. Provider-aware: Anthropic uses content-block cache_control; OpenAI uses prompt_cache_key; others ignore.
type Capabilities ¶
Capabilities describes optional features a model supports.
type Message ¶
type Message struct {
Role Role
Content []control.ContentBlock
}
Message is a canonical Anthropic-shape message (role + content blocks).
type Model ¶
type Model struct {
ID string
Name string
ContextWindow int
MaxOutput int
Pricing Pricing
Capabilities Capabilities
}
Model describes a model the provider exposes.
type Pricing ¶
type Pricing struct {
InputPerMTok float64
OutputPerMTok float64
CacheReadPerMTok float64 // discounted cache hit rate
CacheWritePerMTok float64 // cache-write surcharge
}
Pricing per million tokens, USD.
func PricingForModel ¶
PricingForModel returns the per-million-token rates for a given (providerName, modelID) pair from the static table. Returns (zero, false) for unknown combinations.
type Provider ¶
type Provider interface {
// Name returns the provider's identifier (e.g., "openrouter", "zai").
Name() string
// Chat starts a streaming chat completion. The returned channel
// is closed when the stream terminates (success or error).
//
// The engine consumes events from this channel and re-emits them
// onto the per-session Bus.
Chat(ctx context.Context, req *Request) (<-chan StreamEvent, error)
// Models returns the model catalog this provider can serve. The
// engine surfaces this through /v1/providers/{name}/models.
Models(ctx context.Context) ([]Model, error)
// TokenCount estimates how many tokens the given messages will
// consume against the named model. Used by the compaction
// trigger middleware.
TokenCount(ctx context.Context, model string, msgs []Message) (int, error)
}
Provider is the abstraction every LLM transport implements.
Concrete implementations live in subpackages:
- openrouter (v0.1)
- zai (v0.1)
- copilot (v0.2 placeholder)
- routing (v0.3 RoutingProvider composition)
type Request ¶
type Request struct {
Model string
System string // system prompt (already assembled by middleware)
Messages []Message // canonical history
Tools []ToolSchema // available tools for tool_use
Temperature float64 // 0.0 = deterministic
MaxTokens int // 0 = provider default
CacheHints []CacheBreakpoint // provider-aware cache placement
Extra map[string]interface{} // provider-specific knobs (rarely used)
}
Request is a model invocation. It is provider-agnostic: each adapter translates it to its wire format.
type StreamEvent ¶
type StreamEvent struct {
Kind StreamEventKind
Text string // for KindTextDelta
Thinking []byte // for KindThinkingDelta (opaque, provider-stamped)
ToolUse *control.ToolUse // for KindToolUseStart (Input filled in on Stop)
ToolUseID string // for KindToolUseDelta / KindToolUseStop
InputDelta string // for KindToolUseDelta (concatenates to ToolUse.Input on Stop)
Usage *Usage // for KindUsage
FinishReason string // for KindStop ("stop", "tool_use", "length", "yield")
Err error // for KindError
}
StreamEvent is a single event from a provider's streaming response. Each adapter parses provider-specific wire format and emits these canonical events. The engine's stream parser (engine/stream.go) translates them to control.Event types.
type StreamEventKind ¶
type StreamEventKind int
const ( KindTextDelta StreamEventKind = iota KindThinkingDelta KindToolUseStart KindToolUseDelta KindToolUseStop KindUsage KindStop KindError )
type ToolSchema ¶
ToolSchema is the tool catalog entry surfaced to the model.
type Usage ¶
Usage is the token accounting from a provider's response.
func (*Usage) SetCacheAttribution ¶
func (u *Usage) SetCacheAttribution(c CacheAttribution)
SetCacheAttribution updates the Usage with provider-supplied cache data. Adapters call this when they parse a usage event that includes cache info.
Directories
¶
| Path | Synopsis |
|---|---|
|
Package copilot implements the GitHub Copilot Provider.
|
Package copilot implements the GitHub Copilot Provider. |
|
Package credstore implements credential storage.
|
Package credstore implements credential storage. |
|
Package failover composes a chain of Providers with a circuit-breaker per upstream.
|
Package failover composes a chain of Providers with a circuit-breaker per upstream. |
|
Package helper is part of the GoFastr harness.
|
Package helper is part of the GoFastr harness. |
|
internal
|
|
|
openai
Package openai is an internal OpenAI-compatible adapter used by the OpenRouter and ZAI providers (both speak the same wire shape).
|
Package openai is an internal OpenAI-compatible adapter used by the OpenRouter and ZAI providers (both speak the same wire shape). |
|
Package openrouter is part of the GoFastr harness.
|
Package openrouter is part of the GoFastr harness. |
|
Package routing will implement RoutingProvider: a Provider that composes {router, executors[]} so a single turn can use a cheap model for routing and an expensive model for execution.
|
Package routing will implement RoutingProvider: a Provider that composes {router, executors[]} so a single turn can use a cheap model for routing and an expensive model for execution. |
|
Package zai is part of the GoFastr harness.
|
Package zai is part of the GoFastr harness. |