Documentation
¶
Index ¶
- type BalanceProvider
- type ChatRequest
- type ChatResponse
- type CostTracker
- func (ct *CostTracker) AllSessionCosts() map[string]float64
- func (ct *CostTracker) ExceedsBudget(sessionID string) bool
- func (ct *CostTracker) GlobalCost() float64
- func (ct *CostTracker) MaxBudgetPerSession() float64
- func (ct *CostTracker) Record(sessionID string, cost float64) bool
- func (ct *CostTracker) SessionCost(sessionID string) float64
- type FallbackRule
- type FunctionCall
- type FunctionDef
- type LLMError
- type Message
- type Provider
- type ProviderMetadata
- type Router
- func (r *Router) Complete(ctx context.Context, sessionID string, messages []Message) (*ChatResponse, error)
- func (r *Router) DefaultModel() string
- func (r *Router) HealthCheck(ctx context.Context) error
- func (r *Router) RegisterProvider(p Provider)
- func (r *Router) SetFallbacks(rules []FallbackRule)
- func (r *Router) SetTools(tools []ToolDef)
- type TokenUsage
- type ToolCall
- type ToolDef
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type BalanceProvider ¶
BalanceProvider is an optional interface implemented by providers that can report their remaining credit balance. Returns -1 if the balance is unlimited.
type ChatRequest ¶
type ChatResponse ¶
type ChatResponse struct {
Content string
ToolCalls []ToolCall
TokensUsed TokenUsage
Model string
FinishReason string
}
type CostTracker ¶
type CostTracker struct {
// contains filtered or unexported fields
}
CostTracker tracks token usage and estimated costs per session and globally.
func NewCostTracker ¶
func NewCostTracker(maxPerSession float64) *CostTracker
func (*CostTracker) AllSessionCosts ¶
func (ct *CostTracker) AllSessionCosts() map[string]float64
AllSessionCosts returns a copy of all session costs.
func (*CostTracker) ExceedsBudget ¶
func (ct *CostTracker) ExceedsBudget(sessionID string) bool
ExceedsBudget checks if a session has exceeded its budget.
func (*CostTracker) GlobalCost ¶
func (ct *CostTracker) GlobalCost() float64
GlobalCost returns the total cost across all sessions.
func (*CostTracker) MaxBudgetPerSession ¶
func (ct *CostTracker) MaxBudgetPerSession() float64
MaxBudgetPerSession returns the per-session cost cap.
func (*CostTracker) Record ¶
func (ct *CostTracker) Record(sessionID string, cost float64) bool
Record adds cost for a session. Returns true if within budget.
func (*CostTracker) SessionCost ¶
func (ct *CostTracker) SessionCost(sessionID string) float64
SessionCost returns the total cost for a session.
type FallbackRule ¶
type FallbackRule struct {
Trigger string // "error" | "rate_limit" | "low_funds"
Action string // "switch_provider" | "switch_model" | "wait_and_retry"
Provider string // provider name — for switch_provider
Model string // model name — for switch_model; optional for switch_provider
Threshold float64 // remaining credit threshold in USD — for low_funds
MaxRetries int // number of retry attempts — for wait_and_retry
Backoff string // "exponential" | "constant" — for wait_and_retry
}
FallbackRule describes a single fallback step the router will attempt.
type FunctionCall ¶
FunctionCall is the function name and JSON-encoded arguments within a ToolCall.
type FunctionDef ¶
type FunctionDef struct {
Name string `json:"name"`
Description string `json:"description"`
Parameters map[string]any `json:"parameters"` // JSON Schema object
}
FunctionDef describes the function signature within a ToolDef.
type LLMError ¶
LLMError is returned by providers for API-level failures. Use errors.As to unwrap from wrapped errors.
type Provider ¶
type Provider interface {
ChatCompletion(ctx context.Context, req ChatRequest) (*ChatResponse, error)
Name() string
HealthCheck(ctx context.Context) error
}
Provider defines the interface for LLM backends.
type ProviderMetadata ¶
type Router ¶
type Router struct {
// contains filtered or unexported fields
}
Router selects the appropriate LLM provider for a request.
func NewRouter ¶
func NewRouter(defaultProvider, defaultModel string, costTracker *CostTracker) *Router
func (*Router) DefaultModel ¶
DefaultModel returns the router's default model name.
func (*Router) RegisterProvider ¶
func (*Router) SetFallbacks ¶
func (r *Router) SetFallbacks(rules []FallbackRule)
SetFallbacks configures the ordered list of fallback rules.
type TokenUsage ¶
type ToolCall ¶
type ToolCall struct {
ID string `json:"id"`
Type string `json:"type"` // "function"
Function FunctionCall `json:"function"`
}
ToolCall represents a tool invocation requested by the LLM (OpenAI format).
type ToolDef ¶
type ToolDef struct {
Type string `json:"type"` // "function"
Function FunctionDef `json:"function"`
}
ToolDef describes a tool available for the LLM to call (OpenAI format).
Directories
¶
| Path | Synopsis |
|---|---|
|
Package anthropic implements the llm.Provider interface against the Anthropic Messages API (https://docs.anthropic.com/en/api/messages).
|
Package anthropic implements the llm.Provider interface against the Anthropic Messages API (https://docs.anthropic.com/en/api/messages). |