client

package
v0.1.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 4, 2026 License: MIT Imports: 41 Imported by: 0

Documentation

Overview

Package client provides LLM provider clients for Anthropic, OpenAI, and OpenAI-compatible APIs with streaming, retry, and provider detection.

Package client provides request coalescing for identical concurrent LLM requests. When multiple goroutines send identical requests simultaneously (same provider, model, messages, temperature, max_tokens), the Coalescer deduplicates them into a single API call and broadcasts the result to all waiters.

Index

Examples

Constants

View Source
const (
	// RolePrimary is the default, highest-capability model slot.
	RolePrimary = "primary"
	// RoleWeak is a cheaper/faster model used for auxiliary work such as
	// summarization or classification.
	RoleWeak = "weak"
	// RoleEditor is a model used to revise or refine prior output.
	RoleEditor = "editor"
)

Model role slot names. These identify a logical role that a concrete model fills, letting callers route a request to the appropriate model without hard-coding model ids at the call site.

Variables

View Source
var (
	OpenAICompat = OpenAICompatConfig{
		SupportsStore: true, SupportsDeveloperRole: true,
		SupportsReasoningEffort: true, SupportsUsageInStreaming: true,
		MaxTokensField: "max_completion_tokens",
	}
	GrokCompat = OpenAICompatConfig{
		MaxTokensField: "max_tokens",
	}
	OpenRouterCompat = OpenAICompatConfig{
		ThinkingFormat: "openrouter", MaxTokensField: "max_tokens",
		SupportsUsageInStreaming: true,
	}
	GeminiCompat = OpenAICompatConfig{
		MaxTokensField: "max_tokens", SupportsUsageInStreaming: true,
	}
	ZAICompat = OpenAICompatConfig{
		ThinkingFormat: "zai", MaxTokensField: "max_tokens",
		SupportsUsageInStreaming: true,
	}
	CanopyWaveCompat = OpenAICompatConfig{
		MaxTokensField: "max_tokens",
	}
	OllamaCompat = OpenAICompatConfig{
		MaxTokensField: "max_tokens",
	}
	OpenCodeGoCompat = OpenAICompatConfig{
		MaxTokensField:           "max_tokens",
		SupportsUsageInStreaming: true,
		ThinkingFormat:           "openrouter",
		StripReasoningFromInput:  true,
	}
	KimiCompat = OpenAICompatConfig{
		MaxTokensField:    "max_tokens",
		SupportsCacheRole: true,
	}
	XiaomiCompat = OpenAICompatConfig{
		MaxTokensField: "max_completion_tokens",
	}
	AzureCompat = OpenAICompatConfig{
		MaxTokensField: "max_tokens",
	}
	BedrockCompat = OpenAICompatConfig{
		MaxTokensField: "max_tokens",
	}
	VertexCompat = OpenAICompatConfig{
		MaxTokensField: "max_tokens",
	}
	// DeepSeekCompat: OpenAI-compatible with usage in streaming.
	// The provider rejects reasoning_content in input messages with HTTP 400, so we strip it.
	DeepSeekCompat = OpenAICompatConfig{
		MaxTokensField:           "max_tokens",
		SupportsUsageInStreaming: true,
		StripReasoningFromInput:  true,
	}
)

Per-provider compat configs.

View Source
var CoreProviders = map[string]ProviderRegistryConfig{
	"anthropic": {Name: "anthropic", Type: ProviderTypeAnthropic, EnvKey: "ANTHROPIC_API_KEY", SupportsStreaming: true, SupportsTools: true, SupportsReasoning: true},
	"openai":    {Name: "openai", Type: ProviderTypeOpenAI, BaseURL: "https://api.openai.com/v1", EnvKey: "OPENAI_API_KEY", SupportsStreaming: true, SupportsTools: true, SupportsReasoning: true},
	"azure":     {Name: "azure", Type: ProviderTypeAzure, EnvKey: "AZURE_OPENAI_API_KEY", SupportsStreaming: true, SupportsTools: true, SupportsReasoning: true},
	"bedrock":   {Name: "bedrock", Type: ProviderTypeBedrock, EnvKey: "AWS_SECRET_ACCESS_KEY", SupportsStreaming: true, SupportsTools: true, SupportsReasoning: true},
	"vertex":    {Name: "vertex", Type: ProviderTypeVertex, EnvKey: "VERTEX_ACCESS_TOKEN", SupportsStreaming: true, SupportsTools: true, SupportsReasoning: true},
}

CoreProviders are providers with dedicated SDKs. This map is populated at package init time and must not be written to afterward. All reads are safe for concurrent use without locking.

View Source
var ErrBudgetExceeded = errors.New("eyrie: virtual key budget exceeded")

ErrBudgetExceeded is returned when a virtual key has exhausted its budget.

View Source
var ErrUnknownVirtualKey = errors.New("eyrie: unknown virtual key")

ErrUnknownVirtualKey is returned when a request references a virtual key that the store does not recognize.

View Source
var OpenAICompatibleProviders = map[string]ProviderRegistryConfig{
	"deepseek":               {Name: "deepseek", Type: ProviderTypeOpenAICompatible, BaseURL: "https://api.deepseek.com/v1", EnvKey: "DEEPSEEK_API_KEY", SupportsStreaming: true, SupportsTools: true, SupportsReasoning: true},
	"grok":                   {Name: "grok", Type: ProviderTypeOpenAICompatible, BaseURL: "https://api.x.ai/v1", EnvKey: "XAI_API_KEY", SupportsStreaming: true, SupportsTools: true, SupportsReasoning: true},
	"openrouter":             {Name: "openrouter", Type: ProviderTypeOpenAICompatible, BaseURL: "https://openrouter.ai/api/v1", EnvKey: "OPENROUTER_API_KEY", SupportsStreaming: true, SupportsTools: true, SupportsReasoning: true},
	"zai_payg":               {Name: "zai_payg", Type: ProviderTypeOpenAICompatible, BaseURL: "https://api.z.ai/api/paas/v4", EnvKey: "ZAI_API_KEY", SupportsStreaming: true, SupportsTools: true, SupportsReasoning: true},
	"zai_coding":             {Name: "zai_coding", Type: ProviderTypeOpenAICompatible, BaseURL: "https://api.z.ai/api/coding/paas/v4", EnvKey: "ZAI_CODING_API_KEY", SupportsStreaming: true, SupportsTools: true, SupportsReasoning: true},
	"canopywave":             {Name: "canopywave", Type: ProviderTypeOpenAICompatible, BaseURL: "https://inference.canopywave.io/v1", EnvKey: "CANOPYWAVE_API_KEY", SupportsStreaming: true, SupportsTools: true, SupportsReasoning: true},
	"gemini":                 {Name: "gemini", Type: ProviderTypeOpenAICompatible, BaseURL: "https://generativelanguage.googleapis.com/v1beta/openai", EnvKey: "GEMINI_API_KEY", SupportsStreaming: true, SupportsTools: true, SupportsReasoning: true},
	"ollama":                 {Name: "ollama", Type: ProviderTypeOpenAICompatible, BaseURL: "http://localhost:11434/v1", EnvKey: "OLLAMA_API_KEY", SupportsStreaming: true, SupportsTools: true, SupportsReasoning: false},
	"opencodego":             {Name: "opencodego", Type: ProviderTypeOpenAICompatible, BaseURL: config.DefaultOpenCodeGoBaseURL, EnvKey: "OPENCODEGO_API_KEY", SupportsStreaming: true, SupportsTools: true, SupportsReasoning: true},
	"kimi":                   {Name: "kimi", Type: ProviderTypeOpenAICompatible, BaseURL: config.DefaultKimiOpenAIBaseURL, EnvKey: "MOONSHOT_API_KEY", SupportsStreaming: true, SupportsTools: true, SupportsReasoning: true},
	"xiaomi_mimo_payg":       {Name: "xiaomi_mimo_payg", Type: ProviderTypeOpenAICompatible, BaseURL: config.DefaultXiaomiOpenAIBaseURL, EnvKey: config.EnvXiaomiPaygAPIKey, SupportsStreaming: true, SupportsTools: true, SupportsReasoning: true},
	"xiaomi_mimo_token_plan": {Name: "xiaomi_mimo_token_plan", Type: ProviderTypeOpenAICompatible, BaseURL: "", EnvKey: config.EnvXiaomiTokenPlanAPIKey, SupportsStreaming: true, SupportsTools: true, SupportsReasoning: true},
	"minimax_token_plan":     {Name: "minimax_token_plan", Type: ProviderTypeOpenAICompatible, BaseURL: "https://api.minimax.io/v1", EnvKey: "MINIMAX_TOKEN_PLAN_API_KEY", SupportsStreaming: true, SupportsTools: true, SupportsReasoning: true},
	"minimax_payg":           {Name: "minimax_payg", Type: ProviderTypeOpenAICompatible, BaseURL: "https://api.minimax.io/v1", EnvKey: "MINIMAX_PAYG_API_KEY", SupportsStreaming: true, SupportsTools: true, SupportsReasoning: true},
}

OpenAICompatibleProviders use the OpenAI SDK with custom baseUrl.

View Source
var Version = "dev"

Version is exported here for backwards compatibility. Callers should prefer the canonical eyrie.Version (which is sourced from the repo-root VERSION file). This variable is initialised by the root package via SetVersion to avoid a circular import.

Functions

func ActualCostUSD added in v0.1.1

func ActualCostUSD(model string, usage *EyrieUsage) float64

ActualCostUSD computes the realized USD cost of a completed call from token usage using the same per-model pricing as the cost estimator.

func AddCacheBreakpoints

func AddCacheBreakpoints(messages []EyrieMessage) []anthropicCachedMessage

AddCacheBreakpoints returns a copy of messages with Anthropic cache_control breakpoints applied following the recommended pattern:

  • Breakpoint on the second-to-last message (caches conversation prefix)
  • The last user message is left uncached (always new)

Only applies to messages with role "user" or "assistant". No-op if fewer than 2 messages.

func AnthropicBaseFromOpenAIV1 added in v0.1.1

func AnthropicBaseFromOpenAIV1(openAIBase string) string

AnthropicBaseFromOpenAIV1 strips a trailing /v1 from an OpenAI-compatible base URL.

func ApplyRedactions added in v0.1.1

func ApplyRedactions(response string, violations []GuardrailViolation) string

ApplyRedactions takes the response text and violations, replacing redacted matches with their redaction markers. Non-redact violations are left intact. Match positions are used directly from the violations (captured during Check) so the correct instance of each match is redacted even when the matched text appears multiple times in the response.

func CloseIdleConnections added in v0.1.1

func CloseIdleConnections()

CloseIdleConnections closes idle connections in the shared pool. Call during graceful shutdown.

func DetectProvider

func DetectProvider() string

DetectProvider detects the active provider from the credential store (not process env).

func FormatUsageBar added in v0.1.1

func FormatUsageBar(pct float64, width int) string

func FreezeRegistry

func FreezeRegistry()

FreezeRegistry prevents further provider registrations. Called automatically after first provider lookup.

func NewPooledHTTPClient added in v0.1.1

func NewPooledHTTPClient(timeout time.Duration) *http.Client

NewPooledHTTPClient creates an *http.Client with the shared connection pool transport and the given timeout. All providers should use this instead of constructing raw *http.Client literals.

func ParseCustomHeaders

func ParseCustomHeaders() map[string]string

ParseCustomHeaders parses GRAYCODE_CUSTOM_HEADERS env var into a map.

func RegisterDynamicProvider

func RegisterDynamicProvider(name, baseURL, envKey string) error

RegisterDynamicProvider adds a user-defined OpenAI-compatible provider at runtime. name is the provider key (e.g. "my-local-llm"), baseURL is the API base (e.g. "http://localhost:8080/v1"), and envKey is the environment variable that holds the API key (e.g. "MY_LLM_API_KEY"). If envKey is empty, the provider is treated like ollama (no key required). Returns error if registry is frozen (after first provider lookup).

func ResolveDefaultModel

func ResolveDefaultModel(provider string) string

ResolveDefaultModel resolves the default model for a provider from the catalog.

func ResolveProviderModelEnvOverride

func ResolveProviderModelEnvOverride(provider string) string

ResolveProviderModelEnvOverride resolves the model env override for a provider.

func ResolveRole added in v0.1.1

func ResolveRole(roles ModelRoles, role string) string

ResolveRole returns the model id configured for the given role, defaulting to Primary when the requested role is unknown or its slot is empty. An empty Primary returns "" so the caller's existing default-model logic still applies.

func ResponseHasContent added in v0.1.1

func ResponseHasContent(resp *EyrieResponse) bool

healthFromResponse classifies a non-streaming EyrieResponse. Because the non-streaming response shape does not carry a reasoning field today, the caller passes whether reasoning was observed (e.g. from a thinking field on the raw provider payload); when unknown, pass false. ResponseHasContent reports whether a non-streaming response carries usable text.

func RoleFromContext added in v0.1.1

func RoleFromContext(ctx context.Context) string

RoleFromContext extracts the role from the context, if present.

func SaveCassette added in v0.1.1

func SaveCassette(c *Cassette, path string) error

SaveCassette writes a cassette to a JSON file atomically (temp file + rename).

func SetVersion added in v0.1.1

func SetVersion(v string)

SetVersion is called by the root eyrie package's init to wire the canonical version into this sub-package without creating an import cycle.

func ValidateStructuredOutput added in v0.1.1

func ValidateStructuredOutput(response string, schema map[string]interface{}) error

ValidateStructuredOutput validates a JSON response against a schema. It checks that the response is valid JSON and that all required fields specified in the schema are present with correct types.

func VirtualKeyFromContext added in v0.1.1

func VirtualKeyFromContext(ctx context.Context) string

VirtualKeyFromContext extracts a virtual key id from the context, if present.

func WithRole added in v0.1.1

func WithRole(ctx context.Context, role string) context.Context

WithRole returns a context carrying the named role for a call. The RoleRouter reads this to select the model for the request.

func WithVirtualKey added in v0.1.1

func WithVirtualKey(ctx context.Context, id string) context.Context

WithVirtualKey returns a context carrying the given virtual key id. The BudgetProvider reads this (falling back to ChatOptions.VirtualKeyID) to attribute and enforce spend.

Types

type AdaptiveRateLimitConfig added in v0.1.1

type AdaptiveRateLimitConfig struct {
	// RPMLimit is the maximum requests per minute (0 = no limit).
	RPMLimit int
	// TPMLimit is the maximum tokens per minute (0 = no limit).
	TPMLimit int
	// ThresholdPercent is the percentage of remaining quota below which
	// the provider starts throttling. Default is 10 (i.e., <10% remaining).
	ThresholdPercent int
	// MaxDelay is the maximum time to delay when throttling.
	// Default is 10 seconds. Set to 0 to return an error instead of delaying.
	MaxDelay time.Duration
	// HeaderExtractor is an optional function to extract rate limit info from
	// HTTP response headers. If nil, only internal tracking is used.
	HeaderExtractor HeaderExtractor
}

AdaptiveRateLimitConfig configures the AdaptiveRateLimitProvider.

type AdaptiveRateLimitProvider added in v0.1.1

type AdaptiveRateLimitProvider struct {
	// contains filtered or unexported fields
}

AdaptiveRateLimitProvider wraps any Provider with per-provider adaptive rate limiting. It tracks RPM (requests per minute) and TPM (tokens per minute), using a sliding window approach. When the remaining quota drops below a configurable threshold (default 10%), the provider either delays the request until the window resets or returns a clear error.

Rate limit state can be updated from HTTP response headers when a HeaderExtractor is provided, enabling the wrapper to react to server-reported limits even when they differ from configured values.

AdaptiveRateLimitProvider is safe for concurrent use.

Example
// Create an inner provider (e.g., from NewAnthropicProvider)
var inner Provider = &mockProvider{name: "openai"}

// Wrap with adaptive rate limiting
provider, err := NewAdaptiveRateLimitProvider(inner, AdaptiveRateLimitConfig{
	RPMLimit:         60,    // 60 requests per minute
	TPMLimit:         90000, // 90k tokens per minute
	ThresholdPercent: 10,    // throttle when <10% remaining
	MaxDelay:         5 * time.Second,
	HeaderExtractor:  CommonHeaderExtractor, // parse rate limit headers
})
if err != nil {
	fmt.Printf("Error: %v\n", err)
	return
}

// Use the provider normally
resp, err := provider.Chat(context.Background(), []EyrieMessage{
	{Role: "user", Content: "Hello!"},
}, ChatOptions{Model: "gpt-4"})
if err != nil {
	fmt.Printf("Error: %v\n", err)
	return
}
fmt.Printf("Response: %s\n", resp.Content)

// Check rate limit status
status := provider.Status()
fmt.Printf("RPM used: %d/%d\n", status.RPMUsed, status.RPMLimit)
fmt.Printf("Tokens used: %d/%d\n", status.TPMUsed, status.TPMLimit)
fmt.Printf("Throttle count: %d\n", status.ThrottleCount)

func NewAdaptiveRateLimitProvider added in v0.1.1

func NewAdaptiveRateLimitProvider(inner Provider, config AdaptiveRateLimitConfig) (*AdaptiveRateLimitProvider, error)

NewAdaptiveRateLimitProvider wraps inner with adaptive rate limiting. inner must not be nil (an error is returned otherwise). config may be zero-valued for sensible defaults.

func (*AdaptiveRateLimitProvider) Chat added in v0.1.1

Chat sends a non-streaming chat request with adaptive rate limiting.

func (*AdaptiveRateLimitProvider) Name added in v0.1.1

Name returns the inner provider name suffixed with "/adaptive-ratelimit".

func (*AdaptiveRateLimitProvider) Ping added in v0.1.1

Ping delegates directly to the inner provider.

func (*AdaptiveRateLimitProvider) Status added in v0.1.1

Status returns a snapshot of the current rate limit state.

func (*AdaptiveRateLimitProvider) StreamChat added in v0.1.1

func (a *AdaptiveRateLimitProvider) StreamChat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*StreamResult, error)

StreamChat sends a streaming chat request with adaptive rate limiting.

func (*AdaptiveRateLimitProvider) UpdateFromHeaders added in v0.1.1

func (a *AdaptiveRateLimitProvider) UpdateFromHeaders(h http.Header)

UpdateFromHeaders updates the rate limit state from HTTP response headers. This can be called externally when headers are available (e.g., from a middleware or response interceptor).

type Alert added in v0.1.1

type Alert struct {
	Level     string
	Message   string
	Timestamp time.Time
	Threshold float64
}

Alert represents a usage threshold alert.

type AnthropicClient

type AnthropicClient struct {
	// contains filtered or unexported fields
}

AnthropicClient implements Provider for the Anthropic Messages API.

func NewAnthropicClient

func NewAnthropicClient(apiKey, baseURL string, opts ...ClientOption) *AnthropicClient

NewAnthropicClient creates a configured Anthropic client.

func (*AnthropicClient) Chat

func (c *AnthropicClient) Chat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*EyrieResponse, error)

NOTE: Anthropic does not support a native JSON mode (response_format). Structured output with Anthropic is achieved via the tool-use pattern (defining a tool whose input_schema is your desired output schema). This is not implemented here; opts.ResponseFormat is ignored for Anthropic. Future work: implement tool-use-based structured output for Anthropic.

func (*AnthropicClient) CountTokens added in v0.1.1

func (c *AnthropicClient) CountTokens(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*TokenCountResult, error)

CountTokens counts the number of tokens in a message without generating a response. Uses the same request format as Chat but hits the /v1/messages/count_tokens endpoint. The request body includes model, messages, system, and tools (but not max_tokens or stream).

func (*AnthropicClient) Name

func (c *AnthropicClient) Name() string

Name returns the provider name.

func (*AnthropicClient) Ping

func (c *AnthropicClient) Ping(ctx context.Context) error

Ping checks connectivity to the Anthropic API using a lightweight GET request.

func (*AnthropicClient) StreamChat

func (c *AnthropicClient) StreamChat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*StreamResult, error)

StreamChat sends a streaming message to Anthropic.

type AnthropicClientConfig

type AnthropicClientConfig struct {
	APIKey         string            `json:"-"`
	DefaultHeaders map[string]string `json:"default_headers,omitempty"`
	Timeout        int               `json:"timeout,omitempty"`
	MaxRetries     int               `json:"max_retries,omitempty"`
	Provider       string            `json:"provider,omitempty"`
	BaseURL        string            `json:"base_url,omitempty"`
}

AnthropicClientConfig holds config for creating an Anthropic client.

type AzureClient added in v0.1.1

type AzureClient struct {
	// contains filtered or unexported fields
}

func NewAzureClient added in v0.1.1

func NewAzureClient(apiKey, endpoint, apiVersion string) *AzureClient

func (*AzureClient) Chat added in v0.1.1

func (c *AzureClient) Chat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*EyrieResponse, error)

func (*AzureClient) Name added in v0.1.1

func (c *AzureClient) Name() string

func (*AzureClient) Ping added in v0.1.1

func (c *AzureClient) Ping(ctx context.Context) error

func (*AzureClient) StreamChat added in v0.1.1

func (c *AzureClient) StreamChat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*StreamResult, error)

type BatchClient

type BatchClient struct {
	// contains filtered or unexported fields
}

BatchClient handles Anthropic Message Batches API (50% cost discount).

func NewBatchClient

func NewBatchClient(apiKey, baseURL string) *BatchClient

NewBatchClient creates a batch client for Anthropic's batch API.

func (*BatchClient) Poll

func (bc *BatchClient) Poll(ctx context.Context, batchID string) (*BatchResult, error)

Poll checks the status of a batch. Returns the result when complete.

func (*BatchClient) Submit

func (bc *BatchClient) Submit(ctx context.Context, requests []BatchRequest) (string, error)

Submit sends a batch of requests. Returns the batch ID for polling.

type BatchRequest

type BatchRequest struct {
	CustomID string         `json:"custom_id"`
	Messages []EyrieMessage `json:"messages"`
	Options  ChatOptions    `json:"options"`
}

BatchRequest represents a single request in a batch.

type BatchResponse

type BatchResponse struct {
	CustomID string         `json:"custom_id"`
	Response *EyrieResponse `json:"response,omitempty"`
	Error    string         `json:"error,omitempty"`
}

BatchResponse represents a single response from a batch.

type BatchResult

type BatchResult struct {
	ID        string          `json:"id"`
	Status    string          `json:"status"` // "in_progress", "ended", "failed"
	Responses []BatchResponse `json:"responses,omitempty"`
	CreatedAt time.Time       `json:"created_at"`
}

BatchResult holds the overall batch operation result.

type BedrockClient added in v0.1.1

type BedrockClient struct {
	// contains filtered or unexported fields
}

func NewBedrockClient added in v0.1.1

func NewBedrockClient(accessKeyID, secretAccessKey, sessionToken, region string) *BedrockClient

func (*BedrockClient) Chat added in v0.1.1

func (c *BedrockClient) Chat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*EyrieResponse, error)

func (*BedrockClient) Name added in v0.1.1

func (c *BedrockClient) Name() string

func (*BedrockClient) Ping added in v0.1.1

func (c *BedrockClient) Ping(ctx context.Context) error

func (*BedrockClient) StreamChat added in v0.1.1

func (c *BedrockClient) StreamChat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*StreamResult, error)

type BudgetProvider added in v0.1.1

type BudgetProvider struct {
	// contains filtered or unexported fields
}

BudgetProvider wraps a Provider and enforces per-virtual-key budgets. Before each call it estimates cost and rejects the request if the key is over budget; after a successful call it records the actual spend.

Requests without a virtual key (none in context or options) pass through unmetered, preserving existing behavior.

func NewBudgetProvider added in v0.1.1

func NewBudgetProvider(inner Provider, store BudgetStore) *BudgetProvider

NewBudgetProvider wraps inner with budget enforcement backed by store.

func (*BudgetProvider) Chat added in v0.1.1

func (bp *BudgetProvider) Chat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*EyrieResponse, error)

Chat enforces the budget for the request's virtual key, calls the inner provider, then records actual spend.

func (*BudgetProvider) Name added in v0.1.1

func (bp *BudgetProvider) Name() string

Name returns the inner provider's name.

func (*BudgetProvider) Ping added in v0.1.1

func (bp *BudgetProvider) Ping(ctx context.Context) error

Ping delegates to the inner provider.

func (*BudgetProvider) StreamChat added in v0.1.1

func (bp *BudgetProvider) StreamChat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*StreamResult, error)

StreamChat enforces budget up-front, then streams. Actual streamed usage is recorded from the final usage event if present.

type BudgetStore added in v0.1.1

type BudgetStore interface {
	// CheckBudget returns ErrBudgetExceeded if charging estCostUSD to the key
	// would exceed its budget, ErrUnknownVirtualKey if the key is unknown, or
	// nil if the request may proceed.
	CheckBudget(ctx context.Context, virtualKey string, estCostUSD float64) error
	// RecordUsage records actual spend against a virtual key after a call.
	RecordUsage(ctx context.Context, virtualKey string, costUSD float64, tokensIn, tokensOut int) error
}

BudgetStore is the persistence contract the BudgetProvider depends on. Its methods use only primitive types so both the in-memory store here and the SQLite store in the storage package satisfy it without an import cycle.

type CacheAnalytics added in v0.1.1

type CacheAnalytics struct {
	// contains filtered or unexported fields
}

CacheAnalytics tracks prompt caching effectiveness and cost savings. Shows developers exactly how much money and latency caching saves.

func NewCacheAnalytics added in v0.1.1

func NewCacheAnalytics() *CacheAnalytics

NewCacheAnalytics creates a cache analytics tracker.

func (*CacheAnalytics) FormatSummary added in v0.1.1

func (ca *CacheAnalytics) FormatSummary() string

FormatSummary returns a human-readable summary.

func (*CacheAnalytics) RecordCall added in v0.1.1

func (ca *CacheAnalytics) RecordCall(m CallMetrics)

RecordCall records a call's cache usage from its metrics.

func (*CacheAnalytics) Report added in v0.1.1

func (ca *CacheAnalytics) Report() CacheReport

Report returns the current cache effectiveness report.

type CacheConfig

type CacheConfig struct {
	// MaxAge is how long cache entries remain valid. Default: 5 minutes.
	MaxAge time.Duration
	// MaxSize is the maximum number of cached responses. Default: 100.
	// When exceeded, the least-recently-used entry is evicted.
	MaxSize int
	// Enabled toggles caching. Default: true.
	// When false, the CachedProvider passes all requests through unchanged.
	Enabled bool
	// TemperatureThreshold is the temperature above which responses are not cached.
	// Default: 0.5. Responses with temperature > threshold are expected to vary,
	// so caching them would defeat the purpose.
	TemperatureThreshold float64
}

CacheConfig controls the behavior of CachedProvider.

func DefaultCacheConfig

func DefaultCacheConfig() CacheConfig

DefaultCacheConfig returns a CacheConfig with sensible defaults.

type CacheControlType

type CacheControlType string

CacheControlType is the type of cache control.

const (
	// CacheControlEphemeral caches content for up to 5 minutes.
	CacheControlEphemeral CacheControlType = "ephemeral"
)

type CacheReport added in v0.1.1

type CacheReport struct {
	TotalCalls   int           `json:"total_calls"`
	CacheHits    int           `json:"cache_hits"`
	CacheMisses  int           `json:"cache_misses"`
	HitRate      float64       `json:"hit_rate"`
	TokensSaved  int           `json:"tokens_saved"`
	CostSaved    float64       `json:"cost_saved_usd"`
	LatencySaved time.Duration `json:"latency_saved"`
}

CacheReport summarizes caching effectiveness.

type CacheStatsResult

type CacheStatsResult struct {
	Size    int  `json:"size"`
	MaxSize int  `json:"max_size"`
	Enabled bool `json:"enabled"`
}

CacheStatsResult holds cache statistics.

type CachedContent

type CachedContent struct {
	Type         string      `json:"type"`
	Text         string      `json:"text"`
	CacheControl interface{} `json:"cache_control,omitempty"`
}

CachedContent wraps a content string with cache control metadata.

type CachedProvider

type CachedProvider struct {
	// contains filtered or unexported fields
}

CachedProvider wraps a Provider and caches non-streaming responses based on a hash of the input parameters. Inspired by maximhq/bifrost's caching layer.

CachedProvider is safe for concurrent use.

func NewCachedProvider

func NewCachedProvider(inner Provider, cfg CacheConfig) *CachedProvider

NewCachedProvider wraps inner with a response cache configured by cfg. Zero-value fields in cfg are replaced with defaults.

func (*CachedProvider) CacheStats

func (cp *CachedProvider) CacheStats() CacheStatsResult

CacheStats returns the current number of entries in the cache.

func (*CachedProvider) Chat

func (cp *CachedProvider) Chat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*EyrieResponse, error)

Chat checks the cache first. On a miss, it calls the inner provider and caches the response (if the temperature is not too high).

func (*CachedProvider) ClearCache

func (cp *CachedProvider) ClearCache()

ClearCache removes all cached entries.

func (*CachedProvider) Name

func (cp *CachedProvider) Name() string

Name returns the inner provider's name.

func (*CachedProvider) Ping

func (cp *CachedProvider) Ping(ctx context.Context) error

Ping delegates to the inner provider (no caching).

func (*CachedProvider) SetEnabled

func (cp *CachedProvider) SetEnabled(enabled bool)

SetEnabled toggles caching at runtime.

func (*CachedProvider) StreamChat

func (cp *CachedProvider) StreamChat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*StreamResult, error)

StreamChat delegates to the inner provider without caching. Streaming responses are inherently incremental and not suitable for simple response caching.

type CallMetrics

type CallMetrics struct {
	Model               string    `json:"model"`
	Provider            string    `json:"provider"`
	InputTokens         int       `json:"input_tokens"`
	OutputTokens        int       `json:"output_tokens"`
	CacheReadTokens     int       `json:"cache_read_tokens"`
	CacheCreationTokens int       `json:"cache_creation_tokens"`
	LatencyMs           int64     `json:"latency_ms"`
	Timestamp           time.Time `json:"timestamp"`
}

CallMetrics records telemetry for a single LLM API call.

type CallbackProvider added in v0.1.1

type CallbackProvider struct {
	// contains filtered or unexported fields
}

CallbackProvider wraps any Provider and invokes registered ProviderCallback hooks at the appropriate points in the request lifecycle.

Callbacks are executed in separate goroutines so they never block the main request path. A panic in a callback is recovered and logged; it does not crash the caller.

CallbackProvider is safe for concurrent use.

func NewCallbackProvider added in v0.1.1

func NewCallbackProvider(inner Provider) (*CallbackProvider, error)

NewCallbackProvider wraps the given provider with callback support. The inner provider must not be nil; an error is returned otherwise.

func (*CallbackProvider) AddCallback added in v0.1.1

func (cp *CallbackProvider) AddCallback(cb ProviderCallback)

AddCallback registers a callback. It is safe to call from any goroutine.

func (*CallbackProvider) Callbacks added in v0.1.1

func (cp *CallbackProvider) Callbacks() []ProviderCallback

Callbacks returns a snapshot of the currently registered callbacks.

func (*CallbackProvider) Chat added in v0.1.1

func (cp *CallbackProvider) Chat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*EyrieResponse, error)

Chat sends a non-streaming chat request. OnRequest is called before the request; OnResponse or OnError is called after.

func (*CallbackProvider) Inner added in v0.1.1

func (cp *CallbackProvider) Inner() Provider

Inner returns the wrapped provider.

func (*CallbackProvider) Name added in v0.1.1

func (cp *CallbackProvider) Name() string

Name returns the inner provider name suffixed with "/callbacks".

func (*CallbackProvider) Ping added in v0.1.1

func (cp *CallbackProvider) Ping(ctx context.Context) error

Ping delegates to the inner provider.

func (*CallbackProvider) RemoveCallback added in v0.1.1

func (cp *CallbackProvider) RemoveCallback(cb ProviderCallback) bool

RemoveCallback removes a previously registered callback by identity (pointer comparison). Returns true if the callback was found and removed.

func (*CallbackProvider) SetLogger added in v0.1.1

func (cp *CallbackProvider) SetLogger(l *slog.Logger)

SetLogger sets the logger used for panic-recovery messages.

func (*CallbackProvider) StreamChat added in v0.1.1

func (cp *CallbackProvider) StreamChat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*StreamResult, error)

StreamChat sends a streaming chat request. OnRequest is called before the request. OnError is called if the initial request fails. OnStreamEvent is called for each event in the resulting stream.

type Cassette added in v0.1.1

type Cassette struct {
	Name         string        `json:"name"`
	RecordedAt   time.Time     `json:"recorded_at"`
	Provider     string        `json:"provider"`
	Interactions []Interaction `json:"interactions"`
}

Cassette stores recorded LLM interactions for deterministic replay.

func LoadCassette added in v0.1.1

func LoadCassette(path string) (*Cassette, error)

LoadCassette reads a cassette from a JSON file at path.

type ChatOptions

type ChatOptions struct {
	Provider       string          `json:"provider,omitempty"`
	Model          string          `json:"model,omitempty"`
	Temperature    *float64        `json:"temperature,omitempty"`
	MaxTokens      int             `json:"max_tokens,omitempty"`
	Stream         bool            `json:"stream,omitempty"`
	Tools          []EyrieTool     `json:"tools,omitempty"`
	System         string          `json:"system,omitempty"`
	EnableCaching  bool            `json:"enable_caching,omitempty"`
	ResponseFormat *ResponseFormat `json:"response_format,omitempty"`
	// ReasoningEffort hints how much reasoning the model should perform.
	// Valid values are "low", "medium", or "high" (see ReasoningLow/Medium/High).
	// Only applied for OpenAI-compatible providers whose compat config sets
	// SupportsReasoningEffort.
	ReasoningEffort string `json:"reasoning_effort,omitempty"`
	// ThinkingBudgetTokens enables Anthropic extended thinking with the given
	// token budget when greater than zero. Ignored by other providers.
	ThinkingBudgetTokens int `json:"thinking_budget_tokens,omitempty"`
	// ThinkingMode controls Anthropic thinking behavior: "enabled", "adaptive", or "disabled".
	// When set, takes precedence over ThinkingBudgetTokens (except "enabled" which uses the budget).
	ThinkingMode string `json:"thinking_mode,omitempty"`
	// ThinkingDisplay controls how thinking is shown: "summarized" (default) or "omitted".
	ThinkingDisplay string `json:"thinking_display,omitempty"`
	// GLMThinkingEnabled toggles GLM/Z.ai extended reasoning via the provider's
	// non-OpenAI thinking={"type":"enabled"|"disabled"} request parameter. Only
	// applied for OpenAI-compatible providers whose compat config sets
	// ThinkingFormat to "zai". When nil the parameter is omitted and the model
	// uses its default (GLM defaults to enabled). Ignored by other providers.
	GLMThinkingEnabled *bool `json:"glm_thinking_enabled,omitempty"`
	// VirtualKeyID optionally attributes the request to a logical virtual key
	// for budget enforcement and cost accounting (see BudgetProvider). When
	// empty, the BudgetProvider also checks the request context.
	VirtualKeyID string `json:"virtual_key_id,omitempty"`
	// KimiContextCacheID, when set, prepends a cache-role message to the
	// request for Kimi/Moonshot context caching. Only effective when the
	// provider compat is KimiCompat (SupportsCacheRole true).
	KimiContextCacheID string `json:"kimi_context_cache_id,omitempty"`
	// KimiCacheResetTTL resets the TTL of the cache on use when true.
	// Only effective when KimiContextCacheID is also set.
	KimiCacheResetTTL bool `json:"kimi_cache_reset_ttl,omitempty"`

	// Shared parameters (Anthropic + OpenAI)
	TopP           *float64          `json:"top_p,omitempty"`            // nucleus sampling (0.0-1.0)
	TopK           *int              `json:"top_k,omitempty"`            // top-K sampling (Anthropic only)
	StopSequences  []string          `json:"stop_sequences,omitempty"`   // custom stop sequences
	ToolChoice     *ToolChoiceOption `json:"tool_choice,omitempty"`      // tool use control
	MetadataUserID string            `json:"metadata_user_id,omitempty"` // user ID for abuse detection / monitoring
	ServiceTier    string            `json:"service_tier,omitempty"`     // "auto", "default", "flex", "priority"
	OutputEffort   string            `json:"output_effort,omitempty"`    // "low","medium","high","xhigh","max" (Anthropic)
	OutputSchema   string            `json:"output_schema,omitempty"`    // JSON schema string for structured output (Anthropic)

	// OpenAI-specific parameters
	PresencePenalty  *float64          `json:"presence_penalty,omitempty"`   // -2.0 to 2.0
	FrequencyPenalty *float64          `json:"frequency_penalty,omitempty"`  // -2.0 to 2.0
	N                *int              `json:"n,omitempty"`                  // number of completions (1-128)
	LogProbs         *bool             `json:"logprobs,omitempty"`           // return log probabilities
	TopLogProbs      *int              `json:"top_logprobs,omitempty"`       // 0-20, requires logprobs=true
	Seed             *int              `json:"seed,omitempty"`               // deterministic sampling
	Store            *bool             `json:"store,omitempty"`              // store output for Responses API
	Metadata         map[string]string `json:"metadata,omitempty"`           // developer tags
	Modalities       []string          `json:"modalities,omitempty"`         // "text", "audio"
	AudioConfig      string            `json:"audio_config,omitempty"`       // JSON: {voice, format}
	Prediction       string            `json:"prediction,omitempty"`         // JSON: {type:"content", content:"..."}
	WebSearchOptions string            `json:"web_search_options,omitempty"` // JSON: {search_context_size, user_location}
}

ChatOptions holds options for a chat request.

type ChatProtocol added in v0.1.1

type ChatProtocol int

ChatProtocol selects which existing eyrie client handles a gateway request.

const (
	// ChatProtocolCompletions routes via OpenAIClient (POST /v1/chat/completions).
	ChatProtocolCompletions ChatProtocol = iota
	// ChatProtocolMessages routes via AnthropicClient (POST /v1/messages).
	ChatProtocolMessages
)

type ClientOption

type ClientOption struct {
	// contains filtered or unexported fields
}

ClientOption configures clients.

func WithAPIKey added in v0.1.1

func WithAPIKey(key string) ClientOption

WithAPIKey sets the API key.

func WithBaseURL added in v0.1.1

func WithBaseURL(url string) ClientOption

WithBaseURL sets the base URL.

func WithCoalescing added in v0.1.1

func WithCoalescing(ttl time.Duration) ClientOption

WithCoalescing enables request coalescing for identical concurrent requests. When enabled, multiple goroutines sending identical requests (same provider, model, messages, temperature, max_tokens) will be deduplicated into a single API call, with the result broadcast to all waiters.

The ttl parameter controls how long completed requests remain in the coalescer for potential reuse. A typical value is 100-500ms.

func WithGuardrailType added in v0.1.1

func WithGuardrailType(types ...GuardrailType) ClientOption

WithGuardrailType attaches output guardrails using built-in rules for the specified types. For example, WithGuardrailType(GuardrailPII, GuardrailSecretLeak) enables PII redaction and secret leak blocking with default patterns.

func WithGuardrails added in v0.1.1

func WithGuardrails(rules ...GuardrailRule) ClientOption

WithGuardrails attaches output guardrails to the client. Guardrails run after the LLM response but before returning to the caller. Blocked responses are replaced with an error; redacted responses have matches replaced with asterisks.

func WithHTTPClient

func WithHTTPClient(hc *http.Client) ClientOption

WithHTTPClient sets a custom HTTP client.

func WithLogger

func WithLogger(l *slog.Logger) ClientOption

WithLogger sets the logger.

func WithMaxTokens added in v0.1.1

func WithMaxTokens(n int) ClientOption

WithMaxTokens sets the default max tokens for requests.

func WithMimoAuth added in v0.1.1

func WithMimoAuth() ClientOption

WithMimoAuth uses api-key header per MiMo documentation (OpenAI + Anthropic compat).

func WithModel added in v0.1.1

func WithModel(model string) ClientOption

WithModel sets the default model for requests.

func WithProviderName added in v0.1.1

func WithProviderName(name string) ClientOption

WithProviderName sets the OpenAI client provider name for errors/logging.

func WithRetry

func WithRetry(rc RetryConfig) ClientOption

WithRetry sets retry configuration.

func WithStructuredOutput added in v0.1.1

func WithStructuredOutput(schema map[string]interface{}, maxRetries int) ClientOption

WithStructuredOutput returns a ClientOption that configures structured JSON output. For OpenAI, it sets response_format to json_schema with the given schema. For Anthropic, it enables structured output via the prefill technique.

func WithTemperature added in v0.1.1

func WithTemperature(t float64) ClientOption

WithTemperature sets the default temperature for requests.

func WithTimeout

func WithTimeout(d time.Duration) ClientOption

WithTimeout sets the HTTP client timeout.

type CoalesceKey added in v0.1.1

type CoalesceKey struct {
	Provider    string         `json:"provider"`
	Model       string         `json:"model"`
	Messages    []EyrieMessage `json:"messages"`
	Temperature *float64       `json:"temperature,omitempty"`
	MaxTokens   int            `json:"max_tokens,omitempty"`
}

CoalesceKey uniquely identifies an LLM request for deduplication. It hashes provider, model, messages, temperature, and max_tokens.

func (CoalesceKey) String added in v0.1.1

func (k CoalesceKey) String() string

String returns a stable hash of the coalesce key for map lookup.

type Coalescer added in v0.1.1

type Coalescer struct {
	// contains filtered or unexported fields
}

Coalescer deduplicates identical concurrent LLM requests. It maintains a map of inflight requests indexed by CoalesceKey. When a request arrives that matches an existing inflight request, the caller waits on the existing request's done channel instead of making a new API call.

The Coalescer automatically cleans up completed requests after a TTL. Coalescer is safe for concurrent use.

func NewCoalescer added in v0.1.1

func NewCoalescer(ttl time.Duration) *Coalescer

NewCoalescer creates a new Coalescer with the specified TTL for completed requests. The ttl controls how long completed requests remain in the map for potential reuse.

func (*Coalescer) Coalesce added in v0.1.1

func (c *Coalescer) Coalesce(ctx context.Context, key CoalesceKey, fn func() (*EyrieResponse, error)) (*EyrieResponse, error)

Coalesce deduplicates concurrent identical requests.

If an inflight request exists for the given key, the caller waits on its done channel and returns the same result. Otherwise, a new InflightRequest is created, added to the inflight map, and fn is called to execute the actual request.

The executing goroutine is responsible for:

  1. Calling fn() to get the result
  2. Storing the result in the InflightRequest
  3. Closing the done channel to wake all waiters

All waiting goroutines receive the same response.

func (*Coalescer) Stats added in v0.1.1

func (c *Coalescer) Stats() InflightStats

Stats returns the number of inflight requests (for monitoring/testing).

type CondenseOptions added in v0.1.1

type CondenseOptions struct {
	// MaxSize is the message-count threshold above which condensation runs.
	// When len(messages) <= MaxSize, the history is returned unchanged.
	MaxSize int
	// KeepFirst is the number of leading messages preserved verbatim (e.g. a
	// system prompt and the opening turns). The middle span between the kept
	// head and tail is what gets summarized.
	KeepFirst int
}

CondenseOptions controls how a ConversationCondenser reduces a message history.

type CondenserOption added in v0.1.1

type CondenserOption func(*LLMSummarizingCondenser)

CondenserOption configures an LLMSummarizingCondenser.

func WithCondenserMaxTokens added in v0.1.1

func WithCondenserMaxTokens(n int) CondenserOption

WithCondenserMaxTokens caps the number of tokens requested for the summary.

func WithCondenserPrompt added in v0.1.1

func WithCondenserPrompt(prompt string) CondenserOption

WithCondenserPrompt overrides the default summarization instruction.

func WithCondenserRoles added in v0.1.1

func WithCondenserRoles(roles ModelRoles) CondenserOption

WithCondenserRoles sets the model roles used by the condenser. The Weak role is preferred for the summary call.

type CondensingProvider added in v0.1.1

type CondensingProvider struct {
	// contains filtered or unexported fields
}

CondensingProvider wraps a Provider and runs a ConversationCondenser over the request messages before delegating to the inner provider. It follows the same decorator pattern as BudgetProvider and TracingProvider.

Condensation applies to both Chat and StreamChat. A nil condenser or non-positive CondenseOptions.MaxSize disables condensation (pass-through).

func NewCondensingProvider added in v0.1.1

func NewCondensingProvider(inner Provider, condenser ConversationCondenser, opts CondenseOptions) *CondensingProvider

NewCondensingProvider wraps inner so that request histories are condensed via condenser using the given options. The inner provider must not be nil.

func (*CondensingProvider) Chat added in v0.1.1

func (p *CondensingProvider) Chat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*EyrieResponse, error)

Chat condenses the messages, then delegates to the inner provider.

func (*CondensingProvider) Name added in v0.1.1

func (p *CondensingProvider) Name() string

Name returns the inner provider's name.

func (*CondensingProvider) Ping added in v0.1.1

func (p *CondensingProvider) Ping(ctx context.Context) error

Ping delegates to the inner provider.

func (*CondensingProvider) StreamChat added in v0.1.1

func (p *CondensingProvider) StreamChat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*StreamResult, error)

StreamChat condenses the messages, then delegates to the inner provider.

type ContentPart added in v0.1.1

type ContentPart struct {
	Type       string          `json:"type"`                  // "text", "image_url", "input_audio"
	Text       string          `json:"text,omitempty"`        // for type="text"
	ImageURL   *ImageURLPart   `json:"image_url,omitempty"`   // for type="image_url"
	InputAudio *InputAudioPart `json:"input_audio,omitempty"` // for type="input_audio"
}

ContentPart represents a piece of content in a multi-modal message. Use the helper types (TextPart, ImagePart, AudioPart) to construct these.

type ContinuationConfig

type ContinuationConfig struct {
	// MaxContinuations is the maximum number of continuation calls (default 3).
	MaxContinuations int
	// MaxTotalTokens caps the total output tokens across all continuations (0 = unlimited).
	MaxTotalTokens int
}

ContinuationConfig controls output continuation behavior.

func DefaultContinuationConfig

func DefaultContinuationConfig() ContinuationConfig

DefaultContinuationConfig returns sensible defaults.

type ConversationCondenser added in v0.1.1

type ConversationCondenser interface {
	// Condense returns a reduced copy of messages according to opts. It must
	// not mutate the input slice. When no reduction is needed it may return the
	// input slice unchanged.
	Condense(ctx context.Context, messages []EyrieMessage, opts CondenseOptions) ([]EyrieMessage, error)
}

ConversationCondenser reduces a message history so that long conversations stay within a model's context window while preserving salient information.

type CostEstimate added in v0.1.1

type CostEstimate struct {
	InputTokens   int     `json:"input_tokens"`
	OutputTokens  int     `json:"estimated_output_tokens"`
	InputCostUSD  float64 `json:"input_cost_usd"`
	OutputCostUSD float64 `json:"estimated_output_cost_usd"`
	TotalCostUSD  float64 `json:"estimated_total_cost_usd"`
	Model         string  `json:"model"`
	CacheDiscount float64 `json:"cache_discount_usd"` // potential savings if cached
}

CostEstimate is the pre-call cost prediction.

type CostEstimator added in v0.1.1

type CostEstimator struct{}

CostEstimator estimates the cost of an API call BEFORE sending it. Helps developers set budgets and avoid surprise charges.

func NewCostEstimator added in v0.1.1

func NewCostEstimator() *CostEstimator

NewCostEstimator creates an estimator.

func (*CostEstimator) Estimate added in v0.1.1

func (ce *CostEstimator) Estimate(messages []EyrieMessage, model string, maxOutputTokens int) CostEstimate

Estimate predicts cost for a set of messages + expected output.

func (*CostEstimator) FormatEstimate added in v0.1.1

func (ce *CostEstimator) FormatEstimate(est CostEstimate) string

FormatEstimate returns a human-readable cost estimate.

func (*CostEstimator) IsExpensive added in v0.1.1

func (ce *CostEstimator) IsExpensive(est CostEstimate, threshold float64) bool

IsExpensive returns true if the estimated cost exceeds a threshold.

type DeepSeekClient added in v0.1.1

type DeepSeekClient struct {
	// contains filtered or unexported fields
}

DeepSeekClient uses OpenAI-compatible DeepSeek endpoints first, with optional Anthropic-compat fallback if the OpenAI endpoint is down.

func NewDeepSeekClient added in v0.1.1

func NewDeepSeekClient(apiKey, openAIBase, anthropicBase string, compat *OpenAICompatConfig, opts ...ClientOption) *DeepSeekClient

NewDeepSeekClient builds a DeepSeek provider client. openAIBase is typically "https://api.deepseek.com/v1" anthropicBase is typically "https://api.deepseek.com/anthropic"

func (*DeepSeekClient) Chat added in v0.1.1

func (c *DeepSeekClient) Chat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*EyrieResponse, error)

func (*DeepSeekClient) Name added in v0.1.1

func (c *DeepSeekClient) Name() string

func (*DeepSeekClient) Ping added in v0.1.1

func (c *DeepSeekClient) Ping(ctx context.Context) error

func (*DeepSeekClient) StreamChat added in v0.1.1

func (c *DeepSeekClient) StreamChat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*StreamResult, error)

type DeprecationChecker added in v0.1.1

type DeprecationChecker struct {
	// contains filtered or unexported fields
}

DeprecationChecker warns when using models approaching retirement. Read-only after construction — no mutex needed.

func NewDeprecationChecker added in v0.1.1

func NewDeprecationChecker() *DeprecationChecker

NewDeprecationChecker creates a checker with known deprecations.

func (*DeprecationChecker) Check added in v0.1.1

func (dc *DeprecationChecker) Check(model string) *DeprecationInfo

Check returns deprecation info if the model is deprecated.

func (*DeprecationChecker) Warn added in v0.1.1

func (dc *DeprecationChecker) Warn(model string)

Warn logs a deprecation warning if applicable.

type DeprecationInfo added in v0.1.1

type DeprecationInfo struct {
	Model       string    `json:"model"`
	Deprecated  bool      `json:"deprecated"`
	RetireDate  time.Time `json:"retire_date"`
	Replacement string    `json:"replacement"`
	Message     string    `json:"message"`
}

DeprecationInfo describes a model's deprecation status.

type Embedder added in v0.1.1

type Embedder interface {
	CreateEmbedding(ctx context.Context, req EmbeddingRequest) (*EmbeddingResponse, error)
}

Embedder is the interface for creating embeddings.

type EmbeddingCachedProvider added in v0.1.1

type EmbeddingCachedProvider struct {
	// contains filtered or unexported fields
}

EmbeddingCachedProvider wraps a Provider and serves cached responses when a new request is semantically similar (cosine similarity above a threshold) to a previously seen request. Unlike CachedProvider's exact-match SHA256 keying, this tolerates paraphrases and minor wording changes.

It is safe for concurrent use. StreamChat is passed through unchanged.

func NewEmbeddingCachedProvider added in v0.1.1

func NewEmbeddingCachedProvider(inner Provider, embedder Embedder, cfg SemanticCacheConfig) *EmbeddingCachedProvider

NewEmbeddingCachedProvider wraps inner with a semantic (embedding-similarity) cache. embedder is used to embed requests; cfg zero-value fields are replaced with defaults.

func (*EmbeddingCachedProvider) Chat added in v0.1.1

Chat returns a semantically-cached response on a hit; otherwise calls the inner provider and caches the result.

func (*EmbeddingCachedProvider) Name added in v0.1.1

func (sp *EmbeddingCachedProvider) Name() string

Name returns the inner provider's name.

func (*EmbeddingCachedProvider) Ping added in v0.1.1

Ping delegates to the inner provider.

func (*EmbeddingCachedProvider) Stats added in v0.1.1

Stats returns current cache statistics.

func (*EmbeddingCachedProvider) StreamChat added in v0.1.1

func (sp *EmbeddingCachedProvider) StreamChat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*StreamResult, error)

StreamChat delegates to the inner provider without caching.

type EmbeddingParams

type EmbeddingParams struct {
	Indexing map[string]string `json:"indexing,omitempty"`
	Query    map[string]string `json:"query,omitempty"`
}

EmbeddingParams holds asymmetric params for indexing vs query.

func DefaultEmbeddingParams

func DefaultEmbeddingParams(model string) EmbeddingParams

DefaultEmbeddingParams returns known-good asymmetric params for common embedding models.

type EmbeddingRequest

type EmbeddingRequest struct {
	Model  string            `json:"model"`
	Input  []string          `json:"input"`
	Params map[string]string `json:"params,omitempty"` // indexing or query params
}

EmbeddingRequest represents an embedding API call.

type EmbeddingResponse

type EmbeddingResponse struct {
	Embeddings [][]float32 `json:"embeddings"`
	Model      string      `json:"model"`
	Usage      *EyrieUsage `json:"usage,omitempty"`
}

EmbeddingResponse holds embedding results.

type ExtractOptions added in v0.1.1

type ExtractOptions struct {
	// Chat carries provider/model/temperature for the extraction call. If Model
	// is empty the provider default is used.
	Chat ChatOptions
	// Instruction overrides the default extraction instruction. Use it to scope
	// what relations to extract (e.g. "extract only code dependency relations").
	// When empty, a general noun-constrained instruction is used.
	Instruction string
	// AllowedPredicates, when non-empty, constrains the predicate vocabulary —
	// the model is told to use only these relation types, and triples with other
	// predicates are dropped after extraction. This is the lightweight analogue
	// of CocoIndex's EntityTypeConfig schema constraint.
	AllowedPredicates []string
	// MaxRetries is the schema-validation retry budget. Defaults to 2.
	MaxRetries int
}

ExtractOptions configures triple extraction. The zero value is valid and uses sensible defaults (noun-constrained entities, 2 validation retries).

type EyrieClient

type EyrieClient struct {
	// contains filtered or unexported fields
}

EyrieClient is the universal LLM client. It is safe for concurrent use.

func Client added in v0.1.1

func Client(cfg *EyrieConfig, opts ...ClientOption) *EyrieClient

Client creates an EyrieClient.

func (*EyrieClient) Chat

func (c *EyrieClient) Chat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*EyrieResponse, error)

Chat sends a chat request to the specified (or default) provider.

func (*EyrieClient) ChatWithStructuredOutput added in v0.1.1

func (c *EyrieClient) ChatWithStructuredOutput(ctx context.Context, messages []EyrieMessage, opts ChatOptions, validation SchemaValidation) (*EyrieResponse, error)

ChatWithStructuredOutput sends a chat request with structured output validation. If the response doesn't match the schema, it retries with error feedback.

func (*EyrieClient) CreateEmbedding added in v0.1.1

func (c *EyrieClient) CreateEmbedding(ctx context.Context, req EmbeddingRequest, provider string) (*EmbeddingResponse, error)

CreateEmbedding sends an embedding request to the specified (or default) provider.

func (*EyrieClient) ExtractRelationships added in v0.1.1

func (c *EyrieClient) ExtractRelationships(ctx context.Context, text string, opts ExtractOptions) ([]Relationship, error)

ExtractRelationships extracts subject-predicate-object triples from text using schema-validated structured output with retry. It is a typed convenience layer over ChatWithStructuredOutput, modeled on CocoIndex's ExtractByLlm; yaad and other knowledge-graph consumers can call it instead of hand-rolling extraction prompts and JSON parsing.

func (*EyrieClient) GetProviderInfo

func (c *EyrieClient) GetProviderInfo(provider string) *ProviderRegistryConfig

GetProviderInfo returns config for a provider.

func (*EyrieClient) GetProviders

func (c *EyrieClient) GetProviders() []string

GetProviders lists all available providers.

func (*EyrieClient) Ping

func (c *EyrieClient) Ping(ctx context.Context, provider string) error

Ping checks connectivity to the specified (or default) provider.

func (*EyrieClient) SetAPIKey

func (c *EyrieClient) SetAPIKey(provider, apiKey string)

SetAPIKey sets an API key for a provider.

func (*EyrieClient) StreamChat

func (c *EyrieClient) StreamChat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*StreamResult, error)

StreamChat sends a streaming chat request.

func (*EyrieClient) StreamChatContinue

func (c *EyrieClient) StreamChatContinue(ctx context.Context, messages []EyrieMessage, opts ChatOptions, cfg ContinuationConfig) (*StreamResult, error)

StreamChatContinue is like StreamChat but automatically continues if the response hits max_tokens with text-only content. Continuations are transparent to the caller.

type EyrieConfig

type EyrieConfig struct {
	Provider   string `json:"provider,omitempty"`
	APIKey     string `json:"-"`
	BaseURL    string `json:"base_url,omitempty"`
	Model      string `json:"model,omitempty"`
	MaxRetries int    `json:"max_retries,omitempty"`
}

EyrieConfig holds client configuration.

type EyrieError

type EyrieError struct {
	Provider   string
	Op         string // operation that failed (e.g. "chat", "stream", "ping")
	StatusCode int
	RequestID  string
	Message    string
	Err        error
}

EyrieError is a structured error that preserves provider context, HTTP metadata, and request identification for debugging.

func (*EyrieError) Error

func (e *EyrieError) Error() string

func (*EyrieError) IsAuthError

func (e *EyrieError) IsAuthError() bool

IsAuthError returns true if the error indicates an authentication/authorization problem.

func (*EyrieError) IsRateLimited

func (e *EyrieError) IsRateLimited() bool

IsRateLimited returns true if the error indicates rate limiting.

func (*EyrieError) IsRetriable

func (e *EyrieError) IsRetriable() bool

IsRetriable returns true if the error is likely transient and retrying may help.

func (*EyrieError) Unwrap

func (e *EyrieError) Unwrap() error

type EyrieMessage

type EyrieMessage struct {
	Role         string        `json:"role"`
	Content      string        `json:"content,omitempty"`
	Thinking     string        `json:"thinking,omitempty"`      // chain-of-thought captured from a prior response (never forwarded to providers that reject it)
	ContentParts []ContentPart `json:"content_parts,omitempty"` // multi-modal content (takes precedence over Content/Images)
	Images       []string      `json:"images,omitempty"`
	ToolUse      []ToolCall    `json:"tool_use,omitempty"`     // assistant message with tool calls
	ToolResults  []ToolResult  `json:"tool_results,omitempty"` // user message with one or more tool results
}

EyrieMessage represents a chat message. For simple text messages, set Content directly. For multi-modal messages (images, audio), use ContentParts. When ContentParts is non-empty, it takes precedence over Content and Images. The Images field is retained for backward compatibility.

func BuildStructuredPrompt added in v0.1.1

func BuildStructuredPrompt(messages []EyrieMessage, schema map[string]interface{}) []EyrieMessage

BuildStructuredPrompt adds JSON schema instructions to the message system prompt. It prepends schema requirements to ensure the LLM outputs valid JSON matching the schema.

func MergeConsecutiveRoles

func MergeConsecutiveRoles(messages []EyrieMessage) []EyrieMessage

MergeConsecutiveRoles merges adjacent messages that share the same role by concatenating their content with a newline separator.

Messages with ToolUse or ToolResult are never merged, since those have special provider semantics and must remain separate.

func NewAudioMessage added in v0.1.1

func NewAudioMessage(base64Data, format string) EyrieMessage

NewAudioMessage creates a user message with base64-encoded audio. format should be "wav" or "mp3".

func NewAudioMessageWithText added in v0.1.1

func NewAudioMessageWithText(text, base64Data, format string) EyrieMessage

NewAudioMessageWithText creates a user message with text and base64-encoded audio.

func NewBase64ImageMessage added in v0.1.1

func NewBase64ImageMessage(data, mediaType string) EyrieMessage

NewBase64ImageMessage creates a user message with a base64-encoded image. mediaType should be a MIME type like "image/png" or "image/jpeg".

func NewBase64ImageMessageWithText added in v0.1.1

func NewBase64ImageMessageWithText(text, data, mediaType string) EyrieMessage

NewBase64ImageMessageWithText creates a user message with text and a base64-encoded image.

func NewImageMessage added in v0.1.1

func NewImageMessage(url string) EyrieMessage

NewImageMessage creates a user message with an image from a URL or data URI. The url parameter accepts HTTP(S) URLs or data URIs (data:image/png;base64,...).

func NewImageMessageWithText added in v0.1.1

func NewImageMessageWithText(text, url string) EyrieMessage

NewImageMessageWithText creates a user message with text and an image from a URL or data URI.

func SanitizeMessages

func SanitizeMessages(messages []EyrieMessage) []EyrieMessage

SanitizeMessages inspects messages for orphaned tool_use blocks (assistant messages with tool calls that lack matching tool_result blocks) and injects synthetic error results to prevent 400 errors from providers. This is critical for session resume and compaction scenarios.

type EyrieResponse

type EyrieResponse struct {
	Content        string      `json:"content"`
	Thinking       string      `json:"thinking,omitempty"`
	Usage          *EyrieUsage `json:"usage,omitempty"`
	ToolCalls      []ToolCall  `json:"tool_calls,omitempty"`
	FinishReason   string      `json:"finish_reason"`
	RequestID      string      `json:"request_id,omitempty"`
	OrganizationID string      `json:"organization_id,omitempty"`
}

EyrieResponse is the response from a chat call.

func ChatWithContinuation

func ChatWithContinuation(ctx context.Context, p Provider, messages []EyrieMessage, opts ChatOptions, cfg ContinuationConfig) (*EyrieResponse, error)

ChatWithContinuation calls Chat and automatically continues if stop_reason is "max_tokens". It appends the partial response as an assistant message and retries, accumulating content. Returns the fully assembled response.

type EyrieStreamEvent

type EyrieStreamEvent struct {
	Type       string      `json:"type"` // content, tool_call, tool_input_delta, thinking, done, error, ttft
	Content    string      `json:"content,omitempty"`
	ToolCall   *ToolCall   `json:"tool_call,omitempty"`
	Thinking   string      `json:"thinking,omitempty"`
	Error      string      `json:"error,omitempty"`
	RequestID  string      `json:"request_id,omitempty"`
	Usage      *EyrieUsage `json:"usage,omitempty"`
	StopReason string      `json:"stop_reason,omitempty"`
	TTFTms     int         `json:"ttft_ms,omitempty"` // time-to-first-token milliseconds, set on "done" event
	// TTFT carries time-to-first-token milliseconds on Type=="ttft" events.
	// It is emitted as a dedicated event immediately before the first content
	// or tool-call delta so consumers can measure latency without waiting for
	// the stream to finish.
	TTFT int `json:"ttft,omitempty"`
}

EyrieStreamEvent is a streaming event.

type EyrieTool

type EyrieTool struct {
	Name        string                 `json:"name"`
	Description string                 `json:"description"`
	Parameters  map[string]interface{} `json:"parameters"`
}

EyrieTool represents a tool definition.

type EyrieUsage

type EyrieUsage struct {
	PromptTokens        int `json:"prompt_tokens"`
	CompletionTokens    int `json:"completion_tokens"`
	TotalTokens         int `json:"total_tokens"`
	CacheCreationTokens int `json:"cache_creation_tokens,omitempty"`
	CacheReadTokens     int `json:"cache_read_tokens,omitempty"`
	ThinkingTokens      int `json:"thinking_tokens,omitempty"`
}

EyrieUsage tracks token usage.

type FallbackProvider

type FallbackProvider struct {
	// contains filtered or unexported fields
}

FallbackProvider wraps multiple Providers and automatically falls back to the next one when the current provider returns a retriable error (429, 500, 502, 503, timeout). It does NOT fall back on client errors (400, 401, 403) because those indicate a problem with the request itself, not the provider.

Inspired by BerriAI/litellm's fallback chain feature.

FallbackProvider is safe for concurrent use.

func NewFallbackProvider

func NewFallbackProvider(providers ...Provider) *FallbackProvider

NewFallbackProvider creates a FallbackProvider that tries providers in order. At least one provider must be supplied.

func (*FallbackProvider) Chat

func (fp *FallbackProvider) Chat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*EyrieResponse, error)

Chat sends a non-streaming chat request, falling back through the provider chain on retriable errors. Returns the first successful response.

func (*FallbackProvider) Name

func (fp *FallbackProvider) Name() string

Name returns a composite name listing all providers in the chain.

func (*FallbackProvider) Ping

func (fp *FallbackProvider) Ping(ctx context.Context) error

Ping tries to ping each provider in order, returning nil on the first success.

func (*FallbackProvider) SetLogger

func (fp *FallbackProvider) SetLogger(l *slog.Logger)

SetLogger sets a custom logger for the FallbackProvider.

func (*FallbackProvider) Stats

func (fp *FallbackProvider) Stats() map[string]int64

Stats returns a snapshot of how many times each provider served a request.

func (*FallbackProvider) StreamChat

func (fp *FallbackProvider) StreamChat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*StreamResult, error)

StreamChat sends a streaming chat request, falling back through the provider chain on retriable errors. Returns the first successful stream.

type FeatureSet added in v0.1.1

type FeatureSet struct {
	Thinking         bool `json:"thinking"`
	AdaptiveThinking bool `json:"adaptive_thinking"`
	ToolUse          bool `json:"tool_use"`
	Images           bool `json:"images"`
	Streaming        bool `json:"streaming"`
	Caching          bool `json:"caching"`
	JSON             bool `json:"json_mode"`
	Embeddings       bool `json:"embeddings"`
	MaxContext       int  `json:"max_context"`
	MaxOutput        int  `json:"max_output"`
	Effort           bool `json:"effort"`
	StructuredOutput bool `json:"structured_output"`
	CodeExecution    bool `json:"code_execution"`
	Citations        bool `json:"citations"`
	PDFInput         bool `json:"pdf_input"`
}

FeatureSet describes what a provider or model supports. When the catalog is loaded, per-model values from the live API take precedence over the hardcoded defaults.

type GeminiClient added in v0.1.1

type GeminiClient struct {
	// contains filtered or unexported fields
}

GeminiClient implements Provider for the Google Gemini API.

func NewGeminiClient added in v0.1.1

func NewGeminiClient(apiKey, baseURL string) *GeminiClient

func (*GeminiClient) Chat added in v0.1.1

func (c *GeminiClient) Chat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*EyrieResponse, error)

func (*GeminiClient) Name added in v0.1.1

func (c *GeminiClient) Name() string

func (*GeminiClient) Ping added in v0.1.1

func (c *GeminiClient) Ping(ctx context.Context) error

func (*GeminiClient) StreamChat added in v0.1.1

func (c *GeminiClient) StreamChat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*StreamResult, error)

type GuardrailAction added in v0.1.1

type GuardrailAction string

GuardrailAction determines what happens when a rule matches.

const (
	// GuardrailBlock prevents the response from being returned to the caller.
	GuardrailBlock GuardrailAction = "block"
	// GuardrailRedact replaces the matched content with a redaction marker.
	GuardrailRedact GuardrailAction = "redact"
	// GuardrailWarn allows the response but records the violation.
	GuardrailWarn GuardrailAction = "warn"
)

type GuardrailError added in v0.1.1

type GuardrailError struct {
	Violations []GuardrailViolation `json:"violations"`
	Message    string               `json:"message"`
}

GuardrailError is returned when a guardrail blocks a response.

func (*GuardrailError) Error added in v0.1.1

func (e *GuardrailError) Error() string

type GuardrailProvider added in v0.1.1

type GuardrailProvider struct {
	// contains filtered or unexported fields
}

GuardrailProvider wraps any Provider and runs guardrail checks on LLM responses before returning them to the caller. When a guardrail with Action=Block matches, the response is replaced with an error. When Action=Redact, matched content is scrubbed. When Action=Warn, the violation is logged but the response passes through unchanged.

GuardrailProvider is safe for concurrent use.

func NewGuardrailProvider added in v0.1.1

func NewGuardrailProvider(inner Provider, g *Guardrails) *GuardrailProvider

NewGuardrailProvider wraps the given provider with output guardrails. The inner provider must not be nil. The guardrails parameter may be nil (in which case the wrapper is a no-op).

func (*GuardrailProvider) Chat added in v0.1.1

func (gp *GuardrailProvider) Chat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*EyrieResponse, error)

Chat sends a chat request and validates the response against guardrails.

func (*GuardrailProvider) Inner added in v0.1.1

func (gp *GuardrailProvider) Inner() Provider

Inner returns the wrapped provider.

func (*GuardrailProvider) Name added in v0.1.1

func (gp *GuardrailProvider) Name() string

Name returns the inner provider name suffixed with "/guardrails".

func (*GuardrailProvider) Ping added in v0.1.1

func (gp *GuardrailProvider) Ping(ctx context.Context) error

Ping delegates to the inner provider.

func (*GuardrailProvider) StreamChat added in v0.1.1

func (gp *GuardrailProvider) StreamChat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*StreamResult, error)

StreamChat sends a streaming request and validates content events. Blocked violations cause the stream to be cancelled and an error event emitted. Redactions are applied to individual content chunks.

type GuardrailRule added in v0.1.1

type GuardrailRule struct {
	Type     GuardrailType     `json:"type"`
	Name     string            `json:"name"`
	Pattern  string            `json:"pattern"`
	Action   GuardrailAction   `json:"action"`
	Severity GuardrailSeverity `json:"severity"`
	// contains filtered or unexported fields
}

GuardrailRule defines a single guardrail check.

func AllDefaultRules added in v0.1.1

func AllDefaultRules() []GuardrailRule

AllDefaultRules returns all built-in guardrail rules (PII, secrets, prompt injection, and harmful content).

func DefaultHarmfulContentRules added in v0.1.1

func DefaultHarmfulContentRules() []GuardrailRule

DefaultHarmfulContentRules returns built-in rules for detecting harmful content patterns.

func DefaultPIIRules added in v0.1.1

func DefaultPIIRules() []GuardrailRule

DefaultPIIRules returns built-in rules for detecting PII in responses.

func DefaultPromptInjectionRules added in v0.1.1

func DefaultPromptInjectionRules() []GuardrailRule

DefaultPromptInjectionRules returns built-in rules for detecting prompt injection in responses.

func DefaultSecretLeakRules added in v0.1.1

func DefaultSecretLeakRules() []GuardrailRule

DefaultSecretLeakRules returns built-in rules for detecting leaked secrets.

func RulesForType added in v0.1.1

func RulesForType(t GuardrailType) []GuardrailRule

RulesForType returns the built-in rules for a single GuardrailType.

type GuardrailSeverity added in v0.1.1

type GuardrailSeverity string

GuardrailSeverity indicates how critical a violation is.

const (
	SeverityLow      GuardrailSeverity = "low"
	SeverityMedium   GuardrailSeverity = "medium"
	SeverityHigh     GuardrailSeverity = "high"
	SeverityCritical GuardrailSeverity = "critical"
)

type GuardrailType added in v0.1.1

type GuardrailType string

GuardrailType classifies a guardrail rule.

const (
	// GuardrailPII detects personally identifiable information.
	GuardrailPII GuardrailType = "pii"
	// GuardrailPromptInjection detects prompt injection attempts in responses.
	GuardrailPromptInjection GuardrailType = "prompt_injection"
	// GuardrailHarmfulContent detects harmful or dangerous content patterns.
	GuardrailHarmfulContent GuardrailType = "harmful_content"
	// GuardrailSecretLeak detects leaked secrets, API keys, tokens, and passwords.
	GuardrailSecretLeak GuardrailType = "secret_leak"
	// GuardrailCustom is a user-defined rule with a custom pattern.
	GuardrailCustom GuardrailType = "custom"
)

type GuardrailViolation added in v0.1.1

type GuardrailViolation struct {
	Rule           GuardrailRule `json:"rule"`
	MatchedText    string        `json:"matched_text"`
	RedactedResult string        `json:"redacted_result,omitempty"`
	// contains filtered or unexported fields
}

GuardrailViolation records a single rule match in the response.

type Guardrails added in v0.1.1

type Guardrails struct {
	// contains filtered or unexported fields
}

Guardrails holds registered rules and runs them against LLM responses.

func NewGuardrails added in v0.1.1

func NewGuardrails(rules ...GuardrailRule) *Guardrails

NewGuardrails creates a Guardrails instance with the given rules.

func NewGuardrailsSafe added in v0.1.1

func NewGuardrailsSafe(rules ...GuardrailRule) (*Guardrails, error)

NewGuardrailsSafe creates a Guardrails instance and returns an error if any rule has an invalid pattern. Use this when rules may come from untrusted sources; use NewGuardrails for programmatic rules where invalid patterns indicate a programmer error (matching regexp.MustCompile convention).

func (*Guardrails) AddRule added in v0.1.1

func (g *Guardrails) AddRule(r GuardrailRule)

AddRule registers a guardrail rule. It panics if the pattern is invalid. This follows the regexp.MustCompile convention for programmatic rules where an invalid pattern indicates a programmer error. For rules that may originate from untrusted sources (config files, user input), use AddRuleSafe instead.

func (*Guardrails) AddRuleSafe added in v0.1.1

func (g *Guardrails) AddRuleSafe(r GuardrailRule) error

AddRuleSafe registers a guardrail rule and returns an error if the pattern is invalid, instead of panicking. Use this when rules may come from untrusted sources (config files, user input).

func (*Guardrails) Check added in v0.1.1

func (g *Guardrails) Check(ctx context.Context, response string) ([]GuardrailViolation, error)

Check evaluates all rules against the response text. It returns violations and an error only if a rule with Action=Block matches. For Redact actions, the redacted result is populated in the violation. For Warn actions, the violation is recorded but no error is returned.

func (*Guardrails) Rules added in v0.1.1

func (g *Guardrails) Rules() []GuardrailRule

Rules returns a snapshot of the currently registered rules.

type HeaderExtractor added in v0.1.1

type HeaderExtractor func(h http.Header) *RateLimitHeaders

HeaderExtractor is a function that extracts rate limit information from HTTP response headers. Different providers use different header naming conventions (OpenAI uses x-ratelimit-*, Anthropic uses anthropic-ratelimit-*).

type ImageURLPart added in v0.1.1

type ImageURLPart struct {
	URL    string `json:"url"`
	Detail string `json:"detail,omitempty"` // "auto", "low", "high" (OpenAI-specific)
}

ImageURLPart represents an image content part. URL can be an HTTP(S) URL or a data URI (data:image/png;base64,...).

type InflightRequest added in v0.1.1

type InflightRequest struct {
	// contains filtered or unexported fields
}

InflightRequest represents a pending request that multiple waiters can join. The first goroutine to create this request is responsible for executing it and broadcasting the result to all waiting goroutines.

type InflightStats added in v0.1.1

type InflightStats struct {
	// InflightRequests is the number of unique requests being executed
	InflightRequests int
	// TotalWaiters is the total number of goroutines waiting across all requests
	TotalWaiters int
}

InflightStats contains statistics about inflight coalesced requests.

type InputAudioPart added in v0.1.1

type InputAudioPart struct {
	Data   string `json:"data"`   // base64 encoded audio
	Format string `json:"format"` // "wav", "mp3" (OpenAI) or used to derive media_type (Anthropic)
}

InputAudioPart represents an audio content part (base64 encoded).

type Interaction added in v0.1.1

type Interaction struct {
	Request  RecordedRequest  `json:"request"`
	Response RecordedResponse `json:"response"`
}

Interaction pairs a recorded request with its response.

type LLMSummarizingCondenser added in v0.1.1

type LLMSummarizingCondenser struct {
	// contains filtered or unexported fields
}

LLMSummarizingCondenser condenses a conversation by summarizing its middle span via an LLM call, keeping the first KeepFirst messages and the tail intact. The summary is inserted as a single system note between the head and the tail.

The summary call uses the Weak model role when a ModelRoles is configured (see WithModelRoles / ResolveRole), so summarization runs on a cheaper model.

func NewLLMSummarizingCondenser added in v0.1.1

func NewLLMSummarizingCondenser(provider Provider, opts ...CondenserOption) *LLMSummarizingCondenser

NewLLMSummarizingCondenser creates a condenser that summarizes via the given provider. The provider must not be nil.

func (*LLMSummarizingCondenser) Condense added in v0.1.1

func (c *LLMSummarizingCondenser) Condense(ctx context.Context, messages []EyrieMessage, opts CondenseOptions) ([]EyrieMessage, error)

Condense implements ConversationCondenser. When len(messages) exceeds MaxSize, it keeps the first KeepFirst messages, summarizes the middle span, inserts the summary as a system note, and keeps the remaining tail.

type LazyProvider added in v0.1.3

type LazyProvider struct {
	// contains filtered or unexported fields
}

LazyProvider adapts EyrieClient to the Provider interface without eagerly resolving credentials or constructing a concrete provider.

func NewLazyProvider added in v0.1.3

func NewLazyProvider(cfg *EyrieConfig) *LazyProvider

NewLazyProvider creates a provider wrapper that resolves the concrete provider only when chat or ping operations are invoked.

func (*LazyProvider) Chat added in v0.1.3

func (p *LazyProvider) Chat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*EyrieResponse, error)

func (*LazyProvider) Name added in v0.1.3

func (p *LazyProvider) Name() string

func (*LazyProvider) Ping added in v0.1.3

func (p *LazyProvider) Ping(ctx context.Context) error

func (*LazyProvider) SetAPIKey added in v0.1.3

func (p *LazyProvider) SetAPIKey(provider, apiKey string)

func (*LazyProvider) StreamChat added in v0.1.3

func (p *LazyProvider) StreamChat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*StreamResult, error)

type MemoryBudgetStore added in v0.1.1

type MemoryBudgetStore struct {
	// contains filtered or unexported fields
}

MemoryBudgetStore is an in-memory BudgetStore suitable for tests and single-process use. It is safe for concurrent use.

func NewMemoryBudgetStore added in v0.1.1

func NewMemoryBudgetStore() *MemoryBudgetStore

NewMemoryBudgetStore creates an empty in-memory budget store.

func (*MemoryBudgetStore) CheckBudget added in v0.1.1

func (s *MemoryBudgetStore) CheckBudget(_ context.Context, virtualKey string, estCostUSD float64) error

CheckBudget implements BudgetStore.

func (*MemoryBudgetStore) RecordUsage added in v0.1.1

func (s *MemoryBudgetStore) RecordUsage(_ context.Context, virtualKey string, costUSD float64, tokensIn, tokensOut int) error

RecordUsage implements BudgetStore.

func (*MemoryBudgetStore) SetBudget added in v0.1.1

func (s *MemoryBudgetStore) SetBudget(virtualKey string, limitUSD float64)

SetBudget creates or updates a virtual key with the given USD limit. A non-positive limit means unlimited.

func (*MemoryBudgetStore) Usage added in v0.1.1

func (s *MemoryBudgetStore) Usage(virtualKey string) (usedUSD float64, tokensIn, tokensOut int, ok bool)

Usage returns the recorded spend for a virtual key.

type MetricsCollector

type MetricsCollector struct {
	// contains filtered or unexported fields
}

MetricsCollector stores recent call metrics in a ring buffer.

func NewMetricsCollector

func NewMetricsCollector() *MetricsCollector

NewMetricsCollector creates a new MetricsCollector.

func (*MetricsCollector) Recent

func (mc *MetricsCollector) Recent(n int) []CallMetrics

Recent returns the last n call metrics, most recent first. If fewer than n entries exist, all available entries are returned.

func (*MetricsCollector) Record

func (mc *MetricsCollector) Record(m CallMetrics)

Record adds a new CallMetrics entry to the ring buffer.

func (*MetricsCollector) TotalCost

func (mc *MetricsCollector) TotalCost() float64

TotalCost estimates the total cost across all recorded metrics using a simplified pricing model (per 1M tokens):

  • Input tokens: $3.00 / 1M
  • Output tokens: $15.00 / 1M
  • Cache read tokens: $0.30 / 1M
  • Cache creation tokens: $3.75 / 1M

type MiMoClient added in v0.1.1

type MiMoClient struct {
	// contains filtered or unexported fields
}

MiMoClient uses OpenAI-compatible MiMo endpoints first, with optional Anthropic-compat fallback.

func NewMiMoClient added in v0.1.1

func NewMiMoClient(apiKey, openAIBase, anthropicBase string, compat *OpenAICompatConfig, providerID string, opts ...ClientOption) *MiMoClient

NewMiMoClient builds a MiMo provider client (payg or token_plan gateway).

func (*MiMoClient) Chat added in v0.1.1

func (c *MiMoClient) Chat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*EyrieResponse, error)

func (*MiMoClient) Name added in v0.1.1

func (c *MiMoClient) Name() string

func (*MiMoClient) Ping added in v0.1.1

func (c *MiMoClient) Ping(ctx context.Context) error

func (*MiMoClient) StreamChat added in v0.1.1

func (c *MiMoClient) StreamChat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*StreamResult, error)

type MockCall

type MockCall struct {
	Messages []EyrieMessage
	Options  ChatOptions
}

MockCall records a single call to the mock provider.

type MockMode

type MockMode string

MockMode controls how the mock provider responds.

const (
	// MockModeEcho echoes the last user message back.
	MockModeEcho MockMode = "echo"
	// MockModeFixed returns a fixed response set via MockProvider.Response.
	MockModeFixed MockMode = "fixed"
	// MockModeToolUse returns a tool call response.
	MockModeToolUse MockMode = "tool_use"
	// MockModeError always returns an error.
	MockModeError MockMode = "error"
	// MockModeMaxTokens returns a response with stop_reason=max_tokens (for testing continuation).
	MockModeMaxTokens MockMode = "max_tokens"
)

type MockProvider

type MockProvider struct {
	Mode     MockMode
	Response string // used in MockModeFixed
	ToolName string // used in MockModeToolUse
	ToolArgs map[string]interface{}
	Delay    time.Duration // simulate latency
	Calls    []MockCall    // recorded calls for assertions
	// contains filtered or unexported fields
}

MockProvider is a Provider implementation for testing. It never makes real HTTP requests.

func NewMockProvider

func NewMockProvider(mode MockMode) *MockProvider

NewMockProvider creates a mock provider with the given mode.

func (*MockProvider) CallCount

func (m *MockProvider) CallCount() int

CallCount returns the number of recorded calls.

func (*MockProvider) Chat

func (m *MockProvider) Chat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*EyrieResponse, error)

Chat returns a mock response based on Mode.

func (*MockProvider) LastCall

func (m *MockProvider) LastCall() *MockCall

LastCall returns the most recent recorded call, or nil.

func (*MockProvider) MarshalCalls

func (m *MockProvider) MarshalCalls() string

MarshalCalls returns recorded calls as JSON for debugging.

func (*MockProvider) Name

func (m *MockProvider) Name() string

Name returns "mock".

func (*MockProvider) Ping

func (m *MockProvider) Ping(_ context.Context) error

Ping always succeeds.

func (*MockProvider) Reset

func (m *MockProvider) Reset()

Reset clears recorded calls.

func (*MockProvider) StreamChat

func (m *MockProvider) StreamChat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*StreamResult, error)

StreamChat streams a mock response word by word.

type ModelRoles added in v0.1.1

type ModelRoles struct {
	Primary string `json:"primary,omitempty"`
	Weak    string `json:"weak,omitempty"`
	Editor  string `json:"editor,omitempty"`
}

ModelRoles maps named role slots to concrete model ids. Empty fields fall back to Primary via ResolveRole.

type ModerationOption added in v0.1.1

type ModerationOption func(*ModerationProvider)

ModerationOption configures a ModerationProvider.

func WithBlockedPatterns added in v0.1.1

func WithBlockedPatterns(patterns []string) ModerationOption

WithBlockedPatterns sets regex patterns that will block matching input. Each pattern is compiled as a Go regexp.

func WithCustomChecker added in v0.1.1

func WithCustomChecker(fn func(string) error) ModerationOption

WithCustomChecker sets a custom validation function that receives the concatenated text content of all messages. If the function returns a non-nil error, the request is blocked.

func WithModerationMaxTokens added in v0.1.1

func WithModerationMaxTokens(n int) ModerationOption

WithModerationMaxTokens sets the maximum allowed total token count across all input messages. Token count is estimated as len(strings.Fields(text)) for simplicity. A value of 0 (default) means no limit.

type ModerationProvider added in v0.1.1

type ModerationProvider struct {
	// contains filtered or unexported fields
}

ModerationProvider wraps any Provider and validates input messages before forwarding requests. It checks for blocked patterns (regex), token count limits, and custom safety checks.

ModerationProvider is safe for concurrent use.

func NewModerationProvider added in v0.1.1

func NewModerationProvider(inner Provider, opts ...ModerationOption) *ModerationProvider

NewModerationProvider wraps inner with content moderation. At least one moderation option should be provided or the wrapper is a no-op.

func (*ModerationProvider) Chat added in v0.1.1

func (mp *ModerationProvider) Chat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*EyrieResponse, error)

Chat validates the input messages and forwards to the inner provider.

func (*ModerationProvider) Name added in v0.1.1

func (mp *ModerationProvider) Name() string

Name returns the inner provider name suffixed with "/moderation".

func (*ModerationProvider) Ping added in v0.1.1

func (mp *ModerationProvider) Ping(ctx context.Context) error

Ping delegates to the inner provider.

func (*ModerationProvider) StreamChat added in v0.1.1

func (mp *ModerationProvider) StreamChat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*StreamResult, error)

StreamChat validates the input messages and forwards to the inner provider.

type OpenAIClient

type OpenAIClient struct {
	// contains filtered or unexported fields
}

OpenAIClient implements Provider for OpenAI and OpenAI-compatible APIs.

func NewOpenAIClient

func NewOpenAIClient(apiKey, baseURL string, compat *OpenAICompatConfig, opts ...ClientOption) *OpenAIClient

NewOpenAIClient creates a configured OpenAI/compatible client.

func (*OpenAIClient) Chat

func (c *OpenAIClient) Chat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*EyrieResponse, error)

Chat sends a non-streaming request.

func (*OpenAIClient) CreateEmbedding added in v0.1.1

func (c *OpenAIClient) CreateEmbedding(ctx context.Context, req EmbeddingRequest) (*EmbeddingResponse, error)

CreateEmbedding sends an embedding request to the OpenAI-compatible API endpoint.

func (*OpenAIClient) Name

func (c *OpenAIClient) Name() string

Name returns the provider name.

func (*OpenAIClient) Ping

func (c *OpenAIClient) Ping(ctx context.Context) error

Ping checks connectivity.

func (*OpenAIClient) StreamChat

func (c *OpenAIClient) StreamChat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*StreamResult, error)

StreamChat sends a streaming request.

type OpenAICompatConfig

type OpenAICompatConfig struct {
	SupportsStore                    bool   `json:"supports_store,omitempty"`
	SupportsDeveloperRole            bool   `json:"supports_developer_role,omitempty"`
	SupportsReasoningEffort          bool   `json:"supports_reasoning_effort,omitempty"`
	SupportsUsageInStreaming         bool   `json:"supports_usage_in_streaming,omitempty"`
	SupportsStrictMode               bool   `json:"supports_strict_mode,omitempty"`
	MaxTokensField                   string `json:"max_tokens_field,omitempty"` // "max_tokens" or "max_completion_tokens"
	RequiresToolResultName           bool   `json:"requires_tool_result_name,omitempty"`
	RequiresAssistantAfterToolResult bool   `json:"requires_assistant_after_tool_result,omitempty"`
	RequiresThinkingAsText           bool   `json:"requires_thinking_as_text,omitempty"`
	ThinkingFormat                   string `json:"thinking_format,omitempty"` // "openai", "zai", "qwen", "openrouter"
	// StripReasoningFromInput instructs buildRequestBase to omit the reasoning_content
	// field from assistant messages. DeepSeek (and compatible providers) return HTTP 400
	// if reasoning_content appears in the input context of a multi-turn conversation.
	StripReasoningFromInput bool `json:"strip_reasoning_from_input,omitempty"`
	// SupportsCacheRole enables Kimi/Moonshot context-cache injection: when
	// ChatOptions.KimiContextCacheID is non-empty, buildRequestBase prepends a
	// {"role":"cache","content":<id>} message per the MoonshotAI-Cookbook spec.
	SupportsCacheRole bool `json:"supports_cache_role,omitempty"`
}

OpenAICompatConfig holds provider-specific compatibility flags that control how API requests are constructed for each provider.

type OpenCodeGoClient added in v0.1.1

type OpenCodeGoClient struct {
	// contains filtered or unexported fields
}

OpenCodeGoClient routes OpenCode Go models through OpenAIClient and AnthropicClient per opencode.ai/docs/go, with cross-protocol fallback when the primary path returns no answer text (e.g. reasoning-only MiniMax streams).

func NewOpenCodeGoClient added in v0.1.1

func NewOpenCodeGoClient(apiKey, baseURL string, opts ...ClientOption) *OpenCodeGoClient

NewOpenCodeGoClient builds an OpenCode Go provider client.

func (*OpenCodeGoClient) Chat added in v0.1.1

func (c *OpenCodeGoClient) Chat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*EyrieResponse, error)

func (*OpenCodeGoClient) Name added in v0.1.1

func (c *OpenCodeGoClient) Name() string

func (*OpenCodeGoClient) Ping added in v0.1.1

func (c *OpenCodeGoClient) Ping(ctx context.Context) error

func (*OpenCodeGoClient) StreamChat added in v0.1.1

func (c *OpenCodeGoClient) StreamChat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*StreamResult, error)

type PromptOptimizer added in v0.1.1

type PromptOptimizer struct {
	// contains filtered or unexported fields
}

PromptOptimizer compresses conversation history to reduce input tokens. Keeps the most recent messages intact but summarizes older ones.

func NewPromptOptimizer added in v0.1.1

func NewPromptOptimizer(maxInputTokens int) *PromptOptimizer

NewPromptOptimizer creates an optimizer with a token budget.

func (*PromptOptimizer) Optimize added in v0.1.1

func (po *PromptOptimizer) Optimize(messages []EyrieMessage) []EyrieMessage

Optimize compresses messages to fit within the token budget. Preserves: system message, first user message, last N messages. Summarizes: middle messages.

type ProtocolChatFallback added in v0.1.1

type ProtocolChatFallback func(primaryErr error, primaryResp *EyrieResponse) bool

ProtocolChatFallback decides whether to retry via the alternate protocol after the primary attempt. resp is non-nil only when primary returned err=nil.

type ProtocolRouter added in v0.1.1

type ProtocolRouter struct {
	OpenAI    *OpenAIClient
	Anthropic *AnthropicClient
}

ProtocolRouter picks between OpenAIClient and AnthropicClient for gateways that expose both APIs (OpenCode Go, MiMo). All HTTP work stays in openai.go and anthropic.go; this type only orchestrates routing and fallback.

func (ProtocolRouter) Chat added in v0.1.1

func (r ProtocolRouter) Chat(ctx context.Context, messages []EyrieMessage, opts ChatOptions, primary ChatProtocol, fallback ProtocolChatFallback) (*EyrieResponse, error)

Chat sends a request via the primary protocol, optionally falling back to the alternate protocol when fallback returns true.

func (ProtocolRouter) StreamChat added in v0.1.1

func (r ProtocolRouter) StreamChat(ctx context.Context, messages []EyrieMessage, opts ChatOptions, cfg ProtocolStreamConfig) (*StreamResult, error)

StreamChat streams via the primary protocol. When FallbackOnError matches, the alternate protocol is tried immediately. When ReasoningOnlyFallback is set and the primary is /v1/messages, an empty reasoning-only stream retries via chat/completions.

type ProtocolStreamConfig added in v0.1.1

type ProtocolStreamConfig struct {
	Primary               ChatProtocol
	FallbackOnError       func(error) bool
	ReasoningOnlyFallback bool // retry alternate protocol when primary stream is reasoning-only
}

ProtocolStreamConfig controls streaming across two protocols.

type Provider

type Provider interface {
	// Chat sends a non-streaming chat request.
	Chat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*EyrieResponse, error)
	// StreamChat sends a streaming chat request.
	// The caller must call Close() on the returned StreamResult when done.
	StreamChat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*StreamResult, error)
	// Ping checks connectivity and authentication.
	Ping(ctx context.Context) error
	// Name returns the provider name (e.g. "anthropic", "openai").
	Name() string
}

Provider is the core interface for LLM providers. Implementations must be safe for concurrent use.

func WithRateLimit

func WithRateLimit(p Provider, limiter *RateLimiter) Provider

WithRateLimit wraps a provider with a rate limiter.

type ProviderCallback added in v0.1.1

type ProviderCallback interface {
	// OnRequest is called before each Chat or StreamChat request.
	// The messages and opts parameters must not be modified.
	OnRequest(ctx context.Context, provider string, model string, messages []EyrieMessage, opts ChatOptions)

	// OnResponse is called after a successful Chat request.
	OnResponse(ctx context.Context, provider string, model string, response *EyrieResponse, duration time.Duration)

	// OnError is called after a Chat or StreamChat request fails.
	OnError(ctx context.Context, provider string, model string, err error, duration time.Duration)

	// OnStreamEvent is called for each event emitted during streaming.
	OnStreamEvent(ctx context.Context, provider string, model string, event EyrieStreamEvent)
}

ProviderCallback defines hooks that are invoked at various points during provider request lifecycle. All methods are optional — implement only the ones you need. Implementations MUST be safe for concurrent use.

type ProviderFeatures added in v0.1.1

type ProviderFeatures struct {
	// contains filtered or unexported fields
}

ProviderFeatures tracks which capabilities each provider supports. Prevents sending unsupported features (thinking, tools, images) to providers that don't handle them, avoiding cryptic API errors. Read-only after construction — no mutex needed.

func NewProviderFeatures added in v0.1.1

func NewProviderFeatures() *ProviderFeatures

NewProviderFeatures creates a feature registry. The catalog is the single source of truth for per-model capabilities. The hardcoded map is empty — all values come from the live API via the catalog.

func (*ProviderFeatures) Get added in v0.1.1

func (pf *ProviderFeatures) Get(provider string) FeatureSet

Get returns features for a provider or model. The compiled catalog (populated from the live API) is the single source of truth. Returns zero-value FeatureSet if the catalog is not loaded — caller must ensure the catalog is loaded before querying features.

func (*ProviderFeatures) Supports added in v0.1.1

func (pf *ProviderFeatures) Supports(provider, feature string) bool

Supports checks if a provider supports a specific feature.

type ProviderHealth added in v0.1.1

type ProviderHealth struct {
	// contains filtered or unexported fields
}

ProviderHealth tracks latency and error rates per provider. Enables intelligent routing: prefer the healthiest provider.

func NewProviderHealth added in v0.1.1

func NewProviderHealth() *ProviderHealth

NewProviderHealth creates a health tracker.

func (*ProviderHealth) AllScores added in v0.1.1

func (ph *ProviderHealth) AllScores() []ProviderScore

AllScores returns health scores for all tracked providers.

func (*ProviderHealth) Healthiest added in v0.1.1

func (ph *ProviderHealth) Healthiest(candidates []string) string

Healthiest returns the provider with the best health score.

func (*ProviderHealth) RecordFailure added in v0.1.1

func (ph *ProviderHealth) RecordFailure(provider string, latency time.Duration)

RecordFailure records a failed API call.

func (*ProviderHealth) RecordSuccess added in v0.1.1

func (ph *ProviderHealth) RecordSuccess(provider string, latency time.Duration)

RecordSuccess records a successful API call.

func (*ProviderHealth) Score added in v0.1.1

func (ph *ProviderHealth) Score(provider string) float64

Score returns health score for a provider (0-1).

type ProviderRegistryConfig

type ProviderRegistryConfig struct {
	Name              string              `json:"name"`
	Type              ProviderType        `json:"type"`
	BaseURL           string              `json:"base_url,omitempty"`
	EnvKey            string              `json:"env_key"`
	SupportsStreaming bool                `json:"supports_streaming"`
	SupportsTools     bool                `json:"supports_tools"`
	SupportsReasoning bool                `json:"supports_reasoning"`
	Compat            *OpenAICompatConfig `json:"compat,omitempty"`
}

ProviderRegistryConfig holds provider registry info.

type ProviderScore added in v0.1.1

type ProviderScore struct {
	Name         string  `json:"name"`
	Score        float64 `json:"score"` // 0-1 (1 = perfect health)
	AvgLatencyMs int64   `json:"avg_latency_ms"`
	ErrorRate    float64 `json:"error_rate"`
	IsHealthy    bool    `json:"is_healthy"`
}

ProviderScore represents a provider's health score.

type ProviderType

type ProviderType string

ProviderType classifies providers.

const (
	// ProviderTypeAnthropic uses the Anthropic Messages API.
	ProviderTypeAnthropic ProviderType = "anthropic"
	// ProviderTypeOpenAI uses the OpenAI Chat Completions API.
	ProviderTypeOpenAI ProviderType = "openai"
	// ProviderTypeOpenAICompatible uses OpenAI-compatible APIs with custom base URLs.
	ProviderTypeOpenAICompatible ProviderType = "openai-compatible"
	// ProviderTypeAzure uses Azure OpenAI.
	ProviderTypeAzure ProviderType = "azure"
	// ProviderTypeBedrock uses AWS Bedrock.
	ProviderTypeBedrock ProviderType = "bedrock"
	// ProviderTypeVertex uses Google Vertex AI.
	ProviderTypeVertex ProviderType = "vertex"
)

type RateLimitConfig

type RateLimitConfig struct {
	// RequestsPerMinute is the maximum requests per minute (0 = unlimited).
	RequestsPerMinute int
	// BurstSize is the maximum burst above the steady rate (default = RequestsPerMinute/10, min 1).
	BurstSize int
	// MinInterval is the minimum time between requests (e.g., 5ms for cloud, 0 for local).
	MinInterval time.Duration
}

RateLimitConfig holds rate limit settings for a provider.

type RateLimitHeaders added in v0.1.1

type RateLimitHeaders struct {
	RequestsRemaining int
	RequestsLimit     int
	TokensRemaining   int
	TokensLimit       int
	ResetTime         time.Time
}

RateLimitHeaders contains rate limit information extracted from HTTP response headers. Different providers use different header names; common patterns include OpenAI's x-ratelimit-* and Anthropic's anthropic-ratelimit-* headers.

func CommonHeaderExtractor added in v0.1.1

func CommonHeaderExtractor(h http.Header) *RateLimitHeaders

CommonHeaderExtractor tries to parse rate limit headers from common LLM API providers (OpenAI, Anthropic, and compatible APIs).

type RateLimitState added in v0.1.1

type RateLimitState struct {
	// RPM tracking
	RPMUsed    int       // requests used in the current window
	RPMLimit   int       // maximum requests per minute (0 = unknown)
	RPMResetAt time.Time // when the RPM window resets

	// TPM tracking
	TPMUsed    int       // tokens used in the current window
	TPMLimit   int       // maximum tokens per minute (0 = unknown)
	TPMResetAt time.Time // when the TPM window resets

	// Header-derived remaining counts (from x-ratelimit-remaining)
	RPMRemaining int // -1 if unknown
	TPMRemaining int // -1 if unknown

	// Last updated timestamp
	LastUpdated time.Time

	// Total requests and tokens tracked (lifetime)
	TotalRequests int64
	TotalTokens   int64

	// Number of times the provider was delayed due to near-limit
	ThrottleCount int64
}

RateLimitState holds the current rate limit tracking state for a provider.

type RateLimitedProvider

type RateLimitedProvider struct {
	// contains filtered or unexported fields
}

RateLimitedProvider wraps a Provider with rate limiting.

func (*RateLimitedProvider) Chat

func (r *RateLimitedProvider) Chat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*EyrieResponse, error)

func (*RateLimitedProvider) Name

func (r *RateLimitedProvider) Name() string

func (*RateLimitedProvider) Ping

func (*RateLimitedProvider) StreamChat

func (r *RateLimitedProvider) StreamChat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*StreamResult, error)

type RateLimiter

type RateLimiter struct {
	// contains filtered or unexported fields
}

RateLimiter implements a token bucket rate limiter per provider. It limits the number of requests per second to avoid hitting provider rate limits.

func NewRateLimiter

func NewRateLimiter(defaults RateLimitConfig) *RateLimiter

NewRateLimiter creates a rate limiter with default config applied to all providers.

func (*RateLimiter) SetProviderLimit

func (rl *RateLimiter) SetProviderLimit(provider string, cfg RateLimitConfig)

SetProviderLimit sets a custom rate limit for a specific provider.

func (*RateLimiter) Wait

func (rl *RateLimiter) Wait(ctx context.Context, provider string) error

Wait blocks until a request token is available for the given provider. Returns immediately if no rate limit is configured.

type RecordedRequest added in v0.1.1

type RecordedRequest struct {
	Messages []EyrieMessage `json:"messages"`
	Model    string         `json:"model"`
	System   string         `json:"system,omitempty"`
	Hash     string         `json:"hash"`
}

RecordedRequest captures the essential fields of a chat request.

type RecordedResponse added in v0.1.1

type RecordedResponse struct {
	Content      string      `json:"content,omitempty"`
	ToolCalls    []ToolCall  `json:"tool_calls,omitempty"`
	Usage        *EyrieUsage `json:"usage,omitempty"`
	FinishReason string      `json:"finish_reason,omitempty"`
	Error        string      `json:"error,omitempty"`
}

RecordedResponse captures the response from a provider.

type RecorderMode added in v0.1.1

type RecorderMode string

RecorderMode controls whether the recorder records new interactions or replays existing ones.

const (
	// RecordModeRecord always records new interactions from the inner provider.
	RecordModeRecord RecorderMode = "record"
	// RecordModeReplay always replays from the cassette; never calls the inner provider.
	RecordModeReplay RecorderMode = "replay"
	// RecordModeAuto replays if the cassette file exists, otherwise records.
	RecordModeAuto RecorderMode = "auto"
)

type RecorderProvider added in v0.1.1

type RecorderProvider struct {
	// contains filtered or unexported fields
}

RecorderProvider wraps any Provider to record or replay LLM interactions. It implements the Provider interface and stores interactions in a Cassette.

func NewRecorderProvider added in v0.1.1

func NewRecorderProvider(inner Provider, cassettePath string, mode RecorderMode) (*RecorderProvider, error)

NewRecorderProvider creates a RecorderProvider wrapping inner. In auto mode, if the cassette file exists it loads and replays; otherwise it records. In record mode, a fresh cassette is created. In replay mode, the cassette must exist or an error is returned.

func (*RecorderProvider) Chat added in v0.1.1

func (r *RecorderProvider) Chat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*EyrieResponse, error)

Chat either records a new interaction or replays a stored one.

func (*RecorderProvider) Name added in v0.1.1

func (r *RecorderProvider) Name() string

Name returns the inner provider name suffixed with "/recorder".

func (*RecorderProvider) Ping added in v0.1.1

func (r *RecorderProvider) Ping(ctx context.Context) error

Ping delegates to the inner provider.

func (*RecorderProvider) Save added in v0.1.1

func (r *RecorderProvider) Save() error

Save writes the cassette to its file path.

func (*RecorderProvider) SetRedactor added in v0.1.1

func (r *RecorderProvider) SetRedactor(fn func(string) string)

SetRedactor sets a function that redacts sensitive content before recording.

func (*RecorderProvider) StreamChat added in v0.1.1

func (r *RecorderProvider) StreamChat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*StreamResult, error)

StreamChat either records a new streaming interaction or replays a stored one. In record mode, the real stream is drained and the accumulated response is saved, then a synthetic stream is created to return to the caller. In replay mode, a synthetic stream is created from the stored response.

type Relationship added in v0.1.1

type Relationship struct {
	// Subject is the entity the relation originates from. A noun/noun phrase.
	Subject string `json:"subject"`
	// Predicate is the typed relation (e.g. "depends_on", "authored_by").
	Predicate string `json:"predicate"`
	// Object is the entity the relation points to. A noun/noun phrase.
	Object string `json:"object"`
}

Relationship is a subject-predicate-object triple extracted from text. It is the unit of a knowledge graph: subject and object are entities (nouns), and predicate is the typed relation between them.

This mirrors the typed-extraction pattern popularized by CocoIndex's ExtractByLlm(output_type=list[Relationship]) — instead of free-form text, the model is constrained to emit a list of these triples, validated against a JSON schema with automatic retry.

type RepeatDetector added in v0.1.1

type RepeatDetector struct {

	// MinLength is the minimum accumulated rune count before detection fires.
	// Default 100.
	MinLength int
	// Threshold is the repeatness score below which the stream is aborted.
	// Default 0.5.
	Threshold float64
	// contains filtered or unexported fields
}

RepeatDetector detects streaming repetition using a suffix automaton. GetRepeatness returns the ratio of unique substrings to total possible substrings; when it falls below the threshold after enough tokens have accumulated, the stream is considered stuck in a loop.

Algorithm: SuffixAutomaton from moonpalace/detector/repeat (MIT). uniqueSubstrings = sum of (len - link.len) across all states. total possible = n*(n+1)/2 where n = len(accumulated text). A perfectly non-repetitive string scores 1.0; a fully repeated one scores near 0.0.

func DefaultRepeatDetector added in v0.1.1

func DefaultRepeatDetector() *RepeatDetector

DefaultRepeatDetector returns a RepeatDetector with the moonpalace defaults.

func (*RepeatDetector) Add added in v0.1.1

func (d *RepeatDetector) Add(delta string) bool

Add feeds a content delta. Returns true when repetition is detected and the stream should be aborted.

func (*RepeatDetector) Feed added in v0.1.1

func (d *RepeatDetector) Feed(chunk string)

Feed appends a chunk of text. It does not return a detection signal; call IsRepeating or GetRepeatness after feeding to query state.

func (*RepeatDetector) GetRepeatness added in v0.1.1

func (d *RepeatDetector) GetRepeatness() float64

GetRepeatness returns the unique-substring ratio in [0, 1]. A score near 1.0 means highly varied text; near 0.0 means heavy repetition.

func (*RepeatDetector) IsRepeating added in v0.1.1

func (d *RepeatDetector) IsRepeating() bool

IsRepeating returns true when at least MinLength runes have been accumulated and GetRepeatness is below Threshold.

type RequestLogEntry added in v0.1.1

type RequestLogEntry struct {
	Timestamp    time.Time `json:"timestamp"`
	Provider     string    `json:"provider"`
	Model        string    `json:"model"`
	InputTokens  int       `json:"input_tokens"`
	OutputTokens int       `json:"output_tokens"`
	LatencyMs    int64     `json:"latency_ms"`
	Status       string    `json:"status"` // "success" or "error"
	Error        string    `json:"error,omitempty"`
	CacheHit     bool      `json:"cache_hit"`
}

RequestLogEntry is a single logged API call.

type RequestLogger added in v0.1.1

type RequestLogger struct {
	// contains filtered or unexported fields
}

RequestLogger logs all API requests/responses for debugging.

func NewRequestLogger added in v0.1.1

func NewRequestLogger(enabled bool) *RequestLogger

NewRequestLogger creates a logger.

func (*RequestLogger) Log added in v0.1.1

func (rl *RequestLogger) Log(entry RequestLogEntry)

Log records a request.

func (*RequestLogger) Recent added in v0.1.1

func (rl *RequestLogger) Recent(n int) []RequestLogEntry

Recent returns the last N log entries.

func (*RequestLogger) Summary added in v0.1.1

func (rl *RequestLogger) Summary() string

Summary returns aggregate stats from the log.

type ResponseFormat

type ResponseFormat struct {
	Type   string `json:"type"`             // "json_object" or "json_schema"
	Schema string `json:"schema,omitempty"` // optional JSON schema for structured output
}

ResponseFormat specifies the desired output format for the model response.

type ResponseHealth added in v0.1.1

type ResponseHealth string

ResponseHealth classifies the outcome of a model response so eyrie can turn a confusing "the agent did nothing" symptom into a precise, named diagnostic. It is most valuable for reasoning-capable models behind OpenAI-compatible providers, where a misconfiguration commonly yields thinking tokens but no usable answer.

const (
	// ResponseOK means the response carried usable content and/or tool calls.
	ResponseOK ResponseHealth = "ok"
	// ResponseErrorOnlyReasoning means the model emitted reasoning/thinking
	// tokens but produced zero content and zero tool calls — usually a sign the
	// provider is dropping the post-reasoning answer (wrong thinking-format
	// config, truncated stream, or a reasoning-only model used as a chat model).
	ResponseErrorOnlyReasoning ResponseHealth = "error_only_reasoning"
	// ResponseEmpty means there was no reasoning, no content, and no tool calls.
	ResponseEmpty ResponseHealth = "empty_response"
	// ResponseMalformedStream means the stream ended abnormally (a stream-level
	// error, or no terminal "done"/finish_reason was observed).
	ResponseMalformedStream ResponseHealth = "malformed_stream"
)

func DetectResponseHealth added in v0.1.1

func DetectResponseHealth(sig ResponseSignals) ResponseHealth

DetectResponseHealth classifies a completed response. Order matters: a stream-level error dominates, then the reasoning-only case, then plain empty.

func (ResponseHealth) Diagnostic added in v0.1.1

func (h ResponseHealth) Diagnostic() string

Diagnostic returns a human-readable, actionable description for a non-OK health value, or "" when the response was OK.

func (ResponseHealth) Err added in v0.1.1

func (h ResponseHealth) Err() error

Err returns a non-nil error for a non-OK health value, suitable for surfacing to callers/operators. OK returns nil.

type ResponseSignals added in v0.1.1

type ResponseSignals struct {
	SawReasoning bool // any thinking/reasoning tokens were produced
	ContentLen   int  // length of usable assistant content
	ToolCalls    int  // number of tool calls produced
	FinishReason string
	StreamErr    bool // a stream-level error event was seen
	StreamEnded  bool // a terminal done/finish event was observed
}

ResponseSignals are the minimal observations needed to classify health. They are cheap to gather from both the streaming path (counts of thinking/content/ tool-call events) and the non-streaming path (response field lengths).

type RetryConfig

type RetryConfig struct {
	types.RetryConfig
	RetryOn []int // HTTP status codes to retry on
}

RetryConfig controls retry behavior for HTTP clients. It embeds types.RetryConfig for the core fields and adds RetryOn for HTTP-status-code–driven retry decisions.

func DefaultRetryConfig

func DefaultRetryConfig() RetryConfig

DefaultRetryConfig returns sensible defaults.

func NewRetryConfig added in v0.1.1

func NewRetryConfig(maxRetries int, baseDelay, maxDelay time.Duration, retryOn ...int) RetryConfig

NewRetryConfig constructs a RetryConfig from core fields and optional HTTP status codes to retry on.

type RoleRouter added in v0.1.1

type RoleRouter struct {
	// contains filtered or unexported fields
}

RoleRouter wraps a Provider and overrides ChatOptions.Model with the model configured for the request's role before delegating. The role is taken from the context (see WithRole); when absent, RolePrimary is used. The router only sets a model when the resolved slot is non-empty, so it never clears an explicit opts.Model with an unconfigured role.

RoleRouter follows the same decorator pattern as BudgetProvider and TracingProvider: it is additive and does not change ChatOptions semantics.

func NewRoleRouter added in v0.1.1

func NewRoleRouter(inner Provider, roles ModelRoles) (*RoleRouter, error)

NewRoleRouter wraps inner so that requests are routed to the model configured for their role. The inner provider must not be nil; an error is returned otherwise.

func (*RoleRouter) Chat added in v0.1.1

func (r *RoleRouter) Chat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*EyrieResponse, error)

Chat resolves the request's role to a model, applies it to opts, then delegates to the inner provider.

func (*RoleRouter) Name added in v0.1.1

func (r *RoleRouter) Name() string

Name returns the inner provider's name.

func (*RoleRouter) Ping added in v0.1.1

func (r *RoleRouter) Ping(ctx context.Context) error

Ping delegates to the inner provider.

func (*RoleRouter) StreamChat added in v0.1.1

func (r *RoleRouter) StreamChat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*StreamResult, error)

StreamChat resolves the request's role to a model, applies it to opts, then delegates to the inner provider.

type SSEEvent

type SSEEvent struct {
	Event string
	Data  string
}

SSEEvent represents a single Server-Sent Event.

type SchemaValidation added in v0.1.1

type SchemaValidation struct {
	// Schema is the JSON schema to validate against.
	Schema map[string]interface{}
	// MaxRetries is the maximum number of retry attempts if validation fails.
	MaxRetries int
	// StrictMode enables strict schema validation requiring all fields.
	StrictMode bool
}

SchemaValidation holds configuration for structured output validation with retry.

type SemanticCacheConfig added in v0.1.1

type SemanticCacheConfig struct {
	// MaxAge is how long cache entries remain valid. Default: 5 minutes.
	MaxAge time.Duration
	// MaxSize is the maximum number of cached responses. Default: 100.
	// When exceeded, the least-recently-used entry is evicted.
	MaxSize int
	// Enabled toggles caching. Default: true.
	Enabled bool
	// TemperatureThreshold is the temperature above which responses are not
	// cached. Default: 0.5.
	TemperatureThreshold float64
	// SimilarityThreshold is the minimum cosine similarity (0..1) for a cached
	// entry to be considered a hit. Default: 0.95.
	SimilarityThreshold float64
	// EmbeddingModel is the model passed to the Embedder. Required.
	EmbeddingModel string
}

SemanticCacheConfig controls the behavior of EmbeddingCachedProvider.

func DefaultSemanticCacheConfig added in v0.1.1

func DefaultSemanticCacheConfig() SemanticCacheConfig

DefaultSemanticCacheConfig returns a SemanticCacheConfig with sensible defaults.

type SemanticCacheStats added in v0.1.1

type SemanticCacheStats struct {
	Size    int  `json:"size"`
	MaxSize int  `json:"max_size"`
	Hits    int  `json:"hits"`
	Misses  int  `json:"misses"`
	Enabled bool `json:"enabled"`
}

SemanticCacheStats reports cache occupancy and hit/miss counts.

type StreamGuardrailConfig added in v0.1.1

type StreamGuardrailConfig struct {
	// Enabled turns streaming guardrails on or off.
	Enabled bool `json:"enabled"`
	// MaxChunkSize is the maximum allowed size of a single chunk (in bytes).
	// Chunks exceeding this limit are split. A value of 0 disables splitting.
	MaxChunkSize int `json:"max_chunk_size,omitempty"`
	// AccumulateForPII buffers chunks before running PII detection, since
	// patterns like SSNs or credit-card numbers may span chunk boundaries.
	AccumulateForPII bool `json:"accumulate_for_pii"`
	// BlockOnInjection causes an immediate block when a prompt-injection
	// pattern is detected in any chunk, without waiting for accumulation.
	BlockOnInjection bool `json:"block_on_injection"`
}

StreamGuardrailConfig controls how streaming guardrails behave.

type StreamGuardrailResult added in v0.1.1

type StreamGuardrailResult struct {
	// Chunk is the original chunk text.
	Chunk string `json:"chunk"`
	// Blocked indicates that the chunk (or accumulated content) was blocked
	// by a guardrail with Action=Block.
	Blocked bool `json:"blocked"`
	// Violations lists any rules matched during this chunk's processing.
	Violations []GuardrailViolation `json:"violations,omitempty"`
	// ModifiedChunk is the chunk after redactions have been applied.
	// When no redactions are needed, it equals Chunk.
	ModifiedChunk string `json:"modified_chunk"`
}

StreamGuardrailResult is returned after processing a single chunk.

type StreamGuardrails added in v0.1.1

type StreamGuardrails struct {
	// contains filtered or unexported fields
}

StreamGuardrails validates LLM output chunks incrementally as they arrive, rather than waiting for the full response. It is safe for concurrent use.

func NewStreamGuardrails added in v0.1.1

func NewStreamGuardrails(guardrails *Guardrails, config StreamGuardrailConfig) *StreamGuardrails

NewStreamGuardrails creates a StreamGuardrails with the given guardrail rules and configuration. guardrails must not be nil.

func (*StreamGuardrails) Flush added in v0.1.1

func (sg *StreamGuardrails) Flush() []GuardrailViolation

Flush checks the accumulated buffer for violations that may span chunk boundaries (primarily PII patterns when AccumulateForPII is enabled). It returns all violations found during the entire stream session so far that were not already reported. Call Flush after the last chunk has been processed.

func (*StreamGuardrails) IsBlocked added in v0.1.1

func (sg *StreamGuardrails) IsBlocked() bool

IsBlocked reports whether any guardrail with Action=Block has been triggered during the stream session.

func (*StreamGuardrails) ProcessChunk added in v0.1.1

func (sg *StreamGuardrails) ProcessChunk(chunk string) StreamGuardrailResult

ProcessChunk validates a single streaming chunk against the registered guardrails. If AccumulateForPII is enabled, the chunk is appended to an internal buffer and PII checks are deferred until Flush. If BlockOnInjection is enabled, prompt-injection rules are evaluated immediately on the chunk.

The returned StreamGuardrailResult contains the (possibly redacted) chunk and any violations found.

func (*StreamGuardrails) Reset added in v0.1.1

func (sg *StreamGuardrails) Reset()

Reset clears the internal buffer and accumulated violations.

type StreamMerger added in v0.1.1

type StreamMerger struct {
	StreamFields []string
	IndexFields  []string
	// contains filtered or unexported fields
}

StreamMerger is a schema-agnostic SSE delta merger. It accumulates streaming deltas into a single result map without knowing the provider's schema upfront.

StreamFields are merged by string concatenation (e.g. "content", "arguments"). IndexFields name the key within array elements that identifies their position (e.g. "index" in choices[N].delta), enabling correct out-of-order multi-choice reassembly: element A at index 2 is placed at slot 2 regardless of arrival order.

Pattern ported from moonpalace/merge/merger.go (MIT).

func DefaultStreamMerger added in v0.1.1

func DefaultStreamMerger() *StreamMerger

DefaultStreamMerger returns a StreamMerger with Kimi/OpenAI defaults: StreamFields = ["content", "arguments"], IndexFields = ["index"].

func NewStreamMerger added in v0.1.1

func NewStreamMerger(streamFields, indexFields []string) *StreamMerger

NewStreamMerger returns a StreamMerger with the given stream and index field names.

func (*StreamMerger) Merge added in v0.1.1

func (m *StreamMerger) Merge(delta map[string]interface{}) map[string]interface{}

Merge incorporates a parsed delta map into the accumulated result. It returns the updated accumulator (same map, mutated in place).

func (*StreamMerger) Result added in v0.1.1

func (m *StreamMerger) Result() map[string]interface{}

Result returns the current accumulated state.

type StreamResult

type StreamResult struct {
	Events    <-chan EyrieStreamEvent
	RequestID string
	// contains filtered or unexported fields
}

StreamResult wraps a streaming response with cleanup. Callers must call Close() when done reading events, or cancel the context.

func NewStreamResult added in v0.1.1

func NewStreamResult(events <-chan EyrieStreamEvent, cancel context.CancelFunc) *StreamResult

NewStreamResult creates a StreamResult with a cancel function for resource cleanup.

func StreamChatWithContinuation

func StreamChatWithContinuation(ctx context.Context, p Provider, messages []EyrieMessage, opts ChatOptions, cfg ContinuationConfig) (*StreamResult, error)

StreamChatWithContinuation wraps StreamChat with automatic continuation when the response stops with "max_tokens" and contains only text (no tool calls). It returns a StreamResult whose Events channel transparently continues across multiple LLM calls, emitting a "continuation" event at each boundary.

DEPRECATION NOTE: hawk's Session loop has its own max_tokens recovery (internal/engine/stream.go around the `recoveryCount` loop) that doesn't add a synthetic "Continue." user message, and the eyrie conversation engine (eyrie/conversation.Engine) has its own OutputGroupID-based engine-level continuation. The two engine-level paths produce cleaner conversation shapes (no synthetic user turns) and are the recommended pattern for new code. This client-level helper remains for backwards-compatibility with the embedded eyrie HTTP server and non-hawk consumers; new code should implement continuation at the engine or call-site level instead.

Will be removed in eyrie v0.3.0. See eyrie/CHANGELOG.md for the deprecation timeline.

func (*StreamResult) Close

func (sr *StreamResult) Close()

Close stops the stream and releases resources.

type StreamingTokenCounter added in v0.1.1

type StreamingTokenCounter struct {
	// contains filtered or unexported fields
}

StreamingTokenCounter counts tokens as they stream in real-time. Provides running cost estimate during generation.

func NewStreamingTokenCounter added in v0.1.1

func NewStreamingTokenCounter(model string, inputTokens int) *StreamingTokenCounter

NewStreamingTokenCounter creates a counter for a specific model.

func (*StreamingTokenCounter) AddCached added in v0.1.1

func (stc *StreamingTokenCounter) AddCached(tokens int)

AddCached records cached input tokens.

func (*StreamingTokenCounter) AddOutput added in v0.1.1

func (stc *StreamingTokenCounter) AddOutput(text string)

AddOutput records streamed output tokens.

func (*StreamingTokenCounter) CurrentCost added in v0.1.1

func (stc *StreamingTokenCounter) CurrentCost() float64

CurrentCost returns the running cost so far.

func (*StreamingTokenCounter) Summary added in v0.1.1

func (stc *StreamingTokenCounter) Summary() string

Summary returns current token counts and cost.

type StructuredOutputError added in v0.1.1

type StructuredOutputError struct {
	// Response is the raw response that failed validation.
	Response string
	// ValidationErr is the underlying validation error.
	ValidationErr error
	// Attempt is the attempt number that failed.
	Attempt int
}

StructuredOutputError represents a validation failure with details.

func (*StructuredOutputError) Error added in v0.1.1

func (e *StructuredOutputError) Error() string

type TokenCountResult added in v0.1.1

type TokenCountResult struct {
	InputTokens int `json:"input_tokens"`
}

TokenCountResult holds the result of a token count request.

type ToolCall

type ToolCall struct {
	ID        string                 `json:"id,omitempty"`
	Name      string                 `json:"name"`
	Arguments map[string]interface{} `json:"arguments"`
}

ToolCall represents a tool invocation.

func ParseInlineToolCalls

func ParseInlineToolCalls(text string) (cleanText string, toolCalls []ToolCall)

ParseInlineToolCalls detects and extracts tool calls embedded in text content. Three text-embedded formats are recognized, tried in order:

  • Moonshot/kimi (canopywave): <|tool_calls_section_begin|> <|tool_call_begin|> functions.ToolName:0 <|tool_call_argument_begin|> {"arg":"val"} <|tool_call_end|> <|tool_calls_section_end|>
  • Hermes/Nous (Qwen, and most OpenAI-compatible local models served via vLLM/SGLang/Ollama): each call is a JSON object {"name":...,"arguments":{...}} wrapped in <tool_call>...</tool_call> XML tags, with parallel calls emitted as repeated tag pairs. This is the format Qwen2.5/QwQ emit; Qwen3-Coder uses a structurally similar but distinct "qwen3_xml" variant not handled here.
  • Bare JSON brace-match (last resort): a {"name":...,"arguments":{...}} object found via strings.Index(text, `{"`) / strings.LastIndex(text, "}") without any wrapper tags. Used by some fine-tuned models and direct JSON outputs.

Providers that already surface tool calls through the structured tool_calls channel never reach this path — it is a fallback for models that inline them.

type ToolChoiceOption added in v0.1.1

type ToolChoiceOption struct {
	Type                   string `json:"type"`           // "auto", "any", "tool", "none"
	Name                   string `json:"name,omitempty"` // required when type="tool"
	DisableParallelToolUse bool   `json:"disable_parallel_tool_use,omitempty"`
}

ToolChoiceOption controls how the model uses tools (Anthropic).

type ToolResult

type ToolResult struct {
	ToolUseID string `json:"tool_use_id"`
	Content   string `json:"content"`
	IsError   bool   `json:"is_error,omitempty"`
}

ToolResult represents the result of a tool execution.

type TracingProvider added in v0.1.1

type TracingProvider struct {
	// contains filtered or unexported fields
}

TracingProvider wraps a Provider with OpenTelemetry spans for Chat and StreamChat calls. Use NewTracingProvider to create one.

func NewTracingProvider added in v0.1.1

func NewTracingProvider(inner Provider) *TracingProvider

NewTracingProvider wraps the given provider with OTel tracing.

func (*TracingProvider) Chat added in v0.1.1

func (tp *TracingProvider) Chat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*EyrieResponse, error)

func (*TracingProvider) Name added in v0.1.1

func (tp *TracingProvider) Name() string

func (*TracingProvider) Ping added in v0.1.1

func (tp *TracingProvider) Ping(ctx context.Context) error

func (*TracingProvider) StreamChat added in v0.1.1

func (tp *TracingProvider) StreamChat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*StreamResult, error)

type UsageEntry added in v0.1.1

type UsageEntry struct {
	Tokens    int
	CostUSD   float64
	Timestamp time.Time
	Provider  string
	Model     string
}

UsageEntry represents a single recorded usage event.

type UsageLimitProvider added in v0.1.1

type UsageLimitProvider struct {
	// contains filtered or unexported fields
}

UsageLimitProvider wraps any Provider and enforces token/cost budgets via a UsageTracker. It calls CanProceed() before each Chat/StreamChat request and Record() after successful responses.

If the budget is exhausted, calls return a non-nil error immediately without contacting the upstream provider.

UsageLimitProvider is safe for concurrent use (the underlying UsageTracker is internally synchronised).

func NewUsageLimitProvider added in v0.1.1

func NewUsageLimitProvider(inner Provider, tracker *UsageTracker) (*UsageLimitProvider, error)

NewUsageLimitProvider wraps inner with budget enforcement via tracker. Both arguments must be non-nil; an error is returned otherwise.

func (*UsageLimitProvider) Chat added in v0.1.1

func (u *UsageLimitProvider) Chat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*EyrieResponse, error)

Chat sends a non-streaming chat request. The call is gated by the usage tracker's CanProceed() and the response tokens are recorded on success.

func (*UsageLimitProvider) Name added in v0.1.1

func (u *UsageLimitProvider) Name() string

Name returns the inner provider name suffixed with "/usage-limit".

func (*UsageLimitProvider) Ping added in v0.1.1

func (u *UsageLimitProvider) Ping(ctx context.Context) error

Ping delegates directly to the inner provider (budget is not checked).

func (*UsageLimitProvider) StreamChat added in v0.1.1

func (u *UsageLimitProvider) StreamChat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*StreamResult, error)

StreamChat sends a streaming chat request. The budget check happens before the stream starts. Usage is recorded once the stream delivers a "usage" event (typically the final chunk).

func (*UsageLimitProvider) Tracker added in v0.1.1

func (u *UsageLimitProvider) Tracker() *UsageTracker

Tracker returns the underlying UsageTracker for inspection or configuration.

type UsageSummary added in v0.1.1

type UsageSummary struct {
	HourlyTokens     int
	HourlyRemaining  int
	DailyTokens      int
	DailyRemaining   int
	SessionTokens    int
	SessionRemaining int
	DailyCostUSD     float64
	CostRemaining    float64
	HourlyPct        float64
	DailyPct         float64
}

UsageSummary provides a snapshot of current usage across all windows.

type UsageTracker added in v0.1.1

type UsageTracker struct {
	DailyLimit   int
	HourlyLimit  int
	SessionLimit int
	CostLimitUSD float64

	Alerts []Alert
	// contains filtered or unexported fields
}

UsageTracker tracks API usage across sessions and prevents surprise bills.

func NewUsageTracker added in v0.1.1

func NewUsageTracker() *UsageTracker

NewUsageTracker creates a UsageTracker with sensible defaults.

func (*UsageTracker) CanProceed added in v0.1.1

func (u *UsageTracker) CanProceed() (bool, string)

func (*UsageTracker) CheckThresholds added in v0.1.1

func (u *UsageTracker) CheckThresholds()

func (*UsageTracker) EstimateRemaining added in v0.1.1

func (u *UsageTracker) EstimateRemaining(tokensPerRequest int) int

func (*UsageTracker) FormatSummary added in v0.1.1

func (u *UsageTracker) FormatSummary() string

func (*UsageTracker) GetUsage added in v0.1.1

func (u *UsageTracker) GetUsage() UsageSummary

func (*UsageTracker) PruneOld added in v0.1.1

func (u *UsageTracker) PruneOld()

func (*UsageTracker) Record added in v0.1.1

func (u *UsageTracker) Record(tokens int, costUSD float64, provider, model string)

func (*UsageTracker) Reset added in v0.1.1

func (u *UsageTracker) Reset()

type VertexClient added in v0.1.1

type VertexClient struct {
	// contains filtered or unexported fields
}

func NewVertexClient added in v0.1.1

func NewVertexClient(projectID, region, token string) *VertexClient

func (*VertexClient) Chat added in v0.1.1

func (c *VertexClient) Chat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*EyrieResponse, error)

func (*VertexClient) Name added in v0.1.1

func (c *VertexClient) Name() string

func (*VertexClient) Ping added in v0.1.1

func (c *VertexClient) Ping(ctx context.Context) error

func (*VertexClient) StreamChat added in v0.1.1

func (c *VertexClient) StreamChat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*StreamResult, error)

type WeightedProvider

type WeightedProvider struct {
	// contains filtered or unexported fields
}

WeightedProvider selects a provider based on configured weights, with automatic failover to remaining providers on retriable errors.

WeightedProvider is safe for concurrent use.

func NewWeightedProvider

func NewWeightedProvider(configs []WeightedProviderConfig) (*WeightedProvider, error)

NewWeightedProvider creates a WeightedProvider that selects providers based on the configured weights. At least one provider must be supplied and every weight must be positive; an error is returned otherwise. Weights are normalized to sum to 1.0.

func (*WeightedProvider) Chat

func (wp *WeightedProvider) Chat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*EyrieResponse, error)

Chat sends a non-streaming chat request using weighted random selection with failover on retriable errors.

func (*WeightedProvider) Name

func (wp *WeightedProvider) Name() string

Name returns a composite name showing providers and their weights.

func (*WeightedProvider) Ping

func (wp *WeightedProvider) Ping(ctx context.Context) error

Ping tries to ping each provider, returning nil on the first success.

func (*WeightedProvider) Stats

func (wp *WeightedProvider) Stats() map[string]int64

Stats returns a snapshot of how many times each provider served a request.

func (*WeightedProvider) StreamChat

func (wp *WeightedProvider) StreamChat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*StreamResult, error)

StreamChat sends a streaming chat request using weighted random selection with failover on retriable errors.

type WeightedProviderConfig

type WeightedProviderConfig struct {
	Provider Provider
	Weight   float64 // relative weight (e.g., 0.8 for 80%)
}

WeightedProviderConfig associates a Provider with a selection weight.

type ZAIClient added in v0.1.1

type ZAIClient struct {
	// contains filtered or unexported fields
}

ZAIClient uses the OpenAI-compatible endpoint (paas/v4 or coding/paas/v4) first, with Anthropic-compatible fallback (/api/anthropic) on retriable errors. This provides proper separation for General vs Coding Plan while giving both protocol surfaces (exactly as Xiaomi MiMo does for its plans).

func NewZAIClient added in v0.1.1

func NewZAIClient(apiKey, openAIBase, anthropicBase string, compat *OpenAICompatConfig, providerID string, opts ...ClientOption) *ZAIClient

NewZAIClient builds a Z.AI dual-protocol client for a given plan/region gateway. openAIBase should be the resolved general or coding paas base. anthropicBase should be the resolved /api/anthropic (global or cn). The same apiKey is used for both sides; the plan subscription attached to the key controls quota/billing when using the coding path.

func (*ZAIClient) Chat added in v0.1.1

func (c *ZAIClient) Chat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*EyrieResponse, error)

func (*ZAIClient) Name added in v0.1.1

func (c *ZAIClient) Name() string

func (*ZAIClient) Ping added in v0.1.1

func (c *ZAIClient) Ping(ctx context.Context) error

func (*ZAIClient) StreamChat added in v0.1.1

func (c *ZAIClient) StreamChat(ctx context.Context, messages []EyrieMessage, opts ChatOptions) (*StreamResult, error)

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL