Documentation
¶
Overview ¶
Package llmprovider exposes a single Provider interface that every LLM backend in the project must satisfy. It ships a factory (New) that picks the right Eino model implementation for each supported provider type.
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ErrUsageLimitReached = errors.New("llm provider: subscription usage limit reached")
ErrUsageLimitReached signals that a subscription provider (codex / claude-code) reported its plan quota as exhausted (not a transient rate limit). Callers use errors.Is to detect it and abort immediately instead of rotating credentials or retrying, which would keep hammering an already-flagged account.
Functions ¶
This section is empty.
Types ¶
type ChatModelProvider ¶
type ChatModelProvider interface {
Provider
// ChatModel returns the underlying Eino BaseChatModel.
ChatModel() model.BaseChatModel
}
ChatModelProvider extends Provider with access to the underlying Eino ChatModel for use with ChatModelAgent.
func New ¶
func New(cfg Config) (ChatModelProvider, error)
New builds the appropriate Provider for cfg.ProviderType.
type Config ¶
type Config struct {
ProviderType string
ProviderName string // name of the provider config entry; used by openai-codex to find its pool
ModelName string
BaseURL string // required for llamacpp / ollama; ignored for cloud providers
APIKey string // optional for local providers
Temperature float64 // 0 means the backend default
MaxTokens int // 0 means the backend default
MaxRetries int
Timeout time.Duration
CodexManager *codexauth.Manager // required for openai-codex provider type
ClaudeManager *claudeauth.Manager // required for claude-code provider type
}
Config groups every knob the callers need to set.
type Provider ¶
type Provider interface {
// Generate sends prompt to the model and returns the raw text response.
Generate(ctx context.Context, prompt string) (string, error)
}
Provider is the minimal interface every LLM backend must satisfy.
type Usage ¶ added in v0.4.0
Usage mirrors the subset of Eino's TokenUsage the monitoring layer persists. All fields are zero when the backend did not report usage (e.g. OAuth subscription providers), in which case GenerateWithUsage returns known=false.
type UsageGenerator ¶ added in v0.4.0
type UsageGenerator interface {
// GenerateWithUsage behaves like Generate but also returns token usage and
// whether the backend actually reported it.
GenerateWithUsage(ctx context.Context, prompt string) (string, Usage, bool, error)
}
UsageGenerator is the optional add-on interface implemented by providers that can report token usage alongside the response. The gateway type-asserts for it; providers that don't implement it fall back to plain Generate.
Source Files