Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type BudgetConfig ¶
type BudgetConfig struct {
Enabled bool
MaxTokensPerRequest int
MaxTokensPerMinute int
MaxTokensPerHour int
MaxTokensPerDay int
MaxCostPerRequest float64
MaxCostPerDay float64
AlertThreshold float64
AutoThrottle bool
ThrottleDelay time.Duration
}
BudgetConfig controls token and cost policy assembly.
type CacheConfig ¶
type CacheConfig struct {
Enabled bool
LocalMaxSize int
LocalTTL time.Duration
EnableRedis bool
RedisTTL time.Duration
KeyStrategy string
}
CacheConfig controls prompt-cache assembly.
type Config ¶
type Config struct {
Timeout time.Duration
MaxRetries int
Budget BudgetConfig
Cache CacheConfig
Tool ToolProviderConfig
}
Config controls runtime composition around an already-constructed main provider. It is storage-agnostic and can be reused by external projects.
type Runtime ¶
type Runtime struct {
Gateway llmcore.Gateway
ToolGateway llmcore.Gateway
Provider llm.Provider
ToolProvider llm.Provider
BudgetManager *llmpolicy.TokenBudgetManager
CostTracker *observability.CostTracker
Ledger observability.Ledger
Cache *cache.MultiLevelCache
Metrics *observability.Metrics
PolicyManager *llmpolicy.Manager
}
Runtime groups the LLM-facing runtime dependencies assembled around a main provider chain.
type ToolProviderConfig ¶
type ToolProviderConfig struct {
Provider string
DefaultProvider string
APIKey string
DefaultAPIKey string
BaseURL string
DefaultBaseURL string
Timeout time.Duration
MaxRetries int
}
ToolProviderConfig describes an optional dedicated tool-calling provider. If the provider/api key/baseURL/timeout/max-retry fields are all left empty or zero, the runtime reuses the main provider for tools.
Click to show internal directories.
Click to hide internal directories.