Documentation
¶
Index ¶
- func BuildMainProvider(ctx context.Context, cfg *config.Config, db *gorm.DB, logger *zap.Logger) (llm.Provider, error)
- func RegisterMainProviderBuilder(mode string, builder MainProviderBuilder) error
- func UnregisterMainProviderBuilder(mode string)
- type BudgetConfig
- type CacheConfig
- type Config
- type MainProviderBuilder
- type Runtime
- type ToolProviderConfig
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func BuildMainProvider ¶
func BuildMainProvider(ctx context.Context, cfg *config.Config, db *gorm.DB, logger *zap.Logger) (llm.Provider, error)
BuildMainProvider resolves the configured builder mode and constructs the main text provider for runtime composition.
func RegisterMainProviderBuilder ¶
func RegisterMainProviderBuilder(mode string, builder MainProviderBuilder) error
RegisterMainProviderBuilder registers or replaces a startup builder for a main-provider mode such as "legacy" or "channel_routed".
func UnregisterMainProviderBuilder ¶
func UnregisterMainProviderBuilder(mode string)
UnregisterMainProviderBuilder removes a registered startup builder.
Types ¶
type BudgetConfig ¶
type BudgetConfig struct {
Enabled bool
MaxTokensPerRequest int
MaxTokensPerMinute int
MaxTokensPerHour int
MaxTokensPerDay int
MaxCostPerRequest float64
MaxCostPerDay float64
AlertThreshold float64
AutoThrottle bool
ThrottleDelay time.Duration
}
BudgetConfig controls token and cost policy assembly.
type CacheConfig ¶
type CacheConfig struct {
Enabled bool
LocalMaxSize int
LocalTTL time.Duration
EnableRedis bool
RedisTTL time.Duration
KeyStrategy string
}
CacheConfig controls prompt-cache assembly.
type Config ¶
type Config struct {
Timeout time.Duration
MaxRetries int
Budget BudgetConfig
Cache CacheConfig
Tool ToolProviderConfig
}
Config controls runtime composition around an already-constructed main provider. It is storage-agnostic and can be reused by external projects.
type MainProviderBuilder ¶
type MainProviderBuilder func(ctx context.Context, cfg *config.Config, db *gorm.DB, logger *zap.Logger) (llm.Provider, error)
MainProviderBuilder constructs the handler runtime's main text provider chain for a configured startup mode.
type Runtime ¶
type Runtime struct {
Provider llm.Provider
ToolProvider llm.Provider
BudgetManager *llmpolicy.TokenBudgetManager
CostTracker *observability.CostTracker
Ledger observability.Ledger
Cache *cache.MultiLevelCache
Metrics *observability.Metrics
PolicyManager *llmpolicy.Manager
}
Runtime groups the LLM-facing runtime dependencies assembled around a main provider chain.
type ToolProviderConfig ¶
type ToolProviderConfig struct {
Provider string
DefaultProvider string
APIKey string
DefaultAPIKey string
BaseURL string
DefaultBaseURL string
Timeout time.Duration
MaxRetries int
}
ToolProviderConfig describes an optional dedicated tool-calling provider. If the provider/api key/baseURL/timeout/max-retry fields are all left empty or zero, the runtime reuses the main provider for tools.
Source Files