Documentation
¶
Overview ¶
Package modelinfo manages model metadata caching.
This service provides a caching layer between the dashboard UI and provider backends, reducing API calls and improving performance.
Index ¶
- type ModelView
- type Service
- func (s *Service) DeleteOverride(providerID, modelName string) error
- func (s *Service) GetModelInfo(ctx context.Context, modelID models.ModelId) (*models.ModelInfo, error)
- func (s *Service) GetModelInfos(ctx context.Context, providerID string) ([]models.ModelInfo, error)
- func (s *Service) GetModels(ctx context.Context, providerID string) ([]string, error)
- func (s *Service) InvalidateAll() error
- func (s *Service) InvalidateProvider(providerID string) error
- func (s *Service) IsModelEnabled(providerID, modelName string) bool
- func (s *Service) ListOverrides(providerID string) ([]*models.ModelOverride, error)
- func (s *Service) MergedView(ctx context.Context, providerID string) ([]ModelView, error)
- func (s *Service) PeekMergedView(providerID string) ([]ModelView, error)
- func (s *Service) PeekModelInfos(providerID string) []models.ModelInfo
- func (s *Service) Refresh(ctx context.Context, providerID string) ([]models.ModelInfo, error)
- func (s *Service) RemoveModel(providerID, name string) (bool, error)
- func (s *Service) SetLogger(l *slog.Logger)
- func (s *Service) SetOverride(ov models.ModelOverride) error
- func (s *Service) WarmMissing(ctx context.Context)
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type ModelView ¶
type ModelView struct {
models.ModelInfo
Disabled bool `json:"disabled"`
Custom bool `json:"custom"`
}
ModelView is a model merged with its admin override state.
type Service ¶
type Service struct {
// contains filtered or unexported fields
}
Service manages model metadata caching
func New ¶
func New(database *db.DB, providerSvc *provider.Service, credSvc *credential.Service, cacheTTL time.Duration) *Service
New creates a ModelInfo service
func (*Service) DeleteOverride ¶
DeleteOverride removes the override for one model. For custom models this removes the model itself.
func (*Service) GetModelInfo ¶
func (s *Service) GetModelInfo(ctx context.Context, modelID models.ModelId) (*models.ModelInfo, error)
GetModelInfo retrieves metadata for a specific model
func (*Service) GetModelInfos ¶
GetModelInfos retrieves all model metadata for a provider
func (*Service) InvalidateAll ¶
InvalidateAll clears entire cache
func (*Service) InvalidateProvider ¶
InvalidateProvider clears cache for a specific provider
func (*Service) IsModelEnabled ¶
IsModelEnabled reports whether a model may be routed. Models are enabled unless an override disables them.
func (*Service) ListOverrides ¶
func (s *Service) ListOverrides(providerID string) ([]*models.ModelOverride, error)
ListOverrides returns all overrides of a provider.
func (*Service) MergedView ¶
MergedView merges the cached upstream model list with admin overrides: disabled models are flagged, custom models appended, display metadata applied. Fetches from the upstream on a cache miss — explicit refresh paths only.
func (*Service) PeekMergedView ¶
PeekMergedView is MergedView over the cache (stale entries count) with no upstream fetch. Browsing paths use it: upstream sync is opt-in (models_auto_sync) or an explicit user action.
func (*Service) PeekModelInfos ¶
PeekModelInfos returns the cached model list without triggering an upstream fetch; stale entries count, and a memory miss hydrates from the persisted record. A true miss returns nil — callers that merely display a count must not ping the provider (auto-sync is opt-in via config.models_auto_sync; everything else syncs on explicit user action or the startup warm).
func (*Service) Refresh ¶ added in v0.1.0
Refresh forces an upstream fetch and stores it. The previous cache is kept when the fetch fails: until a new list arrives the old one counts as current.
func (*Service) RemoveModel ¶ added in v0.1.0
RemoveModel drops one model from the cached list after the upstream reports it does not exist. A missing provider record is a no-op. Reports whether anything was removed.
func (*Service) SetOverride ¶
func (s *Service) SetOverride(ov models.ModelOverride) error
SetOverride creates or replaces the override for one model.
func (*Service) WarmMissing ¶
WarmMissing creates model caches for providers that have none. Runs in the background at startup; refresh afterwards stays opt-in (models_auto_sync) or manual.