modelinfo

package
v0.3.9 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 9, 2026 License: MIT Imports: 13 Imported by: 0

Documentation

Overview

Package modelinfo manages model metadata caching.

This service provides a caching layer between the dashboard UI and provider backends, reducing API calls and improving performance.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type ModelView

type ModelView struct {
	models.ModelInfo
	Disabled bool `json:"disabled"`
	Custom   bool `json:"custom"`
}

ModelView is a model merged with its admin override state.

type Service

type Service struct {
	// contains filtered or unexported fields
}

Service manages model metadata caching

func New

func New(database *db.DB, providerSvc *provider.Service, credSvc *credential.Service, cacheTTL time.Duration) *Service

New creates a ModelInfo service

func (*Service) DeleteOverride

func (s *Service) DeleteOverride(providerID, modelName string) error

DeleteOverride removes the override for one model. For custom models this removes the model itself.

func (*Service) GetModelInfo

func (s *Service) GetModelInfo(ctx context.Context, modelID models.ModelId) (*models.ModelInfo, error)

GetModelInfo retrieves metadata for a specific model

func (*Service) GetModelInfos

func (s *Service) GetModelInfos(ctx context.Context, providerID string) ([]models.ModelInfo, error)

GetModelInfos retrieves all model metadata for a provider

func (*Service) GetModels

func (s *Service) GetModels(ctx context.Context, providerID string) ([]string, error)

GetModels retrieves just the model names for a provider

func (*Service) InvalidateAll

func (s *Service) InvalidateAll() error

InvalidateAll clears entire cache

func (*Service) InvalidateProvider

func (s *Service) InvalidateProvider(providerID string) error

InvalidateProvider clears cache for a specific provider

func (*Service) IsModelEnabled

func (s *Service) IsModelEnabled(providerID, modelName string) (bool, error)

IsModelEnabled reports whether a model may be routed. Models are enabled unless an override disables them. Missing overrides allow routing; storage failures deny routing.

func (*Service) ListOverrides

func (s *Service) ListOverrides(providerID string) ([]*models.ModelOverride, error)

ListOverrides returns all overrides of a provider.

func (*Service) MergedView

func (s *Service) MergedView(ctx context.Context, providerID string) ([]ModelView, error)

MergedView merges the cached upstream model list with admin overrides: disabled models are flagged, custom models appended, display metadata applied. Fetches from the upstream on a cache miss — explicit refresh paths only.

func (*Service) PeekMergedView

func (s *Service) PeekMergedView(providerID string) ([]ModelView, error)

PeekMergedView is MergedView over the cache (stale entries count) with no upstream fetch. Browsing paths use it: upstream sync is opt-in (models_auto_sync) or an explicit user action.

func (*Service) PeekModelInfos

func (s *Service) PeekModelInfos(providerID string) []models.ModelInfo

PeekModelInfos returns the cached model list without triggering an upstream fetch; stale entries count, and a memory miss hydrates from the persisted record. A true miss returns nil — callers that merely display a count must not ping the provider (auto-sync is opt-in via config.models_auto_sync; everything else syncs on explicit user action or the startup warm).

func (*Service) Refresh added in v0.1.0

func (s *Service) Refresh(ctx context.Context, providerID string) ([]models.ModelInfo, error)

Refresh forces an upstream fetch and stores it. The previous cache is kept when the fetch fails: until a new list arrives the old one counts as current.

func (*Service) RemoveModel added in v0.1.0

func (s *Service) RemoveModel(providerID, name string) (bool, error)

RemoveModel drops one model from the cached list after the upstream reports it does not exist. A missing provider record is a no-op. Reports whether anything was removed.

func (*Service) SetLogger

func (s *Service) SetLogger(l *slog.Logger)

SetLogger wires structured logging (called once from server.New).

func (*Service) SetOverride

func (s *Service) SetOverride(ov models.ModelOverride) error

SetOverride creates or replaces the override for one model.

func (*Service) WarmMissing

func (s *Service) WarmMissing(ctx context.Context)

WarmMissing creates model caches for providers that have none. Runs in the background at startup; refresh afterwards stays opt-in (models_auto_sync) or manual.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL