Documentation
¶
Overview ¶
Package catalog is the single source of truth for built-in LLM model metadata (context window, max output tokens, pricing, capabilities).
This package exists so consumers outside the factory (e.g. the agent package resolving token budgets) can read catalog entries without depending on provider client packages.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func BuildCatalog ¶
BuildCatalog returns the static catalog of all known models across providers. Callers get a fresh map each time; the pointed-to ModelInfo entries are freshly allocated and safe to mutate by the caller.
Fields:
- ShowInDropdown: false for bedrock and azure-openai (endpoint-configured, not user-selected)
- IsReasoning: true for o-series, Magistral, extended/adaptive thinking models
- Capabilities: "text", "vision", "tool-use", "thinking", "image-gen"
func Lookup ¶
Lookup returns the ModelInfo for a given provider + model ID, or nil when the pair is not in the currently registered default source. This is the package-level convenience form; it delegates to DefaultSource().Lookup with context.Background.
At startup the default source is the built-in StaticSource. Callers can extend it via Register — for example to consult a DB-backed catalog first and fall back to the built-in map:
catalog.Register(catalog.MultiSource{
catalog.NewCachedSource(myDBSource, 5*time.Minute),
catalog.StaticSource(),
})
This is an exact-match lookup at each source; callers that need prefix or version-suffix matching must handle that above.
func NormalizeProvider ¶
NormalizeProvider returns the canonical form of a provider name as used in catalog keys. Callers may pass "azureopenai" or "azure-openai" interchangeably.
func Register ¶
func Register(s Source)
Register replaces the package-level default source used by Lookup. The new source is consulted by every subsequent call into the package-level Lookup, including calls from factory and agent that were previously hardcoded to the built-in catalog.
Typical usage at program startup:
catalog.Register(catalog.MultiSource{
catalog.NewCachedSource(myDBSource, 5*time.Minute),
catalog.StaticSource(),
})
Calling Register(nil) restores the built-in StaticSource.
Types ¶
type CachedSource ¶
type CachedSource struct {
// contains filtered or unexported fields
}
CachedSource wraps another Source with a TTL-based read-through cache. Both positive (found) and negative (nil) results are cached to avoid retrying misses on every chat request.
Safe for concurrent use. List calls bypass the cache (the cost of caching a whole enumerated catalog rarely pays off and introduces staleness risk on admin writes).
Concurrent misses on the same key collapse into a single inner.Lookup via singleflight, so a thundering-herd of chat requests for a newly-seen model only hits the backing source once.
func NewCachedSource ¶
func NewCachedSource(inner Source, ttl time.Duration) *CachedSource
NewCachedSource wraps inner with a TTL cache. A ttl of zero disables expiration (entries live until Invalidate). A negative ttl behaves like zero. Passing a nil inner panics — the caller's mistake, not runtime data.
func (*CachedSource) Invalidate ¶
func (c *CachedSource) Invalidate()
Invalidate drops every cached entry. Call this from admin write paths that mutate the underlying source so the next Lookup reflects the change.
func (*CachedSource) InvalidateModel ¶
func (c *CachedSource) InvalidateModel(provider, modelID string)
InvalidateModel drops the cached entry for a single (provider, modelID). Useful when the admin write path knows exactly which model changed.
type MultiSource ¶
type MultiSource []Source
MultiSource is an ordered chain of Sources. Lookup returns the first non-nil result; List merges every underlying source, with earlier entries winning on (provider, modelID) collisions so dynamic sources can shadow built-in entries.
MultiSource itself holds no state; each entry is consulted on every call. Wrap individual entries in CachedSource to add per-Source caching.
type Source ¶
type Source interface {
// Lookup returns the ModelInfo for (provider, modelID), or nil when the
// pair is not in this source. Callers fall back to the next source in
// the chain on a nil return.
Lookup(ctx context.Context, provider, modelID string) *loomv1.ModelInfo
// List returns every known ModelInfo grouped by provider. A Source that
// cannot enumerate (e.g. a lazy remote lookup) may return nil or an
// empty map; callers must tolerate either.
List(ctx context.Context) map[string][]*loomv1.ModelInfo
}
Source is a dynamic model-metadata provider. Implementations may back their data with the built-in static map, a database, a remote service, a config file, etc. Register a Source at program startup to extend the package-level Lookup function beyond the built-in catalog.
Implementations must be safe for concurrent use.
func DefaultSource ¶
func DefaultSource() Source
DefaultSource returns the currently registered default Source. Tests and advanced callers can use this to compose on top of whatever has already been registered.
func StaticSource ¶
func StaticSource() Source
StaticSource returns a Source backed by the built-in catalog map returned by BuildCatalog. Use this as the last (fallback) entry in a MultiSource so dynamic sources can override or extend the built-in entries.
Source Files