modelcatalog

package
v1.3.22 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 21, 2026 License: Apache-2.0 Imports: 20 Imported by: 9

Documentation

Overview

Package modelcatalog composes three subpackages — datasheet (pricing + model parameters + capabilities), live (per-(provider, keyID) list-models cache), and keyconfig (per-provider allow/block/aliases derived from keys) — into the ModelCatalog facade that consumers (governance, telemetry, logging, server, etc.) use.

The composer owns I/O orchestration: the hourly pricing sync ticker, the distributed lock used during sync, and the gossip after-sync hook. Subpackages perform no I/O directly — they expose Load/Sync methods the composer calls.

Editing the model pool. These methods are the push surface server.go orchestrates against — fetched list-models responses go into live, configstore key edits go into keyconfig, and the composed pool is what the read methods in models.go return.

Index

Constants

View Source 
const (
	DefaultSyncInterval           = datasheet.DefaultSyncInterval
	MinimumPricingSyncIntervalSec = int64(3600)

	ConfigLastPricingSyncKey    = "LastModelPricingSync"
	ConfigLastParamsSyncKey     = "LastModelParametersSync"
	ConfigLastMCPLibrarySyncKey = "LastMCPLibrarySync"
)
View Source 
const (
	ScopeKindGlobal                = datasheet.ScopeKindGlobal
	ScopeKindProvider              = datasheet.ScopeKindProvider
	ScopeKindProviderKey           = datasheet.ScopeKindProviderKey
	ScopeKindVirtualKey            = datasheet.ScopeKindVirtualKey
	ScopeKindVirtualKeyProvider    = datasheet.ScopeKindVirtualKeyProvider
	ScopeKindVirtualKeyProviderKey = datasheet.ScopeKindVirtualKeyProviderKey

	MatchTypeExact    = datasheet.MatchTypeExact
	MatchTypeWildcard = datasheet.MatchTypeWildcard
)

Scope kind constants re-exported for callers that compare by value.

View Source 
const (
	DefaultPricingURL             = datasheet.DefaultURL
	DefaultModelParametersURL     = datasheet.DefaultModelParametersURL
	DefaultPricingTimeout         = datasheet.DefaultPricingTimeout
	DefaultModelParametersTimeout = datasheet.DefaultModelParametersTimeout

	DefaultMCPLibraryURL     = "https://getbifrost.ai/mcp-library"
	DefaultMCPLibraryTimeout = 45 * time.Second
)

Sync timing defaults re-exported from datasheet for consumers of the historical constants.

Variables

This section is empty.

Functions

func Slugify added in v1.3.19 

func Slugify(name string) string

Slugify derives a URL/identifier-safe slug from a display name: lowercase, non-alphanumeric runs collapsed to a single "-", and leading/trailing "-" trimmed. Used to key custom library entries off their name so the existing unique slug index detects duplicates. Returns "" for names with no alphanumeric content (the caller rejects an empty slug).

NOTE: only ASCII letters and digits are retained — accented/unicode characters (e.g. "données") are stripped, which may produce unexpected slugs for non-ASCII names. This is intentional to keep slugs URL-safe without a transliteration dependency; callers should be aware of this limitation.

func SyncMCPLibrary added in v1.3.19 

func SyncMCPLibrary(ctx context.Context, url string, store configstore.ConfigStore) (int, error)

SyncMCPLibrary fetches the MCP server catalog from url, parses the JSON payload, and upserts each row into the mcp_library table keyed by slug. Returns the number of rows upserted.

The function is intentionally stateless and operates directly on the ConfigStore so it can be called from both the force-sync handler and the background worker without needing a dedicated manager struct.

Types

type AliasOwner added in v1.3.19 

type AliasOwner = keyconfig.AliasOwner

Type re-exports so external callers can continue importing the legacy names (PricingEntry, PricingOptions, etc.) without changing imports. Internally these live in the datasheet / keyconfig subpackages.

type Config 

type Config struct {
	PricingURL          *string `json:"pricing_url,omitempty"`
	PricingSyncInterval *int64  `json:"pricing_sync_interval,omitempty"` // seconds
	ModelParametersURL  *string `json:"model_parameters_url,omitempty"`

	// MCPLibraryURL overrides the endpoint the MCP server library catalog is
	// synced from. Empty/nil uses DefaultMCPLibraryURL. Mirrors PricingURL: the
	// default ships out of the box and the user can point it at a custom source.
	MCPLibraryURL          *string `json:"mcp_library_url,omitempty"`
	MCPLibrarySyncInterval *int64  `json:"mcp_library_sync_interval,omitempty"` // seconds
}

Config is the model pricing configuration.

type KeyConfigEntry added in v1.3.19 

type KeyConfigEntry = keyconfig.KeyEntry

Type re-exports so external callers can continue importing the legacy names (PricingEntry, PricingOptions, etc.) without changing imports. Internally these live in the datasheet / keyconfig subpackages.

type MCPLibraryEntry added in v1.3.19 

type MCPLibraryEntry struct {
	Name               string                    `json:"name"`
	Description        string                    `json:"description,omitempty"`
	Category           string                    `json:"category,omitempty"`
	ConnectionType     schemas.MCPConnectionType `json:"connection_type"`
	ConnectionURL      string                    `json:"connection_url,omitempty"`
	StdioConfig        *schemas.MCPStdioConfig   `json:"stdio_config,omitempty"`
	AuthType           schemas.MCPAuthType       `json:"auth_type,omitempty"`
	RequiredHeaderKeys []string                  `json:"required_header_keys,omitempty"`
	IconURL            string                    `json:"icon_url,omitempty"`
	DocsURL            string                    `json:"docs_url,omitempty"`
	Publisher          string                    `json:"publisher,omitempty"`
	Tags               []string                  `json:"tags,omitempty"`
	Metadata           map[string]any            `json:"metadata,omitempty"`
}

MCPLibraryEntry is the JSON shape a single server has in the remote MCP library catalog (the payload fetched from DefaultMCPLibraryURL / custom URL). The catalog carries no slug; it is derived from Name at sync time via Slugify. The remaining fields map onto TableMCPLibrary minus the DB-managed fields (ID, Slug, CreatedAt, UpdatedAt).

type MCPLibraryPayload added in v1.3.19 

type MCPLibraryPayload struct {
	Servers       []MCPLibraryEntry `json:"servers"`
	LastUpdatedAt string            `json:"lastUpdatedAt,omitempty"`
}

MCPLibraryPayload is the top-level JSON envelope returned by the remote MCP library catalog endpoint.

type MatchType added in v1.3.0 

type MatchType = datasheet.MatchType

Type re-exports so external callers can continue importing the legacy names (PricingEntry, PricingOptions, etc.) without changing imports. Internally these live in the datasheet / keyconfig subpackages.

type ModelCatalog 

type ModelCatalog struct {
	// contains filtered or unexported fields
}

func Init 

func Init(ctx context.Context, config *Config, configStore configstore.ConfigStore, logger schemas.Logger) (*ModelCatalog, error)

func NewTestCatalog added in v1.2.19 

func NewTestCatalog(baseModelIndex map[string]string) *ModelCatalog

NewTestCatalog constructs a minimal ModelCatalog for unit tests. Does not start background workers or hit external services.

func NewTestCatalogWithDatasheet added in v1.3.22 

func NewTestCatalogWithDatasheet(ds *datasheet.Store) *ModelCatalog

NewTestCatalogWithDatasheet wraps a caller-provided datasheet.Store (e.g. one loaded from a local testdata pricing file via datasheet.New(...) + LoadFromURLIntoMemory) in a ModelCatalog, so tests in other packages can exercise real pricing/cost computation without reaching the network.

func (*ModelCatalog) AllowedModelsForProvider added in v1.3.19 

func (mc *ModelCatalog) AllowedModelsForProvider(provider schemas.ModelProvider) schemas.WhiteList

AllowedModelsForProvider returns the aggregated whitelist for the provider (union of enabled keys' Models minus per-key blacklists, or ["*"] when any key is unrestricted). Used by the load balancer to know what each provider can serve without re-walking the configstore.

func (*ModelCatalog) BlacklistedModelsForProvider added in v1.3.19 

func (mc *ModelCatalog) BlacklistedModelsForProvider(provider schemas.ModelProvider) schemas.BlackList

BlacklistedModelsForProvider returns the intersection of enabled keys' BlacklistedModels for the provider — a model is included only when every enabled key blacklists it.

func (*ModelCatalog) CalculateCost 

func (mc *ModelCatalog) CalculateCost(result *schemas.BifrostResponse, scopes *PricingLookupScopes) float64

CalculateCost computes the dollar cost for a Bifrost response.

func (*ModelCatalog) CalculateCostForUsage added in v1.3.22 

func (mc *ModelCatalog) CalculateCostForUsage(usage *schemas.BifrostLLMUsage, provider schemas.ModelProvider, model string, requestType schemas.RequestType, scopes *PricingLookupScopes) float64

CalculateCostForUsage computes the dollar cost from a bare usage object when no full BifrostResponse is available — used to bill partial usage carried on a failed/cancelled request (BifrostError.ExtraFields.BilledUsage).

func (*ModelCatalog) Cleanup 

func (mc *ModelCatalog) Cleanup() error

func (*ModelCatalog) ConfiguredProviders added in v1.3.19 

func (mc *ModelCatalog) ConfiguredProviders() []schemas.ModelProvider

ConfiguredProviders returns every provider with at least one entry in keyconfig. Used by the load balancer's provider selection where the configured-provider set is the routing-eligible universe.

func (*ModelCatalog) DeletePricingOverride added in v1.3.0 

func (mc *ModelCatalog) DeletePricingOverride(id string)

func (*ModelCatalog) ForceReloadMCPLibrary added in v1.3.19 

func (mc *ModelCatalog) ForceReloadMCPLibrary(ctx context.Context) (int, error)

ForceReloadMCPLibrary triggers an immediate MCP library sync from the current configured source and advances the MCP library sync timer on success.

func (*ModelCatalog) ForceReloadPricing added in v1.1.48 

func (mc *ModelCatalog) ForceReloadPricing(ctx context.Context) error

ForceReloadPricing triggers an immediate URL→DB→memory sync for pricing and model parameters in parallel, fires the gossip hook, and resets the ticker so the next scheduled sync waits a full interval from now.

Behavior change from pre-refactor: this no longer touches the live list-models cache. List-models refresh is now driven by key/provider edits, not by pricing reloads.

func (*ModelCatalog) GetBaseModelName added in v1.2.19 

func (mc *ModelCatalog) GetBaseModelName(model string) string

func (*ModelCatalog) GetDistinctBaseModelNames added in v1.2.19 

func (mc *ModelCatalog) GetDistinctBaseModelNames() []string

GetDistinctBaseModelNames returns all unique base model names from the datasheet. Used by governance for cross-provider model selection.

func (*ModelCatalog) GetModelCapabilityEntryForModel added in v1.2.35 

func (mc *ModelCatalog) GetModelCapabilityEntryForModel(model string, provider schemas.ModelProvider) *PricingEntry

GetModelCapabilityEntryForModel returns capability metadata for a (model, provider) pair. Prefers chat, then responses, then text-completion entries; falls back to the lexicographically first available mode for deterministic behavior.

func (*ModelCatalog) GetModelsForProvider 

func (mc *ModelCatalog) GetModelsForProvider(provider schemas.ModelProvider) []string

GetModelsForProvider returns the effective allowed model set for the provider. Filtered live entries are authoritative when present (they were pre-gated by ListModelsPipeline against the key's allow/block/aliases); otherwise the datasheet view is filtered by the keyconfig aggregates.

func (*ModelCatalog) GetPricingEntryForModel added in v1.1.28 

func (mc *ModelCatalog) GetPricingEntryForModel(model string, provider schemas.ModelProvider) *PricingEntry

GetPricingEntryForModel returns any pricing entry for the model across known modes. Used by the inference handler to enrich list-models responses.

func (*ModelCatalog) GetProvidersForModel 

func (mc *ModelCatalog) GetProvidersForModel(model string) []schemas.ModelProvider

GetProvidersForModel returns every provider that can serve the model. Composes across stores and applies the cross-provider special cases (openrouter / vertex / groq-gpt / bedrock-claude) preserved verbatim from the pre-refactor implementation.

func (*ModelCatalog) GetSupportedParameters added in v1.3.2 

func (mc *ModelCatalog) GetSupportedParameters(model string) []string

func (*ModelCatalog) GetUnfilteredModelsForProvider added in v1.2.22 

func (mc *ModelCatalog) GetUnfilteredModelsForProvider(provider schemas.ModelProvider) []string

GetUnfilteredModelsForProvider returns the raw catalog view (no gate applied): union of live unfiltered entries and the datasheet view.

func (*ModelCatalog) InvalidateLive added in v1.3.19 

func (mc *ModelCatalog) InvalidateLive(provider schemas.ModelProvider, keyID string)

InvalidateLive drops both filtered + unfiltered live entries for one key.

func (*ModelCatalog) InvalidateLiveProvider added in v1.3.19 

func (mc *ModelCatalog) InvalidateLiveProvider(provider schemas.ModelProvider)

InvalidateLiveProvider drops all live entries for a provider.

func (*ModelCatalog) IsModelAllowedForProvider added in v1.2.9 

func (mc *ModelCatalog) IsModelAllowedForProvider(provider schemas.ModelProvider, model string, providerConfig *configstore.ProviderConfig, allowedModels schemas.WhiteList) bool

IsModelAllowedForProvider checks whether the model is allowed for the provider given an explicit allowedModels list (used by VK governance checks, not by the static keyconfig allow set).

  • allowedModels=["*"]: defer to GetProvidersForModel (with custom-provider fast path when list-models is disabled).
  • allowedModels=[]: deny-by-default.
  • explicit allowedModels: direct or provider-prefixed match against the provider's catalog.

func (*ModelCatalog) IsRequestTypeSupported added in v1.3.2 

func (mc *ModelCatalog) IsRequestTypeSupported(model string, provider schemas.ModelProvider, requestType schemas.RequestType) bool

IsRequestTypeSupported preserves the historical (model, provider, requestType) signature; provider is ignored (the underlying datasheet index is keyed by model only).

func (*ModelCatalog) IsSameModel added in v1.2.19 

func (mc *ModelCatalog) IsSameModel(model1, model2 string) bool

func (*ModelCatalog) IsTextCompletionSupported added in v1.2.11 

func (mc *ModelCatalog) IsTextCompletionSupported(model string, provider schemas.ModelProvider) bool

func (*ModelCatalog) KeyConfigEntries added in v1.3.19 

func (mc *ModelCatalog) KeyConfigEntries(provider schemas.ModelProvider) []KeyConfigEntry

KeyConfigEntries returns the per-key entries for one provider (used by orchestration to know which keys to fan list-models calls across).

func (*ModelCatalog) KeysAllowingModel added in v1.3.19 

func (mc *ModelCatalog) KeysAllowingModel(provider schemas.ModelProvider, model string) []string

KeysAllowingModel returns the IDs of enabled keys that can serve the model.

func (*ModelCatalog) RefineModelForProvider added in v1.1.21 

func (mc *ModelCatalog) RefineModelForProvider(provider schemas.ModelProvider, model string) (string, error)

RefineModelForProvider refines a model identifier for providers that need a leading "provider/" segment (Groq, Replicate). Returns the original model unchanged when no refinement applies, or an error when multiple catalog candidates match ambiguously.

func (*ModelCatalog) ReloadFromDB added in v1.3.2 

func (mc *ModelCatalog) ReloadFromDB(ctx context.Context) error

ReloadFromDB reloads pricing + model-parameters caches from the database. Gossip handler on non-leader pods.

func (*ModelCatalog) ReloadPricing 

func (mc *ModelCatalog) ReloadPricing(ctx context.Context) error

ReloadPricing re-reads the pricing table into the in-memory cache. The management API uses this after a batched write so the new attributes are observable immediately. The 24-hour ticker still owns refreshing pricing fields from the upstream datasheet; this just refreshes the cache.

func (*ModelCatalog) RemoveKeyConfigForProvider added in v1.3.19 

func (mc *ModelCatalog) RemoveKeyConfigForProvider(provider schemas.ModelProvider)

RemoveKeyConfigForProvider drops keyconfig state for the provider.

func (*ModelCatalog) ReplaceKeyConfig added in v1.3.19 

func (mc *ModelCatalog) ReplaceKeyConfig(snapshot map[schemas.ModelProvider][]schemas.Key)

ReplaceKeyConfig atomically resets the keyconfig snapshot for all providers.

func (*ModelCatalog) ResolveAlias added in v1.3.19 

func (mc *ModelCatalog) ResolveAlias(provider schemas.ModelProvider, model string) (AliasOwner, bool)

ResolveAlias returns which key owns an alias on the provider and its AliasConfig.

func (*ModelCatalog) SetAfterSyncHook added in v1.3.2 

func (mc *ModelCatalog) SetAfterSyncHook(fn func(ctx context.Context))

SetAfterSyncHook registers a callback invoked after every successful URL → DB pricing sync. In enterprise this broadcasts a gossip message so other pods reload from DB.

func (*ModelCatalog) SetKeyConfigForProvider added in v1.3.19 

func (mc *ModelCatalog) SetKeyConfigForProvider(provider schemas.ModelProvider, keys []schemas.Key)

SetKeyConfigForProvider replaces the keyconfig snapshot for one provider.

func (*ModelCatalog) SetPricingOverrides added in v1.3.0 

func (mc *ModelCatalog) SetPricingOverrides(rows []configstoreTables.TablePricingOverride) error

func (*ModelCatalog) SetShouldSyncGate added in v1.3.2 

func (mc *ModelCatalog) SetShouldSyncGate(fn func(ctx context.Context) bool)

func (*ModelCatalog) UpdateSyncConfig added in v1.3.2 

func (mc *ModelCatalog) UpdateSyncConfig(ctx context.Context, config *Config) error

UpdateSyncConfig updates the pricing/params URLs and sync interval, restarts the background sync worker, then runs a full sync cycle.

func (*ModelCatalog) UpsertLive added in v1.3.19 

func (mc *ModelCatalog) UpsertLive(provider schemas.ModelProvider, keyID string, unfiltered bool, models []string)

UpsertLive caches one (provider, keyID, unfiltered) list-models response.

func (*ModelCatalog) UpsertLiveFromResponse added in v1.3.19 

func (mc *ModelCatalog) UpsertLiveFromResponse(provider schemas.ModelProvider, keyID string, unfiltered bool, resp *schemas.BifrostListModelsResponse)

UpsertLiveFromResponse extracts model IDs from a BifrostListModelsResponse (parsing "provider/model" prefixes, filtering by provider match, deduplicating) and pushes them into the live cache. A nil resp is a no-op so callers can't accidentally clear an existing cache entry by handing in a missing response.

func (*ModelCatalog) UpsertModelPricingAttributes added in v1.3.14 

func (mc *ModelCatalog) UpsertModelPricingAttributes(ctx context.Context, model string, provider schemas.ModelProvider, attrs map[string]string) (int64, error)

UpsertModelPricingAttributes writes additional_attributes for every row matching (model, provider) and reloads the pricing cache.

func (*ModelCatalog) UpsertPricingOverrides added in v1.3.0 

func (mc *ModelCatalog) UpsertPricingOverrides(rows ...*configstoreTables.TablePricingOverride) error

type PricingEntry 

type PricingEntry = datasheet.Entry

Type re-exports so external callers can continue importing the legacy names (PricingEntry, PricingOptions, etc.) without changing imports. Internally these live in the datasheet / keyconfig subpackages.

type PricingLookupScopes added in v1.3.0 

type PricingLookupScopes = datasheet.LookupScopes

Type re-exports so external callers can continue importing the legacy names (PricingEntry, PricingOptions, etc.) without changing imports. Internally these live in the datasheet / keyconfig subpackages.

func PricingLookupScopesFromContext added in v1.3.0 

func PricingLookupScopesFromContext(ctx *schemas.BifrostContext, provider string) *PricingLookupScopes

PricingLookupScopesFromContext is re-exported so callers don't have to change their imports.

type PricingOptions added in v1.3.0 

type PricingOptions = datasheet.Options

Type re-exports so external callers can continue importing the legacy names (PricingEntry, PricingOptions, etc.) without changing imports. Internally these live in the datasheet / keyconfig subpackages.

type PricingOverride added in v1.3.0 

type PricingOverride = datasheet.Override

Type re-exports so external callers can continue importing the legacy names (PricingEntry, PricingOptions, etc.) without changing imports. Internally these live in the datasheet / keyconfig subpackages.

type ScopeKind added in v1.3.0 

type ScopeKind = datasheet.ScopeKind

Type re-exports so external callers can continue importing the legacy names (PricingEntry, PricingOptions, etc.) without changing imports. Internally these live in the datasheet / keyconfig subpackages.

Source Files

View all Source files

Directories

Path Synopsis
Package pricing owns the pricing/model-parameters catalog (compartments A + B + E): canonical pricing rows fetched from the upstream datasheet, per-provider datasheet-derived model views, supported request types and parameters, and scoped pricing overrides.
 Package pricing owns the pricing/model-parameters catalog (compartments A + B + E): canonical pricing rows fetched from the upstream datasheet, per-provider datasheet-derived model views, supported request types and parameters, and scoped pricing overrides.
Package keyconfig caches per-key configuration (allowed/blacklisted models, aliases) for every configured provider.
 Package keyconfig caches per-key configuration (allowed/blacklisted models, aliases) for every configured provider.
Package live caches the response of provider /v1/models calls per (provider, keyID, unfiltered).
 Package live caches the response of provider /v1/models calls per (provider, keyID, unfiltered).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.