modelcatalog

package
v1.5.5 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 30, 2026 License: Apache-2.0 Imports: 21 Imported by: 9

Documentation

Overview

Package modelcatalog composes three subpackages — datasheet (pricing + model parameters + capabilities), live (per-(provider, keyID) list-models cache), and keyconfig (per-provider allow/block/aliases derived from keys) — into the ModelCatalog facade that consumers (governance, telemetry, logging, server, etc.) use.

The composer owns I/O orchestration: the hourly pricing sync ticker, the distributed lock used during sync, and the gossip after-sync hook. Subpackages perform no I/O directly — they expose Load/Sync methods the composer calls.

Editing the model pool. These methods are the push surface server.go orchestrates against — fetched list-models responses go into live, configstore key edits go into keyconfig, and the composed pool is what the read methods in models.go return.

Index

Constants

View Source 
const (
	DefaultSyncInterval           = datasheet.DefaultSyncInterval
	MinimumPricingSyncIntervalSec = int64(3600)

	// DefaultLiveModelsSyncInterval is how often the gateway re-fetches each
	// provider's list-models response in the background.
	DefaultLiveModelsSyncInterval = 60 * time.Minute
	// MinimumLiveModelsSyncIntervalSec is the floor for a non-zero live models
	// sync interval. Deliberately far below MinimumPricingSyncIntervalSec:
	// pricing is a single shared datasheet fetch on a 24h cadence, whereas an
	// operator fronting a fast-moving aggregator may legitimately want model
	// discovery to run every few minutes.
	MinimumLiveModelsSyncIntervalSec = int64(60)
	// LiveModelsSyncDisabled is the sentinel for "never refresh in the
	// background". Distinct from a negative value, which is treated as
	// corrupted config and falls back to the default.
	LiveModelsSyncDisabled = int64(0)

	ConfigLastPricingSyncKey    = "LastModelPricingSync"
	ConfigLastParamsSyncKey     = "LastModelParametersSync"
	ConfigLastMCPLibrarySyncKey = "LastMCPLibrarySync"
)
View Source 
const (
	ScopeKindGlobal                = datasheet.ScopeKindGlobal
	ScopeKindProvider              = datasheet.ScopeKindProvider
	ScopeKindProviderKey           = datasheet.ScopeKindProviderKey
	ScopeKindVirtualKey            = datasheet.ScopeKindVirtualKey
	ScopeKindVirtualKeyProvider    = datasheet.ScopeKindVirtualKeyProvider
	ScopeKindVirtualKeyProviderKey = datasheet.ScopeKindVirtualKeyProviderKey
	ScopeKindUser                  = datasheet.ScopeKindUser
	ScopeKindUserProvider          = datasheet.ScopeKindUserProvider
	ScopeKindUserProviderKey       = datasheet.ScopeKindUserProviderKey

	MatchTypeExact    = datasheet.MatchTypeExact
	MatchTypeWildcard = datasheet.MatchTypeWildcard
)

Scope kind constants re-exported for callers that compare by value.

View Source 
const (
	DefaultPricingURL             = datasheet.DefaultURL
	DefaultModelParametersURL     = datasheet.DefaultModelParametersURL
	DefaultPricingTimeout         = datasheet.DefaultPricingTimeout
	DefaultModelParametersTimeout = datasheet.DefaultModelParametersTimeout

	DefaultMCPLibraryURL     = "https://getbifrost.ai/mcp-library"
	DefaultMCPLibraryTimeout = 45 * time.Second
)

Sync timing defaults re-exported from datasheet for consumers of the historical constants.

Variables

This section is empty.

Functions

func ApplyModelInfo added in v1.5.5 

func ApplyModelInfo(model *schemas.Model, entry *PricingEntry)

ApplyModelInfo merges catalog metadata from entry into model, filling only fields the caller has not already populated. Provider-reported values always win: the catalog is a backfill for what a provider's list-models response leaves out, never an override of what it did report.

Exported so the list-models handlers can enrich provider responses through the same mapping that GetModelInfo uses.

Every reference-typed field is cloned on the way in. An Entry's maps, pointers and slices alias the datasheet's shared pricing row (reading a struct out of that map is a shallow copy), so assigning them straight through would hand callers - including third-party plugins via ctx.GetModelInfo - a live handle into catalog state. A write through such a handle silently rewrites the catalog for every later request, and racing the pricing sync on the map is a fatal concurrent map access, not a recoverable panic.

func Slugify added in v1.3.19 

func Slugify(name string) string

Slugify derives a URL/identifier-safe slug from a display name: lowercase, non-alphanumeric runs collapsed to a single "-", and leading/trailing "-" trimmed. Used to key custom library entries off their name so the existing unique slug index detects duplicates. Returns "" for names with no alphanumeric content (the caller rejects an empty slug).

NOTE: only ASCII letters and digits are retained — accented/unicode characters (e.g. "données") are stripped, which may produce unexpected slugs for non-ASCII names. This is intentional to keep slugs URL-safe without a transliteration dependency; callers should be aware of this limitation.

func SyncMCPLibrary added in v1.3.19 

func SyncMCPLibrary(ctx context.Context, url string, store configstore.ConfigStore) (int, error)

SyncMCPLibrary fetches the MCP server catalog from url, parses the JSON payload, and upserts each row into the mcp_library table keyed by slug. Returns the number of rows upserted.

The function is intentionally stateless and operates directly on the ConfigStore so it can be called from both the force-sync handler and the background worker without needing a dedicated manager struct.

Types

type AliasOwner added in v1.3.19 

type AliasOwner = keyconfig.AliasOwner

Type re-exports so external callers can continue importing the legacy names (PricingEntry, PricingOptions, etc.) without changing imports. Internally these live in the datasheet / keyconfig subpackages.

type Config 

type Config struct {
	PricingURL          *string `json:"pricing_url,omitempty"`
	PricingSyncInterval *int64  `json:"pricing_sync_interval,omitempty"` // seconds
	ModelParametersURL  *string `json:"model_parameters_url,omitempty"`

	// MCPLibraryURL overrides the endpoint the MCP server library catalog is
	// synced from. Empty/nil uses DefaultMCPLibraryURL. Mirrors PricingURL: the
	// default ships out of the box and the user can point it at a custom source.
	MCPLibraryURL          *string `json:"mcp_library_url,omitempty"`
	MCPLibrarySyncInterval *int64  `json:"mcp_library_sync_interval,omitempty"` // seconds

	// LiveModelsSyncInterval is how often each provider's list-models response
	// is re-fetched in the background, in seconds. Nil uses
	// DefaultLiveModelsSyncInterval; LiveModelsSyncDisabled (0) turns the
	// background refresher off entirely.
	//
	// Unlike the sync intervals above, nothing in this package acts on it: the
	// live model cache is per-process and the refresh has to go through the
	// Bifrost routing pipeline, so the HTTP server owns that ticker. This field
	// is the transport's configuration channel, carried here to keep every
	// catalog cadence in one place.
	LiveModelsSyncInterval *int64 `json:"live_models_sync_interval,omitempty"` // seconds
}

Config is the model pricing configuration.

type KeyConfigEntry added in v1.3.19 

type KeyConfigEntry = keyconfig.KeyEntry

Type re-exports so external callers can continue importing the legacy names (PricingEntry, PricingOptions, etc.) without changing imports. Internally these live in the datasheet / keyconfig subpackages.

type MCPLibraryEntry added in v1.3.19 

type MCPLibraryEntry struct {
	Name               string                    `json:"name"`
	Description        string                    `json:"description,omitempty"`
	Category           string                    `json:"category,omitempty"`
	ConnectionType     schemas.MCPConnectionType `json:"connection_type"`
	ConnectionURL      string                    `json:"connection_url,omitempty"`
	StdioConfig        *schemas.MCPStdioConfig   `json:"stdio_config,omitempty"`
	AuthType           schemas.MCPAuthType       `json:"auth_type,omitempty"`
	RequiredHeaderKeys []string                  `json:"required_header_keys,omitempty"`
	IconURL            string                    `json:"icon_url,omitempty"`
	DocsURL            string                    `json:"docs_url,omitempty"`
	Publisher          string                    `json:"publisher,omitempty"`
	Tags               []string                  `json:"tags,omitempty"`
	Metadata           map[string]any            `json:"metadata,omitempty"`
}

MCPLibraryEntry is the JSON shape a single server has in the remote MCP library catalog (the payload fetched from DefaultMCPLibraryURL / custom URL). The catalog carries no slug; it is derived from Name at sync time via Slugify. The remaining fields map onto TableMCPLibrary minus the DB-managed fields (ID, Slug, CreatedAt, UpdatedAt).

type MCPLibraryPayload added in v1.3.19 

type MCPLibraryPayload struct {
	Servers       []MCPLibraryEntry `json:"servers"`
	LastUpdatedAt string            `json:"lastUpdatedAt,omitempty"`
}

MCPLibraryPayload is the top-level JSON envelope returned by the remote MCP library catalog endpoint.

type MatchType added in v1.3.0 

type MatchType = datasheet.MatchType

Type re-exports so external callers can continue importing the legacy names (PricingEntry, PricingOptions, etc.) without changing imports. Internally these live in the datasheet / keyconfig subpackages.

type ModelCatalog 

type ModelCatalog struct {
	// contains filtered or unexported fields
}

func Init 

func Init(ctx context.Context, config *Config, configStore configstore.ConfigStore, logger schemas.Logger) (*ModelCatalog, error)

func NewTestCatalog added in v1.2.19 

func NewTestCatalog(baseModelIndex map[string]string) *ModelCatalog

NewTestCatalog constructs a minimal ModelCatalog for unit tests. Does not start background workers or hit external services.

func NewTestCatalogWithDatasheet added in v1.3.22 

func NewTestCatalogWithDatasheet(ds *datasheet.Store) *ModelCatalog

NewTestCatalogWithDatasheet wraps a caller-provided datasheet.Store (e.g. one loaded from a local testdata pricing file via datasheet.New(...) + LoadFromURLIntoMemory) in a ModelCatalog, so tests in other packages can exercise real pricing/cost computation without reaching the network.

func (*ModelCatalog) AllowedModelsForProvider added in v1.3.19 

func (mc *ModelCatalog) AllowedModelsForProvider(provider schemas.ModelProvider) schemas.WhiteList

AllowedModelsForProvider returns the aggregated whitelist for the provider (union of enabled keys' Models minus per-key blacklists, or ["*"] when any key is unrestricted). Used by the load balancer to know what each provider can serve without re-walking the configstore.

func (*ModelCatalog) BlacklistedModelsForProvider added in v1.3.19 

func (mc *ModelCatalog) BlacklistedModelsForProvider(provider schemas.ModelProvider) schemas.BlackList

BlacklistedModelsForProvider returns the intersection of enabled keys' BlacklistedModels for the provider — a model is included only when every enabled key blacklists it.

func (*ModelCatalog) CalculateCost 

func (mc *ModelCatalog) CalculateCost(result *schemas.BifrostResponse, scopes *PricingLookupScopes) float64

CalculateCost computes the dollar cost for a Bifrost response.

func (*ModelCatalog) CalculateCostForUsage added in v1.3.22 

func (mc *ModelCatalog) CalculateCostForUsage(usage *schemas.BifrostLLMUsage, provider schemas.ModelProvider, model string, requestType schemas.RequestType, scopes *PricingLookupScopes) float64

CalculateCostForUsage computes the dollar cost from a bare usage object when no full BifrostResponse is available — used to bill partial usage carried on a failed/cancelled request (BifrostError.ExtraFields.BilledUsage).

func (*ModelCatalog) CalculateRequestCost added in v1.5.5 

func (mc *ModelCatalog) CalculateRequestCost(ctx *schemas.BifrostContext, resp *schemas.BifrostResponse) float64

CalculateRequestCost returns the dollar cost of resp, resolving governance pricing overrides (virtual key / user / provider key scopes) from ctx.

This is the plugin-facing entry point behind ctx.CalculateCost. It is a thin wrapper over CalculateCost so tiered pricing, batch/priority/flex rates and the provider/mode fallback chain all apply unchanged.

Must be called synchronously while ctx is still live; the scope lookup reads request identity off the context.

func (*ModelCatalog) Cleanup 

func (mc *ModelCatalog) Cleanup() error

func (*ModelCatalog) ConfiguredProviders added in v1.3.19 

func (mc *ModelCatalog) ConfiguredProviders() []schemas.ModelProvider

ConfiguredProviders returns every provider with at least one entry in keyconfig. Used by the load balancer's provider selection where the configured-provider set is the routing-eligible universe.

func (*ModelCatalog) DeletePricingOverride added in v1.3.0 

func (mc *ModelCatalog) DeletePricingOverride(id string)

func (*ModelCatalog) ForceReloadMCPLibrary added in v1.3.19 

func (mc *ModelCatalog) ForceReloadMCPLibrary(ctx context.Context) (int, error)

ForceReloadMCPLibrary triggers an immediate MCP library sync from the current configured source and advances the MCP library sync timer on success.

func (*ModelCatalog) ForceReloadPricing added in v1.1.48 

func (mc *ModelCatalog) ForceReloadPricing(ctx context.Context) error

ForceReloadPricing triggers an immediate URL→DB→memory sync for pricing and model parameters in parallel, fires the gossip hook, and resets the ticker so the next scheduled sync waits a full interval from now.

Behavior change from pre-refactor: this no longer touches the live list-models cache. List-models refresh is now driven by key/provider edits, not by pricing reloads.

func (*ModelCatalog) GetBaseModelName added in v1.2.19 

func (mc *ModelCatalog) GetBaseModelName(model string) string

func (*ModelCatalog) GetDistinctBaseModelNames added in v1.2.19 

func (mc *ModelCatalog) GetDistinctBaseModelNames() []string

GetDistinctBaseModelNames returns all unique base model names from the datasheet. Used by governance for cross-provider model selection.

func (*ModelCatalog) GetModelCapabilityEntryForModel added in v1.2.35 

func (mc *ModelCatalog) GetModelCapabilityEntryForModel(model string, provider schemas.ModelProvider) *PricingEntry

GetModelCapabilityEntryForModel returns capability metadata for a (model, provider) pair. Prefers chat, then responses, then text-completion entries; falls back to the lexicographically first available mode for deterministic behavior.

func (*ModelCatalog) GetModelInfo added in v1.5.5 

func (mc *ModelCatalog) GetModelInfo(provider schemas.ModelProvider, model string) *schemas.Model

GetModelInfo returns pricing and capability metadata for a (provider, model) pair in the same shape the /v1/models endpoint reports, or nil when the catalog has no entry for it.

Lookup order is pricing-row first (exact model+provider across every request mode), then the capability entry, which additionally resolves through the canonical base model and the wider model family. That ordering keeps dated model IDs like "gpt-4o-2024-08-06" resolvable even when only the family row carries capability data.

The returned *schemas.Model is freshly allocated and owned by the caller.

func (*ModelCatalog) GetModelsForProvider 

func (mc *ModelCatalog) GetModelsForProvider(provider schemas.ModelProvider) []string

GetModelsForProvider returns the effective allowed model set for the provider. Filtered live entries are authoritative when present (they were pre-gated by ListModelsPipeline against the key's allow/block/aliases); otherwise the datasheet view is filtered by the keyconfig aggregates.

func (*ModelCatalog) GetPricingEntryForModel added in v1.1.28 

func (mc *ModelCatalog) GetPricingEntryForModel(model string, provider schemas.ModelProvider) *PricingEntry

GetPricingEntryForModel returns any pricing entry for the model across known modes. Used by the inference handler to enrich list-models responses.

func (*ModelCatalog) GetProvidersForModel 

func (mc *ModelCatalog) GetProvidersForModel(model string) []schemas.ModelProvider

GetProvidersForModel returns every provider that can serve the model. Composes across stores and applies the cross-provider special cases (openrouter / vertex / groq-gpt / bedrock-claude) preserved verbatim from the pre-refactor implementation.

func (*ModelCatalog) GetSupportedParameters added in v1.3.2 

func (mc *ModelCatalog) GetSupportedParameters(model string) []string

func (*ModelCatalog) GetUnfilteredModelsForProvider added in v1.2.22 

func (mc *ModelCatalog) GetUnfilteredModelsForProvider(provider schemas.ModelProvider) []string

GetUnfilteredModelsForProvider returns the raw catalog view (no gate applied): union of live unfiltered entries and the datasheet view.

func (*ModelCatalog) InvalidateLive added in v1.3.19 

func (mc *ModelCatalog) InvalidateLive(provider schemas.ModelProvider, keyID string)

InvalidateLive drops both filtered + unfiltered live entries for one key.

func (*ModelCatalog) InvalidateLiveProvider added in v1.3.19 

func (mc *ModelCatalog) InvalidateLiveProvider(provider schemas.ModelProvider)

InvalidateLiveProvider drops all live entries for a provider.

func (*ModelCatalog) IsModelAllowedForProvider added in v1.2.9 

func (mc *ModelCatalog) IsModelAllowedForProvider(provider schemas.ModelProvider, model string, providerConfig *configstore.ProviderConfig, allowedModels schemas.WhiteList) bool

IsModelAllowedForProvider checks whether the model is allowed for the provider given an explicit allowedModels list (used by VK governance checks, not by the static keyconfig allow set).

  • allowedModels=["*"]: defer to GetProvidersForModel (with custom-provider fast path when list-models is disabled).
  • allowedModels=[]: deny-by-default.
  • explicit allowedModels: direct or provider-prefixed match against the provider's catalog.

func (*ModelCatalog) IsRequestTypeSupported added in v1.3.2 

func (mc *ModelCatalog) IsRequestTypeSupported(model string, provider schemas.ModelProvider, requestType schemas.RequestType) bool

IsRequestTypeSupported preserves the historical (model, provider, requestType) signature; provider is ignored (the underlying datasheet index is keyed by model only).

func (*ModelCatalog) IsSameModel added in v1.2.19 

func (mc *ModelCatalog) IsSameModel(model1, model2 string) bool

func (*ModelCatalog) IsTextCompletionSupported added in v1.2.11 

func (mc *ModelCatalog) IsTextCompletionSupported(model string, provider schemas.ModelProvider) bool

func (*ModelCatalog) KeyConfigEntries added in v1.3.19 

func (mc *ModelCatalog) KeyConfigEntries(provider schemas.ModelProvider) []KeyConfigEntry

KeyConfigEntries returns the per-key entries for one provider (used by orchestration to know which keys to fan list-models calls across).

func (*ModelCatalog) KeysAllowingModel added in v1.3.19 

func (mc *ModelCatalog) KeysAllowingModel(provider schemas.ModelProvider, model string) []string

KeysAllowingModel returns the IDs of enabled keys that can serve the model.

func (*ModelCatalog) LiveGeneration added in v1.5.5 

func (mc *ModelCatalog) LiveGeneration(provider schemas.ModelProvider) uint64

LiveGeneration returns the live cache's invalidation counter for the provider. Read it before issuing a list-models fetch and pass it back to UpsertLiveFromResponseIfCurrent when the response lands.

func (*ModelCatalog) RefineModelForProvider added in v1.1.21 

func (mc *ModelCatalog) RefineModelForProvider(provider schemas.ModelProvider, model string) (string, error)

RefineModelForProvider refines a model identifier for providers whose catalog names carry a leading "provider/" segment (Groq, Replicate, Perplexity, OpenRouter), resolving a bare request like "gpt-oss-120b" to the provider's catalog slug ("openai/gpt-oss-120b"). Returns the original model unchanged when no refinement applies, or an error when multiple catalog candidates match ambiguously.

Idempotent: a model that already carries a known provider prefix is the refined form itself — routing plugins may refine the same request more than once, so it is returned unchanged without a catalog scan. The one exception is a model carrying the TARGET provider's own prefix (e.g. a fallback entry built from another provider's refined form, or a double-prefixed input): the prefix is stripped and the bare remainder re-refined, so canonical own-prefixed names ("perplexity/sonar", "openrouter/auto") round-trip through the catalog scan unchanged.

func (*ModelCatalog) ReloadFromDB added in v1.3.2 

func (mc *ModelCatalog) ReloadFromDB(ctx context.Context) error

ReloadFromDB reloads pricing + model-parameters caches from the database. Gossip handler on non-leader pods.

func (*ModelCatalog) ReloadPricing 

func (mc *ModelCatalog) ReloadPricing(ctx context.Context) error

ReloadPricing re-reads the pricing table into the in-memory cache. The management API uses this after a batched write so the new attributes are observable immediately. The 24-hour ticker still owns refreshing pricing fields from the upstream datasheet; this just refreshes the cache.

func (*ModelCatalog) RemoveKeyConfigForProvider added in v1.3.19 

func (mc *ModelCatalog) RemoveKeyConfigForProvider(provider schemas.ModelProvider)

RemoveKeyConfigForProvider drops keyconfig state for the provider.

func (*ModelCatalog) ReplaceKeyConfig added in v1.3.19 

func (mc *ModelCatalog) ReplaceKeyConfig(snapshot map[schemas.ModelProvider][]schemas.Key)

ReplaceKeyConfig atomically resets the keyconfig snapshot for all providers.

func (*ModelCatalog) ResolveAlias added in v1.3.19 

func (mc *ModelCatalog) ResolveAlias(provider schemas.ModelProvider, model string) (AliasOwner, bool)

ResolveAlias returns which key owns an alias on the provider and its AliasConfig.

func (*ModelCatalog) ResolveModelParameters added in v1.5.5 

func (mc *ModelCatalog) ResolveModelParameters(ctx context.Context, model string) (*configstoreTables.TableModelParameters, error)

ResolveModelParameters reads the model-parameters row for model, resolving provider-qualified or bare aliases to the datasheet's stored key (exact → provider-prefix-stripped → base model → provider-qualified variants).

func (*ModelCatalog) RetainLiveKeys added in v1.5.5 

func (mc *ModelCatalog) RetainLiveKeys(provider schemas.ModelProvider, keep map[string]struct{})

RetainLiveKeys prunes the provider's live entries down to the keys in keep, leaving those keys' cached models in place. Used before a re-fetch so a failed list-models call cannot empty a catalog that was valid a moment ago.

func (*ModelCatalog) SetAfterSyncHook added in v1.3.2 

func (mc *ModelCatalog) SetAfterSyncHook(fn func(ctx context.Context))

SetAfterSyncHook registers a callback invoked after every successful URL → DB pricing sync. In enterprise this broadcasts a gossip message so other pods reload from DB.

func (*ModelCatalog) SetKeyConfigForProvider added in v1.3.19 

func (mc *ModelCatalog) SetKeyConfigForProvider(provider schemas.ModelProvider, keys []schemas.Key)

SetKeyConfigForProvider replaces the keyconfig snapshot for one provider.

func (*ModelCatalog) SetPricingOverrides added in v1.3.0 

func (mc *ModelCatalog) SetPricingOverrides(rows []configstoreTables.TablePricingOverride) error

func (*ModelCatalog) SetShouldSyncGate added in v1.3.2 

func (mc *ModelCatalog) SetShouldSyncGate(fn func(ctx context.Context) bool)

func (*ModelCatalog) UpdateSyncConfig added in v1.3.2 

func (mc *ModelCatalog) UpdateSyncConfig(ctx context.Context, config *Config) error

UpdateSyncConfig updates the pricing/params URLs and sync interval, restarts the background sync worker, then runs a full sync cycle.

func (*ModelCatalog) UpsertLive added in v1.3.19 

func (mc *ModelCatalog) UpsertLive(provider schemas.ModelProvider, keyID string, unfiltered bool, models []string)

UpsertLive caches one (provider, keyID, unfiltered) list-models response.

func (*ModelCatalog) UpsertLiveFromResponse added in v1.3.19 

func (mc *ModelCatalog) UpsertLiveFromResponse(provider schemas.ModelProvider, keyID string, unfiltered bool, resp *schemas.BifrostListModelsResponse)

UpsertLiveFromResponse extracts model IDs from a BifrostListModelsResponse (parsing "provider/model" prefixes, filtering by provider match, deduplicating) and pushes them into the live cache. A nil resp is a no-op so callers can't accidentally clear an existing cache entry by handing in a missing response.

func (*ModelCatalog) UpsertLiveFromResponseIfCurrent added in v1.5.5 

func (mc *ModelCatalog) UpsertLiveFromResponseIfCurrent(provider schemas.ModelProvider, keyID string, unfiltered bool, resp *schemas.BifrostListModelsResponse, gen uint64) bool

UpsertLiveFromResponseIfCurrent is UpsertLiveFromResponse with a staleness guard: the write is dropped, and false returned, if the provider was invalidated (key deleted or disabled, provider removed, entries pruned) after gen was read. A nil resp is a no-op and also reports false, matching UpsertLiveFromResponse's "never clear a cache entry with a missing response" contract.

func (*ModelCatalog) UpsertModelPricingAttributes added in v1.3.14 

func (mc *ModelCatalog) UpsertModelPricingAttributes(ctx context.Context, model string, provider schemas.ModelProvider, attrs map[string]string) (int64, error)

UpsertModelPricingAttributes writes additional_attributes for every row matching (model, provider) and reloads the pricing cache.

func (*ModelCatalog) UpsertPricingOverrides added in v1.3.0 

func (mc *ModelCatalog) UpsertPricingOverrides(rows ...*configstoreTables.TablePricingOverride) error

type PricingEntry 

type PricingEntry = datasheet.Entry

Type re-exports so external callers can continue importing the legacy names (PricingEntry, PricingOptions, etc.) without changing imports. Internally these live in the datasheet / keyconfig subpackages.

type PricingLookupScopes added in v1.3.0 

type PricingLookupScopes = datasheet.LookupScopes

Type re-exports so external callers can continue importing the legacy names (PricingEntry, PricingOptions, etc.) without changing imports. Internally these live in the datasheet / keyconfig subpackages.

func PricingLookupScopesFromContext added in v1.3.0 

func PricingLookupScopesFromContext(ctx *schemas.BifrostContext, provider string) *PricingLookupScopes

PricingLookupScopesFromContext is re-exported so callers don't have to change their imports.

type PricingOptions added in v1.3.0 

type PricingOptions = datasheet.Options

Type re-exports so external callers can continue importing the legacy names (PricingEntry, PricingOptions, etc.) without changing imports. Internally these live in the datasheet / keyconfig subpackages.

type PricingOverride added in v1.3.0 

type PricingOverride = datasheet.Override

Type re-exports so external callers can continue importing the legacy names (PricingEntry, PricingOptions, etc.) without changing imports. Internally these live in the datasheet / keyconfig subpackages.

type ScopeKind added in v1.3.0 

type ScopeKind = datasheet.ScopeKind

Type re-exports so external callers can continue importing the legacy names (PricingEntry, PricingOptions, etc.) without changing imports. Internally these live in the datasheet / keyconfig subpackages.

Source Files

View all Source files

Directories

Path Synopsis
Package pricing owns the pricing/model-parameters catalog (compartments A + B + E): canonical pricing rows fetched from the upstream datasheet, per-provider datasheet-derived model views, supported request types and parameters, and scoped pricing overrides.
 Package pricing owns the pricing/model-parameters catalog (compartments A + B + E): canonical pricing rows fetched from the upstream datasheet, per-provider datasheet-derived model views, supported request types and parameters, and scoped pricing overrides.
Package keyconfig caches per-key configuration (allowed/blacklisted models, aliases) for every configured provider.
 Package keyconfig caches per-key configuration (allowed/blacklisted models, aliases) for every configured provider.
Package live caches the response of provider /v1/models calls per (provider, keyID, unfiltered).
 Package live caches the response of provider /v1/models calls per (provider, keyID, unfiltered).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.