modelcatalog

package
v1.2.10 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 15, 2026 License: Apache-2.0 Imports: 13 Imported by: 6

Documentation

Overview

Package modelcatalog provides a pricing manager for the framework.

Index

Constants

View Source 
const (
	DefaultPricingSyncInterval = 24 * time.Hour
	ConfigLastPricingSyncKey   = "LastModelPricingSync"
	DefaultPricingURL          = "https://getbifrost.ai/datasheet"
	DefaultPricingTimeout      = 45 * time.Second
)
View Source 
const (
	TokenTierAbove128K = 128000
	TokenTierAbove200K = 200000
)

Default sync interval and config key

Variables

This section is empty.

Functions

This section is empty.

Types

type Config 

type Config struct {
	PricingURL          *string        `json:"pricing_url,omitempty"`
	PricingSyncInterval *time.Duration `json:"pricing_sync_interval,omitempty"`
}

Config is the model pricing configuration.

type ModelCatalog 

type ModelCatalog struct {
	// contains filtered or unexported fields
}

func Init 

func Init(ctx context.Context, config *Config, configStore configstore.ConfigStore, shouldSyncPricingFunc ShouldSyncPricingFunc, logger schemas.Logger) (*ModelCatalog, error)

Init initializes the pricing manager

func (*ModelCatalog) AddModelDataToPool 

func (mc *ModelCatalog) AddModelDataToPool(modelData *schemas.BifrostListModelsResponse)

AddModelDataToPool adds model data to the model pool.

func (*ModelCatalog) CalculateCost 

func (mc *ModelCatalog) CalculateCost(result *schemas.BifrostResponse) float64

CalculateCost calculates the cost of a Bifrost response

func (*ModelCatalog) CalculateCostFromUsage 

func (mc *ModelCatalog) CalculateCostFromUsage(provider string, model string, deployment string, usage *schemas.BifrostLLMUsage, requestType schemas.RequestType, isBatch bool, audioSeconds *int, audioTokenDetails *schemas.TranscriptionUsageInputTokenDetails, imageUsage *schemas.ImageUsage) float64

CalculateCostFromUsage calculates cost in dollars using pricing manager and usage data with conditional pricing

func (*ModelCatalog) CalculateCostWithCacheDebug 

func (mc *ModelCatalog) CalculateCostWithCacheDebug(result *schemas.BifrostResponse) float64

CalculateCostWithCacheDebug calculates the cost of a Bifrost response with cache debug information

func (*ModelCatalog) Cleanup 

func (mc *ModelCatalog) Cleanup() error

Cleanup cleans up the model catalog

func (*ModelCatalog) DeleteModelDataForProvider added in v1.1.21 

func (mc *ModelCatalog) DeleteModelDataForProvider(provider schemas.ModelProvider)

DeleteModelDataForProvider deletes all model data from the pool for a given provider

func (*ModelCatalog) ForceReloadPricing added in v1.1.48 

func (mc *ModelCatalog) ForceReloadPricing(ctx context.Context) error

func (*ModelCatalog) GetModelsForProvider 

func (mc *ModelCatalog) GetModelsForProvider(provider schemas.ModelProvider) []string

GetModelsForProvider returns all available models for a given provider (thread-safe)

func (*ModelCatalog) GetPricingEntryForModel added in v1.1.28 

func (mc *ModelCatalog) GetPricingEntryForModel(model string, provider schemas.ModelProvider) *PricingEntry

GetPricingEntryForModel returns the pricing data

func (*ModelCatalog) GetProvidersForModel 

func (mc *ModelCatalog) GetProvidersForModel(model string) []schemas.ModelProvider

GetProvidersForModel returns all providers for a given model (thread-safe)

func (*ModelCatalog) IsModelAllowedForProvider added in v1.2.9 

func (mc *ModelCatalog) IsModelAllowedForProvider(provider schemas.ModelProvider, model string, allowedModels []string) bool

IsModelAllowedForProvider checks if a model is allowed for a specific provider based on the allowed models list and catalog data. It handles all cross-provider logic including provider-prefixed models and special routing rules.

Parameters:

  • provider: The provider to check against
  • model: The model name (without provider prefix, e.g., "gpt-4o" or "claude-3-5-sonnet")
  • allowedModels: List of allowed model names (can be empty, can include provider prefixes)

Behavior:

  • If allowedModels is empty: Uses model catalog to check if provider supports the model (delegates to GetProvidersForModel which handles all cross-provider logic)
  • If allowedModels is not empty: Checks if model matches any entry in the list Provider-specific validation:
  • Direct matches: "gpt-4o" in allowedModels for any provider
  • Prefixed matches: Only if the prefixed model exists in provider's catalog (e.g., "openai/gpt-4o" in allowedModels only matches if openrouter's catalog contains "openai/gpt-4o" AND the model part matches the request)

Returns:

  • bool: true if the model is allowed for the provider, false otherwise

Examples: 

// Empty allowedModels - uses catalog
mc.IsModelAllowedForProvider("openrouter", "claude-3-5-sonnet", []string{})
// Returns: true (catalog knows openrouter has "anthropic/claude-3-5-sonnet")

// Explicit allowedModels with prefix - validates against catalog
mc.IsModelAllowedForProvider("openrouter", "gpt-4o", []string{"openai/gpt-4o"})
// Returns: true (openrouter's catalog contains "openai/gpt-4o" AND model part is "gpt-4o")

// Explicit allowedModels with prefix - wrong model
mc.IsModelAllowedForProvider("openrouter", "claude-3-5-sonnet", []string{"openai/gpt-4o"})
// Returns: false (model part "gpt-4o" doesn't match request "claude-3-5-sonnet")

// Explicit allowedModels without prefix
mc.IsModelAllowedForProvider("openai", "gpt-4o", []string{"gpt-4o"})
// Returns: true (direct match)

func (*ModelCatalog) RefineModelForProvider added in v1.1.21 

func (mc *ModelCatalog) RefineModelForProvider(provider schemas.ModelProvider, model string) string

RefineModelForProvider refines the model for a given provider. e.g. "gpt-oss-120b" for groq provider -> "openai/gpt-oss-120b"

func (*ModelCatalog) ReloadPricing 

func (mc *ModelCatalog) ReloadPricing(ctx context.Context, config *Config) error

ReloadPricing reloads the pricing manager from config

type PricingEntry 

type PricingEntry struct {
	// Basic pricing
	InputCostPerToken  float64 `json:"input_cost_per_token"`
	OutputCostPerToken float64 `json:"output_cost_per_token"`
	Provider           string  `json:"provider"`
	Mode               string  `json:"mode"`
	// Additional pricing for media
	InputCostPerVideoPerSecond *float64 `json:"input_cost_per_video_per_second,omitempty"`
	InputCostPerAudioPerSecond *float64 `json:"input_cost_per_audio_per_second,omitempty"`
	// Character-based pricing
	InputCostPerCharacter  *float64 `json:"input_cost_per_character,omitempty"`
	OutputCostPerCharacter *float64 `json:"output_cost_per_character,omitempty"`
	// Pricing above 128k tokens
	InputCostPerTokenAbove128kTokens          *float64 `json:"input_cost_per_token_above_128k_tokens,omitempty"`
	InputCostPerCharacterAbove128kTokens      *float64 `json:"input_cost_per_character_above_128k_tokens,omitempty"`
	InputCostPerImageAbove128kTokens          *float64 `json:"input_cost_per_image_above_128k_tokens,omitempty"`
	InputCostPerVideoPerSecondAbove128kTokens *float64 `json:"input_cost_per_video_per_second_above_128k_tokens,omitempty"`
	InputCostPerAudioPerSecondAbove128kTokens *float64 `json:"input_cost_per_audio_per_second_above_128k_tokens,omitempty"`
	OutputCostPerTokenAbove128kTokens         *float64 `json:"output_cost_per_token_above_128k_tokens,omitempty"`
	OutputCostPerCharacterAbove128kTokens     *float64 `json:"output_cost_per_character_above_128k_tokens,omitempty"`
	//Pricing above 200k tokens
	InputCostPerTokenAbove200kTokens           *float64 `json:"input_cost_per_token_above_200k_tokens,omitempty"`
	OutputCostPerTokenAbove200kTokens          *float64 `json:"output_cost_per_token_above_200k_tokens,omitempty"`
	CacheCreationInputTokenCostAbove200kTokens *float64 `json:"cache_creation_input_token_cost_above_200k_tokens,omitempty"`
	CacheReadInputTokenCostAbove200kTokens     *float64 `json:"cache_read_input_token_cost_above_200k_tokens,omitempty"`
	// Cache and batch pricing
	CacheReadInputTokenCost   *float64 `json:"cache_read_input_token_cost,omitempty"`
	InputCostPerTokenBatches  *float64 `json:"input_cost_per_token_batches,omitempty"`
	OutputCostPerTokenBatches *float64 `json:"output_cost_per_token_batches,omitempty"`
	// Image generation pricing
	InputCostPerImageToken       *float64 `json:"input_cost_per_image_token,omitempty"`
	OutputCostPerImageToken      *float64 `json:"output_cost_per_image_token,omitempty"`
	InputCostPerImage            *float64 `json:"input_cost_per_image,omitempty"`
	OutputCostPerImage           *float64 `json:"output_cost_per_image,omitempty"`
	CacheReadInputImageTokenCost *float64 `json:"cache_read_input_image_token_cost,omitempty"`
}

PricingEntry represents a single model's pricing information

type ShouldSyncPricingFunc added in v1.2.0 

type ShouldSyncPricingFunc func(ctx context.Context) bool

ShouldSyncPricingFunc is a function that determines if pricing data should be synced It returns a boolean indicating if syncing is needed It is completely optional and can be nil if not needed syncPricing function will be called if this function returns true

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.