modelcatalog

package
v1.1.59 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 6, 2026 License: Apache-2.0 Imports: 13 Imported by: 6

Documentation

Overview

Package modelcatalog provides a pricing manager for the framework.

Index

Constants

View Source 
const (
	DefaultPricingSyncInterval = 24 * time.Hour
	ConfigLastPricingSyncKey   = "LastModelPricingSync"
	DefaultPricingURL          = "https://getbifrost.ai/datasheet"
	DefaultPricingTimeout      = 45 * time.Second
)
View Source 
const (
	TokenTierAbove128K = 128000
	TokenTierAbove200K = 200000
)

Default sync interval and config key

Variables

This section is empty.

Functions

This section is empty.

Types

type Config 

type Config struct {
	PricingURL          *string        `json:"pricing_url,omitempty"`
	PricingSyncInterval *time.Duration `json:"pricing_sync_interval,omitempty"`
}

Config is the model pricing configuration.

type ModelCatalog 

type ModelCatalog struct {
	// contains filtered or unexported fields
}

func Init 

func Init(ctx context.Context, config *Config, configStore configstore.ConfigStore, logger schemas.Logger) (*ModelCatalog, error)

Init initializes the pricing manager

func (*ModelCatalog) AddModelDataToPool 

func (mc *ModelCatalog) AddModelDataToPool(modelData *schemas.BifrostListModelsResponse)

AddModelDataToPool adds model data to the model pool.

func (*ModelCatalog) CalculateCost 

func (mc *ModelCatalog) CalculateCost(result *schemas.BifrostResponse) float64

CalculateCost calculates the cost of a Bifrost response

func (*ModelCatalog) CalculateCostFromUsage 

func (mc *ModelCatalog) CalculateCostFromUsage(provider string, model string, deployment string, usage *schemas.BifrostLLMUsage, requestType schemas.RequestType, isBatch bool, audioSeconds *int, audioTokenDetails *schemas.TranscriptionUsageInputTokenDetails) float64

CalculateCostFromUsage calculates cost in dollars using pricing manager and usage data with conditional pricing

func (*ModelCatalog) CalculateCostWithCacheDebug 

func (mc *ModelCatalog) CalculateCostWithCacheDebug(result *schemas.BifrostResponse) float64

CalculateCostWithCacheDebug calculates the cost of a Bifrost response with cache debug information

func (*ModelCatalog) Cleanup 

func (mc *ModelCatalog) Cleanup() error

Cleanup cleans up the model catalog

func (*ModelCatalog) DeleteModelDataForProvider added in v1.1.21 

func (mc *ModelCatalog) DeleteModelDataForProvider(provider schemas.ModelProvider)

DeleteModelDataForProvider deletes all model data from the pool for a given provider

func (*ModelCatalog) ForceReloadPricing added in v1.1.48 

func (mc *ModelCatalog) ForceReloadPricing(ctx context.Context) error

func (*ModelCatalog) GetModelsForProvider 

func (mc *ModelCatalog) GetModelsForProvider(provider schemas.ModelProvider) []string

GetModelsForProvider returns all available models for a given provider (thread-safe)

func (*ModelCatalog) GetPricingEntryForModel added in v1.1.28 

func (mc *ModelCatalog) GetPricingEntryForModel(model string, provider schemas.ModelProvider) *PricingEntry

GetPricingEntryForModel returns the pricing data

func (*ModelCatalog) GetProvidersForModel 

func (mc *ModelCatalog) GetProvidersForModel(model string) []schemas.ModelProvider

GetProvidersForModel returns all providers for a given model (thread-safe)

func (*ModelCatalog) RefineModelForProvider added in v1.1.21 

func (mc *ModelCatalog) RefineModelForProvider(provider schemas.ModelProvider, model string) string

RefineModelForProvider refines the model for a given provider. e.g. "gpt-oss-120b" for groq provider -> "openai/gpt-oss-120b"

func (*ModelCatalog) ReloadPricing 

func (mc *ModelCatalog) ReloadPricing(ctx context.Context, config *Config) error

ReloadPricing reloads the pricing manager from config

type PricingEntry 

type PricingEntry struct {
	// Basic pricing
	InputCostPerToken  float64 `json:"input_cost_per_token"`
	OutputCostPerToken float64 `json:"output_cost_per_token"`
	Provider           string  `json:"provider"`
	Mode               string  `json:"mode"`
	// Additional pricing for media
	InputCostPerImage          *float64 `json:"input_cost_per_image,omitempty"`
	InputCostPerVideoPerSecond *float64 `json:"input_cost_per_video_per_second,omitempty"`
	InputCostPerAudioPerSecond *float64 `json:"input_cost_per_audio_per_second,omitempty"`
	// Character-based pricing
	InputCostPerCharacter  *float64 `json:"input_cost_per_character,omitempty"`
	OutputCostPerCharacter *float64 `json:"output_cost_per_character,omitempty"`
	// Pricing above 128k tokens
	InputCostPerTokenAbove128kTokens          *float64 `json:"input_cost_per_token_above_128k_tokens,omitempty"`
	InputCostPerCharacterAbove128kTokens      *float64 `json:"input_cost_per_character_above_128k_tokens,omitempty"`
	InputCostPerImageAbove128kTokens          *float64 `json:"input_cost_per_image_above_128k_tokens,omitempty"`
	InputCostPerVideoPerSecondAbove128kTokens *float64 `json:"input_cost_per_video_per_second_above_128k_tokens,omitempty"`
	InputCostPerAudioPerSecondAbove128kTokens *float64 `json:"input_cost_per_audio_per_second_above_128k_tokens,omitempty"`
	OutputCostPerTokenAbove128kTokens         *float64 `json:"output_cost_per_token_above_128k_tokens,omitempty"`
	OutputCostPerCharacterAbove128kTokens     *float64 `json:"output_cost_per_character_above_128k_tokens,omitempty"`
	//Pricing above 200k tokens
	InputCostPerTokenAbove200kTokens           *float64 `json:"input_cost_per_token_above_200k_tokens,omitempty"`
	OutputCostPerTokenAbove200kTokens          *float64 `json:"output_cost_per_token_above_200k_tokens,omitempty"`
	CacheCreationInputTokenCostAbove200kTokens *float64 `json:"cache_creation_input_token_cost_above_200k_tokens,omitempty"`
	CacheReadInputTokenCostAbove200kTokens     *float64 `json:"cache_read_input_token_cost_above_200k_tokens,omitempty"`
	// Cache and batch pricing
	CacheReadInputTokenCost   *float64 `json:"cache_read_input_token_cost,omitempty"`
	InputCostPerTokenBatches  *float64 `json:"input_cost_per_token_batches,omitempty"`
	OutputCostPerTokenBatches *float64 `json:"output_cost_per_token_batches,omitempty"`
}

PricingEntry represents a single model's pricing information

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.