Documentation
¶
Overview ¶
Package modelsdev provides an adapter for loading model pricing data from models.dev (https://models.dev/api.json).
The adapter supports loading from a local file, fetching from the URL, or refreshing the data on-demand.
Index ¶
- func NewLookup(opts ...Option) (llmproxy.CostLookup, error)
- type Adapter
- func (a *Adapter) FindProviderForModel(model string) string
- func (a *Adapter) GetCostLookup() llmproxy.CostLookup
- func (a *Adapter) GetModel(provider string, model string) *Model
- func (a *Adapter) GetProvider(provider string) *Provider
- func (a *Adapter) ListModels(provider string) []string
- func (a *Adapter) ListProviders() []string
- func (a *Adapter) Load(ctx context.Context) error
- func (a *Adapter) Lookup(provider string, model string) (llmproxy.CostInfo, bool)
- func (a *Adapter) Refresh(ctx context.Context) error
- type Cost
- type Limit
- type Modalities
- type Model
- type ModelsDevJSON
- type Option
- type Provider
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func NewLookup ¶
func NewLookup(opts ...Option) (llmproxy.CostLookup, error)
NewLookup is a convenience function that creates an adapter and returns just the CostLookup function.
Example:
lookup, _ := modelsdev.NewLookup(modelsdev.WithFile("models.json"))
billing := interceptors.NewBilling(lookup, onResult)
Types ¶
type Adapter ¶
type Adapter struct {
// contains filtered or unexported fields
}
Adapter loads and caches model pricing data from models.dev.
func LoadFromFile ¶
LoadFromFile is a convenience function that creates an adapter and loads data from a file in one step.
func LoadFromURL ¶
LoadFromURL is a convenience function that creates an adapter and fetches data from the URL in one step.
func New ¶
New creates a new models.dev adapter. Data is not loaded until Load() or Lookup() is called.
Example - fetch from URL:
adapter := modelsdev.New() adapter.Load(context.Background()) // optional, loads on first lookup
Example - load from file:
adapter := modelsdev.New(modelsdev.WithFile("models.json"))
Example - custom URL, TTL, and markup:
adapter := modelsdev.New(
modelsdev.WithURL("https://internal.example.com/models.json"),
modelsdev.WithTTL(30*time.Minute),
modelsdev.WithMarkup(1.2), // 20% markup
)
func (*Adapter) FindProviderForModel ¶ added in v0.0.6
FindProviderForModel searches all providers to find which one has the given model. Returns the provider ID if found, or empty string if not found. This is useful for provider detection when the model name is known but provider is not.
If data is not loaded or TTL expired, it loads automatically. Note: Auto-load uses context.Background() - call Load() explicitly to control the context for initial load.
func (*Adapter) GetCostLookup ¶
func (a *Adapter) GetCostLookup() llmproxy.CostLookup
GetCostLookup returns a CostLookup function for use with interceptors.
Example:
adapter := modelsdev.New(modelsdev.WithFile("models.json"))
billing := interceptors.NewBilling(adapter.GetCostLookup(), func(r llmproxy.BillingResult) {
log.Printf("Cost: $%.6f", r.TotalCost)
})
func (*Adapter) GetModel ¶
GetModel returns model information for a provider/model. Returns nil if not found.
func (*Adapter) GetProvider ¶
GetProvider returns provider information. Returns nil if not found.
func (*Adapter) ListModels ¶
ListModels returns all model IDs for a provider.
func (*Adapter) ListProviders ¶
ListProviders returns all provider IDs.
func (*Adapter) Load ¶
Load fetches or reads the models.dev JSON data. If a file path is set, it reads from the file. Otherwise, it fetches from the URL.
type Cost ¶
type Cost struct {
Input float64 `json:"input,omitempty"`
Output float64 `json:"output,omitempty"`
Reasoning float64 `json:"reasoning,omitempty"`
CacheRead float64 `json:"cache_read,omitempty"`
CacheWrite float64 `json:"cache_write,omitempty"`
InputAudio float64 `json:"input_audio,omitempty"`
OutputAudio float64 `json:"output_audio,omitempty"`
}
Cost represents pricing in USD per 1M tokens.
type Limit ¶
type Limit struct {
Context int `json:"context,omitempty"`
Input int `json:"input,omitempty"`
Output int `json:"output,omitempty"`
}
Limit represents token limits.
type Modalities ¶
type Modalities struct {
Input []string `json:"input,omitempty"`
Output []string `json:"output,omitempty"`
}
Modalities represents supported input/output types.
type Model ¶
type Model struct {
ID string `json:"id"`
Name string `json:"name"`
Family string `json:"family,omitempty"`
Attachment bool `json:"attachment"`
Reasoning bool `json:"reasoning"`
ToolCall bool `json:"tool_call"`
Temperature bool `json:"temperature,omitempty"`
Knowledge string `json:"knowledge,omitempty"`
ReleaseDate string `json:"release_date"`
LastUpdated string `json:"last_updated"`
OpenWeights bool `json:"open_weights"`
StructuredOutput bool `json:"structured_output,omitempty"`
Status string `json:"status,omitempty"`
Cost *Cost `json:"cost,omitempty"`
Limit *Limit `json:"limit,omitempty"`
Modalities *Modalities `json:"modalities,omitempty"`
}
Model represents a model in models.dev.
type ModelsDevJSON ¶
ModelsDevJSON is the root structure of models.dev/api.json.
type Option ¶
type Option func(*Adapter)
Option configures the Adapter.
func WithHTTPClient ¶
WithHTTPClient sets a custom HTTP client for fetching.
func WithMarkup ¶
WithMarkup sets a markup multiplier applied to all prices. For example:
- 1.0 = no markup (default)
- 1.2 = 20% markup (price * 1.2)
- 1.5 = 50% markup (price * 1.5)
This is useful for reselling API access with a profit margin.
Source Files