Documentation
¶
Overview ¶
Package modelfamily resolves a version-free model family to one concrete model ID at compile time. It decides; it performs no I/O. The catalog, the price table and the data-policy check are supplied by the caller.
Index ¶
- Constants
- func FetchVercelCatalog(ctx context.Context, client *http.Client, baseURL, apiKey string) (Catalog, *Prices, error)
- func FetchVercelEndpointMax(ctx context.Context, client *http.Client, baseURL, apiKey, id string) (maxInput, maxOutput float64, err error)
- func LockKey(provider, name string) string
- func PlanProvider(provider string) bool
- type Catalog
- type Decision
- type Family
- func (f Family) Choose(catalog Catalog, prices *Prices, policy func(id string) error) Decision
- func (f Family) Eligible(id string, prices *Prices, policy func(id string) error) string
- func (f Family) Member(id string) (Version, bool)
- func (f Family) Reuse(entry LockEntry, prices *Prices, policy func(id string) error) string
- func (f Family) ReuseRecorded(entry LockEntry) string
- func (f Family) Validate() error
- type Lock
- type LockEntry
- type Price
- type Prices
- type Skipped
- type Version
Constants ¶
const ( ReasonCatalogIncomplete = "catalog_incomplete" ReasonNoMember = "no_member_in_catalog" ReasonVersionTie = "version_tie" ReasonUnpriced = "unpriced" ReasonPriceEstimate = "price_not_a_ceiling" ReasonOverInput = "input_price_over_ceiling" ReasonOverOutput = "output_price_over_ceiling" ReasonPolicy = "data_policy" ReasonNoEligible = "no_eligible_member" )
Reasons a candidate is passed over or a family cannot be resolved.
const ContextTiersAll = "all"
ContextTiersAll marks a price entry that encodes every context-length tier the table knows, so its maximum rates can bound a choice.
const LockFileName = "models.lock"
LockFileName is the lock's name beside the pod file.
const MaxPriceExportBytes = 1 << 20
MaxPriceExportBytes bounds the price export read from cllama.
const VercelOfficialBaseURL = "https://ai-gateway.vercel.sh/v1"
VercelOfficialBaseURL is the AI Gateway's API root.
Variables ¶
This section is empty.
Functions ¶
func FetchVercelCatalog ¶ added in v0.35.0
func FetchVercelCatalog(ctx context.Context, client *http.Client, baseURL, apiKey string) (Catalog, *Prices, error)
FetchVercelCatalog reads the AI Gateway's model list with the team's availability annotations. The members it returns are language models the team may use; the price table holds, for each that states a whole price, its highest listed input and output rate. A model with a missing or malformed price or tier is listed with no price, so it cannot be chosen. An annotated listing that the gateway itself calls partial is incomplete.
func FetchVercelEndpointMax ¶ added in v0.35.0
func FetchVercelEndpointMax(ctx context.Context, client *http.Client, baseURL, apiKey, id string) (maxInput, maxOutput float64, err error)
FetchVercelEndpointMax returns the highest prompt and completion price, per million tokens, among the providers that serve a gateway model. The gateway may route to any of them. A listing with no endpoint, or with one whose price is missing or malformed, yields no price.
func PlanProvider ¶ added in v0.35.0
PlanProvider reports whether a provider is a plan sign-in route, served from a subscription and not billed per token.
Types ¶
type Catalog ¶
type Catalog struct {
// Complete is true only for one successful read of the whole listing.
Complete bool
IDs []string
}
Catalog is a provider's model listing as read by the caller.
func FetchCatalog ¶
func FetchCatalog(ctx context.Context, client *http.Client, baseURL, apiKey string) (Catalog, error)
FetchCatalog reads a provider's model listing from its OpenAI-compatible "models" endpoint. The key is sent to that endpoint only: redirects are not followed. The read is bounded in time and size. Anything short of one whole, well-formed listing returns an incomplete catalog, which moves nothing; the error says why and never carries the key or reply text.
type Decision ¶
type Decision struct {
// ID is the chosen model, empty when nothing could be chosen.
ID string
// Price is the exact entry the choice was bounded with.
Price Price
// Reason says why nothing was chosen.
Reason string
// Skipped lists members with a higher version than the choice, or every
// member when nothing was chosen.
Skipped []Skipped
}
Decision is the outcome of resolving one family.
type Family ¶
type Family struct {
Name string
Provider string
// Pattern is a literal model ID with exactly one "*" where the version is.
Pattern string
// MaxInput and MaxOutput are absolute ceilings in USD per million tokens
// on the listed standard rates, long-context tier included.
MaxInput float64
MaxOutput float64
}
Family is an operator's statement of which models a slot may use.
func (Family) Choose ¶
Choose picks the highest-versioned member of the family in the catalog that is exactly priced within the ceilings and passes the data policy. It chooses nothing from an incomplete catalog, and nothing when two members share a version.
func (Family) Eligible ¶
Eligible reports why a member may not be used, or "" when it may. It is the single rule applied to a catalog candidate and to a reused lock.
func (Family) Member ¶
Member reports whether id belongs to the family and returns its version. An ID is a member only if it is the pattern's prefix, a version and the pattern's suffix, in full. A dated snapshot or any other variant is not.
func (Family) Reuse ¶
Reuse reports whether a locked choice may stand when the catalog cannot be read. Only the catalog's freshness is waived. The entry must have been made for this same provider and pattern, and its model must still be a member that is exactly priced within the current ceilings and passes the current data policy. It returns "" when the entry may be reused.
func (Family) ReuseRecorded ¶ added in v0.35.0
ReuseRecorded is Reuse for a provider whose prices come from its own catalog. With the catalog unavailable there is no price to read, so the rates recorded when the model was chosen are held to the current ceilings.
type Lock ¶
Lock records which concrete model each family resolved to.
type LockEntry ¶
type LockEntry struct {
Provider string `json:"provider"`
Pattern string `json:"pattern"`
ID string `json:"id"`
// PreviousID is the model the family resolved to before this one.
PreviousID string `json:"previous_id,omitempty"`
// MaxInput and MaxOutput are the listed maximum rates the choice was
// bounded with, in USD per million tokens.
MaxInput float64 `json:"max_input"`
MaxOutput float64 `json:"max_output"`
// CatalogAt is when the provider's complete listing was read.
CatalogAt time.Time `json:"catalog_at"`
// CllamaTag and CllamaImageID name the image whose price table was used.
CllamaTag string `json:"cllama_tag,omitempty"`
CllamaImageID string `json:"cllama_image_id,omitempty"`
// PriceSource is "catalog" when the rates came from the provider's own
// listing and not from cllama's table.
PriceSource string `json:"price_source,omitempty"`
}
LockEntry is one family's resolution and the evidence it rests on.
type Price ¶
type Price struct {
Provider string `json:"provider"`
Model string `json:"model"`
Input float64 `json:"input"`
Output float64 `json:"output"`
MaxInput float64 `json:"max_input"`
MaxOutput float64 `json:"max_output"`
ContextTiers string `json:"context_tiers"`
}
Price is one exact entry of cllama's compiled price table.
type Prices ¶
Prices is the price table exported by the pinned cllama.
func ParsePrices ¶
ParsePrices reads the output of "cllama prices --json". It accepts only the schema and scope this package bounds against, and refuses a table with an invalid rate or two entries for one model.