modeldata

package
v0.28.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 13, 2026 License: MIT Imports: 16 Imported by: 0

Documentation

Overview

Package modeldata keeps the files the LLM gateway engine prices requests and looks up model capabilities from: a pricing datasheet and a model-parameters file.

By default the engine downloads both from its vendor's website. Obot instead keeps them in the gateway's data dir and points the engine at them with file:// URLs, so the engine never fetches them itself and always has something to boot with: with an empty store and nothing to load, the engine exits the process.

The files are converted from LiteLLM's model_prices_and_context_window.json (MIT; see data/LICENSE.litellm). Obot's images and release binaries embed a snapshot downloaded at build time (see Bundle), and Obot refreshes the files from a URL that serves the same conversion, Obot's upgrade server by default. The engine re-reads them on its own schedule.

Index

Constants

View Source
const (

	// RefreshInterval is how often Obot downloads the files from the refresh URL.
	RefreshInterval = 6 * time.Hour
)

Variables

This section is empty.

Functions

func Bundle

func Bundle(ctx context.Context, baseURL, dir string) error

Bundle downloads the model data baseURL serves into dir, compressed, for a build to embed.

func Download

func Download(ctx context.Context, client *http.Client, dataDir, baseURL string) error

Download replaces the gateway's model data files with the ones baseURL serves, if both pass validation.

func FileURL

func FileURL(path string) string

FileURL returns the file:// URL of an absolute path.

func Paths

func Paths(dataDir string) (pricing, params string)

Paths returns the files the engine reads model pricing and capabilities from.

func Prepare

func Prepare(ctx context.Context, dataDir, refreshURL string) error

Prepare makes sure the gateway's data dir has model pricing and capability files for the engine to read.

  • With a bundled snapshot, the snapshot is written unless the files there are newer, as they are once a refresh has downloaded them.
  • Without one, as in a build from source, files already there are kept. Otherwise Prepare tries to download them from refreshURL, and failing that writes empty files: the gateway starts with no model prices until a refresh succeeds.

func Refresh

func Refresh(ctx context.Context, dataDir, baseURL string, interval time.Duration)

Refresh downloads the model pricing and capability files from baseURL every interval until ctx ends. A download that fails or doesn't pass validation leaves the current files in place.

func RefreshURL

func RefreshURL(raw string) (string, error)

RefreshURL validates the configured model data URL. Empty or "off" turns refreshing off.

Types

This section is empty.

Directories

Path Synopsis
Package modeldatatest provides model data for tests: a few real entries, converted the way Obot's model data is, so tests that need model prices don't depend on the data a build downloads.
Package modeldatatest provides model data for tests: a few real entries, converted the way Obot's model data is, so tests that need model prices don't depend on the data a build downloads.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL