models

package
v0.1.12-alpha Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 1, 2026 License: Apache-2.0 Imports: 18 Imported by: 0

Documentation

Index

Constants

View Source
const (
	ListingLive     = "live"
	ListingWaitlist = "waitlist"
	ListingHidden   = "hidden"
)

Listing status values describe how a model is presented on the public catalog. They are independent of the operational lifecycle in Status.

View Source
const AutoModelID = "auto"

AutoModelID is the explicit way to ask for hardware-based selection. An empty model id means the same thing.

View Source
const UnmeasuredVRAMFloorMB int64 = 8192

Fits reports whether the host satisfies a manifest's hardware floor. When VRAM is unknown the VRAM floor cannot be checked and is skipped; the caller is responsible for telling the operator that happened. UnmeasuredVRAMFloorMB is the VRAM an unmeasurable host is assumed to have: the platform minimum, not unlimited. Unmeasured VRAM used to mean "ignore the VRAM requirement", which on a 64GB-RAM machine with an unreadable card selected the largest model in the catalog and would OOM on any ordinary GPU. Assuming the floor picks something that will actually run; an operator who knows the real number can still pass --model explicitly.

View Source
const VRAMUnmeasured int64 = -1

VRAMUnmeasured marks a host whose VRAM could not be established. It is distinct from zero so "no measurement" never reads as "no memory".

Variables

This section is empty.

Functions

func Fits

func Fits(manifest Manifest, host HostCapacity) bool

func LicenseTextFor

func LicenseTextFor(identifier string) (string, bool)

LicenseTextFor returns the vendored license text for a manifest license identifier, and whether one exists.

func ReadCatalogFile

func ReadCatalogFile(catalogDir, name string) ([]byte, string, error)

ReadCatalogFile reads a catalog file from disk when present, falling back to the catalog embedded in the binary. Released binaries run from arbitrary working directories with no repo checkout, so the embedded copy is what makes `thirdshift start` work after a plain install.

Types

type Artifact

type Artifact struct {
	URL       string
	SHA256    string
	SizeBytes int64
}

type Cache

type Cache struct {
	Dir        string
	QuotaBytes int64
	HTTPClient *http.Client
}

func (Cache) Ensure

func (c Cache) Ensure(ctx context.Context, artifact Artifact, activeSHA256 string) (string, error)

func (Cache) EnsureLicense

func (c Cache) EnsureLicense(artifactSHA256, text string) (string, error)

EnsureLicense writes a model's license text next to its cached artifact as <sha256>.LICENSE.txt. Licenses that require distribution must accompany the model materials on every host, so callers treat failure as fatal to model preparation.

type Capabilities

type Capabilities struct {
	ChatCompletions bool
	Streaming       bool
	Tools           bool
	Embeddings      bool
}

type Hardware

type Hardware struct {
	MinVRAMMB          int
	MinRAMMB           int
	MinDiskMB          int
	EligibleGPUClasses []string
}

type HostCapacity

type HostCapacity struct {
	VRAMTotalMB int64
	RAMTotalMB  int64
	DiskFreeMB  int64
}

HostCapacity is the measured host, in megabytes. VRAMTotalMB of zero or less means VRAM could not be established; selection then falls back to RAM alone and says so, rather than assuming a size.

type License

type License struct {
	Identifier          string
	ReviewedAt          string
	Notes               string
	DistributeWithModel bool
	Attribution         *LicenseAttribution
}

type LicenseAttribution

type LicenseAttribution struct {
	DisplayText string
	NoticeText  string
	LicenseURL  string
	AUPURL      string
}

LicenseAttribution carries operator-facing attribution requirements a model license imposes on public surfaces (e.g. "Built with Llama").

type Limits

type Limits struct {
	MaxInputTokens  int
	MaxOutputTokens int
	MaxRequestBytes int
}

type Listing

type Listing struct {
	Status                        string
	Description                   string
	ExpectedOutputTokensPerSecond *float64
	MarketComparison              *MarketComparison
}

type Manifest

type Manifest struct {
	SchemaVersion int
	ModelID       string
	DisplayName   string
	Description   string
	Status        string
	Listing       Listing
	Source        Source
	License       License
	Runtime       Runtime
	Hardware      Hardware
	Limits        Limits
	Capabilities  Capabilities
	Policy        Policy
	Pricing       Pricing
	Verification  Verification
}

func LoadCatalogManifest

func LoadCatalogManifest(catalogDir, modelID string) (Manifest, string, error)

func LoadSelectableManifests

func LoadSelectableManifests(catalogDir string) ([]Manifest, error)

LoadSelectableManifests lists every manifest that parses and is not hidden. Names come from the catalog directory when it exists and from the embedded catalog otherwise, because a released binary runs from an arbitrary working directory with no repo checkout — the same reason ReadCatalogFile falls back. Unparseable manifests (placeholders) are skipped, matching catalog sync.

func ParseManifestFile

func ParseManifestFile(path string) (Manifest, error)

func (Manifest) CatalogDescription

func (m Manifest) CatalogDescription() string

CatalogDescription returns the manifest's public one-liner, preferring the listing block over the legacy top-level description. It stays empty when the manifest has neither so readers can fall back to the display name.

func (Manifest) ListingStatus

func (m Manifest) ListingStatus() string

ListingStatus returns the normalized public listing status, defaulting to live so manifests written before the listing block keep their behavior.

type MarketComparison

type MarketComparison struct {
	TypicalInputPerMillionUSD  float64
	TypicalOutputPerMillionUSD float64
	SourceNote                 string
}

type Policy

type Policy struct {
	DataClass            string
	ContentFilterProfile string
}

type Pricing

type Pricing struct {
	CustomerInputPerMillionUSD            float64
	CustomerOutputPerMillionUSD           float64
	HostCreditPerMillionAcceptedOutputUSD float64
}

type Runtime

type Runtime struct {
	Engine          string
	BuildID         string
	ReleaseManifest string
	BinarySHA256    string
	Arguments       RuntimeArguments
}

type RuntimeArguments

type RuntimeArguments struct {
	ContextSize int
	BatchSize   int
	GPULayers   string
	Device      string
	Parallel    int
	Host        string
	Port        string
}

type Selection

type Selection struct {
	ModelID  string
	Manifest Manifest
	Reason   string
	// VRAMAssumed is true when VRAM could not be measured and the choice rests
	// on RAM alone. Callers should surface this prominently.
	VRAMAssumed bool
}

Selection is the outcome of auto-selection, carrying the reason so the node can log why this model was chosen.

func ResolveModel

func ResolveModel(requested string, manifests []Manifest, host HostCapacity) (Selection, error)

ResolveModel applies the model flag. An explicit id is honored but still checked against the hardware, so a host fails fast with an actionable message instead of starting something that will exhaust VRAM mid-job.

func SelectModel

func SelectModel(manifests []Manifest, host HostCapacity) (Selection, error)

SelectModel picks the largest catalog model the host can run.

type Source

type Source struct {
	Provider   string
	Repository string
	Revision   string
	File       string
	URL        string
	SHA256     string
	SizeBytes  int64
}

type Verification

type Verification struct {
	DuplicateSampleRate float64
	ChallengeRate       float64
}

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL