Documentation
¶
Overview ¶
Package modelstore resolves model names to on-disk paths within a modeld node's models directory. It exists so a session request can omit Path: a remote node has no idea what filesystem layout the runtime that sent the request uses, so the node must resolve ModelName+Type against its own storage instead of trusting a caller-supplied path.
Layout mirrors what the runtime's local catalog providers already scan (runtime/modelrepo/llama, runtime/modelrepo/openvino), so an existing local models directory works unchanged as a node's models dir:
<dir>/<name>/model.gguf (llama) <dir>/<name>/openvino_model.xml (openvino, or openvino_language_model.xml)
Index ¶
- Constants
- Variables
- func Dir(dataRoot, override string) string
- func FileDigest(path string) (string, error)
- func Resolve(dir, name, backendType, wantDigest string) (path string, err error)
- type Admin
- func (a *Admin) DiskStats(ctx context.Context) (DiskStats, error)
- func (a *Admin) ListModels(_ context.Context) ([]NodeModel, error)
- func (a *Admin) ReceiveModel(_ context.Context, manifest PushManifest, r io.Reader) (PushResult, error)
- func (a *Admin) RemoveModel(_ context.Context, name string) error
- type DiskStats
- type NodeModel
- type PushFormat
- type PushManifest
- type PushResult
Constants ¶
const DefaultSubdir = "models"
DefaultSubdir is the models directory name under a data root when no explicit override is configured.
Variables ¶
var ( // ErrModelNotFound is returned when no model matching the requested // name+type exists in the models directory. ErrModelNotFound = transport.ErrModelNotFound // ErrDigestMismatch is returned when a caller-supplied digest does not // match the on-disk content. Only enforced when the caller supplies a // non-empty digest to verify against. ErrDigestMismatch = transport.ErrDigestMismatch // ErrUnsupportedType is returned for a backend type this package does not // know how to resolve. ErrUnsupportedType = transport.ErrUnsupportedModelType )
These alias the transport package's canonical sentinels rather than defining new error values: model resolution errors surface both in-process (slot.Service callers checking modelstore.ErrModelNotFound directly) and across the gRPC wire (Admin.ReceiveModel's errors, encoded/decoded via transport/grpc's sentinel table). A local, unrelated error value here would lose its identity crossing the wire.
Functions ¶
func Dir ¶
Dir resolves the models directory: an explicit override if non-empty, otherwise <dataRoot>/<DefaultSubdir>.
func FileDigest ¶
FileDigest returns the hex-encoded sha256 digest of a file's content.
func Resolve ¶
Resolve finds the on-disk path for a model by name and backend type within dir. When wantDigest is non-empty and the backend type supports digest verification, the resolved file's content digest is compared and a mismatch is rejected — this guards against a stale or wrong file answering under a name the caller expects to be a specific model.
OpenVINO IR models are directories; digest verification for them is not yet implemented (their content-addressing arrives with node-side push in a later phase), so wantDigest is ignored for backend type "openvino".
Types ¶
type Admin ¶
type Admin struct {
// contains filtered or unexported fields
}
Admin implements node-side model store management: listing, removal, disk stats, and receiving pushed model blobs. Per the runtime's model distribution design, a node never fetches from an external source — the runtime is the only source of model bytes, and Admin is purely a local sink/inventory over the models directory.
func (*Admin) DiskStats ¶
DiskStats reports free/used/total bytes on the filesystem backing the models directory.
func (*Admin) ListModels ¶
ListModels enumerates every model in the models directory.
func (*Admin) ReceiveModel ¶
func (a *Admin) ReceiveModel(_ context.Context, manifest PushManifest, r io.Reader) (PushResult, error)
ReceiveModel is the sink side of PushModel: it consumes r (the raw byte stream described by manifest), verifies it against manifest.Digest, and atomically installs it as the named model. Idempotent: an existing model with a matching digest is kept and the newly received bytes discarded.
Deciding whether to push at all — skipping when the model is already present with a matching digest — is the caller's job (the runtime's reconcile loop checks ListModels first, since it can do that without transferring any bytes). ReceiveModel always accepts and verifies a full stream so there is exactly one, always-correct write path; it is not an optimization for the already-present case.
At most one push per model name may be in flight at a time; a concurrent second push for the same name is rejected immediately.
type DiskStats ¶
DiskStats reports free/used space on the filesystem backing the models directory, so the runtime can decide whether a push would fit before sending gigabytes of model weights.
type NodeModel ¶
type NodeModel struct {
Name string
Type string // "llama" | "openvino"
Digest string // sha256 hex; empty for openvino (see Resolve doc)
SizeBytes int64
// ContextLength is the model's trained context ceiling, from a header-only
// metadata parse (no device query, no capacity planner) — 0 if that parse
// failed or was skipped; see ListModels.
ContextLength int
}
NodeModel is one model as observed on a node's models directory.
type PushFormat ¶
type PushFormat string
PushFormat identifies how a PushModel byte stream is laid out.
const ( // PushFormatFile is a single-file model (llama GGUF), written as // <dir>/<name>/model.gguf. PushFormatFile PushFormat = "file" // PushFormatTar is a directory model (OpenVINO IR) sent as an uncompressed // tar stream and unpacked into <dir>/<name>/. PushFormatTar PushFormat = "tar" )
type PushManifest ¶
type PushManifest struct {
Name string
Type string // "llama" | "openvino"
Digest string
TotalBytes int64
Format PushFormat
}
PushManifest describes an incoming model push. Digest is the sha256 of the raw byte stream exactly as sent (the tar stream itself for PushFormatTar, not a hash of the unpacked contents) — the receiver verifies it as bytes arrive, before touching the models directory.
type PushResult ¶
type PushResult struct {
// AlreadyPresent means a model with this name and a matching digest was
// already installed; the newly received bytes were verified then
// discarded rather than replacing it.
AlreadyPresent bool
BytesWritten int64
}
PushResult reports what ReceiveModel actually did.
Source Files