provision

package
v0.1.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 7, 2026 License: MIT Imports: 9 Imported by: 0

Documentation

Overview

Package provision installs a local inference runtime that Flynn fetches itself, so a machine with no runtime can still run a model with no manual setup step. It pairs with the pure inference core: that package decides which runtime version is safe to run, this one obtains a pinned, safe build and places it on disk.

The mechanics of fetching, verifying, extracting, and atomically installing a pinned archive are the generic acquire layer's job; this package adds the model-runtime policy on top: a release is gated against the inference advisory floor before any network access, and a build is installed under a versioned directory so versions coexist.

It does not run the runtime. Launching the installed binary, which is the code-execution surface a malicious model targets, is the caller's job and happens inside the sandbox. This package only guarantees the bytes on disk are the pinned, gate-approved build.

Index

Constants

View Source
const (
	// ArchiveZip is a .zip archive (the Windows release form).
	ArchiveZip = acquire.ArchiveZip
	// ArchiveTarGz is a gzip-compressed tar (the Linux and macOS release form).
	ArchiveTarGz = acquire.ArchiveTarGz
)

Variables

This section is empty.

Functions

func FetchModelDir

func FetchModelDir(ctx context.Context, dl *fetch.Downloader, files []ModelFile, destDir string) (string, error)

FetchModelDir downloads every file of a multi-file model into destDir, each verified and traversal-guarded, reusing any file already present. It returns destDir on success. It refuses an empty manifest (a model with no files is not a model) and an entry whose name escapes the directory, so the directory it returns holds exactly the manifest's files and nothing outside it. The download cap for a file is its known size plus a small margin, so a server that returns more than the manifest claims is refused rather than written.

func ModelDirPresent

func ModelDirPresent(files []ModelFile, destDir string) bool

ModelDirPresent reports whether every file of a manifest is already present under destDir, so a caller can skip the fetch entirely (and the network) when the model is fully on disk.

Types

type Acquired

type Acquired struct {
	// Runtime is the runtime this is, matching inference.Runtime.Name.
	Runtime string
	// Version is the gate-approved version that was acquired.
	Version inference.Version
	// Binary is the server executable path for a binary-archive runtime; empty for a
	// container runtime.
	Binary string
	// Image is the digest-pinned image for a container runtime; its zero value means this
	// is a binary runtime.
	Image AcquiredImage
	// FromCache is true when the runtime was already present and nothing was downloaded or
	// pulled.
	FromCache bool
}

Acquired is what the launch layer needs to run a runtime once it has been obtained, in the one shape that covers both runtime worlds. Exactly one of Binary or Image is set: a binary-archive runtime resolves to an executable on disk, a container runtime resolves to a digest-pinned image the engine runs.

func (Acquired) IsContainer

func (a Acquired) IsContainer() bool

IsContainer reports whether this resolves to a container image rather than a binary.

type AcquiredImage

type AcquiredImage struct {
	// Ref is the image reference without the digest, e.g. "vllm/vllm-openai:v0.11.1".
	Ref string
	// Digest is the content digest the image is pinned to, "sha256:" then 64 hex chars.
	Digest string
}

AcquiredImage is a digest-pinned container image a container runtime resolves to. The digest is the trust anchor (an image is content-addressed, so a moved tag cannot substitute different bytes); the ref is recorded for diagnostics. It mirrors the shape the sandbox container tier validates again before it runs the image, so the pin is enforced at acquisition and re-enforced at run.

type ArchiveKind

type ArchiveKind = acquire.ArchiveKind

ArchiveKind is the container format a runtime release ships in. Its values match acquire.ArchiveKind, so a runtime release maps to the generic acquire layer directly.

type ArchiveProvisioner

type ArchiveProvisioner struct {
	// Release is the pinned build to install.
	Release Release
	// DestDir is where installed builds live.
	DestDir string
	// Downloader is the verified fetch path the archive is downloaded through.
	Downloader *fetch.Downloader
}

ArchiveProvisioner acquires a runtime shipped as a pinned binary archive: the llama.cpp shape. It is the Provisioner adapter over Install, so the existing download-verify-extract path is one strategy among several rather than the only one.

func (ArchiveProvisioner) Acquire

func (p ArchiveProvisioner) Acquire(ctx context.Context) (Acquired, error)

Acquire installs the release (gating it before any download) and reports its binary.

func (ArchiveProvisioner) Runtime

func (p ArchiveProvisioner) Runtime() string

Runtime is the archive's runtime name.

type ContainerProvisioner

type ContainerProvisioner struct {
	// RuntimeName is the runtime this image is, e.g. "vllm".
	RuntimeName string
	// Version is the runtime version the pinned digest corresponds to. Blessing a digest
	// asserts which version it is; that version is gated against the advisory floor, so a
	// digest for a known-vulnerable build is refused before it is pulled.
	Version inference.Version
	// Ref and Digest pin the image. Digest is required and must be a well-formed sha256.
	Ref    string
	Digest string
	// Pull ensures the image is present; nil treats the image as already present.
	Pull ImagePuller
}

ContainerProvisioner acquires a runtime shipped as a container image: the vLLM shape. A blessed image pinned by content digest is the trust anchor, so the Python and CUDA contents that are awkward to verify piecemeal are sealed inside one digest-addressed artifact. Upgrading is blessing a new digest, nothing more.

With Pull set the image is pulled by digest at acquisition; with Pull nil the strategy is the zero-cost path that drives an image the engine already has (the engine resolves the pinned digest on first run), so the same type serves both "obtain it" and "use what is present". Either way the version is gated and the digest is checked before anything runs.

func (ContainerProvisioner) Acquire

Acquire gates the version and the digest, ensures the image is present when a puller is set, and reports the digest-pinned image. It refuses before any pull when the version is below the floor or the image is not pinned to a well-formed digest.

func (ContainerProvisioner) Runtime

func (p ContainerProvisioner) Runtime() string

Runtime is the container runtime's name.

type DetectProvisioner

type DetectProvisioner struct {
	// RuntimeName is the runtime to detect, matching inference.Runtime.Name; its known
	// version format is used to parse the located runtime's version output.
	RuntimeName string
	// Locate finds the present runtime and reports its raw version output.
	Locate Locator
}

DetectProvisioner drives a binary runtime already present on the host: the zero-cost fast path. It does not download anything; it locates the runtime, parses the version its command prints, and gates that version, so an already-present but vulnerable build is still refused. The container equivalent is a ContainerProvisioner with a nil puller, so this type covers the binary case.

func (DetectProvisioner) Acquire

func (p DetectProvisioner) Acquire(ctx context.Context) (Acquired, error)

Acquire locates the runtime, gates its version, and reports its binary. It refuses when the runtime is absent or its present version is below the advisory floor.

func (DetectProvisioner) Runtime

func (p DetectProvisioner) Runtime() string

Runtime is the detected runtime's name.

type ImagePuller

type ImagePuller func(ctx context.Context, ref, digest string) error

ImagePuller ensures a digest-pinned image is present on the host, pulling it by digest if it is not. It is injected so the container acquisition policy is testable without an engine; the real puller runs the engine's pull, which resolves the ref to the pinned digest and verifies the bytes against it, so the pull is tamper-evident on its own.

type Installed

type Installed struct {
	// BinPath is the absolute path to the runnable server executable.
	BinPath string
	// Version is the build's version.
	Version inference.Version
	// FromCache is true when the build was already installed and was reused.
	FromCache bool
}

Installed describes a runtime build present on disk after Install.

func Install

func Install(ctx context.Context, dl *fetch.Downloader, rel Release, destDir string) (Installed, error)

Install ensures the release's runtime build is present under destDir and returns the path to its server binary. It is idempotent: a build already extracted at its versioned location is reused without a download. The release is gated before any network access, so a build that would be refused at run time is never fetched; the download, digest verification, traversal-guarded extraction, and atomic install are the acquire layer's.

type Locator

type Locator func(ctx context.Context) (binPath, rawVersion string, ok bool)

Locator finds a runtime already present on the host and returns a handle to it (a binary path) and the raw output of its version command, or ok=false when it is absent. It is injected so detection is testable without the real tool installed.

type ModelFile

type ModelFile struct {
	// Name is the file's path within the model directory, for example "model.safetensors" or
	// "tokenizer.json". It is relative and must not escape the directory.
	Name string
	// URL is the https source of the file.
	URL string
	// SHA256 is the pinned digest the file is verified against. Empty pins on fetch (the
	// computed digest is recorded but nothing pre-pinned was checked), the lower-trust path
	// the single-file fetch also allows.
	SHA256 string
	// SizeBytes is the file's known size, used as the per-file download cap. 0 uses the
	// downloader's default ceiling.
	SizeBytes int64
}

ModelFile is one file of a multi-file model: its name within the model directory and where to fetch it from, verified against a pinned digest. It is the per-file unit a safetensors model's manifest is made of.

type Provisioner

type Provisioner interface {
	// Runtime is the inference runtime this provisions, matching inference.Runtime.Name.
	Runtime() string
	// Acquire obtains the runtime and returns the handle launch consumes. It reuses an
	// already-present build or image rather than re-fetching where it can.
	Acquire(ctx context.Context) (Acquired, error)
}

Provisioner obtains a runtime and reports what launch needs to run it. Each strategy gates the runtime version before any network access, so a runtime that would be refused at run time is never downloaded or pulled.

type Release

type Release struct {
	// Runtime is the inference runtime this build is, matching inference.Runtime.Name.
	Runtime string
	// Version is the build's version, used to gate it and to name its install dir.
	Version inference.Version
	// GOOS and GOARCH are the platform this build targets (Go's runtime.GOOS/GOARCH).
	GOOS, GOARCH string
	// URL is the https source of the release archive.
	URL string
	// SHA256 is the pinned digest the downloaded archive must match.
	SHA256 string
	// SizeBytes is the archive's known size, used as the download cap.
	SizeBytes int64
	// Archive is the archive's container format.
	Archive ArchiveKind
	// BinName is the server executable to locate inside the extracted archive (for
	// example "llama-server" or "llama-server.exe"). Its sibling libraries are extracted
	// alongside it, so the located binary is runnable in place.
	BinName string
}

Release is a single pinned runtime build for one OS and architecture: where to get it, the digest it must match, and which executable inside it is the server to run. A release is data, fixed at build time, so the set of builds Flynn will install is auditable and cannot be redirected at runtime.

func ReleaseFor

func ReleaseFor(runtime, goos, goarch string) (Release, bool)

ReleaseFor returns the pinned runtime build Flynn would install for a platform, and whether one exists. The runtime is matched by name (matching inference.Runtime.Name), the platform by Go's runtime.GOOS and runtime.GOARCH.

func Releases

func Releases() []Release

Releases returns a copy of the full pinned release set, for reporting which builds Flynn can install.

func (Release) Gate

func (r Release) Gate() error

Gate reports the error from the version gate for this release, or nil when the build is safe to run. A release should never be installed if it does not pass, so a caller can refuse before fetching anything.

type VLLMRelease

type VLLMRelease struct {
	// Ref is the image reference without the digest, for diagnostics and the pull.
	Ref string
	// Digest is the content digest the image is pinned to ("sha256:" + 64 hex).
	Digest string
	// Version is the vLLM version this digest is, gated against the inference floor.
	Version inference.Version
}

VLLMRelease is the blessed vLLM image: the reference it is pulled by, the content digest it is pinned to, and the version that digest is. The version is gated against the advisory floor before the image is pulled, so a digest blessed as a known-vulnerable build is refused, exactly as a binary Release is gated before it is fetched.

func VLLMImage

func VLLMImage() VLLMRelease

VLLMImage returns the blessed vLLM image release.

func (VLLMRelease) Gate

func (r VLLMRelease) Gate() error

Gate reports the advisory-floor error for this image's version, or nil when the version is safe to run, so a caller refuses a known-vulnerable image before pulling it.

func (VLLMRelease) Provisioner

func (r VLLMRelease) Provisioner(pull ImagePuller) ContainerProvisioner

Provisioner builds the container provisioner that acquires this image, pulling it by digest through pull (the engine verifies the bytes against the digest). A nil pull drives an image the engine already has.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL