lockfile

package
v0.19.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 4, 2026 License: Apache-2.0 Imports: 12 Imported by: 0

Documentation

Overview

Package lockfile defines the on-disk weights.lock format and operations on it: parsing, loading, canonical serialization, and entry-level equality checks.

The lockfile is Cog's source-of-truth for imported weights. It captures the source (URI + fingerprint + include/exclude), the resulting content (setDigest, files, layers), and the assembled OCI manifest digest. Everything downstream — OCI manifests, the runtime /.cog/weights.json, registry state validation — is a projection of these fields.

Index

Constants

View Source
const Version = 1

Version is the current lockfile format version.

It is an integer; monotonic bumps (1 → 2) signal schema changes. Pre-release "v1" string versions have no migration path.

View Source
const WeightsLockFilename = "weights.lock"

WeightsLockFilename is the default filename for the weights lock file.

Variables

This section is empty.

Functions

func EntriesEqual

func EntriesEqual(a, b *WeightLockEntry) bool

EntriesEqual reports whether two entries are identical in both content and source. ImportedAt is intentionally excluded — it is a consequence of an import being written, not an input to the equality check.

A lockfile entry is safe to leave unchanged only when both a and b are non-nil and every field (besides ImportedAt) agrees.

func PruneLockfile

func PruneLockfile(lockPath string, keep []string) error

PruneLockfile removes lockfile entries whose names are not in keep. It is a no-op when the lockfile does not exist or when nothing would change, avoiding unnecessary file rewrites (which churn git diffs).

func WithLock

func WithLock(ctx context.Context, lockPath string, fn func() error) (retErr error)

WithLock runs fn while holding an exclusive cross-process advisory lock on a sibling guard file. Concurrent `cog weights import` runs against the same project would otherwise race load+mutate+save and lose writes.

ctx cancellation interrupts a blocking lock acquisition so a wedged peer can't make import un-cancellable.

Types

type ConfigWeight

type ConfigWeight struct {
	Name    string
	URI     string
	Target  string
	Include []string
	Exclude []string
}

ConfigWeight is the lockfile package's view of a weight declaration from cog.yaml. It carries only the user-intent fields that affect whether a lockfile entry is stale. Callers must normalize URI and sort Include/Exclude before constructing a ConfigWeight — CheckDrift does byte-exact comparison.

type DriftKind

type DriftKind string

DriftKind classifies how a weight declaration has drifted from the lockfile.

const (
	// DriftOrphaned means the lockfile has an entry with no matching
	// config declaration — the weight was removed from cog.yaml.
	DriftOrphaned DriftKind = "orphaned"
	// DriftPending means config declares a weight that has no lockfile
	// entry — the weight has never been imported.
	DriftPending DriftKind = "pending"
	// DriftConfigChanged means both config and lockfile have the weight
	// but a user-intent field (URI, target, include, exclude) differs.
	DriftConfigChanged DriftKind = "config-changed"
)

type DriftResult

type DriftResult struct {
	Name    string
	Kind    DriftKind
	Details string // human-readable detail, e.g. "target: /old → /new"
}

DriftResult describes a single config-vs-lockfile mismatch.

func CheckDrift

func CheckDrift(lock *WeightsLock, configWeights []ConfigWeight) []DriftResult

CheckDrift compares config declarations against lockfile entries and returns every mismatch. The result is empty when config and lockfile agree. A nil lock is treated as an empty lockfile (every config weight is "pending"). The function is pure: no I/O, no network.

type RuntimeWeightEntry

type RuntimeWeightEntry struct {
	Name      string `json:"name"`
	Target    string `json:"target"`
	SetDigest string `json:"setDigest"`
}

RuntimeWeightEntry is one weight in the runtime manifest. Three fields per entry: name, target, and the content-addressable set digest.

type RuntimeWeightsManifest

type RuntimeWeightsManifest struct {
	Weights []RuntimeWeightEntry `json:"weights"`
}

RuntimeWeightsManifest is the in-image /.cog/weights.json file that signals managed weights to coglet. It is a minimal projection of the lockfile: only the fields coglet needs to know which weights to expect and where (spec §3.3).

type WeightLockEntry

type WeightLockEntry struct {
	// Name is the weight's logical name (e.g. "z-image-turbo").
	Name string `json:"name"`
	// Target is the container mount path for this weight.
	Target string `json:"target"`
	// Source records where the weight came from and how it was filtered.
	Source WeightLockSource `json:"source"`
	// Digest is the sha256 digest of the assembled OCI manifest.
	Digest string `json:"digest"`
	// SetDigest is the weight set digest (spec §2.4): a content-addressable
	// identifier for the file set, independent of packing strategy.
	SetDigest string `json:"setDigest"`
	// Size is the total uncompressed size of all files in bytes (sum of
	// layer sizeUncompressed).
	Size int64 `json:"size"`
	// SizeCompressed is the total compressed layer size in bytes (sum of
	// layer size) — the bytes the registry stores.
	SizeCompressed int64 `json:"sizeCompressed"`
	// Files is the per-file index, sorted by path. Each entry records the
	// file's size, content digest, and which layer contains it.
	Files []WeightLockFile `json:"files"`
	// Layers is the set of packed tar layers, sorted by digest. Layer
	// emission order from the packer is not guaranteed stable (future
	// concurrency) — sorting produces deterministic output.
	Layers []WeightLockLayer `json:"layers"`
}

WeightLockEntry is one declared weight in the lockfile.

The entry carries everything needed to reproduce the OCI artifacts:

  • identity of the source (Source block)
  • content-addressable identity of the file set (SetDigest)
  • per-file index mapping each file to its layer (Files)
  • intrinsic layer properties for the manifest (Layers)
  • the assembled manifest digest (Digest)

No annotations are stored here; OCI presentation annotations are derived at manifest-build time from the typed fields (name, target, setDigest, etc.).

func (*WeightLockEntry) ComputeSetDigest

func (e *WeightLockEntry) ComputeSetDigest() string

ComputeSetDigest returns the weight set digest (spec §2.4): the dirhash of the entry's file set. ComputeSetDigest canonicalizes the entry in place before hashing, so Files order at call time does not affect the result.

type WeightLockFile

type WeightLockFile struct {
	// Path is the file path relative to the weight source directory,
	// with forward slashes regardless of host OS.
	Path string `json:"path"`
	// Size is the file's uncompressed size in bytes.
	Size int64 `json:"size"`
	// Digest is the sha256 content digest of the file (hex-encoded with
	// the "sha256:" prefix).
	Digest string `json:"digest"`
	// Layer is the digest of the layer containing this file.
	Layer string `json:"layer"`
}

WeightLockFile is a single file in a WeightLockEntry's Files index.

This mirrors the config blob entry shape (spec §2.3) so the config blob can be projected directly from Files without a second walk of the source directory.

func (WeightLockFile) DirhashParts

func (f WeightLockFile) DirhashParts() weightsource.DirhashPart

DirhashParts implements weightsource.Dirhashable so WeightLockFile slices can be passed directly to weightsource.DirHash.

type WeightLockLayer

type WeightLockLayer struct {
	// Digest is the sha256 digest of the layer blob.
	Digest string `json:"digest"`
	// MediaType is the OCI layer media type
	// (application/vnd.oci.image.layer.v1.tar or .tar+gzip).
	MediaType string `json:"mediaType"`
	// Size is the size of the layer blob in bytes (the bytes the
	// registry stores, post-compression for gzip layers).
	Size int64 `json:"size"`
	// SizeUncompressed is the sum of regular-file bytes in the layer,
	// matching the definition used for run.cog.weight.size.uncompressed
	// on index descriptors.
	SizeUncompressed int64 `json:"sizeUncompressed"`
}

WeightLockLayer is an intrinsic description of a single packed tar layer.

Only intrinsic properties live here — digest, mediaType, compressed size (Size), uncompressed size (SizeUncompressed). Layer content type ("bundle" vs "file") is not stored; it is derivable from Files (one file referencing the layer = single-file layer, many = bundle). Annotations are an OCI presentation detail and never stored in the lockfile.

type WeightLockSource

type WeightLockSource struct {
	// URI is the normalized source URI (e.g. file://./weights,
	// hf://org/model, s3://bucket/prefix/).
	URI string `json:"uri"`
	// Fingerprint is the source's version identity at import time.
	// Scheme-prefixed (sha256:, commit:, etag:, …).
	Fingerprint weightsource.Fingerprint `json:"fingerprint"`
	// Include is the sorted list of glob-style include patterns applied
	// to the source. Sorted because order is not semantically meaningful
	// (the patterns are a set, not a sequence) and canonicalizing here
	// keeps the lockfile stable across reorderings in cog.yaml. Empty
	// patterns are serialized as [] so the shape is stable.
	Include []string `json:"include"`
	// Exclude is the sorted list of exclude patterns, same shape as Include.
	Exclude []string `json:"exclude"`
	// ImportedAt is the wall-clock time of the import that produced this
	// entry. It is informational only — it never participates in
	// equality checks (see EntriesEqual).
	ImportedAt time.Time `json:"importedAt"`
}

WeightLockSource records provenance for a WeightLockEntry.

An import is a pure function of (source URI, source fingerprint, include/exclude). Recording all four inputs plus the import timestamp makes the lockfile self-contained: given these fields and the source at Fingerprint, you can deterministically reproduce the Files/Layers that the entry describes.

type WeightsLock

type WeightsLock struct {
	Version int `json:"version"`
	// EnvelopeFormat is the sha256 digest (with "sha256:" prefix)
	// identifying the packer configuration that produced — or, on
	// the next import, will produce — the recorded layer digests.
	//
	// Cog stamps the current envelope digest into the lockfile on
	// every rewrite. On a subsequent import a mismatch (including a
	// missing/empty value, treated as "no match") forces the builder
	// to recompute layer digests from the local content store
	// instead of trusting the cached entry. See
	// pkg/model/envelope.go for what feeds into the digest.
	//
	// Empty when the lockfile has never been written by a
	// version of cog that knows about this field — that empty
	// value compares unequal to any current envelope digest, which
	// is exactly the "force a recompute" behavior we want.
	EnvelopeFormat string            `json:"envelopeFormat"`
	Weights        []WeightLockEntry `json:"weights"`
}

WeightsLock is the parsed representation of a weights.lock file.

The serialized form is stable and deterministic: Weights is kept in insertion order (matching cog.yaml), every entry's Files slice is sorted by path, and every entry's Layers slice is sorted by digest. Regenerating the lockfile from the same source produces byte-identical output, which is what makes weights.lock safe to check into git.

func LoadWeightsLock

func LoadWeightsLock(path string) (*WeightsLock, error)

LoadWeightsLock loads a weights.lock file from disk.

func ParseWeightsLock

func ParseWeightsLock(data []byte) (*WeightsLock, error)

ParseWeightsLock parses a weights.lock JSON document and rejects anything that is not a supported lockfile version. It also rejects lockfiles with duplicate weight names or duplicate targets — those would yield non-deterministic behavior in FindWeight, Pull, Prepare, and Retain.

func (*WeightsLock) FindWeight

func (wl *WeightsLock) FindWeight(name string) *WeightLockEntry

FindWeight returns the lockfile entry with the given name, or nil if no such entry exists.

func (*WeightsLock) Marshal

func (wl *WeightsLock) Marshal() ([]byte, error)

Marshal serializes the lockfile to canonical JSON bytes. It applies the sort + normalization rules described on Save. Marshal mutates the receiver's entries in place (sorting their Files and Layers); this is safe because the sort order is the canonical order.

func (*WeightsLock) Retain

func (wl *WeightsLock) Retain(keep []string)

Retain removes any entries whose Name is not in keep. The order of surviving entries is preserved. Retain is used after a full import pass to prune weights that were removed from cog.yaml.

func (*WeightsLock) RuntimeManifest

func (wl *WeightsLock) RuntimeManifest() *RuntimeWeightsManifest

RuntimeManifest projects the lockfile into the minimal runtime manifest written to /.cog/weights.json (spec §3.3). The result contains only the fields coglet needs: name, target, and setDigest per weight.

func (*WeightsLock) Save

func (wl *WeightsLock) Save(path string) error

Save writes the weights.lock to disk in canonical JSON form.

Save is deterministic: for any given WeightsLock value, repeated calls produce byte-identical output. It sorts each entry's Files by path and Layers by digest before serializing, normalizes empty Include/Exclude slices to [] (never omitted), and emits standard two-space indent.

The write is atomic: bytes are written to a sibling temp file in the same directory and renamed into place. A killed cog process or concurrent writer cannot leave a half-written lockfile that would later block `cog push` / `cog predict` / `cog train`.

func (*WeightsLock) Upsert

func (wl *WeightsLock) Upsert(entry WeightLockEntry)

Upsert inserts or replaces the entry with the matching Name. It leaves all other entries in place and untouched.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL