pkgstore

package
v0.0.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 18, 2026 License: GPL-3.0 Imports: 15 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ConfigHash 

func ConfigHash(linkingToKeys map[string]string) string

ConfigHash computes the config hash from a LinkingTo dependency map. Entries are sorted by package name to ensure deterministic output. An empty map produces the canonical empty config hash.

func IngestContext 

func IngestContext(
	descPath string, lf *Lockfile,
) (configHash string, linkingToKeys map[string]string,
	sourceCompiled bool, linkingToNames []string, err error)

IngestContext extracts compile-time context from an installed package's DESCRIPTION and computes the config hash from the lockfile. Returns the config hash, LinkingTo store key map, source_compiled flag, and sorted LinkingTo package names.

func ParseDCF 

func ParseDCF(path string) (map[string]string, error)

ParseDCF reads a Debian Control File (DESCRIPTION) into a map.

func PlatformFromLockfile 

func PlatformFromLockfile(lf *Lockfile) string

PlatformFromLockfile derives the platform prefix from a pak lockfile. Uses per-package fields (short form) rather than top-level fields (which contain long human-readable strings like "R version 4.5.2 (2025-10-31)").

func ReadPackageManifest 

func ReadPackageManifest(libDir string) (map[string]string, error)

ReadPackageManifest reads the per-worker package manifest.

func ReadStoreManifest 

func ReadStoreManifest(path string) (map[string]string, error)

ReadStoreManifest reads a store-manifest from the given path. The path should be the full file path (e.g., "{bundle}/store-manifest.json").

func RecoverPlatform 

func RecoverPlatform(storePath string) string

RecoverPlatform scans the store root for existing platform directories (e.g., "4.5-x86_64-pc-linux-gnu/") to restore the platform after a server restart.

func SpawnEvictionSweeper 

func SpawnEvictionSweeper(ctx context.Context, store *Store, retention time.Duration)

SpawnEvictionSweeper starts a background goroutine that periodically evicts stale store entries.

func SplitStoreRef 

func SplitStoreRef(ref string) (sourceHash, configHash string, err error)

SplitStoreRef splits a compound store ref "sourceHash/configHash" into its two components. Returns an error if the ref is malformed.

func StoreKey 

func StoreKey(entry LockfileEntry) (string, error)

StoreKey computes the curated hash for a pak lockfile entry. The hash is SHA-256 of a NUL-delimited string of identity fields.

Hash input format: 

{RemoteType}\0{field1}\0{field2}[...\0{fieldN}]

func StoreRef 

func StoreRef(sourceHash, configHash string) string

StoreRef returns a compound "sourceHash/configHash" string for use in the per-worker package manifest (.packages.json).

func UpdatePackageManifest 

func UpdatePackageManifest(libDir string, additions map[string]string) error

UpdatePackageManifest adds entries to the per-worker package manifest. Existing entries are preserved; additions overwrite on key collision.

func WriteConfigMeta 

func WriteConfigMeta(path string, meta ConfigMeta) error

WriteConfigMeta writes a per-config sidecar file.

func WritePackageManifest 

func WritePackageManifest(libDir string, manifest map[string]string) error

WritePackageManifest writes a per-worker package manifest to libDir.

func WriteStoreConfigs 

func WriteStoreConfigs(path string, sc StoreConfigs) error

WriteStoreConfigs atomically writes configs.json via write-to-temp + rename.

func WriteStoreManifest 

func WriteStoreManifest(dir string, manifest map[string]string) error

WriteStoreManifest writes the store-manifest ({package: "sourceHash/configHash"}) to the given directory. Called by `store ingest` at the end of a build.

Types

type ConfigMeta 

type ConfigMeta struct {
	CreatedAt time.Time `json:"created_at"`
}

ConfigMeta represents the per-config sidecar file.

type Lockfile 

type Lockfile struct {
	LockfileVersion int             `json:"lockfile_version"`
	RVersion        string          `json:"r_version"` // "R version 4.5.2 (2025-10-31)"
	OS              string          `json:"os"`        // "Ubuntu 24.04.2 LTS" (human-readable)
	Platform        string          `json:"platform"`  // "x86_64-pc-linux-gnu" (R triple)
	Packages        []LockfileEntry `json:"packages"`
}

Lockfile is the top-level pak lockfile structure.

func ReadLockfile 

func ReadLockfile(path string) (*Lockfile, error)

ReadLockfile reads and validates a pak lockfile from the given path.

func (*Lockfile) Validate 

func (lf *Lockfile) Validate() error

Validate checks that the pak lockfile has the structure and fields we depend on.

type LockfileEntry 

type LockfileEntry struct {
	Package          string           `json:"package"`
	Version          string           `json:"version"`
	Type             string           `json:"type"` // "standard", "github", etc.
	NeedsCompilation bool             `json:"needscompilation"`
	Metadata         LockfileMetadata `json:"metadata"`
	SHA256           string           `json:"sha256"`   // archive hash (top-level)
	Platform         string           `json:"platform"` // per-pkg R triple, e.g. "x86_64-pc-linux-gnu"
	RVersion         string           `json:"rversion"` // per-pkg short R version, e.g. "4.5"
}

LockfileEntry is a single package record from the pak lockfile.

func (LockfileEntry) IsMetaEntry 

func (e LockfileEntry) IsMetaEntry() bool

IsMetaEntry reports whether this lockfile entry is a pak pseudo-package (e.g. "deps::/app") rather than a real package. Meta-entries are skipped during validation and store operations.

func (LockfileEntry) Validate 

func (e LockfileEntry) Validate() error

Validate checks that a lockfile entry has the fields needed for store key computation and platform detection. Meta-entries (deps, local, installed) are skipped.

type LockfileMetadata 

type LockfileMetadata struct {
	RemoteType   string `json:"RemoteType"`
	RemoteSha    string `json:"RemoteSha"`
	RemoteSubdir string `json:"RemoteSubdir,omitempty"`
}

LockfileMetadata holds the nested metadata object from a pak lockfile entry. Only the fields needed for store key computation are mapped.

type PopulateStats 

type PopulateStats struct {
	Hits        int
	Misses      int
	ABIHits     int
	ABIRebuilds int
}

PopulateStats reports cache hit/miss counts from a populate run.

type Store 

type Store struct {
	// contains filtered or unexported fields
}

Store is a content-addressable package store keyed by {platform}/{package}/{source_hash}/{config_hash}.

func NewStore 

func NewStore(root string) *Store

func (*Store) Acquire 

func (s *Store) Acquire(ctx context.Context, pkg, sourceHash string, staleThreshold time.Duration) error

Acquire takes a file-based lock for a package source hash. Blocks with jittered backoff until the lock is acquired or the context is cancelled. Stale locks older than staleThreshold are removed and re-attempted.

func (*Store) AssembleLibrary 

func (s *Store) AssembleLibrary(
	libDir string, storeManifest map[string]string,
) (missing []string, err error)

AssembleLibrary creates a library directory by hard-linking packages from the store based on a pre-computed store-manifest. After linking, it writes a .packages.json manifest initialized as a copy of the store-manifest.

func (*Store) CleanupStagingDir 

func (s *Store) CleanupStagingDir(dir string) error

CleanupStagingDir removes a staging directory.

func (*Store) CleanupWorkerLib 

func (s *Store) CleanupWorkerLib(workerID string) error

CleanupWorkerLib removes a worker's library directory.

func (*Store) ConfigMetaPath 

func (s *Store) ConfigMetaPath(pkg, sourceHash, configHash string) string

ConfigMetaPath returns the config sidecar file path.

func (*Store) ConfigsPath 

func (s *Store) ConfigsPath(pkg, sourceHash string) string

ConfigsPath returns the path to configs.json for a source hash.

func (*Store) CreateStagingDir 

func (s *Store) CreateStagingDir() (string, error)

CreateStagingDir creates a staging directory under the store root. Used by runtime package installation to provide a working directory on the same filesystem as the store (enabling atomic rename and hardlinks).

func (*Store) EvictStale 

func (s *Store) EvictStale(ctx context.Context, retention time.Duration) (int, error)

EvictStale removes config entries whose sidecar mtime is older than the retention cutoff. Returns the number of configs removed.

func (*Store) Has 

func (s *Store) Has(pkg, sourceHash, configHash string) bool

Has reports whether the store contains a specific config for a package.

func (*Store) Ingest 

func (s *Store) Ingest(pkg, sourceHash, configHash, srcDir string) error

Ingest atomically moves an installed package tree into the store as a config entry. No-op if the config already exists. srcDir must be on the same filesystem as the store (for atomic rename).

func (*Store) IngestPackages 

func (s *Store) IngestPackages(ctx context.Context, lf *Lockfile, lib string, refManifest map[string]string) (map[string]string, error)

IngestPackages iterates over lockfile entries, ingests any new or changed packages into the store, and returns the complete store-manifest map. Packages already present in refManifest with matching compound refs are skipped. Remaining refManifest entries are carried forward so the manifest is complete for container transfer.

func (*Store) LockPath 

func (s *Store) LockPath(pkg, sourceHash string) string

LockPath returns the lock directory path for a package source hash.

func (*Store) Path 

func (s *Store) Path(pkg, sourceHash, configHash string) string

Path returns the config directory (installed package tree) path.

func (*Store) Platform 

func (s *Store) Platform() string

func (*Store) PopulateBuild 

func (s *Store) PopulateBuild(
	lf *Lockfile, lib string, refManifest map[string]string,
) (PopulateStats, error)

PopulateBuild is the standard build-time populate: for each lockfile entry, check the store for a matching config and hardlink hits into the build library.

func (*Store) PopulateRuntime 

func (s *Store) PopulateRuntime(
	lf *Lockfile, lib, refLib string, refManifest map[string]string,
) (PopulateStats, error)

PopulateRuntime extends the build-time populate with worker library pre-population and the reverse-LinkingTo ABI check. After this function, the staging directory contains a complete library minus packages that need ABI recompilation.

func (*Store) Release 

func (s *Store) Release(pkg, sourceHash string)

Release removes the lock directory for a package source hash.

func (*Store) ResolveConfig 

func (s *Store) ResolveConfig(
	pkg, sourceHash string, lf *Lockfile,
) (configHash string, ok bool)

ResolveConfig reads configs.json for a package's source hash and returns the config hash that matches the current lockfile's LinkingTo store keys. Returns ("", false) if no matching config exists (miss) or if configs.json doesn't exist (never seen).

func (*Store) Root 

func (s *Store) Root() string

func (*Store) SetPlatform 

func (s *Store) SetPlatform(p string)

func (*Store) SourceDir 

func (s *Store) SourceDir(pkg, sourceHash string) string

SourceDir returns the source-hash directory for a package.

func (*Store) Touch 

func (s *Store) Touch(pkg, sourceHash, configHash string)

Touch updates the mtime of a config's sidecar file. Used for last-accessed tracking — the eviction sweeper removes config entries whose sidecar mtime exceeds the retention window.

func (*Store) WorkerLibDir 

func (s *Store) WorkerLibDir(workerID string) string

WorkerLibDir returns the host-side library directory for a worker.

func (*Store) WriteIngestMeta 

func (s *Store) WriteIngestMeta(
	entry LockfileEntry, lf *Lockfile,
	sourceHash, configHash string, linkingToKeys map[string]string,
	sourceCompiled bool, linkingToNames []string,
) error

WriteIngestMeta writes the config sidecar and updates configs.json for a newly ingested package config.

type StoreConfigs 

type StoreConfigs struct {
	SourceCompiled bool                         `json:"source_compiled"`
	LinkingTo      []string                     `json:"linkingto"`
	Configs        map[string]map[string]string `json:"configs"`
}

StoreConfigs represents the configs.json file at the source-hash level.

func ReadStoreConfigs 

func ReadStoreConfigs(path string) (StoreConfigs, error)

ReadStoreConfigs reads a configs.json file.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.