store

package
v0.24.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 20, 2026 License: MIT Imports: 21 Imported by: 0

Documentation

Overview

Package store provides reference implementations of the artifactkit storage interfaces: a filesystem blob CAS, a SQLite/postgres/mysql metadata index, and the credential tables. It is the only module that pulls a database driver, so the protocol adapters stay dependency-light.

Index

Constants

View Source
const (
	BlobFilesystem = "filesystem"
	BlobS3         = "s3"
)
View Source
const (
	KindSQLite   = "sqlite"
	KindPostgres = "postgres"
	KindMySQL    = "mysql"
)

Metadata backend kinds (shared with easyvcs).

Variables

This section is empty.

Functions

func BlobBackends added in v0.16.0

func BlobBackends() []string

BlobBackends lists the registered backend names (filesystem plus any).

func EnsureDir

func EnsureDir(dir string) error

EnsureDir creates a directory (and parents) if it does not exist.

func OpenBlobStore

func OpenBlobStore(backend, dir string) (artifactkit.BlobStore, error)

OpenBlobStore returns a BlobStore for the given backend. "filesystem" (default; a filesystem CAS rooted at dir) is built in. Other backends (e.g. "s3") register a constructor via RegisterBlobBackend from their own module, so this package never imports a heavy cloud SDK and the adapter module graph stays lean. Requesting an unregistered backend fails startup loudly — a silent fallback would put multi-GB blobs on the wrong storage.

func RegisterBlobBackend added in v0.16.0

func RegisterBlobBackend(name string, build func(dir string) (artifactkit.BlobStore, error))

RegisterBlobBackend installs a constructor for a backend name. A separate module (e.g. s3blob) calls this from its init() so consumers enable it with a blank import; OpenBlobStore then resolves it by name.

func RunReaper added in v0.17.0

func RunReaper(ctx context.Context, s *Store, blobs artifactkit.BlobStore, interval, grace time.Duration, logf func(string, ...any))

RunReaper runs the storage reaper on a ticker until ctx is cancelled: it expires negative cache entries and reclaims orphan blobs. It is the shared entry point for both the standalone server (cmd) and easylab's embedded registry, so a long-running deployment always reclaims space. interval <= 0 disables it. logf may be nil.

Types

type BlobStat added in v0.16.0

type BlobStat struct {
	Objects int64 `json:"objects"`
	Bytes   int64 `json:"bytes"`
}

BlobStat is the CAS-wide total.

type DriverConfig

type DriverConfig struct {
	Kind string
	DSN  string
}

DriverConfig selects the metadata backend. Kind is "sqlite", "postgres", or "mysql"/"mariadb" (aliased to mysql). It shares EASYVCS_DB_DRIVER / DSN with the easyvcs engine so one deployment configures both.

type FileBlobStore

type FileBlobStore struct {
	// contains filtered or unexported fields
}

FileBlobStore is a content-addressed store: one file per digest, addressed as `sha256/<first-two>/<rest>`, with a `.hashes.json` sidecar recording the hash set computed at write time. Bytes never enter RAM (atomic rename from a temp file), so multi-GB layers are safe.

func NewFileBlobStore

func NewFileBlobStore(dir string) (*FileBlobStore, error)

NewFileBlobStore opens a filesystem CAS rooted at dir.

func (*FileBlobStore) Delete

func (s *FileBlobStore) Delete(ctx context.Context, digest string) error

Delete implements BlobStore.

func (*FileBlobStore) Hashes added in v0.24.0

func (s *FileBlobStore) Hashes(ctx context.Context, digest string) (artifactkit.Hashes, bool, error)

Hashes implements BlobStore: the hash set recorded at write time. ok=false when no sidecar exists (a blob written before this feature, or by another tool); the caller may then fall back to computing it.

func (*FileBlobStore) List

func (s *FileBlobStore) List(ctx context.Context) ([]artifactkit.BlobInfo, error)

List implements BlobStore.

func (*FileBlobStore) Open

func (s *FileBlobStore) Open(ctx context.Context, digest string) (io.ReadSeekCloser, error)

Open implements BlobStore.

func (*FileBlobStore) Put added in v0.24.0

func (s *FileBlobStore) Put(ctx context.Context, r io.Reader, wantDigest string) (artifactkit.Stored, bool, error)

Put implements BlobStore: stream to a temp file while computing every hash in one pass, verify against wantDigest (when set), write the hash sidecar, then publish with an atomic rename. A partially written file is never visible. Returns (stored=false) when the blob already existed.

func (*FileBlobStore) Stat

func (s *FileBlobStore) Stat(ctx context.Context, digest string) (*artifactkit.BlobInfo, error)

Stat implements BlobStore.

type FormatStat added in v0.16.0

type FormatStat struct {
	Format  string `json:"format"`
	Rows    int64  `json:"rows"`
	Blobs   int64  `json:"blobs"` // distinct digests referenced
	Bytes   int64  `json:"bytes"`
	Expired int64  `json:"expired"` // mutable entries past ExpiresAt
}

FormatStat is one format's row count and referenced bytes.

type GCStats added in v0.16.0

type GCStats struct {
	NegativeExpired int   // negative (404/410) rows removed
	OrphanBlobs     int   // CAS blobs with no referencing row removed
	BlobsKept       int   // blobs still referenced
	BytesFreed      int64 // bytes reclaimed from orphan blobs
}

GCStats summarizes one garbage-collection pass.

type Stats added in v0.16.0

type Stats struct {
	Formats []FormatStat `json:"formats"`
	Blobs   BlobStat     `json:"blobs"`
}

Stats summarizes the registry's on-disk footprint: row counts and bytes per format, and the CAS totals. It backs the admin /stats endpoint so an operator can see growth without scanning the filesystem by hand.

type Store

type Store struct {
	// contains filtered or unexported fields
}

Store is a metadata (IndexStore) engine backed by GORM, so it can run on sqlite (default, via glebarez/sqlite which wraps modernc — no cgo), postgres (pgx), or mysql/mariadb (go-sql-driver) with one code path. Blob bytes live separately (see FileBlobStore / the blob factory), never inline here.

func OpenSQLite

func OpenSQLite(path string) (*Store, error)

OpenSQLite opens the metadata store at a SQLite path (legacy helper; the convenient wrapper over OpenStore for the default backend).

func OpenStore

func OpenStore(cfg DriverConfig) (*Store, error)

OpenStore opens the metadata store on the configured backend. The semantic layer (artifactkit protocol handlers) is agnostic — only this boundary changes. sqlite uses the same pure-Go dialector (glebarez/sqlite) as easyvcs, so the "sqlite" database/sql driver is registered exactly once per process.

func (*Store) AuthTokenCount added in v0.3.0

func (s *Store) AuthTokenCount(ctx context.Context) (int64, error)

AuthTokenCount returns the number of registered credentials (diagnostics).

func (*Store) Close

func (s *Store) Close() error

Close implements IndexStore.

func (*Store) CreateAuthToken added in v0.3.0

func (s *Store) CreateAuthToken(ctx context.Context, token string, userID int64, level string) error

CreateAuthToken registers a credential for a user. The plaintext is hashed on the way in and never persisted. level is "read" or "write".

func (*Store) CreateAuthUser added in v0.3.0

func (s *Store) CreateAuthUser(ctx context.Context, username string) (int64, error)

CreateAuthUser inserts a user (idempotent on the username).

func (*Store) DB

func (s *Store) DB() *gorm.DB

DB exposes the underlying handle so callers may coordinate with a blob store when needed. Kept for compatibility; blob stores no longer reuse it by default (blob lives on the filesystem / S3).

func (*Store) Delete

func (s *Store) Delete(ctx context.Context, format, repository, version string) error

Delete implements IndexStore.

func (*Store) DeleteRepo

func (s *Store) DeleteRepo(ctx context.Context, format, repository string) (int, error)

DeleteRepo implements IndexStore.

func (*Store) DeleteTarget added in v0.13.0

func (s *Store) DeleteTarget(ctx context.Context, id string) error

DeleteTarget implements targets.Store.

func (*Store) DeleteUpload

func (s *Store) DeleteUpload(ctx context.Context, id string) error

DeleteUpload implements IndexStore.

func (*Store) ExpireNegative added in v0.16.0

func (s *Store) ExpireNegative(ctx context.Context, now time.Time) (int, error)

ExpireNegative removes netcache negative (404/410) rows whose TTL has passed. These hold no bytes (the blob is empty), so deleting them never violates "fetch once" — a later request re-probes the origin, as intended.

func (*Store) Get

func (s *Store) Get(ctx context.Context, format, repository, version string) (artifactkit.Artifact, error)

Get implements IndexStore.

func (*Store) GetMeta

func (s *Store) GetMeta(ctx context.Context, format, repository string) ([]byte, error)

GetMeta implements IndexStore.

func (*Store) GetUpload

func (s *Store) GetUpload(ctx context.Context, id string) (artifactkit.UploadRecord, error)

GetUpload implements IndexStore.

func (*Store) ListArtifacts added in v0.19.0

func (s *Store) ListArtifacts(ctx context.Context, format, repository string) ([]artifactkit.Artifact, error)

ListArtifacts implements IndexStore: every row for a repository in one query.

func (*Store) ListPackages

func (s *Store) ListPackages(ctx context.Context) ([]artifactkit.PackageSummary, error)

ListPackages implements IndexStore.

func (*Store) ListRepositories

func (s *Store) ListRepositories(ctx context.Context) ([]string, error)

ListRepositories implements IndexStore.

func (*Store) ListRepositoriesByFormat

func (s *Store) ListRepositoriesByFormat(ctx context.Context, format string) ([]string, error)

ListRepositoriesByFormat implements IndexStore.

func (*Store) ListTargets added in v0.13.0

func (s *Store) ListTargets(ctx context.Context) ([]targets.Target, error)

ListTargets implements targets.Store over the metadata DB.

func (*Store) ListUploads

func (s *Store) ListUploads(ctx context.Context) ([]string, error)

ListUploads implements IndexStore.

func (*Store) ListVersions

func (s *Store) ListVersions(ctx context.Context, format, repository string) ([]string, error)

ListVersions implements IndexStore.

func (*Store) LookupToken added in v0.3.0

func (s *Store) LookupToken(ctx context.Context, token string) (artifactkit.Principal, bool)

LookupToken implements artifactkit.TokenStore.

func (*Store) LookupUsername added in v0.3.0

func (s *Store) LookupUsername(ctx context.Context, username string) (artifactkit.Principal, bool)

LookupUsername implements artifactkit.TokenStore: the strongest credential level registered for the user (write wins).

func (*Store) OpenInstance added in v0.3.0

func (s *Store) OpenInstance(ctx context.Context) bool

OpenInstance implements artifactkit.TokenStore: no users registered means a fresh single-user deployment where anonymous write is permitted.

func (*Store) Put

func (s *Store) Put(ctx context.Context, a artifactkit.Artifact) error

Put implements IndexStore.

func (*Store) PutTarget added in v0.13.0

func (s *Store) PutTarget(ctx context.Context, t targets.Target) error

PutTarget implements targets.Store (upsert by ID).

func (*Store) ReapOrphanBlobs added in v0.16.0

func (s *Store) ReapOrphanBlobs(ctx context.Context, blobs artifactkit.BlobStore, grace time.Duration) (GCStats, error)

ReapOrphanBlobs deletes CAS blobs that no artifact row references and whose file is older than grace (so a blob being written by an in-flight fetch is never removed). A non-positive grace uses a 1h default. minAge protects recently-added blobs from a race with a concurrent Put that has not yet been indexed.

func (*Store) ReferencedDigests added in v0.17.0

func (s *Store) ReferencedDigests(ctx context.Context) (map[string]bool, error)

ReferencedDigests implements artifactkit.IndexStore: every CAS digest any artifact row references (primary Digest plus each blob descriptor), in one pass over the table.

func (*Store) SaveUpload

func (s *Store) SaveUpload(ctx context.Context, u artifactkit.UploadRecord) error

SaveUpload implements IndexStore.

func (*Store) SetMeta

func (s *Store) SetMeta(ctx context.Context, format, repository string, data []byte) error

SetMeta implements IndexStore.

func (*Store) Stats added in v0.16.0

func (s *Store) Stats(ctx context.Context, blobs artifactkit.BlobStore) (Stats, error)

Stats computes the registry footprint. The per-format "bytes" sums each distinct digest once per format (shared CAS bytes counted per referencing format, which is what an operator wants for attribution).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL