Documentation
¶
Overview ¶
Package store provides reference implementations of the artifactkit storage interfaces: a filesystem blob CAS, a SQLite/postgres/mysql metadata index, and the credential tables. It is the only module that pulls a database driver, so the protocol adapters stay dependency-light.
Index ¶
- Constants
- func BlobBackends() []string
- func EnsureDir(dir string) error
- func OpenBlobStore(backend, dir string) (artifactkit.BlobStore, error)
- func RegisterBlobBackend(name string, build func(dir string) (artifactkit.BlobStore, error))
- func RunReaper(ctx context.Context, s *Store, blobs artifactkit.BlobStore, ...)
- type BlobStat
- type DriverConfig
- type FileBlobStore
- func (s *FileBlobStore) Delete(ctx context.Context, digest string) error
- func (s *FileBlobStore) Hashes(ctx context.Context, digest string) (artifactkit.Hashes, bool, error)
- func (s *FileBlobStore) List(ctx context.Context) ([]artifactkit.BlobInfo, error)
- func (s *FileBlobStore) Open(ctx context.Context, digest string) (io.ReadSeekCloser, error)
- func (s *FileBlobStore) Put(ctx context.Context, r io.Reader, wantDigest string) (artifactkit.Stored, bool, error)
- func (s *FileBlobStore) Stat(ctx context.Context, digest string) (*artifactkit.BlobInfo, error)
- type FormatStat
- type GCStats
- type Stats
- type Store
- func (s *Store) AuthTokenCount(ctx context.Context) (int64, error)
- func (s *Store) Close() error
- func (s *Store) CreateAuthToken(ctx context.Context, token string, userID int64, level string) error
- func (s *Store) CreateAuthUser(ctx context.Context, username string) (int64, error)
- func (s *Store) DB() *gorm.DB
- func (s *Store) Delete(ctx context.Context, format, repository, version string) error
- func (s *Store) DeleteRepo(ctx context.Context, format, repository string) (int, error)
- func (s *Store) DeleteTarget(ctx context.Context, id string) error
- func (s *Store) DeleteUpload(ctx context.Context, id string) error
- func (s *Store) ExpireNegative(ctx context.Context, now time.Time) (int, error)
- func (s *Store) Get(ctx context.Context, format, repository, version string) (artifactkit.Artifact, error)
- func (s *Store) GetMeta(ctx context.Context, format, repository string) ([]byte, error)
- func (s *Store) GetUpload(ctx context.Context, id string) (artifactkit.UploadRecord, error)
- func (s *Store) ListArtifacts(ctx context.Context, format, repository string) ([]artifactkit.Artifact, error)
- func (s *Store) ListPackages(ctx context.Context) ([]artifactkit.PackageSummary, error)
- func (s *Store) ListRepositories(ctx context.Context) ([]string, error)
- func (s *Store) ListRepositoriesByFormat(ctx context.Context, format string) ([]string, error)
- func (s *Store) ListTargets(ctx context.Context) ([]targets.Target, error)
- func (s *Store) ListUploads(ctx context.Context) ([]string, error)
- func (s *Store) ListVersions(ctx context.Context, format, repository string) ([]string, error)
- func (s *Store) LookupToken(ctx context.Context, token string) (artifactkit.Principal, bool)
- func (s *Store) LookupUsername(ctx context.Context, username string) (artifactkit.Principal, bool)
- func (s *Store) OpenInstance(ctx context.Context) bool
- func (s *Store) Put(ctx context.Context, a artifactkit.Artifact) error
- func (s *Store) PutTarget(ctx context.Context, t targets.Target) error
- func (s *Store) ReapOrphanBlobs(ctx context.Context, blobs artifactkit.BlobStore, grace time.Duration) (GCStats, error)
- func (s *Store) ReferencedDigests(ctx context.Context) (map[string]bool, error)
- func (s *Store) SaveUpload(ctx context.Context, u artifactkit.UploadRecord) error
- func (s *Store) SetMeta(ctx context.Context, format, repository string, data []byte) error
- func (s *Store) Stats(ctx context.Context, blobs artifactkit.BlobStore) (Stats, error)
Constants ¶
const ( BlobFilesystem = "filesystem" BlobS3 = "s3" )
const ( KindSQLite = "sqlite" KindPostgres = "postgres" KindMySQL = "mysql" )
Metadata backend kinds (shared with easyvcs).
Variables ¶
This section is empty.
Functions ¶
func BlobBackends ¶ added in v0.16.0
func BlobBackends() []string
BlobBackends lists the registered backend names (filesystem plus any).
func OpenBlobStore ¶
func OpenBlobStore(backend, dir string) (artifactkit.BlobStore, error)
OpenBlobStore returns a BlobStore for the given backend. "filesystem" (default; a filesystem CAS rooted at dir) is built in. Other backends (e.g. "s3") register a constructor via RegisterBlobBackend from their own module, so this package never imports a heavy cloud SDK and the adapter module graph stays lean. Requesting an unregistered backend fails startup loudly — a silent fallback would put multi-GB blobs on the wrong storage.
func RegisterBlobBackend ¶ added in v0.16.0
func RegisterBlobBackend(name string, build func(dir string) (artifactkit.BlobStore, error))
RegisterBlobBackend installs a constructor for a backend name. A separate module (e.g. s3blob) calls this from its init() so consumers enable it with a blank import; OpenBlobStore then resolves it by name.
func RunReaper ¶ added in v0.17.0
func RunReaper(ctx context.Context, s *Store, blobs artifactkit.BlobStore, interval, grace time.Duration, logf func(string, ...any))
RunReaper runs the storage reaper on a ticker until ctx is cancelled: it expires negative cache entries and reclaims orphan blobs. It is the shared entry point for both the standalone server (cmd) and easylab's embedded registry, so a long-running deployment always reclaims space. interval <= 0 disables it. logf may be nil.
Types ¶
type DriverConfig ¶
DriverConfig selects the metadata backend. Kind is "sqlite", "postgres", or "mysql"/"mariadb" (aliased to mysql). It shares EASYVCS_DB_DRIVER / DSN with the easyvcs engine so one deployment configures both.
type FileBlobStore ¶
type FileBlobStore struct {
// contains filtered or unexported fields
}
FileBlobStore is a content-addressed store: one file per digest, addressed as `sha256/<first-two>/<rest>`, with a `.hashes.json` sidecar recording the hash set computed at write time. Bytes never enter RAM (atomic rename from a temp file), so multi-GB layers are safe.
func NewFileBlobStore ¶
func NewFileBlobStore(dir string) (*FileBlobStore, error)
NewFileBlobStore opens a filesystem CAS rooted at dir.
func (*FileBlobStore) Delete ¶
func (s *FileBlobStore) Delete(ctx context.Context, digest string) error
Delete implements BlobStore.
func (*FileBlobStore) Hashes ¶ added in v0.24.0
func (s *FileBlobStore) Hashes(ctx context.Context, digest string) (artifactkit.Hashes, bool, error)
Hashes implements BlobStore: the hash set recorded at write time. ok=false when no sidecar exists (a blob written before this feature, or by another tool); the caller may then fall back to computing it.
func (*FileBlobStore) List ¶
func (s *FileBlobStore) List(ctx context.Context) ([]artifactkit.BlobInfo, error)
List implements BlobStore.
func (*FileBlobStore) Open ¶
func (s *FileBlobStore) Open(ctx context.Context, digest string) (io.ReadSeekCloser, error)
Open implements BlobStore.
func (*FileBlobStore) Put ¶ added in v0.24.0
func (s *FileBlobStore) Put(ctx context.Context, r io.Reader, wantDigest string) (artifactkit.Stored, bool, error)
Put implements BlobStore: stream to a temp file while computing every hash in one pass, verify against wantDigest (when set), write the hash sidecar, then publish with an atomic rename. A partially written file is never visible. Returns (stored=false) when the blob already existed.
type FormatStat ¶ added in v0.16.0
type FormatStat struct {
Format string `json:"format"`
Rows int64 `json:"rows"`
Blobs int64 `json:"blobs"` // distinct digests referenced
Bytes int64 `json:"bytes"`
Expired int64 `json:"expired"` // mutable entries past ExpiresAt
}
FormatStat is one format's row count and referenced bytes.
type GCStats ¶ added in v0.16.0
type GCStats struct {
NegativeExpired int // negative (404/410) rows removed
OrphanBlobs int // CAS blobs with no referencing row removed
BlobsKept int // blobs still referenced
BytesFreed int64 // bytes reclaimed from orphan blobs
}
GCStats summarizes one garbage-collection pass.
type Stats ¶ added in v0.16.0
type Stats struct {
Formats []FormatStat `json:"formats"`
Blobs BlobStat `json:"blobs"`
}
Stats summarizes the registry's on-disk footprint: row counts and bytes per format, and the CAS totals. It backs the admin /stats endpoint so an operator can see growth without scanning the filesystem by hand.
type Store ¶
type Store struct {
// contains filtered or unexported fields
}
Store is a metadata (IndexStore) engine backed by GORM, so it can run on sqlite (default, via glebarez/sqlite which wraps modernc — no cgo), postgres (pgx), or mysql/mariadb (go-sql-driver) with one code path. Blob bytes live separately (see FileBlobStore / the blob factory), never inline here.
func OpenSQLite ¶
OpenSQLite opens the metadata store at a SQLite path (legacy helper; the convenient wrapper over OpenStore for the default backend).
func OpenStore ¶
func OpenStore(cfg DriverConfig) (*Store, error)
OpenStore opens the metadata store on the configured backend. The semantic layer (artifactkit protocol handlers) is agnostic — only this boundary changes. sqlite uses the same pure-Go dialector (glebarez/sqlite) as easyvcs, so the "sqlite" database/sql driver is registered exactly once per process.
func (*Store) AuthTokenCount ¶ added in v0.3.0
AuthTokenCount returns the number of registered credentials (diagnostics).
func (*Store) CreateAuthToken ¶ added in v0.3.0
func (s *Store) CreateAuthToken(ctx context.Context, token string, userID int64, level string) error
CreateAuthToken registers a credential for a user. The plaintext is hashed on the way in and never persisted. level is "read" or "write".
func (*Store) CreateAuthUser ¶ added in v0.3.0
CreateAuthUser inserts a user (idempotent on the username).
func (*Store) DB ¶
DB exposes the underlying handle so callers may coordinate with a blob store when needed. Kept for compatibility; blob stores no longer reuse it by default (blob lives on the filesystem / S3).
func (*Store) DeleteRepo ¶
DeleteRepo implements IndexStore.
func (*Store) DeleteTarget ¶ added in v0.13.0
DeleteTarget implements targets.Store.
func (*Store) DeleteUpload ¶
DeleteUpload implements IndexStore.
func (*Store) ExpireNegative ¶ added in v0.16.0
ExpireNegative removes netcache negative (404/410) rows whose TTL has passed. These hold no bytes (the blob is empty), so deleting them never violates "fetch once" — a later request re-probes the origin, as intended.
func (*Store) Get ¶
func (s *Store) Get(ctx context.Context, format, repository, version string) (artifactkit.Artifact, error)
Get implements IndexStore.
func (*Store) ListArtifacts ¶ added in v0.19.0
func (s *Store) ListArtifacts(ctx context.Context, format, repository string) ([]artifactkit.Artifact, error)
ListArtifacts implements IndexStore: every row for a repository in one query.
func (*Store) ListPackages ¶
ListPackages implements IndexStore.
func (*Store) ListRepositories ¶
ListRepositories implements IndexStore.
func (*Store) ListRepositoriesByFormat ¶
ListRepositoriesByFormat implements IndexStore.
func (*Store) ListTargets ¶ added in v0.13.0
ListTargets implements targets.Store over the metadata DB.
func (*Store) ListUploads ¶
ListUploads implements IndexStore.
func (*Store) ListVersions ¶
ListVersions implements IndexStore.
func (*Store) LookupToken ¶ added in v0.3.0
LookupToken implements artifactkit.TokenStore.
func (*Store) LookupUsername ¶ added in v0.3.0
LookupUsername implements artifactkit.TokenStore: the strongest credential level registered for the user (write wins).
func (*Store) OpenInstance ¶ added in v0.3.0
OpenInstance implements artifactkit.TokenStore: no users registered means a fresh single-user deployment where anonymous write is permitted.
func (*Store) ReapOrphanBlobs ¶ added in v0.16.0
func (s *Store) ReapOrphanBlobs(ctx context.Context, blobs artifactkit.BlobStore, grace time.Duration) (GCStats, error)
ReapOrphanBlobs deletes CAS blobs that no artifact row references and whose file is older than grace (so a blob being written by an in-flight fetch is never removed). A non-positive grace uses a 1h default. minAge protects recently-added blobs from a race with a concurrent Put that has not yet been indexed.
func (*Store) ReferencedDigests ¶ added in v0.17.0
ReferencedDigests implements artifactkit.IndexStore: every CAS digest any artifact row references (primary Digest plus each blob descriptor), in one pass over the table.
func (*Store) SaveUpload ¶
SaveUpload implements IndexStore.