Documentation
¶
Overview ¶
GitStore implements Store on top of a Git repository.
Its mirror clone is bare: fetch-only, with no working tree and no HEAD to manage (git.CloneOrUpdateBareMirror). Publish exports that mirror's tree for the requested revision via git.ExportTree (handling regular files, symlinks and submodules), then decrypts every SOPS-encrypted file the export produced (decrypt.go) before the artifact is published.
OCIStore implements Store on top of an OCI artifact reference (a fixed repository and tag, e.g. "ghcr.io/org/repo:latest"). See the package doc on git_store.go for how OCIStore's role fits into the Phase 2 plan.
Package store maps mutable source references to immutable revisions and materialized artifact directories. GitStore and OCIStore keep source-specific fetching, verification, and export details behind the Store interface.
Index ¶
- Constants
- Variables
- func ArtifactDirName(revision Revision) string
- type Artifact
- type GCOptions
- type GCResult
- type GitStore
- func (s *GitStore) List() ([]Artifact, error)
- func (s *GitStore) Lookup(revision Revision) (Artifact, bool, error)
- func (s *GitStore) MirrorDir() string
- func (s *GitStore) Publish(ctx context.Context, revision Revision) (Artifact, error)
- func (s *GitStore) Resolve(ctx context.Context, ref string) (Revision, error)
- type GitStoreOptions
- type OCIStore
- type OCIStoreOptions
- type Revision
- type Store
Constants ¶
const ArtifactsSubdir = "artifacts"
ArtifactsSubdir is the fixed directory name, relative to a store's base directory, that holds one subdirectory per published revision.
const LiveSubdir = "live"
LiveSubdir is the fixed directory name, relative to a store's base directory, that holds the mutable copies of deployed files that are updated in place instead of per revision (e.g. bind mounts that are excluded from recreation with recreate.ignore).
const MirrorSubdir = "mirror"
MirrorSubdir is the fixed directory name, relative to a store's base directory, that holds GitStore's underlying bare mirror clone. Exported so the startup migration pass (internal/migration) can recognize and lay out the same directory a fresh GitStore would use, without importing GitStore itself.
const SubmodulesSubdir = "submodules"
SubmodulesSubdir is the fixed directory name for cached Git submodule mirrors.
Variables ¶
var ErrRevisionNotFound = errors.New("source store: revision not found")
ErrRevisionNotFound is returned by Publish when the requested Revision is not reachable in the store's local state and cannot be materialized without an unbounded fetch. Callers should Resolve the owning reference first, which fetches exactly what is needed to reach it.
Functions ¶
func ArtifactDirName ¶
ArtifactDirName exposes the revision-to-directory-name encoding applied by artifactPath, for callers that need to locate or lay out an artifact directory without going through a Store.
Types ¶
type Artifact ¶
type Artifact struct {
// Revision is the immutable revision this artifact was published for.
Revision Revision
// Path is the absolute directory holding the revision's deployable
// contents.
Path string
}
Artifact is a materialized, read-only directory for one Revision. Once Publish returns an Artifact, its Path is never written to again by the store that produced it - readers may use it without holding any lock.
func ListArtifacts ¶
ListArtifacts returns every artifact already published under baseDir, without requiring a fully constructed Store. GitStore.List and OCIStore.List are both thin wrappers over this same function; it is exported directly for callers (e.g. garbage collection, see internal/gc) that need to inspect a base directory without holding the per-source-type options (remote URL, credentials, ...) a real Store requires.
type GCOptions ¶
type GCOptions struct {
// RetentionRecords is the number of most-recently-modified unreferenced
// artifacts that are always kept, regardless of RetentionTTL.
// Mirrors Flux's --artifact-retention-records.
RetentionRecords int
// RetentionTTL is how long an unreferenced artifact beyond
// RetentionRecords is kept before Sweep is allowed to remove it.
// Mirrors Flux's --artifact-retention-ttl.
RetentionTTL time.Duration
}
GCOptions configures artifact garbage collection for one store's base directory (see Sweep). It mirrors Flux's artifact-retention model: RetentionRecords always wins over RetentionTTL, so a repository that is simply quiet for a long time never loses its most recent artifacts.
type GCResult ¶
type GCResult struct {
// Kept lists every artifact Sweep decided not to remove: referenced
// (live), within RetentionRecords, or younger than RetentionTTL.
Kept []Artifact
// Removed lists every artifact Sweep deleted.
Removed []Artifact
}
GCResult reports what Sweep did for one base directory.
func Sweep ¶
Sweep removes unreferenced, expired artifacts under baseDir.
live is the hard safety check: any revision present in it - regardless of how it got there - is always kept, no matter its age or position in the retention ranking. Callers are expected to build live from every revision currently referenced by a deployed stack (across every configured Docker context) plus every revision currently being prepared for a deployment that has not finished yet (see the source package's in-flight tracker); Sweep itself has no way to tell the two apart and does not need to.
Artifacts not in live are ranked by their directory's modification time, newest first. The newest RetentionRecords of them are always kept; the rest are removed once older than RetentionTTL, measured against now (an explicit clock so tests are deterministic).
Candidates for removal are never anything outside "<baseDir>/artifacts/<revision>" (via listArtifacts) - the mirror clone, its lock file, the submodule cache, and any in-progress temporary publish directory are never listed as candidates in the first place, so they can never be removed here. Once a candidate is actually removed, its "<path>.lock" sibling - left behind by DeployStack's per-artifact cross-process lock - is removed alongside it.
A single artifact that fails to stat or remove is recorded as kept and does not stop the sweep of the others; all such errors are joined and returned alongside the (still valid) result.
type GitStore ¶
type GitStore struct {
// contains filtered or unexported fields
}
GitStore is a Git-backed Store. See the package doc for its current bridging scope.
func NewGitStore ¶
func NewGitStore(opts GitStoreOptions) (*GitStore, error)
NewGitStore returns a GitStore configured by opts.
func (*GitStore) MirrorDir ¶
MirrorDir returns the absolute path of the store's bare mirror clone. Read-only callers that need a *git.Repository handle for history queries (e.g. resolving a different reference, or GetLatestCommit/ GetChangedFilesBetweenCommits-style lookups) can open it directly with git.OpenRepository. Such reads should still hold a shared sourcecache.AcquireSharedPathLock(mirrorDir) for their duration: Resolve/ Publish only exclude each other via the matching exclusive lock, so an unguarded read can otherwise observe the mirror mid-fetch from a concurrent stack sharing this repository.
func (*GitStore) Publish ¶
Publish exports the mirror's tree at revision into a read-only artifact directory. revision must already be reachable in the mirror - normally guaranteed by a prior Resolve call for the reference that named it. If it is not reachable, Publish returns ErrRevisionNotFound rather than silently doing an unbounded fetch to find it.
type GitStoreOptions ¶
type GitStoreOptions struct {
// Log receives debug/info output. Required.
Log *slog.Logger
// CloneURL is the repository to fetch from. Required.
CloneURL string
// BaseDir is the store's private directory: it holds the mirror clone
// at "<BaseDir>/mirror" and published artifacts under
// "<BaseDir>/artifacts/<revision>". Required.
BaseDir string
Private bool
SSHPrivateKey string
SSHPrivateKeyPassphrase string
AccessToken string
SkipTLSVerify bool
ProxyOptions transport.ProxyOptions
// CloneSubmodules enables submodule materialization in Publish, via
// git.ExportTree's own fetch-and-recurse logic. It has no effect on
// Resolve/the mirror clone itself: the mirror never needs its
// submodules checked out, since Publish reads tree objects, not the
// mirror's worktree.
CloneSubmodules bool
// Depth is the shallow-clone depth passed to the underlying sync.
// 0 clones full history.
Depth int
}
GitStoreOptions configures a GitStore. It mirrors the parameters that git.CloneOrUpdateRepository already takes, so a GitStore can be constructed from the same values callers use today.
type OCIStore ¶
type OCIStore struct {
// contains filtered or unexported fields
}
OCIStore is an OCI-backed Store.
func NewOCIStore ¶
func NewOCIStore(opts OCIStoreOptions) (*OCIStore, error)
NewOCIStore returns an OCIStore configured by opts.
func (*OCIStore) Publish ¶
Publish pulls and extracts the artifact pinned to revision into a read-only artifact directory. revision is passed to the pull as the expected digest, so the artifact actually materialized is always the one Resolve verified - regardless of what the store's tag currently points to.
func (*OCIStore) Resolve ¶
Resolve resolves ref - a digest, or empty/"latest" to mean "whatever the store's configured tag currently points to" - against the registry, and verifies it against the store's cosign trust policy before returning it. A digest that fails verification is never returned as a Revision.
type OCIStoreOptions ¶
type OCIStoreOptions struct {
// Log receives debug/info output. Defaults to slog.Default().
Log *slog.Logger
// ArtifactRef is the OCI reference to resolve and pull from,
// e.g. "ghcr.io/org/repo:latest". Required.
ArtifactRef string
// BaseDir is the store's private directory: published artifacts live
// under "<BaseDir>/artifacts/<digest>". Required.
BaseDir string
// CustomTarget selects a non-default ".doco-cd.<target>.y(a)ml" config
// inside the artifact, matching the deployment's CustomTarget setting.
CustomTarget string
TrustPolicy config.OciTrustPolicy
TrustPolicyOverride config.OciTrustPolicyOverride
VerifyMaxWorkers uint
}
OCIStoreOptions configures an OCIStore.
type Revision ¶
type Revision string
Revision is an immutable identifier for one version of a source: a Git commit SHA, or an OCI manifest digest. Unlike a reference (a branch, tag, or "latest"), resolving the same Revision twice always yields the same content.
func ArtifactRoot ¶
ArtifactRoot returns the artifact directory ("<baseDir>/artifacts/<revision>") that path is nested under, and the revision it was published at, if any. It performs no filesystem access - it is a pure path computation for callers that only have a descendant path (e.g. a working directory recorded in a container label) and need to recover the artifact root/revision it was published under.
type Store ¶
type Store interface {
// Resolve maps ref (a branch, tag, "latest", or already a Revision) to
// the Revision it currently points to. It may contact the remote (a Git
// fetch, a registry HEAD/manifest request) but never modifies a
// previously published Artifact.
Resolve(ctx context.Context, ref string) (Revision, error)
// Publish materializes revision into a read-only Artifact directory and
// returns it. If an Artifact for revision has already been published,
// it is returned as-is without redoing the work. Publish is safe to
// call concurrently for the same or different revisions.
Publish(ctx context.Context, revision Revision) (Artifact, error)
// Lookup returns the Artifact for revision if it has already been
// published, without publishing it. The second return value is false
// if no artifact exists yet.
Lookup(revision Revision) (Artifact, bool, error)
// List returns every Artifact currently published by this store, e.g.
// for garbage collection or migration.
List() ([]Artifact, error)
}
Store maps a source's mutable references to immutable revisions, and immutable revisions to materialized artifact directories.
Implementations must make Publish safe for concurrent callers publishing the same Revision (idempotent, at-most-once materialization) and different Revisions (no shared mutable state between them) at the same time. Resolve and Lookup take no lock; they either contact the remote or inspect already-immutable local state.