Documentation
¶
Overview ¶
Package artifacts defines Harbor's content-addressed blob store — the mandatory routing target for any output above the heavy-output threshold (default 32 KB; RFC §6.10).
The surface is a single mandatory `ArtifactStore` interface (eight methods including `Close`); there is NO `NoOp` fallback. V1 ships two drivers — an in-memory floor for dev/embedded use and a filesystem driver for single-binary production deployments. Harbor also ships SQLite-blob, Postgres-blob, and the S3-style driver; all four downstream drivers inherit the conformance suite verbatim.
Identity model. `ArtifactScope` is a flat `(TenantID, UserID, SessionID, TaskID)` tuple — deliberately distinct from the runtime's `identity.Quadruple{Identity, RunID}` shape. Keeping `ArtifactScope` as a flat-string struct lets the store stay dependency-free of `internal/identity`.
THE READ KEY IS THE ISOLATION TRIPLE. `Get` / `GetRef` / `Exists` / `Delete` match `(TenantID, UserID, SessionID, id)`. `TaskID` is a PROVENANCE ANNOTATION — it records which task produced the bytes; it is not an isolation principal and does not participate in resolution. The isolation boundary is `(tenant, user, session)` and a task runs WITHIN it rather than widening or narrowing it.
Two consequences follow, stated rather than left to be discovered:
- An artifact written under one run's scope is readable by a SIBLING RUN IN THE SAME SESSION. That is the intent: the session is the innermost isolation scope, the session-artifact manifest already enumerates across the runs inside it, and nothing crosses a session, a user or a tenant. The content-addressed `ID` is what makes it safe — within one session two artifacts sharing an id ARE the same bytes, so dropping a field from the read key cannot merge distinct content; it can only stop an artifact being hidden from itself.
- The WRITE/dedup key narrows with the read key. Two tasks storing identical bytes in one session collapse to ONE artifact, and `ArtifactRef.Scope.TaskID` is therefore FIRST-WRITER-WINS. See `ArtifactStore.List` for what that costs a `TaskID` filter.
Identity is mandatory at the API boundary. Empty tenant / user / session each return wrapped `ErrIdentityRequired` from Put*, Get, GetRef, Exists and Delete. Empty `TaskID` is acceptable for session-scoped artifacts (parallel to `state.StateStore`'s session-vs-run rule). `List` is a FILTER rather than a key: it requires a tenant and treats the remaining empty fields as wildcards WITHIN that tenant, so an unscoped all-tenants listing is not expressible at the store boundary.
Get / GetRef return `(value, found, err)`. Found-false is NOT an error — the consumer pattern is "Exists → fetch." `ErrNotFound` is reserved for actual error contexts (e.g. corrupted indexing); the conformance suite tests the `(nil, false, nil)` shape explicitly.
Audit redaction is upstream. The store stores opaque bytes and never re-redacts; mixing redaction into a leaf would couple the store to the audit subsystem and split responsibility.
Index ¶
- Constants
- Variables
- func Register(name string, factory Factory)
- func RegisteredDrivers() []string
- func Validate(scope ArtifactScope) error
- func ValidateFilter(filter ArtifactScope) error
- type ArtifactRef
- type ArtifactScope
- type ArtifactStore
- type Factory
- type Presigner
- type PutOpts
- type ScopedArtifacts
- func (s *ScopedArtifacts) Delete(ctx context.Context, id string) (bool, error)
- func (s *ScopedArtifacts) Exists(ctx context.Context, id string) (bool, error)
- func (s *ScopedArtifacts) Get(ctx context.Context, id string) ([]byte, bool, error)
- func (s *ScopedArtifacts) GetRef(ctx context.Context, id string) (*ArtifactRef, bool, error)
- func (s *ScopedArtifacts) List(ctx context.Context) ([]ArtifactRef, error)
- func (s *ScopedArtifacts) PutBytes(ctx context.Context, data []byte, opts PutOpts) (ArtifactRef, error)
- func (s *ScopedArtifacts) PutText(ctx context.Context, text string, opts PutOpts) (ArtifactRef, error)
- func (s *ScopedArtifacts) Scope() ArtifactScope
Constants ¶
const DefaultDriver = "inmem"
DefaultDriver is the floor — the in-memory driver. The SQLite-blob, Postgres-blob, and S3-style drivers register additional names; `Open` switches on `cfg.Driver`.
Variables ¶
var ( // ErrNotFound — reserved for error contexts (e.g. corrupted // secondary index pointing at an absent primary). Get / GetRef // found-false is NOT this error; it is `(nil, false, nil)`. ErrNotFound = errors.New("artifacts: ref not found") // ErrScopeMismatch — `ScopedArtifacts` saw a returned ref whose // scope differs from the facade's fixed scope. Should be // impossible by construction; surfaced loudly when it isn't. ErrScopeMismatch = errors.New("artifacts: scope mismatch") // ErrIdentityRequired — Put*/Get/GetRef/Exists/Delete called // with a scope missing tenant/user/session, or List called with a // filter missing a tenant. ErrIdentityRequired = errors.New("artifacts: identity required (tenant/user/session)") // ErrInvalidScope — scope failed structural validation outside // the identity-required dimension (reserved; not currently // returned by V1 drivers). ErrInvalidScope = errors.New("artifacts: invalid scope") // ErrUnknownDriver — Open was asked for a driver name no // registered factory handles. ErrUnknownDriver = errors.New("artifacts: unknown driver") // ErrStoreClosed — any method called after Close. ErrStoreClosed = errors.New("artifacts: store is closed") )
Sentinel errors. Callers compare via errors.Is.
var ErrPresignUnsupported = errors.New("artifacts: presigned URLs not supported by this driver")
ErrPresignUnsupported is returned (wrapped) when a caller asks an `ArtifactStore` for presigned URLs via type-assertion to `Presigner` and the underlying driver does not implement the capability. The error's presence is part of the failure-loud contract — silent fallback to byte-streaming would mask backend configuration mistakes.
Functions ¶
func Register ¶
Register installs a driver factory under name. Drivers self-register from their package init(); cmd/harbor blank-imports the production drivers to trigger registration. Per AGENTS.md §4.4.
Re-registering the same name panics — the registration model is write-once-at-init and a duplicate signals a build mis-configuration.
func RegisteredDrivers ¶
func RegisteredDrivers() []string
RegisteredDrivers returns a sorted list of driver names. Useful for boot-log output and surfacing in error messages.
func Validate ¶
func Validate(scope ArtifactScope) error
Validate is the package-level helper that mirrors `ArtifactScope.Validate`. Returns wrapped `ErrIdentityRequired` when any of tenant / user / session is empty. Empty `TaskID` is accepted.
func ValidateFilter ¶ added in v1.23.0
func ValidateFilter(filter ArtifactScope) error
ValidateFilter is the package-level helper that mirrors `ArtifactScope.ValidateFilter` — `List`'s precondition. Returns wrapped `ErrIdentityRequired` when the tenant is empty.
Types ¶
type ArtifactRef ¶
type ArtifactRef struct {
ID string
MimeType string
SizeBytes int64
Filename string
SHA256 string
Scope ArtifactScope
Namespace string
Source map[string]any
}
ArtifactRef is the canonical reference returned by Put* and resolved by GetRef. `ID` is content-addressed: `{namespace}_{sha256_hex[:12]}`. Re-uploading identical bytes within the same isolation triple returns the existing ref (no duplicate storage) EVEN WHEN the two Puts carry different `Scope.TaskID` values — so `Scope.TaskID` on a resolved ref is the FIRST writer's stamp, not necessarily the caller's.
`SHA256` carries the full hex digest (64 chars). `SizeBytes` is the length of the stored bytes. `Source` is opaque caller metadata — drivers persist it as-is; for the FS driver, values must be JSON-encodable (non-encodable values cause Put to fail at marshal time).
type ArtifactScope ¶
ArtifactScope carries the `(tenant, user, session)` isolation triple that OWNS an artifact plus the `TaskID` provenance annotation that records which task produced it. All four fields are flat strings; the consumer (tool dispatcher) is responsible for translating the runtime's `identity.Quadruple` (whose `RunID` becomes `TaskID`) into this shape.
Mandatory at the API boundary: `TenantID`, `UserID`, `SessionID` must be non-empty for Put*, Get, GetRef, Exists and Delete. Empty `TaskID` is acceptable for session-scoped artifacts, and a populated one never narrows a read — see `Triple`.
func (ArtifactScope) Equal ¶
func (s ArtifactScope) Equal(other ArtifactScope) bool
Equal reports whether two scopes are field-for-field equal, INCLUDING the `TaskID` provenance annotation. It is therefore NOT the read key: two scopes that differ only in `TaskID` address the same artifact but are not `Equal`. Use `EqualTriple` for a resolution-side comparison; `Equal` remains the right answer when the question really is "is this the same stamp."
func (ArtifactScope) EqualTriple ¶ added in v1.23.0
func (s ArtifactScope) EqualTriple(other ArtifactScope) bool
EqualTriple reports whether two scopes name the same isolation triple, ignoring the `TaskID` provenance annotation. This is the comparison that matches the read key.
func (ArtifactScope) Triple ¶ added in v1.23.0
func (s ArtifactScope) Triple() ArtifactScope
Triple returns the scope with `TaskID` cleared — the READ KEY. Drivers key `Get` / `GetRef` / `Exists` / `Delete` on this value plus the id, so a caller who stamped a task and a caller who did not resolve the same artifact.
func (ArtifactScope) Validate ¶
func (s ArtifactScope) Validate() error
Validate returns wrapped `ErrIdentityRequired` when any of tenant / user / session is empty. Empty `TaskID` is accepted.
Use the package-level `Validate(scope)` helper when you don't have an `ArtifactScope` value handy; both call sites converge on the same rule.
func (ArtifactScope) ValidateFilter ¶ added in v1.23.0
func (s ArtifactScope) ValidateFilter() error
ValidateFilter is `List`'s precondition. A list filter is a predicate over a result set rather than an identity, so an empty `UserID` / `SessionID` / `TaskID` stays a wildcard — but the TENANT is required, because without it the zero-value scope is a legal all-tenants filter at the store boundary and every discovery surface built on `List` would inherit that. A caller who legitimately reads another tenant names that tenant explicitly, under the admin-scope gate its Protocol surface enforces.
type ArtifactStore ¶
type ArtifactStore interface {
// PutBytes stores data under scope, returning the canonical ref.
// The ref's `ID` is `{namespace}_{sha256_hex[:12]}`. Re-Put with
// identical (triple, namespace, bytes) is a no-op that returns the
// EXISTING ref — including when the new call carries a different
// `scope.TaskID`, in which case the returned ref carries the first
// writer's stamp and nothing is stored under the caller's task.
PutBytes(ctx context.Context, scope ArtifactScope, data []byte, opts PutOpts) (ArtifactRef, error)
// PutText is a thin wrapper over PutBytes that stores `text` as
// UTF-8 bytes. Recovered via Get as bytes. MimeType defaults to
// `text/plain; charset=utf-8` when opts.MimeType is empty.
PutText(ctx context.Context, scope ArtifactScope, text string, opts PutOpts) (ArtifactRef, error)
// Get returns the bytes for `id` within `scope`'s isolation triple.
// `scope.TaskID` is IGNORED: a caller that stamped a task and a
// caller that did not read the same artifact. Found-false indicates
// the ref does not exist under that triple; it is NOT an error.
// ErrNotFound is reserved for actual error contexts.
Get(ctx context.Context, scope ArtifactScope, id string) ([]byte, bool, error)
// GetRef returns the metadata-only ref for `id` within `scope`'s
// isolation triple. `scope.TaskID` is IGNORED for resolution; the
// returned `Ref.Scope.TaskID` is the stored provenance stamp, which
// is the first writer's and may differ from the caller's. Same
// found-false semantics as Get.
GetRef(ctx context.Context, scope ArtifactScope, id string) (*ArtifactRef, bool, error)
// Exists reports whether `id` is stored under `scope`'s isolation
// triple. `scope.TaskID` is IGNORED. Cheaper than GetRef when the
// caller only needs presence.
Exists(ctx context.Context, scope ArtifactScope, id string) (bool, error)
// Delete removes `id` from `scope`'s isolation triple and returns
// whether anything existed before delete. `scope.TaskID` is IGNORED,
// and EVERY stored copy under the triple is removed — a Delete that
// reported success while leaving a copy a later Get resolves would
// be the silent degradation CLAUDE.md §13 forbids. Idempotent:
// Delete on absent returns `(false, nil)`.
Delete(ctx context.Context, scope ArtifactScope, id string) (bool, error)
// List returns refs whose scope matches `filter`. `filter.TenantID`
// is REQUIRED (wrapped `ErrIdentityRequired` otherwise) — an
// unscoped all-tenants listing is not expressible here. Every other
// empty field is a wildcard within that tenant:
// `ArtifactScope{TenantID: "A"}` lists every artifact under tenant A
// across users / sessions / tasks.
//
// THE TaskID FILTER IS LOSSY, AND SAYING SO IS PART OF THE CONTRACT.
// Because the write key is the isolation triple, two tasks storing
// identical bytes in one session collapse to ONE artifact carrying
// the FIRST writer's stamp. So `filter.TaskID = "B"` does not return
// an artifact whose bytes run B stored, if run A stored them first.
// This is inherent to a content-addressed store rather than an
// implementation shortfall: the id is derived from the bytes, so
// "which run produced these bytes" has no single answer once two
// runs produce them. `filter.TaskID` answers "which artifacts is
// this run the recorded producer of", which is a weaker question
// than "which artifacts did this run write".
//
// Order is not specified; callers that need stability sort the
// returned slice themselves.
List(ctx context.Context, filter ArtifactScope) ([]ArtifactRef, error)
// Close releases driver resources. Subsequent calls return
// wrapped `ErrStoreClosed`. Implementations MUST honour ctx
// during long teardowns (none of V1's drivers have any).
Close(ctx context.Context) error
}
ArtifactStore is Harbor's mandatory content-addressed blob store. All eight methods are required; there is no `Supports*` capability ceremony (AGENTS.md §4.4). Implementations MUST be safe for N concurrent goroutines on a single shared instance; the conformance suite's `Concurrent_PutGet_NoRace` is the gate.
Identity is enforced at the API boundary: every Put*/Get/GetRef/ Exists/Delete validates `scope`'s triple before touching storage, and `List` requires at least a tenant.
KEY VERSUS FILTER — the distinction the interface is built on. `Get` / `GetRef` / `Exists` / `Delete` take an IDENTITY: they resolve on `(TenantID, UserID, SessionID, id)` and IGNORE `scope.TaskID`, which is a provenance annotation. `List` takes a PREDICATE over a result set: every empty field below the tenant is a wildcard, `TaskID` included. The two are deliberately different things and the difference is stated here so they stop being accidentally different.
Get / GetRef return `(value, found, err)`. Found-false is NOT an error.
func Open ¶
func Open(_ context.Context, cfg config.ArtifactsConfig) (ArtifactStore, error)
Open returns the ArtifactStore built by the factory whose name matches cfg.Driver (defaults to DefaultDriver when cfg.Driver is empty).
func OpenDriver ¶
func OpenDriver(name string, cfg config.ArtifactsConfig) (ArtifactStore, error)
OpenDriver opens a specific driver by name; useful for tests that want to exercise the registry against a non-default driver.
type Factory ¶
type Factory func(config.ArtifactsConfig) (ArtifactStore, error)
Factory builds an ArtifactStore from an ArtifactsConfig. Drivers expose one Factory each via init() → Register.
type Presigner ¶
type Presigner interface {
// PresignGet returns a time-bounded HTTPS URL the caller can hand
// to a downstream consumer for direct download of the artifact's
// bytes. Read-side only — there is no PresignPut / PresignDelete
// counterpart at V1 (write-side presigned URLs are an attack
// surface; documented in the phase plan).
//
// Returns a wrapped error when:
// - the scope fails identity validation (`ErrIdentityRequired`),
// - the artifact does not exist in this scope (`ErrNotFound`),
// - `expiry` is out of the `[1 minute, 7 days]` range,
// - the underlying signer fails.
PresignGet(ctx context.Context, scope ArtifactScope, id string, expiry time.Duration) (string, error)
}
Presigner is an OPTIONAL capability interface that backends with native presigned-URL support implement. This is the explicit exception to the "no optional capabilities" rule in AGENTS.md §4.4: only S3-compatible object stores have presigned URLs natively, and the capability cannot be reasonably faked by the other V1 drivers (InMem / FS / SQLite-blob / Postgres-blob) without bolting a separate signing service onto Harbor — which is out of V1 scope.
Per RFC §6.10, presigned URLs are the read-side hand-off path for media-class artifacts: the runtime hands a Console / Protocol client a time-bounded HTTPS URL the client downloads directly from object storage, bypassing the runtime's bytes path entirely.
Callers that need presigned URLs type-assert the `ArtifactStore` they hold to `Presigner`; absence is a typed error (`ErrPresignUnsupported`), not a silent fallback. The S3 driver is the only V1 driver implementing this capability.
Identity is mandatory at the Presigner boundary just like every other ArtifactStore method: implementations MUST validate the `scope` and reject with `ErrIdentityRequired` when any of tenant/user/session is empty.
`expiry` is bounded — implementations MUST reject expiries shorter than 1 minute or longer than 7 days (S3's documented limit). Out- of-range expiries return a clear error rather than being silently clamped (AGENTS.md §5: fail loudly).
type PutOpts ¶
PutOpts carries optional metadata for Put* calls.
`Namespace` is a logical bucket that participates in `ID` computation, so the same bytes under different namespaces produce distinct refs. Callers SHOULD provide a namespace; drivers default to `"default"` when empty.
`Filename` is metadata only — never used in path construction. The FS driver's path-safety guard rejects traversal regardless.
`Source` values must be JSON-encodable when targeting the FS driver (it persists `Source` to a sibling `.meta.json`). Use Go primitives, slices, and maps; non-encodable values (functions, channels, cyclic graphs) cause Put to fail at marshal time.
type ScopedArtifacts ¶
type ScopedArtifacts struct {
// contains filtered or unexported fields
}
ScopedArtifacts is the immutable facade tools and runtime use to access the artifact store. It carries a fixed `ArtifactScope` (set at construction, never mutated) and:
- Auto-stamps the scope on every Put*: callers do not pass scope at the facade boundary, so they cannot accidentally write to a different one.
- Scope-checks on `GetRef`, and ONLY on `GetRef`: if the underlying store returned a ref outside the facade's isolation triple, the facade returns wrapped `ErrScopeMismatch` rather than leaking cross-scope bytes. `Get` / `Exists` / `Delete` / `List` pass straight through — the DRIVERS are the enforcement point, and they resolve on the triple. (This list used to claim all five methods checked; only one ever did. The claim is corrected rather than the other four grown a check they do not need.)
The check compares the ISOLATION TRIPLE, not the whole scope. A ref carries the producing task's `TaskID` as a provenance stamp, and a facade constructed without one — the ordinary case for a tool reading back what an earlier run in the same session wrote — must not read that stamp as a mismatch. Comparing the full scope here would refuse exactly the read the triple-keyed store exists to allow.
Construction fails loud: `NewScoped` panics when the scope fails `Validate`. The facade's invariant is "every operation has a valid fixed scope"; rejecting at construction is the simplest, loudest failure mode (AGENTS.md §5: "fail loudly"). Acceptable per design — the alternative (returning an error from a constructor on every call site) accumulates boilerplate that masks the actual bug: callers that built a facade without a complete identity triple.
func NewScoped ¶
func NewScoped(store ArtifactStore, scope ArtifactScope) *ScopedArtifacts
NewScoped wraps `store` with a fixed `scope`. Panics with a wrapped `ErrIdentityRequired` if scope is invalid (empty tenant/user/session).
Tools and runtime construct ScopedArtifacts at the consumer boundary (e.g. the tool dispatcher); they then never see the raw `ArtifactScope` again.
func (*ScopedArtifacts) Delete ¶
Delete removes `id` from the facade's isolation triple. Returns whether anything existed before delete; idempotent.
func (*ScopedArtifacts) Exists ¶
Exists reports whether `id` is stored under the facade's isolation triple.
func (*ScopedArtifacts) Get ¶
Get returns the bytes for `id` within the facade's isolation triple. Found-false is NOT an error. The store resolves on the triple, so a facade constructed with or without a `TaskID` reads the same bytes.
func (*ScopedArtifacts) GetRef ¶
func (s *ScopedArtifacts) GetRef(ctx context.Context, id string) (*ArtifactRef, bool, error)
GetRef returns the ref for `id` within the facade's isolation triple. Found-false is NOT an error. If the underlying store returns a ref from a DIFFERENT triple, GetRef returns wrapped `ErrScopeMismatch` (defensive — V1 drivers resolve on the triple, so this can only fire on a driver bug or a future cross-scope driver).
The comparison is `EqualTriple`, not `Equal`: `Ref.Scope.TaskID` is the producing task's provenance stamp and is expected to differ from the facade's whenever a sibling run in the same session wrote the bytes. Comparing the full scope would turn the read the reconciled key exists to enable into an error.
func (*ScopedArtifacts) List ¶
func (s *ScopedArtifacts) List(ctx context.Context) ([]ArtifactRef, error)
List returns every artifact under the facade's isolation TRIPLE — the facade's own `TaskID` is deliberately dropped from the filter.
The facade's reads resolve on the triple, so listing on the full scope would enumerate a strict subset of what the same facade can `Get`: the caller would be shown rows for its own task and hidden from rows a sibling run wrote that it can nonetheless fetch. Listing and reading answer the same question here, which is the whole point of the reconciled key. Callers that want a per-task provenance filter call the underlying store's `List` with a `TaskID` — and read that method's godoc on why the answer is first-writer-lossy.
func (*ScopedArtifacts) PutBytes ¶
func (s *ScopedArtifacts) PutBytes(ctx context.Context, data []byte, opts PutOpts) (ArtifactRef, error)
PutBytes stores data under the facade's scope. The scope is stamped onto the returned ref automatically.
func (*ScopedArtifacts) PutText ¶
func (s *ScopedArtifacts) PutText(ctx context.Context, text string, opts PutOpts) (ArtifactRef, error)
PutText stores text under the facade's scope.
func (*ScopedArtifacts) Scope ¶
func (s *ScopedArtifacts) Scope() ArtifactScope
Scope returns the fixed scope this facade was constructed with. Useful for tests / diagnostics; the value is immutable.
Directories
¶
| Path | Synopsis |
|---|---|
|
Package conformancetest exposes the canonical correctness suite every artifacts.ArtifactStore driver must pass.
|
Package conformancetest exposes the canonical correctness suite every artifacts.ArtifactStore driver must pass. |
|
drivers
|
|
|
fs
Package fs is Harbor's filesystem ArtifactStore driver.
|
Package fs is Harbor's filesystem ArtifactStore driver. |
|
inmem
Package inmem is Harbor's V1 in-memory ArtifactStore driver.
|
Package inmem is Harbor's V1 in-memory ArtifactStore driver. |
|
postgres
Package postgres is Harbor's V1 Postgres-backed ArtifactStore driver.
|
Package postgres is Harbor's V1 Postgres-backed ArtifactStore driver. |
|
s3
Package s3 is Harbor's S3-compatible ArtifactStore driver.
|
Package s3 is Harbor's S3-compatible ArtifactStore driver. |
|
sqlite
Package sqlite is Harbor's SQLite-backed `artifacts.ArtifactStore` driver.
|
Package sqlite is Harbor's SQLite-backed `artifacts.ArtifactStore` driver. |