artifacts

package
v1.23.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 27, 2026 License: Apache-2.0 Imports: 8 Imported by: 0

Documentation

Overview

Package artifacts defines Harbor's content-addressed blob store — the mandatory routing target for any output above the heavy-output threshold (default 32 KB; RFC §6.10).

The surface is a single mandatory `ArtifactStore` interface (eight methods including `Close`); there is NO `NoOp` fallback. V1 ships two drivers — an in-memory floor for dev/embedded use and a filesystem driver for single-binary production deployments. Harbor also ships SQLite-blob, Postgres-blob, and the S3-style driver; all four downstream drivers inherit the conformance suite verbatim.

Identity model. `ArtifactScope` is a flat `(TenantID, UserID, SessionID, TaskID)` tuple — deliberately distinct from the runtime's `identity.Quadruple{Identity, RunID}` shape. Keeping `ArtifactScope` as a flat-string struct lets the store stay dependency-free of `internal/identity`.

THE READ KEY IS THE ISOLATION TRIPLE. `Get` / `GetRef` / `Exists` / `Delete` match `(TenantID, UserID, SessionID, id)`. `TaskID` is a PROVENANCE ANNOTATION — it records which task produced the bytes; it is not an isolation principal and does not participate in resolution. The isolation boundary is `(tenant, user, session)` and a task runs WITHIN it rather than widening or narrowing it.

Two consequences follow, stated rather than left to be discovered:

  1. An artifact written under one run's scope is readable by a SIBLING RUN IN THE SAME SESSION. That is the intent: the session is the innermost isolation scope, the session-artifact manifest already enumerates across the runs inside it, and nothing crosses a session, a user or a tenant. The content-addressed `ID` is what makes it safe — within one session two artifacts sharing an id ARE the same bytes, so dropping a field from the read key cannot merge distinct content; it can only stop an artifact being hidden from itself.
  2. The WRITE/dedup key narrows with the read key. Two tasks storing identical bytes in one session collapse to ONE artifact, and `ArtifactRef.Scope.TaskID` is therefore FIRST-WRITER-WINS. See `ArtifactStore.List` for what that costs a `TaskID` filter.

Identity is mandatory at the API boundary. Empty tenant / user / session each return wrapped `ErrIdentityRequired` from Put*, Get, GetRef, Exists and Delete. Empty `TaskID` is acceptable for session-scoped artifacts (parallel to `state.StateStore`'s session-vs-run rule). `List` is a FILTER rather than a key: it requires a tenant and treats the remaining empty fields as wildcards WITHIN that tenant, so an unscoped all-tenants listing is not expressible at the store boundary.

Get / GetRef return `(value, found, err)`. Found-false is NOT an error — the consumer pattern is "Exists → fetch." `ErrNotFound` is reserved for actual error contexts (e.g. corrupted indexing); the conformance suite tests the `(nil, false, nil)` shape explicitly.

Audit redaction is upstream. The store stores opaque bytes and never re-redacts; mixing redaction into a leaf would couple the store to the audit subsystem and split responsibility.

Index

Constants

View Source
const DefaultDriver = "inmem"

DefaultDriver is the floor — the in-memory driver. The SQLite-blob, Postgres-blob, and S3-style drivers register additional names; `Open` switches on `cfg.Driver`.

Variables

View Source
var (
	// ErrNotFound — reserved for error contexts (e.g. corrupted
	// secondary index pointing at an absent primary). Get / GetRef
	// found-false is NOT this error; it is `(nil, false, nil)`.
	ErrNotFound = errors.New("artifacts: ref not found")
	// ErrScopeMismatch — `ScopedArtifacts` saw a returned ref whose
	// scope differs from the facade's fixed scope. Should be
	// impossible by construction; surfaced loudly when it isn't.
	ErrScopeMismatch = errors.New("artifacts: scope mismatch")
	// ErrIdentityRequired — Put*/Get/GetRef/Exists/Delete called
	// with a scope missing tenant/user/session, or List called with a
	// filter missing a tenant.
	ErrIdentityRequired = errors.New("artifacts: identity required (tenant/user/session)")
	// ErrInvalidScope — scope failed structural validation outside
	// the identity-required dimension (reserved; not currently
	// returned by V1 drivers).
	ErrInvalidScope = errors.New("artifacts: invalid scope")
	// ErrUnknownDriver — Open was asked for a driver name no
	// registered factory handles.
	ErrUnknownDriver = errors.New("artifacts: unknown driver")
	// ErrStoreClosed — any method called after Close.
	ErrStoreClosed = errors.New("artifacts: store is closed")
)

Sentinel errors. Callers compare via errors.Is.

View Source
var ErrPresignUnsupported = errors.New("artifacts: presigned URLs not supported by this driver")

ErrPresignUnsupported is returned (wrapped) when a caller asks an `ArtifactStore` for presigned URLs via type-assertion to `Presigner` and the underlying driver does not implement the capability. The error's presence is part of the failure-loud contract — silent fallback to byte-streaming would mask backend configuration mistakes.

Functions

func Register

func Register(name string, factory Factory)

Register installs a driver factory under name. Drivers self-register from their package init(); cmd/harbor blank-imports the production drivers to trigger registration. Per AGENTS.md §4.4.

Re-registering the same name panics — the registration model is write-once-at-init and a duplicate signals a build mis-configuration.

func RegisteredDrivers

func RegisteredDrivers() []string

RegisteredDrivers returns a sorted list of driver names. Useful for boot-log output and surfacing in error messages.

func Validate

func Validate(scope ArtifactScope) error

Validate is the package-level helper that mirrors `ArtifactScope.Validate`. Returns wrapped `ErrIdentityRequired` when any of tenant / user / session is empty. Empty `TaskID` is accepted.

func ValidateFilter added in v1.23.0

func ValidateFilter(filter ArtifactScope) error

ValidateFilter is the package-level helper that mirrors `ArtifactScope.ValidateFilter` — `List`'s precondition. Returns wrapped `ErrIdentityRequired` when the tenant is empty.

Types

type ArtifactRef

type ArtifactRef struct {
	ID        string
	MimeType  string
	SizeBytes int64
	Filename  string
	SHA256    string
	Scope     ArtifactScope
	Namespace string
	Source    map[string]any
}

ArtifactRef is the canonical reference returned by Put* and resolved by GetRef. `ID` is content-addressed: `{namespace}_{sha256_hex[:12]}`. Re-uploading identical bytes within the same isolation triple returns the existing ref (no duplicate storage) EVEN WHEN the two Puts carry different `Scope.TaskID` values — so `Scope.TaskID` on a resolved ref is the FIRST writer's stamp, not necessarily the caller's.

`SHA256` carries the full hex digest (64 chars). `SizeBytes` is the length of the stored bytes. `Source` is opaque caller metadata — drivers persist it as-is; for the FS driver, values must be JSON-encodable (non-encodable values cause Put to fail at marshal time).

type ArtifactScope

type ArtifactScope struct {
	TenantID  string
	UserID    string
	SessionID string
	TaskID    string
}

ArtifactScope carries the `(tenant, user, session)` isolation triple that OWNS an artifact plus the `TaskID` provenance annotation that records which task produced it. All four fields are flat strings; the consumer (tool dispatcher) is responsible for translating the runtime's `identity.Quadruple` (whose `RunID` becomes `TaskID`) into this shape.

Mandatory at the API boundary: `TenantID`, `UserID`, `SessionID` must be non-empty for Put*, Get, GetRef, Exists and Delete. Empty `TaskID` is acceptable for session-scoped artifacts, and a populated one never narrows a read — see `Triple`.

func (ArtifactScope) Equal

func (s ArtifactScope) Equal(other ArtifactScope) bool

Equal reports whether two scopes are field-for-field equal, INCLUDING the `TaskID` provenance annotation. It is therefore NOT the read key: two scopes that differ only in `TaskID` address the same artifact but are not `Equal`. Use `EqualTriple` for a resolution-side comparison; `Equal` remains the right answer when the question really is "is this the same stamp."

func (ArtifactScope) EqualTriple added in v1.23.0

func (s ArtifactScope) EqualTriple(other ArtifactScope) bool

EqualTriple reports whether two scopes name the same isolation triple, ignoring the `TaskID` provenance annotation. This is the comparison that matches the read key.

func (ArtifactScope) Triple added in v1.23.0

func (s ArtifactScope) Triple() ArtifactScope

Triple returns the scope with `TaskID` cleared — the READ KEY. Drivers key `Get` / `GetRef` / `Exists` / `Delete` on this value plus the id, so a caller who stamped a task and a caller who did not resolve the same artifact.

func (ArtifactScope) Validate

func (s ArtifactScope) Validate() error

Validate returns wrapped `ErrIdentityRequired` when any of tenant / user / session is empty. Empty `TaskID` is accepted.

Use the package-level `Validate(scope)` helper when you don't have an `ArtifactScope` value handy; both call sites converge on the same rule.

func (ArtifactScope) ValidateFilter added in v1.23.0

func (s ArtifactScope) ValidateFilter() error

ValidateFilter is `List`'s precondition. A list filter is a predicate over a result set rather than an identity, so an empty `UserID` / `SessionID` / `TaskID` stays a wildcard — but the TENANT is required, because without it the zero-value scope is a legal all-tenants filter at the store boundary and every discovery surface built on `List` would inherit that. A caller who legitimately reads another tenant names that tenant explicitly, under the admin-scope gate its Protocol surface enforces.

type ArtifactStore

type ArtifactStore interface {
	// PutBytes stores data under scope, returning the canonical ref.
	// The ref's `ID` is `{namespace}_{sha256_hex[:12]}`. Re-Put with
	// identical (triple, namespace, bytes) is a no-op that returns the
	// EXISTING ref — including when the new call carries a different
	// `scope.TaskID`, in which case the returned ref carries the first
	// writer's stamp and nothing is stored under the caller's task.
	PutBytes(ctx context.Context, scope ArtifactScope, data []byte, opts PutOpts) (ArtifactRef, error)

	// PutText is a thin wrapper over PutBytes that stores `text` as
	// UTF-8 bytes. Recovered via Get as bytes. MimeType defaults to
	// `text/plain; charset=utf-8` when opts.MimeType is empty.
	PutText(ctx context.Context, scope ArtifactScope, text string, opts PutOpts) (ArtifactRef, error)

	// Get returns the bytes for `id` within `scope`'s isolation triple.
	// `scope.TaskID` is IGNORED: a caller that stamped a task and a
	// caller that did not read the same artifact. Found-false indicates
	// the ref does not exist under that triple; it is NOT an error.
	// ErrNotFound is reserved for actual error contexts.
	Get(ctx context.Context, scope ArtifactScope, id string) ([]byte, bool, error)

	// GetRef returns the metadata-only ref for `id` within `scope`'s
	// isolation triple. `scope.TaskID` is IGNORED for resolution; the
	// returned `Ref.Scope.TaskID` is the stored provenance stamp, which
	// is the first writer's and may differ from the caller's. Same
	// found-false semantics as Get.
	GetRef(ctx context.Context, scope ArtifactScope, id string) (*ArtifactRef, bool, error)

	// Exists reports whether `id` is stored under `scope`'s isolation
	// triple. `scope.TaskID` is IGNORED. Cheaper than GetRef when the
	// caller only needs presence.
	Exists(ctx context.Context, scope ArtifactScope, id string) (bool, error)

	// Delete removes `id` from `scope`'s isolation triple and returns
	// whether anything existed before delete. `scope.TaskID` is IGNORED,
	// and EVERY stored copy under the triple is removed — a Delete that
	// reported success while leaving a copy a later Get resolves would
	// be the silent degradation CLAUDE.md §13 forbids. Idempotent:
	// Delete on absent returns `(false, nil)`.
	Delete(ctx context.Context, scope ArtifactScope, id string) (bool, error)

	// List returns refs whose scope matches `filter`. `filter.TenantID`
	// is REQUIRED (wrapped `ErrIdentityRequired` otherwise) — an
	// unscoped all-tenants listing is not expressible here. Every other
	// empty field is a wildcard within that tenant:
	// `ArtifactScope{TenantID: "A"}` lists every artifact under tenant A
	// across users / sessions / tasks.
	//
	// THE TaskID FILTER IS LOSSY, AND SAYING SO IS PART OF THE CONTRACT.
	// Because the write key is the isolation triple, two tasks storing
	// identical bytes in one session collapse to ONE artifact carrying
	// the FIRST writer's stamp. So `filter.TaskID = "B"` does not return
	// an artifact whose bytes run B stored, if run A stored them first.
	// This is inherent to a content-addressed store rather than an
	// implementation shortfall: the id is derived from the bytes, so
	// "which run produced these bytes" has no single answer once two
	// runs produce them. `filter.TaskID` answers "which artifacts is
	// this run the recorded producer of", which is a weaker question
	// than "which artifacts did this run write".
	//
	// Order is not specified; callers that need stability sort the
	// returned slice themselves.
	List(ctx context.Context, filter ArtifactScope) ([]ArtifactRef, error)

	// Close releases driver resources. Subsequent calls return
	// wrapped `ErrStoreClosed`. Implementations MUST honour ctx
	// during long teardowns (none of V1's drivers have any).
	Close(ctx context.Context) error
}

ArtifactStore is Harbor's mandatory content-addressed blob store. All eight methods are required; there is no `Supports*` capability ceremony (AGENTS.md §4.4). Implementations MUST be safe for N concurrent goroutines on a single shared instance; the conformance suite's `Concurrent_PutGet_NoRace` is the gate.

Identity is enforced at the API boundary: every Put*/Get/GetRef/ Exists/Delete validates `scope`'s triple before touching storage, and `List` requires at least a tenant.

KEY VERSUS FILTER — the distinction the interface is built on. `Get` / `GetRef` / `Exists` / `Delete` take an IDENTITY: they resolve on `(TenantID, UserID, SessionID, id)` and IGNORE `scope.TaskID`, which is a provenance annotation. `List` takes a PREDICATE over a result set: every empty field below the tenant is a wildcard, `TaskID` included. The two are deliberately different things and the difference is stated here so they stop being accidentally different.

Get / GetRef return `(value, found, err)`. Found-false is NOT an error.

func Open

Open returns the ArtifactStore built by the factory whose name matches cfg.Driver (defaults to DefaultDriver when cfg.Driver is empty).

func OpenDriver

func OpenDriver(name string, cfg config.ArtifactsConfig) (ArtifactStore, error)

OpenDriver opens a specific driver by name; useful for tests that want to exercise the registry against a non-default driver.

type Factory

type Factory func(config.ArtifactsConfig) (ArtifactStore, error)

Factory builds an ArtifactStore from an ArtifactsConfig. Drivers expose one Factory each via init() → Register.

type Presigner

type Presigner interface {
	// PresignGet returns a time-bounded HTTPS URL the caller can hand
	// to a downstream consumer for direct download of the artifact's
	// bytes. Read-side only — there is no PresignPut / PresignDelete
	// counterpart at V1 (write-side presigned URLs are an attack
	// surface; documented in the phase plan).
	//
	// Returns a wrapped error when:
	//   - the scope fails identity validation (`ErrIdentityRequired`),
	//   - the artifact does not exist in this scope (`ErrNotFound`),
	//   - `expiry` is out of the `[1 minute, 7 days]` range,
	//   - the underlying signer fails.
	PresignGet(ctx context.Context, scope ArtifactScope, id string, expiry time.Duration) (string, error)
}

Presigner is an OPTIONAL capability interface that backends with native presigned-URL support implement. This is the explicit exception to the "no optional capabilities" rule in AGENTS.md §4.4: only S3-compatible object stores have presigned URLs natively, and the capability cannot be reasonably faked by the other V1 drivers (InMem / FS / SQLite-blob / Postgres-blob) without bolting a separate signing service onto Harbor — which is out of V1 scope.

Per RFC §6.10, presigned URLs are the read-side hand-off path for media-class artifacts: the runtime hands a Console / Protocol client a time-bounded HTTPS URL the client downloads directly from object storage, bypassing the runtime's bytes path entirely.

Callers that need presigned URLs type-assert the `ArtifactStore` they hold to `Presigner`; absence is a typed error (`ErrPresignUnsupported`), not a silent fallback. The S3 driver is the only V1 driver implementing this capability.

Identity is mandatory at the Presigner boundary just like every other ArtifactStore method: implementations MUST validate the `scope` and reject with `ErrIdentityRequired` when any of tenant/user/session is empty.

`expiry` is bounded — implementations MUST reject expiries shorter than 1 minute or longer than 7 days (S3's documented limit). Out- of-range expiries return a clear error rather than being silently clamped (AGENTS.md §5: fail loudly).

type PutOpts

type PutOpts struct {
	MimeType  string
	Filename  string
	Namespace string
	Source    map[string]any
}

PutOpts carries optional metadata for Put* calls.

`Namespace` is a logical bucket that participates in `ID` computation, so the same bytes under different namespaces produce distinct refs. Callers SHOULD provide a namespace; drivers default to `"default"` when empty.

`Filename` is metadata only — never used in path construction. The FS driver's path-safety guard rejects traversal regardless.

`Source` values must be JSON-encodable when targeting the FS driver (it persists `Source` to a sibling `.meta.json`). Use Go primitives, slices, and maps; non-encodable values (functions, channels, cyclic graphs) cause Put to fail at marshal time.

type ScopedArtifacts

type ScopedArtifacts struct {
	// contains filtered or unexported fields
}

ScopedArtifacts is the immutable facade tools and runtime use to access the artifact store. It carries a fixed `ArtifactScope` (set at construction, never mutated) and:

  • Auto-stamps the scope on every Put*: callers do not pass scope at the facade boundary, so they cannot accidentally write to a different one.
  • Scope-checks on `GetRef`, and ONLY on `GetRef`: if the underlying store returned a ref outside the facade's isolation triple, the facade returns wrapped `ErrScopeMismatch` rather than leaking cross-scope bytes. `Get` / `Exists` / `Delete` / `List` pass straight through — the DRIVERS are the enforcement point, and they resolve on the triple. (This list used to claim all five methods checked; only one ever did. The claim is corrected rather than the other four grown a check they do not need.)

The check compares the ISOLATION TRIPLE, not the whole scope. A ref carries the producing task's `TaskID` as a provenance stamp, and a facade constructed without one — the ordinary case for a tool reading back what an earlier run in the same session wrote — must not read that stamp as a mismatch. Comparing the full scope here would refuse exactly the read the triple-keyed store exists to allow.

Construction fails loud: `NewScoped` panics when the scope fails `Validate`. The facade's invariant is "every operation has a valid fixed scope"; rejecting at construction is the simplest, loudest failure mode (AGENTS.md §5: "fail loudly"). Acceptable per design — the alternative (returning an error from a constructor on every call site) accumulates boilerplate that masks the actual bug: callers that built a facade without a complete identity triple.

func NewScoped

func NewScoped(store ArtifactStore, scope ArtifactScope) *ScopedArtifacts

NewScoped wraps `store` with a fixed `scope`. Panics with a wrapped `ErrIdentityRequired` if scope is invalid (empty tenant/user/session).

Tools and runtime construct ScopedArtifacts at the consumer boundary (e.g. the tool dispatcher); they then never see the raw `ArtifactScope` again.

func (*ScopedArtifacts) Delete

func (s *ScopedArtifacts) Delete(ctx context.Context, id string) (bool, error)

Delete removes `id` from the facade's isolation triple. Returns whether anything existed before delete; idempotent.

func (*ScopedArtifacts) Exists

func (s *ScopedArtifacts) Exists(ctx context.Context, id string) (bool, error)

Exists reports whether `id` is stored under the facade's isolation triple.

func (*ScopedArtifacts) Get

func (s *ScopedArtifacts) Get(ctx context.Context, id string) ([]byte, bool, error)

Get returns the bytes for `id` within the facade's isolation triple. Found-false is NOT an error. The store resolves on the triple, so a facade constructed with or without a `TaskID` reads the same bytes.

func (*ScopedArtifacts) GetRef

func (s *ScopedArtifacts) GetRef(ctx context.Context, id string) (*ArtifactRef, bool, error)

GetRef returns the ref for `id` within the facade's isolation triple. Found-false is NOT an error. If the underlying store returns a ref from a DIFFERENT triple, GetRef returns wrapped `ErrScopeMismatch` (defensive — V1 drivers resolve on the triple, so this can only fire on a driver bug or a future cross-scope driver).

The comparison is `EqualTriple`, not `Equal`: `Ref.Scope.TaskID` is the producing task's provenance stamp and is expected to differ from the facade's whenever a sibling run in the same session wrote the bytes. Comparing the full scope would turn the read the reconciled key exists to enable into an error.

func (*ScopedArtifacts) List

func (s *ScopedArtifacts) List(ctx context.Context) ([]ArtifactRef, error)

List returns every artifact under the facade's isolation TRIPLE — the facade's own `TaskID` is deliberately dropped from the filter.

The facade's reads resolve on the triple, so listing on the full scope would enumerate a strict subset of what the same facade can `Get`: the caller would be shown rows for its own task and hidden from rows a sibling run wrote that it can nonetheless fetch. Listing and reading answer the same question here, which is the whole point of the reconciled key. Callers that want a per-task provenance filter call the underlying store's `List` with a `TaskID` — and read that method's godoc on why the answer is first-writer-lossy.

func (*ScopedArtifacts) PutBytes

func (s *ScopedArtifacts) PutBytes(ctx context.Context, data []byte, opts PutOpts) (ArtifactRef, error)

PutBytes stores data under the facade's scope. The scope is stamped onto the returned ref automatically.

func (*ScopedArtifacts) PutText

func (s *ScopedArtifacts) PutText(ctx context.Context, text string, opts PutOpts) (ArtifactRef, error)

PutText stores text under the facade's scope.

func (*ScopedArtifacts) Scope

func (s *ScopedArtifacts) Scope() ArtifactScope

Scope returns the fixed scope this facade was constructed with. Useful for tests / diagnostics; the value is immutable.

Directories

Path Synopsis
Package conformancetest exposes the canonical correctness suite every artifacts.ArtifactStore driver must pass.
Package conformancetest exposes the canonical correctness suite every artifacts.ArtifactStore driver must pass.
drivers
fs
Package fs is Harbor's filesystem ArtifactStore driver.
Package fs is Harbor's filesystem ArtifactStore driver.
inmem
Package inmem is Harbor's V1 in-memory ArtifactStore driver.
Package inmem is Harbor's V1 in-memory ArtifactStore driver.
postgres
Package postgres is Harbor's V1 Postgres-backed ArtifactStore driver.
Package postgres is Harbor's V1 Postgres-backed ArtifactStore driver.
s3
Package s3 is Harbor's S3-compatible ArtifactStore driver.
Package s3 is Harbor's S3-compatible ArtifactStore driver.
sqlite
Package sqlite is Harbor's SQLite-backed `artifacts.ArtifactStore` driver.
Package sqlite is Harbor's SQLite-backed `artifacts.ArtifactStore` driver.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL