Help us shape the future of Go! Take the 2026 Go Developer Survey

artifact

package
v1.9.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 26, 2026 License: MIT Imports: 10 Imported by: 0

Documentation

Overview

Package artifact defines Dispatch's data plane: tracked references to objects in external storage, the pluggable Backend interface those objects live behind, and the Store contract that persists their metadata and ownership links.

This package is a leaf. It imports only the root dispatch package, the id package, and stdlib. The staging middleware, which needs job and middleware, lives in the artifact/staging sub-package so that job may import artifact without a cycle.

Artifacts come in two lifecycles. Durable artifacts are written by the application and merely tracked here; Dispatch reads them and never deletes them. Ephemeral artifacts are created by Dispatch on a handler's behalf, refcounted through links, and swept once every owner is terminal and the retention window has passed.

Index

Constants

View Source
const DefaultEphemeralPrefix = "ephemeral"

DefaultEphemeralPrefix is where Dispatch-owned objects live when no prefix is configured.

Variables

View Source
var (
	// ErrNotFound means the artifact or its underlying object does not
	// exist. Retrying a fetch of something that no longer exists cannot
	// succeed, so it unwraps to dispatch.ErrPermanent.
	ErrNotFound error = &categoryError{
		msg:    "dispatch/artifact: not found",
		parent: dispatch.ErrPermanent,
	}

	// ErrPermissionDenied means the backend refused the operation as
	// unauthorized. Nothing changes until the credentials or the backend's
	// access policy do, so it unwraps to dispatch.ErrPermanent too.
	ErrPermissionDenied error = &categoryError{
		msg:    "dispatch/artifact: permission denied",
		parent: dispatch.ErrPermanent,
	}

	// ErrExists means an artifact already exists for this owner, name,
	// and a prior attempt. Create with IfAbsent returns it so a retried
	// handler can skip recomputation.
	ErrExists = errors.New("dispatch/artifact: already exists")

	// ErrSizeExceeded means a bound artifact is larger than the input
	// declaration's MaxSize.
	ErrSizeExceeded = errors.New("dispatch/artifact: size exceeds declared maximum")

	// ErrImmutable means an attempt was made to delete or overwrite a
	// durable artifact through a path reserved for ephemeral ones.
	ErrImmutable = errors.New("dispatch/artifact: durable artifacts are immutable")

	// ErrNoBackend means no storage backend is configured. Every artifact
	// operation is a no-op in this state and Dispatch behaves exactly as
	// it did before the artifact plane existed.
	ErrNoBackend = errors.New("dispatch/artifact: no backend configured")

	// ErrUnbound means a required input declaration has no binding on the
	// job being executed.
	ErrUnbound = errors.New("dispatch/artifact: required input not bound")

	// ErrUndeclared means a binding was supplied for a name the job
	// definition does not declare.
	ErrUndeclared = errors.New("dispatch/artifact: binding has no matching declaration")
)

Both permanent sentinels below unwrap to dispatch.ErrPermanent, so the executor sends a job that hits one straight to the dead letter queue instead of retrying it. A caller chooses how precisely to match:

errors.Is(err, artifact.ErrNotFound)  // the object is gone
errors.Is(err, dispatch.ErrPermanent) // don't retry -- also true

Match dispatch.ErrPermanent when the question is whether to retry, and the specific sentinel when the answer changes what you do. Code that asks "is this missing" in order to decide "should I retry" is asking the wrong question: a permission failure is just as permanent, and answering it with a retry loop wastes the same budget a deleted input would.

Functions

func Required

func Required(s *InputSpec)

Required marks an input as mandatory.

func StageAsPath

func StageAsPath(s *InputSpec)

StageAsPath pre-downloads the input and exposes it as a file path.

func StageLazy

func StageLazy(s *InputSpec)

StageLazy skips the download; the handler streams via Open.

func SupportsPresign

func SupportsPresign(backend Backend, ref Ref) bool

SupportsPresign reports the backend's current signing capability.

func TotalMaxSize

func TotalMaxSize(specs []InputSpec) int64

TotalMaxSize sums the declared limits. It is zero when any declaration is unbounded, since the total is then unknown rather than small.

The engine uses this to reject, at registration, a definition whose inputs could never fit the staging budget — so an unstageable job fails on a developer's machine instead of at 3am.

func ValidateInputs

func ValidateInputs(specs []InputSpec) error

ValidateInputs checks a definition's declarations as a set, rejecting invalid names and duplicates.

func WithAccessor

func WithAccessor(ctx context.Context, a Accessor) context.Context

WithAccessor attaches an Accessor to a context.

Types

type Accessor

type Accessor interface {
	// Path returns the local file path of a staged input. It returns an
	// empty string for an input that was not declared, was not bound, or
	// was declared lazy.
	Path(name string) string

	// Open streams an input's bytes. It works for both staging modes:
	// a path-staged input reads from local disk, a lazy one from the
	// backend.
	Open(ctx context.Context, name string) (io.ReadCloser, error)

	// Ref returns the artifact bound to an input.
	Ref(name string) (Ref, bool)

	// Create begins writing an output owned by the running job. Outputs
	// are imperative rather than declared so a handler can produce a
	// number of them it only discovers at run time.
	Create(ctx context.Context, name string, opts ...CreateOption) (*CommitWriter, error)

	// Existing returns an artifact a previous attempt committed under this
	// name, letting a retried handler skip work it already did.
	Existing(ctx context.Context, name string) (Ref, bool)
}

Accessor is the handler-facing face of the artifact plane. A handler obtains one with From(ctx).

func From

func From(ctx context.Context) Accessor

From returns the Accessor for the running job.

It never returns nil. When the artifact plane is disabled, or the job declared no inputs, it returns a no-op Accessor so a handler calling From(ctx).Path("x") gets an empty string rather than a panic.

type Artifact

type Artifact struct {
	ID          id.ArtifactID `json:"id"`
	Backend     string        `json:"backend"`
	Bucket      string        `json:"bucket"`
	Key         string        `json:"key"`
	Size        int64         `json:"size"`
	ContentHash string        `json:"content_hash,omitempty"`
	ContentType string        `json:"content_type,omitempty"`
	Lifecycle   Lifecycle     `json:"lifecycle"`
	ScopeAppID  string        `json:"scope_app_id,omitempty"`
	ScopeOrgID  string        `json:"scope_org_id,omitempty"`
	ExpiresAt   *time.Time    `json:"expires_at,omitempty"`
	CreatedAt   time.Time     `json:"created_at"`
	DeletedAt   *time.Time    `json:"deleted_at,omitempty"`
}

Artifact is a tracked object in external storage.

func (*Artifact) Clone

func (a *Artifact) Clone() *Artifact

Clone returns a deep copy so stores can hand out values callers may safely mutate.

func (*Artifact) IsDeleted

func (a *Artifact) IsDeleted() bool

IsDeleted reports whether the artifact has been soft-deleted by the sweeper. A soft-deleted artifact is no longer served but its bytes survive until the purge pass.

func (*Artifact) Ref

func (a *Artifact) Ref() Ref

Ref returns a lightweight handle to this artifact.

type Backend

type Backend interface {
	// Name returns the backend's identifier, recorded in Artifact.Backend.
	Name() string

	// Open returns a reader over the object's bytes. It returns
	// ErrNotFound if the object does not exist.
	Open(ctx context.Context, ref Ref) (io.ReadCloser, error)

	// Create begins writing a new object. The bytes are not visible until
	// Commit. Callers must call Commit or Abort.
	Create(ctx context.Context, bucket, key string) (Writer, error)

	// Stat reports the object's size and content type without reading it.
	// It returns ErrNotFound if the object does not exist.
	Stat(ctx context.Context, ref Ref) (ObjectInfo, error)

	// Delete removes the object. Deleting a missing object is not an error.
	Delete(ctx context.Context, ref Ref) error
}

Backend is the pluggable object-storage contract behind an artifact. Dispatch ships an adapter for Trove; any store can implement this.

type CommitWriter

type CommitWriter struct {
	// contains filtered or unexported fields
}

CommitWriter writes an ephemeral artifact's bytes and, on Commit, records the artifact and its link atomically.

func (*CommitWriter) Abort

func (w *CommitWriter) Abort() error

Abort discards the pending object. It is a no-op after Commit, so `defer w.Abort()` is the correct idiom alongside a successful commit.

func (*CommitWriter) Commit

func (w *CommitWriter) Commit(ctx context.Context) (Ref, error)

Commit finalises the object, inserts the artifact, and links it to the owner as an output. The store writes both in one operation, so a zero-link artifact cannot result from a normal race.

func (*CommitWriter) Write

func (w *CommitWriter) Write(p []byte) (int, error)

Write appends bytes to the pending object.

type CreateOption

type CreateOption func(*CreateOptions)

CreateOption configures Create.

func ContentType

func ContentType(ct string) CreateOption

ContentType records the media type of the object being written.

func IfAbsent

func IfAbsent() CreateOption

IfAbsent makes Create return ErrExists when a previous attempt of the same owner already committed this name.

This is what lets a retried handler skip work it already did: a job splitting a 400-page PDF can resume at page 317 instead of re-rendering the 316 pages a prior attempt committed.

func Retain

func Retain(d time.Duration) CreateOption

Retain overrides the default retention for this artifact.

func Scope

func Scope(appID, orgID string) CreateOption

Scope tags the created artifact with a tenant application and org.

type CreateOptions

type CreateOptions struct {
	ContentType string
	ScopeAppID  string
	ScopeOrgID  string
	Retention   time.Duration
	IfAbsent    bool
}

CreateOptions configures creation of an ephemeral artifact.

type InputOption

type InputOption func(*InputSpec)

InputOption configures an InputSpec.

func MaxSize

func MaxSize(bytes int64) InputOption

MaxSize caps the size of a bound artifact. The limit is enforced at enqueue, so an oversized input never becomes a failed job.

type InputSpec

type InputSpec struct {
	// Name identifies the input in the handler and in bindings.
	Name string
	// Required fails the job when no binding is supplied.
	Required bool
	// MaxSize rejects an oversized binding at enqueue. Zero means no limit.
	MaxSize int64
	// Mode selects staging behaviour.
	Mode StageMode
}

InputSpec declares one artifact a job consumes.

Declaring inputs, rather than burying refs in an opaque payload, is what lets the engine know the total input size before it schedules the job, validate bindings at enqueue instead of at run time, and stage the bytes before the handler is ever called.

func FindInput

func FindInput(specs []InputSpec, name string) (InputSpec, bool)

FindInput returns the declaration with the given name.

func Input

func Input(name string, opts ...InputOption) InputSpec

Input declares an artifact input on a job definition.

func (InputSpec) Validate

func (s InputSpec) Validate() error

Validate reports whether the declaration is usable.

The name becomes both a path component in the storage key and a filename in the staging directory, so anything that could escape either is rejected here rather than at run time.

type Lifecycle

type Lifecycle string

Lifecycle determines whether Dispatch may delete an artifact's bytes.

const (
	// Durable artifacts are written by the application and merely tracked
	// by Dispatch. They are read-only here and are never swept.
	Durable Lifecycle = "durable"

	// Ephemeral artifacts are created by Dispatch on a handler's behalf.
	// They are refcounted through links and swept once every owner is
	// terminal and the retention window has passed.
	Ephemeral Lifecycle = "ephemeral"
)

func (Lifecycle) Valid

func (l Lifecycle) Valid() bool

Valid reports whether the lifecycle is a recognised value.

type Link struct {
	ArtifactID id.ArtifactID `json:"artifact_id"`
	OwnerKind  OwnerKind     `json:"owner_kind"`
	OwnerID    string        `json:"owner_id"`
	Role       Role          `json:"role"`
	Name       string        `json:"name"`
	Attempt    int           `json:"attempt"`
	CreatedAt  time.Time     `json:"created_at"`
}

Link records that an owner references an artifact in a given role. Attempt scopes the link to one execution attempt so a retried job's outputs do not collide with its previous attempt's.

func (*Link) Clone

func (l *Link) Clone() *Link

Clone returns a copy of the link.

func (*Link) Owner

func (l *Link) Owner() OwnerRef

Owner returns the OwnerRef this link belongs to.

type ListOpts

type ListOpts struct {
	// Limit is the maximum number of artifacts to return. Zero means no limit.
	Limit int
	// Offset is the number of artifacts to skip.
	Offset int
	// Lifecycle filters by lifecycle. Empty means all.
	Lifecycle Lifecycle
	// ScopeAppID filters by tenant application. Empty means all.
	ScopeAppID string
	// ScopeOrgID filters by tenant organization. Empty means all.
	ScopeOrgID string
	// IncludeDeleted includes soft-deleted artifacts. Default false.
	IncludeDeleted bool
}

ListOpts controls pagination and filtering for artifact list queries.

type ObjectInfo

type ObjectInfo struct {
	Size        int64
	ContentType string
	ETag        string
}

ObjectInfo is what a Backend reports about a stored object.

type OwnerKind

type OwnerKind string

OwnerKind identifies which entity owns a link.

const (
	// OwnerJob links an artifact to a job.
	OwnerJob OwnerKind = "job"
	// OwnerRun links an artifact to a workflow run.
	OwnerRun OwnerKind = "run"
	// OwnerStep links an artifact to a single workflow step.
	OwnerStep OwnerKind = "step"
)

func (OwnerKind) Valid

func (k OwnerKind) Valid() bool

Valid reports whether the owner kind is a recognised value.

type OwnerRef

type OwnerRef struct {
	Kind OwnerKind
	ID   string
}

OwnerRef identifies a link owner.

func (OwnerRef) Valid

func (o OwnerRef) Valid() bool

Valid reports whether the owner reference is usable.

type Page

type Page struct {
	Artifacts  []*Artifact
	NextCursor string
	Complete   bool
}

Page is one page of ListArtifactsPage. NextCursor and Complete mean what they mean on job.Page.

type PageLister

type PageLister interface {
	ListArtifactsPage(ctx context.Context, opts PageOpts) (Page, error)
}

PageLister is the cursor-paged artifact read. ListArtifacts is kept for existing callers.

type PageOpts

type PageOpts struct {
	Lifecycle      Lifecycle
	ScopeAppID     string
	ScopeOrgID     string
	IncludeDeleted bool
	Cursor         string
	Limit          int
}

PageOpts filters and pages ListArtifactsPage, newest first by ID. The filters mean what they mean on ListOpts; only the paging differs.

func (PageOpts) Match

func (o PageOpts) Match(a *Artifact) bool

Match reports whether a passes every filter in o.

type PresignSupport

type PresignSupport interface{ SupportsPresign(ref Ref) bool }

PresignSupport reports whether the driver for an artifact can sign URLs.

type Presigner

type Presigner interface {
	// PresignGet returns a time-limited URL granting read access.
	PresignGet(ctx context.Context, ref Ref, ttl time.Duration) (string, error)
}

Presigner is an optional Backend capability for direct client access.

It is what lets a remote worker fetch a large object straight from object storage instead of streaming it through the coordinator, which would otherwise make the coordinator a bandwidth bottleneck.

type RangeReader

type RangeReader interface {
	// OpenRange returns a reader over n bytes starting at off. A negative
	// n reads to the end.
	OpenRange(ctx context.Context, ref Ref, off, n int64) (io.ReadCloser, error)
}

RangeReader is an optional Backend capability for partial reads.

type RecordReader

type RecordReader interface {
	GetArtifactRecord(ctx context.Context, artifactID id.ArtifactID) (*Artifact, error)
}

RecordReader inspects metadata, including soft-deleted records. It never serves bytes.

type Ref

type Ref struct {
	ID          id.ArtifactID `json:"id"`
	Backend     string        `json:"backend"`
	Bucket      string        `json:"bucket"`
	Key         string        `json:"key"`
	Size        int64         `json:"size"`
	ContentHash string        `json:"content_hash,omitempty"`
}

Ref is a lightweight handle to a tracked artifact. It is what callers pass to Bind, what handlers receive from Commit, and what workflow steps store in checkpoints — small enough to serialise freely.

func (Ref) IsZero

func (r Ref) IsZero() bool

IsZero reports whether the ref is unset.

type RegisterOption

type RegisterOption func(*RegisterOptions)

RegisterOption configures Register.

func WithContentType

func WithContentType(ct string) RegisterOption

WithContentType records the artifact's media type.

func WithScope

func WithScope(appID, orgID string) RegisterOption

WithScope tags the artifact with a tenant application and organization.

type RegisterOptions

type RegisterOptions struct {
	ScopeAppID  string
	ScopeOrgID  string
	ContentType string
}

RegisterOptions configures registration of a durable artifact.

type Role

type Role string

Role describes how an owner relates to an artifact.

const (
	// RoleInput marks an artifact consumed by the owner.
	RoleInput Role = "input"
	// RoleOutput marks an artifact produced by the owner.
	RoleOutput Role = "output"
	// RoleIntermediate marks an artifact passed between workflow steps.
	RoleIntermediate Role = "intermediate"
)

func (Role) Valid

func (r Role) Valid() bool

Valid reports whether the role is a recognised value.

type Service

type Service struct {
	// contains filtered or unexported fields
}

Service is the operational face of the artifact plane. It pairs a Store with a Backend and owns the rules that keep the two consistent: registration is idempotent, ephemeral keys embed the attempt, and — on the CommitWriter path (Create/CreateFenced/Commit) — an artifact row is never written without its link. Register is the exception: it writes a durable row for a pre-existing backend object with a nil link, which Store.CreateArtifact's own contract supports, since there is no job/attempt to link it to yet.

func NewService

func NewService(store Store, backend Backend, opts ...ServiceOption) *Service

NewService creates a Service. A nil backend leaves the artifact plane disabled: every method returns ErrNoBackend and Dispatch behaves exactly as it did before artifacts existed.

func (*Service) Backend

func (s *Service) Backend() Backend

Backend returns the underlying object storage.

func (*Service) Create

func (s *Service) Create(
	ctx context.Context,
	owner OwnerRef,
	attempt int,
	name string,
	opts ...CreateOption,
) (*CommitWriter, error)

Create begins writing an ephemeral artifact owned by owner.

The returned writer publishes nothing until Commit; Abort discards it. With IfAbsent, a name a prior attempt already committed returns ErrExists so the caller can skip the work.

func (*Service) CreateFenced

func (s *Service) CreateFenced(
	ctx context.Context,
	owner OwnerRef,
	attempt int,
	name string,
	fenceToken string,
	opts ...CreateOption,
) (*CommitWriter, error)

CreateFenced is Create, except the storage key additionally incorporates fenceToken, so two callers racing to commit the same (owner, attempt, name) under different fenceTokens can never collide on the same backend object.

This exists for a caller whose own claim to "the current holder of (owner, attempt)" is itself fenced — a worker committing an out-of-process rung's outputs under a lease epoch is the motivating case (see worker.Runner.commitOutputFile). Two workers can each believe they hold the same job at the same RetryCount at once: a lease reclaim races a worker that has not yet noticed its lease expired and is still finishing a long attempt. Create's key is a pure function of (owner, attempt, name), so both would resolve to the identical backend object — whichever Commit lands second would silently overwrite the first's bytes, behind a store row that still claims the first writer's size, with no error surfaced to anyone. A distinct fenceToken per holder (their lease epoch) gives each holder its own object instead: the loser's write lands next to the winner's rather than on top of it, and the store's own uniqueness check on (backend, bucket, key) can then only ever reject an actual repeat of the SAME holder recommitting the SAME name, never one holder clobbering another's bytes purely by losing a race.

fenceToken is never recorded on Link or Artifact — Attempt keeps meaning exactly what it means everywhere else in this package — it only ever changes the storage key most callers never see. A caller with no fence to offer should use Create; an empty fenceToken here is refused rather than silently behaving like Create, so a caller that meant to fence but passed a zero value fails loudly instead of losing the protection without noticing.

func (*Service) DefaultBucket

func (s *Service) DefaultBucket() string

DefaultBucket returns the bucket ephemeral objects are written to.

func (*Service) Enabled

func (s *Service) Enabled() bool

Enabled reports whether a backend is configured.

func (*Service) EphemeralKey

func (s *Service) EphemeralKey(owner OwnerRef, attempt int, name string) string

EphemeralKey returns the storage key for an owner's output.

The attempt is part of the key because Commit is attempt-scoped while storage coordinates are unique: without it, a retried job writing the same name would collide with its own previous attempt.

func (*Service) FindCommitted

func (s *Service) FindCommitted(ctx context.Context, owner OwnerRef, attempt int, name, fenceToken string) (Ref, error)

FindCommitted returns the artifact already sitting at the exact storage coordinates Create or CreateFenced would use for (owner, attempt, name, fenceToken), if one exists. It returns ErrNotFound when none does.

This is narrower than FindExisting on purpose: FindExisting answers "has ANY attempt committed this name," which is what a handler's own Existing/IfAbsent check wants. FindCommitted answers "did THIS EXACT caller already commit THIS EXACT thing" — which is what lets a caller recognise its own earlier, successful partial work as a no-op to skip rather than a collision to fail on, without also treating a DIFFERENT holder's commit of the same name under a different fenceToken as anything but what it is: a separate object at a separate key. See worker.Runner.commitOutputFile, the motivating caller: a retry of a launch-failure-classified commit failure reuses the identical (owner, attempt) — RetryCount does not advance for a launch failure — so without this, re-committing a name that already landed in an earlier, partially-failed pass would collide with itself on every subsequent attempt.

func (*Service) FindExisting

func (s *Service) FindExisting(ctx context.Context, owner OwnerRef, name string) (Ref, error)

FindExisting returns the artifact a previous attempt committed under this owner and name, across all attempts.

func (*Service) Get

func (s *Service) Get(ctx context.Context, artifactID id.ArtifactID) (*Artifact, error)

Get resolves a ref to its stored artifact.

func (s *Service) Link(
	ctx context.Context,
	ref Ref,
	owner OwnerRef,
	role Role,
	name string,
	attempt int,
) error

Link records that an owner references an existing artifact. This is how a declared input is attributed to the job that consumed it.

func (*Service) Open

func (s *Service) Open(ctx context.Context, ref Ref) (io.ReadCloser, error)

Open streams an artifact's bytes.

func (*Service) Register

func (s *Service) Register(ctx context.Context, bucket, key string, opts ...RegisterOption) (Ref, error)

Register records an object the application already uploaded, returning a durable Ref that Dispatch will read but never delete.

Register does not hash the object. Hashing a multi-gigabyte file would turn enqueue into a full read pass; the hash is filled in later by the staging cache, which is already streaming every byte to disk. Until then the artifact is identified by its storage coordinates.

Registering the same coordinates twice returns the existing ref rather than an error, so callers can register unconditionally.

func (*Service) Store

func (s *Service) Store() Store

Store returns the underlying persistence layer.

type ServiceOption

type ServiceOption func(*Service)

ServiceOption configures a Service.

func WithDefaultBucket

func WithDefaultBucket(bucket string) ServiceOption

WithDefaultBucket sets the bucket ephemeral objects are written to.

func WithEphemeralPrefix

func WithEphemeralPrefix(prefix string) ServiceOption

WithEphemeralPrefix sets the key prefix for Dispatch-owned objects.

func WithRetention

func WithRetention(d time.Duration) ServiceOption

WithRetention sets the default retention applied to ephemeral artifacts that do not carry their own expiry.

type StageMode

type StageMode int

StageMode determines how a declared input is presented to the handler.

const (
	// StageModePath materialises the artifact to local disk before the
	// handler runs, and Accessor.Path returns the file path.
	//
	// This is the default because the libraries that read Dispatch's heavy
	// inputs — CAD kernels, mesh importers, PDF engines — seek and
	// memory-map. They need a file, not a stream.
	StageModePath StageMode = iota

	// StageModeLazy downloads nothing up front. The handler calls
	// Accessor.Open to stream the bytes if and when it needs them.
	//
	// Right for data read once, front to back. Wrong for a multi-gigabyte
	// model a native library will seek around in.
	StageModeLazy
)

func (StageMode) String

func (m StageMode) String() string

String renders the mode for logs and errors.

type Store

type Store interface {
	// CreateArtifact inserts an artifact and, when link is non-nil, its
	// first link atomically. Returns ErrExists if an artifact already
	// exists at the same backend, bucket, and key.
	CreateArtifact(ctx context.Context, a *Artifact, link *Link) error

	// GetArtifact retrieves an artifact by ID. Returns ErrNotFound if it
	// does not exist or has been soft-deleted.
	GetArtifact(ctx context.Context, artifactID id.ArtifactID) (*Artifact, error)

	// FindArtifactByKey retrieves an artifact by its storage coordinates.
	// Returns ErrNotFound if none exists.
	FindArtifactByKey(ctx context.Context, backend, bucket, key string) (*Artifact, error)

	// UpdateArtifact persists changes to size, content hash, content type,
	// and expiry. It must not permit changing lifecycle.
	UpdateArtifact(ctx context.Context, a *Artifact) error

	// ListArtifacts returns artifacts matching the given options.
	ListArtifacts(ctx context.Context, opts ListOpts) ([]*Artifact, error)

	// LinkArtifact records that an owner references an artifact. Linking
	// the same artifact, owner, name, and attempt twice is a no-op rather
	// than an error.
	LinkArtifact(ctx context.Context, link *Link) error

	// ListLinks returns every link belonging to the given owner.
	ListLinks(ctx context.Context, owner OwnerRef) ([]*Link, error)

	// FindLinkByName returns the link for an owner and name with the
	// highest attempt number, breaking ties by CreatedAt descending so
	// resolution is deterministic and favours the later writer. This is
	// what IfAbsent uses to detect that a prior attempt already produced
	// an output. Returns ErrNotFound if no attempt has produced it.
	//
	// Ties happen: track D's lease reclaim increments EvictCount, never
	// RetryCount, so a reclaimed (zombie) holder and the worker it was
	// fenced out for can both commit a link at the same (OwnerKind,
	// OwnerID, Name, Attempt) — CreateFenced closes the storage-key
	// collision between them, but nothing stops two Link rows with
	// different ArtifactIDs at the identical tuple. The CreatedAt
	// tie-break is the cheap half of the fix: it makes which of the two
	// wins deterministic instead of backend-dependent map/query order.
	// The complete fix — carrying the fence token on Link itself, so the
	// zombie's write is rejected rather than merely outrun — is a schema
	// change across all five backends and remains open.
	FindLinkByName(ctx context.Context, owner OwnerRef, name string) (*Link, error)

	// ListArtifactsByOwner returns the artifacts linked to an owner,
	// optionally filtered by role. An empty role returns all.
	ListArtifactsByOwner(ctx context.Context, owner OwnerRef, role Role) ([]*Artifact, error)

	// SweepEphemeral marks eligible ephemeral artifacts as deleted and
	// returns them.
	SweepEphemeral(ctx context.Context, opts SweepOpts) ([]*Artifact, error)

	// SweepOrphans marks ephemeral artifacts that have no links at all and
	// were created before the cutoff.
	SweepOrphans(ctx context.Context, cutoff time.Time, limit int) ([]*Artifact, error)

	// ListPurgeable returns soft-deleted artifacts whose deletion is older
	// than grace, so their bytes may be removed from the backend.
	ListPurgeable(ctx context.Context, grace time.Duration, limit int) ([]*Artifact, error)

	// PurgeArtifact hard-deletes an artifact row and its links after the
	// bytes have been removed from the backend.
	PurgeArtifact(ctx context.Context, artifactID id.ArtifactID) error
}

Store defines the persistence contract for artifacts and their links.

Implementations must guarantee that CreateArtifact inserts the artifact and its link atomically, so a zero-link artifact can only result from a partial failure and never from a normal race.

SweepEphemeral and SweepOrphans must constrain themselves to Lifecycle == Ephemeral using a literal, never a value threaded through from a caller. Durable artifacts must be unreachable from both.

type SweepOpts

type SweepOpts struct {
	// Retention is the grace period after the last owner reaches a
	// terminal state before an artifact becomes eligible.
	Retention time.Duration
	// Limit caps how many artifacts a single sweep call may mark.
	// Zero means no limit.
	Limit int
	// DryRun computes eligibility and returns the artifacts that would be
	// marked without modifying anything.
	DryRun bool
}

SweepOpts controls a lifecycle sweep.

type Writer

type Writer interface {
	io.Writer

	// Commit finalises the object and returns its logical info.
	Commit(ctx context.Context) (ObjectInfo, error)

	// Abort discards the partial object. It is a no-op after Commit.
	Abort() error
}

Writer accumulates bytes for a new object.

Commit reports the logical size of the bytes written, which may differ from what the backend stored — compression and encryption middleware change the stored form, and the artifact row records what the handler produced.

Abort after a successful Commit is a no-op, so `defer w.Abort()` is the correct idiom.

Directories

Path Synopsis
Package artifacttest provides a shared conformance suite and test doubles for artifact storage.
Package artifacttest provides a shared conformance suite and test doubles for artifact storage.
Package cache is the worker-local staging cache for artifact inputs.
Package cache is the worker-local staging cache for artifact inputs.
Package staging wires the artifact plane into job execution.
Package staging wires the artifact plane into job execution.
Package sweeper reclaims the storage Dispatch owns.
Package sweeper reclaims the storage Dispatch owns.
Package trove adapts a *trove.Trove instance as an artifact.Backend.
Package trove adapts a *trove.Trove instance as an artifact.Backend.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL