Documentation
¶
Overview ¶
Package artifactfs is a crash-safe, local-filesystem implementation of go-workflow's values.ArtifactStore.
Layout. Every artifact lives in a content-bound directory objects/<scope>/<sha256(ownerID)>/<artifactID>/ holding a "payload" file and a strictly decoded "manifest.json"; in-flight writes live under staging/ (prefix "partial-"). Directories are mode 0700 and files 0600. Owner ids and absolute paths never enter a reference: the URI is artifact://<authority>/<scope>/<sha256(ownerID)>/<artifactID>, where the artifact id is the SHA-256 of the artifact's immutable identity (authority, owner scope and hash, digest, media type, size, producer, redaction, retention, timestamps).
Durability. Put streams the source through a size bound and SHA-256 into staging, fsyncs payload, manifest and directory, then publishes the directory with one atomic rename; identical concurrent Puts converge on one artifact. Reads verify the complete payload digest before releasing bytes.
Fail-closed checks. The identities of the root, objects and staging directories are captured by New and revalidated on every operation, symlinks and non-regular files are rejected on every path component, and authorization happens before any local path is resolved and again after the manifest is strictly loaded. The authority string is persisted identity: it is required, has no default, and must never change for an existing root.
The on-disk format is frozen; changing it requires a manifest version bump. References of other authorities are routed only to delegates approved with WithExternal and are never deleted through them.
Example ¶
package main
import (
"context"
"fmt"
"io"
"os"
"path/filepath"
"strings"
"time"
"github.com/hollis-labs/libs/workflow/engine/host/artifactfs"
"github.com/hollis-labs/libs/workflow/engine/values"
)
func main() {
// The root must be a real path: New rejects any symlink component.
tmp, _ := os.MkdirTemp("", "artifacts-")
defer func() { _ = os.RemoveAll(tmp) }()
root, err := filepath.EvalSymlinks(tmp)
if err != nil {
panic(err)
}
// The authority is persisted identity: pick it once, never change it.
store, err := artifactfs.New(root, "example-local", artifactfs.OwnerClaimAuthorizer("example-local", ""))
if err != nil {
panic(err)
}
ctx := context.Background()
now := time.Now().UTC()
access := values.ArtifactAccess{Principal: "worker", RunID: "run-1", At: now}
request := values.ArtifactPutRequest{
Store: "example-local",
Owner: values.ArtifactOwner{Scope: values.ArtifactOwnerRun, ID: "run-1"},
Metadata: values.Metadata{
Producer: values.Producer{Kind: "node_output", Reference: "invocation-1", Output: "result"},
MediaType: "text/plain", Redaction: values.RedactionPrivate, Retention: values.RetentionRun,
},
MaxBytes: 1 << 20, CreatedAt: now, Access: access,
}
value, err := values.CaptureValue(ctx, store, request, strings.NewReader("hello, artifact"), values.CapturePolicy{
Mode: values.CaptureArtifactOnly, InlineLimit: values.DefaultInlineLimit,
})
if err != nil {
panic(err)
}
reader, err := store.Open(ctx, access, *value.Artifact)
if err != nil {
panic(err)
}
defer func() { _ = reader.Close() }()
body, _ := io.ReadAll(reader)
fmt.Println(string(body), reader.Verified())
}
Output: hello, artifact true
Index ¶
- func OwnerClaimAuthorizer(authority, maintenancePrincipal string) values.ArtifactAuthorizer
- type Option
- type Store
- func (store *Store) Cleanup(ctx context.Context, request values.ArtifactCleanupRequest) (values.ArtifactCleanupResult, error)
- func (store *Store) Delete(ctx context.Context, request values.ArtifactDeleteRequest) (values.ArtifactCleanupResult, error)
- func (store *Store) Open(ctx context.Context, access values.ArtifactAccess, ref values.ArtifactRef) (values.ArtifactReadCloser, error)
- func (store *Store) Put(ctx context.Context, request values.ArtifactPutRequest, source io.Reader) (values.ArtifactMetadata, error)
- func (store *Store) Stat(ctx context.Context, access values.ArtifactAccess, ref values.ArtifactRef) (values.ArtifactMetadata, error)
Examples ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func OwnerClaimAuthorizer ¶
func OwnerClaimAuthorizer(authority, maintenancePrincipal string) values.ArtifactAuthorizer
OwnerClaimAuthorizer returns a strict default ownership policy for a Store whose local authority is authority (owner ids are hashed into the reference, so the policy needs it). A principal can access a run or project artifact only with the exact corresponding claim (Access.RunID or Access.ProjectID equal to the owner id, or hashing to the owner segment of the reference when the owner is not yet known). Global cleanup is reserved for the maintenance principal; an empty maintenancePrincipal means "system".
Types ¶
type Option ¶
type Option func(*config)
Option configures New.
func WithExternal ¶
func WithExternal(authority string, delegate values.ArtifactStore) Option
WithExternal approves a foreign authority: Stat and Open of references whose Store is authority are passed to delegate (after this store's own authorization, and re-authorized against the metadata the delegate returns), and Delete or Cleanup of such references preserves them without calling the delegate. The authority must be canonical and differ from the store's own.
type Store ¶
type Store struct {
// contains filtered or unexported fields
}
Store is a local, immutable go-workflow artifact store. Each artifact is stored as a content-bound directory containing a payload and a strictly decoded manifest. Owner identities are hashed before they enter a path or URI. References of other authorities are routed only to delegates approved with WithExternal and are never deleted through them.
func New ¶
func New(root, authority string, authorizer values.ArtifactAuthorizer, opts ...Option) (*Store, error)
New constructs a fail-closed store below root. authority is the persisted identity of the store: it is part of every ArtifactRef (Store field and URI host) and of every artifact id, so it has no default and must never change for an existing root. It must match ^[a-z][a-z0-9.-]{0,62}$. authorizer is the host's authorization boundary and is required.
func (*Store) Cleanup ¶
func (store *Store) Cleanup(ctx context.Context, request values.ArtifactCleanupRequest) (values.ArtifactCleanupResult, error)
Cleanup applies go-workflow's owner, expiry, partial, none, and external cleanup boundaries. External bytes belong to their delegate and are never deleted here.
func (*Store) Delete ¶
func (store *Store) Delete(ctx context.Context, request values.ArtifactDeleteRequest) (values.ArtifactCleanupResult, error)
Delete removes one exact local reference. Repeated deletion is successful and reports already_absent.
func (*Store) Open ¶
func (store *Store) Open(ctx context.Context, access values.ArtifactAccess, ref values.ArtifactRef) (values.ArtifactReadCloser, error)
Open returns a streaming reader that independently verifies the digest as the caller consumes it. The stored bytes are verified once before exposure.
func (*Store) Put ¶
func (store *Store) Put(ctx context.Context, request values.ArtifactPutRequest, source io.Reader) (values.ArtifactMetadata, error)
Put streams one bounded artifact into an atomic staging directory before publishing its immutable content-bound location.
func (*Store) Stat ¶
func (store *Store) Stat(ctx context.Context, access values.ArtifactAccess, ref values.ArtifactRef) (values.ArtifactMetadata, error)
Stat validates authorization, ownership, expiry, immutable metadata, size, and payload digest before returning metadata.