artifactfs

package
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 16, 2026 License: MIT, MIT Imports: 16 Imported by: 0

Documentation

Overview

Package artifactfs is a crash-safe, local-filesystem implementation of go-workflow's values.ArtifactStore.

Layout. Every artifact lives in a content-bound directory objects/<scope>/<sha256(ownerID)>/<artifactID>/ holding a "payload" file and a strictly decoded "manifest.json"; in-flight writes live under staging/ (prefix "partial-"). Directories are mode 0700 and files 0600. Owner ids and absolute paths never enter a reference: the URI is artifact://<authority>/<scope>/<sha256(ownerID)>/<artifactID>, where the artifact id is the SHA-256 of the artifact's immutable identity (authority, owner scope and hash, digest, media type, size, producer, redaction, retention, timestamps).

Durability. Put streams the source through a size bound and SHA-256 into staging, fsyncs payload, manifest and directory, then publishes the directory with one atomic rename; identical concurrent Puts converge on one artifact. Reads verify the complete payload digest before releasing bytes.

Fail-closed checks. The identities of the root, objects and staging directories are captured by New and revalidated on every operation, symlinks and non-regular files are rejected on every path component, and authorization happens before any local path is resolved and again after the manifest is strictly loaded. The authority string is persisted identity: it is required, has no default, and must never change for an existing root.

The on-disk format is frozen; changing it requires a manifest version bump. References of other authorities are routed only to delegates approved with WithExternal and are never deleted through them.

Example
package main

import (
	"context"
	"fmt"
	"io"
	"os"
	"path/filepath"
	"strings"
	"time"

	"github.com/hollis-labs/libs/workflow/engine/host/artifactfs"
	"github.com/hollis-labs/libs/workflow/engine/values"
)

func main() {
	// The root must be a real path: New rejects any symlink component.
	tmp, _ := os.MkdirTemp("", "artifacts-")
	defer func() { _ = os.RemoveAll(tmp) }()
	root, err := filepath.EvalSymlinks(tmp)
	if err != nil {
		panic(err)
	}

	// The authority is persisted identity: pick it once, never change it.
	store, err := artifactfs.New(root, "example-local", artifactfs.OwnerClaimAuthorizer("example-local", ""))
	if err != nil {
		panic(err)
	}

	ctx := context.Background()
	now := time.Now().UTC()
	access := values.ArtifactAccess{Principal: "worker", RunID: "run-1", At: now}
	request := values.ArtifactPutRequest{
		Store: "example-local",
		Owner: values.ArtifactOwner{Scope: values.ArtifactOwnerRun, ID: "run-1"},
		Metadata: values.Metadata{
			Producer:  values.Producer{Kind: "node_output", Reference: "invocation-1", Output: "result"},
			MediaType: "text/plain", Redaction: values.RedactionPrivate, Retention: values.RetentionRun,
		},
		MaxBytes: 1 << 20, CreatedAt: now, Access: access,
	}
	value, err := values.CaptureValue(ctx, store, request, strings.NewReader("hello, artifact"), values.CapturePolicy{
		Mode: values.CaptureArtifactOnly, InlineLimit: values.DefaultInlineLimit,
	})
	if err != nil {
		panic(err)
	}

	reader, err := store.Open(ctx, access, *value.Artifact)
	if err != nil {
		panic(err)
	}
	defer func() { _ = reader.Close() }()
	body, _ := io.ReadAll(reader)
	fmt.Println(string(body), reader.Verified())
}
Output:
hello, artifact true

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func OwnerClaimAuthorizer

func OwnerClaimAuthorizer(authority, maintenancePrincipal string) values.ArtifactAuthorizer

OwnerClaimAuthorizer returns a strict default ownership policy for a Store whose local authority is authority (owner ids are hashed into the reference, so the policy needs it). A principal can access a run or project artifact only with the exact corresponding claim (Access.RunID or Access.ProjectID equal to the owner id, or hashing to the owner segment of the reference when the owner is not yet known). Global cleanup is reserved for the maintenance principal; an empty maintenancePrincipal means "system".

Types

type Option

type Option func(*config)

Option configures New.

func WithExternal

func WithExternal(authority string, delegate values.ArtifactStore) Option

WithExternal approves a foreign authority: Stat and Open of references whose Store is authority are passed to delegate (after this store's own authorization, and re-authorized against the metadata the delegate returns), and Delete or Cleanup of such references preserves them without calling the delegate. The authority must be canonical and differ from the store's own.

type Store

type Store struct {
	// contains filtered or unexported fields
}

Store is a local, immutable go-workflow artifact store. Each artifact is stored as a content-bound directory containing a payload and a strictly decoded manifest. Owner identities are hashed before they enter a path or URI. References of other authorities are routed only to delegates approved with WithExternal and are never deleted through them.

func New

func New(root, authority string, authorizer values.ArtifactAuthorizer, opts ...Option) (*Store, error)

New constructs a fail-closed store below root. authority is the persisted identity of the store: it is part of every ArtifactRef (Store field and URI host) and of every artifact id, so it has no default and must never change for an existing root. It must match ^[a-z][a-z0-9.-]{0,62}$. authorizer is the host's authorization boundary and is required.

func (*Store) Cleanup

Cleanup applies go-workflow's owner, expiry, partial, none, and external cleanup boundaries. External bytes belong to their delegate and are never deleted here.

func (*Store) Delete

Delete removes one exact local reference. Repeated deletion is successful and reports already_absent.

func (*Store) Open

Open returns a streaming reader that independently verifies the digest as the caller consumes it. The stored bytes are verified once before exposure.

func (*Store) Put

func (store *Store) Put(ctx context.Context, request values.ArtifactPutRequest, source io.Reader) (values.ArtifactMetadata, error)

Put streams one bounded artifact into an atomic staging directory before publishing its immutable content-bound location.

func (*Store) Stat

Stat validates authorization, ownership, expiry, immutable metadata, size, and payload digest before returning metadata.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL