payloads

package
v0.4.7 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 3, 2026 License: MIT Imports: 7 Imported by: 0

Documentation

Overview

Package payloads is the built-in file `payloads` dataset — the node-readable per-file index of the files subsystem (docs/files.md).

One row per file, living in a plaintext (node-readable) derived child object under the file's owner:

  • cleartext, node-readable: id (the fileId), rootCid, size, networkSign, author — everything a filenode-v2 broker needs for refcount / GC / quota / durability without any space key;
  • one sealed field `enc` — AES-GCM under a key derived from the space read key — grouping the member-only secrets {key, name, sha256, mime, inline}.

The object's tree changes ship UNencrypted at the any-sync level (object.PlaintextSpec); the privacy boundary is the `enc` field, sealed and unsealed by this package. `enc` stays sealed at rest in the materialized row; typed readers unseal at read time, so a keyless reader materializes the row like everyone else and simply can't open `enc`.

Index

Constants

View Source
const (
	FieldRootCid     = "rootCid"     // string; ABSENT for inline rows
	FieldSize        = "size"        // number; plaintext byte size
	FieldNetworkSign = "networkSign" // string; absent until durable; requires rootCid
	FieldAuthor      = "author"      // string; derived from the change signer
	FieldObjectId    = "objectId"    // string; the object this file is bound to (cleartext so the files view can filter by parent)
	FieldEnc         = "enc"         // object {kid, ct}; sealed member-only secrets
)

Row field names. Full names, matching the row-field convention of the objects dataset (author/createdAt/spaceId/modifiedAt/modifiedBy); short keys are a change-envelope concern, not a row concern.

View Source
const (
	EncKeyId         = "kid" // ACL key-record id the blob was sealed under
	EncKeyCiphertext = "ct"  // AES-GCM sealed EncPayload (nonce prepended)
)

Keys inside the cleartext `enc` wrapper object.

View Source
const ChangeType = "payloads"

ChangeType is the tree-root change type of a payloads object. It is the plaintext-class anchor: the root is immutable, signed, and cleartext, so keyed and keyless readers agree the object is node-readable and restricted to this dataset.

View Source
const Dataset = "payloads"

Dataset is the CRDT dataset name; on disk the collection is `<payloadsObjectId>_payloads`.

View Source
const HandlerVersion = "payloads-v1"

HandlerVersion is the DataVersion stamped on every payloads change. Hardcoded handler identifier (same convention as the other built-in datasets); bump the suffix when validation rules must reject stale writers.

View Source
const InlineMaxSize = 4096

InlineMaxSize is the inline-tier cutoff: a file strictly smaller than this rides inside `enc` (Inline bytes) with no rootCid / S3 / networkSign — the filenode never sees it.

View Source
const MaxNetworkSignLen = 2048

MaxNetworkSignLen bounds the recorded networkSign — it is an opaque receipt string ("{fileNetworkId}/{sign(rootCid)}"), not a blob.

View Source
const WellKnownDeriveSeed = "builtin:payloads"

WellKnownDeriveSeed is the ChangePayload seed of a payloads object whose owner is a SIGNED object. Combined with ParentId (the owner object) it derives the same deterministic child id on every peer, and binds the payloads object to the owner so any-sync cascade-deletes it with the owner.

Variables

View Source
var ErrNoKey = errors.New("payloads: no key")

ErrNoKey is the sentinel a KeyProvider returns when it cannot resolve a sealing key (keyless reader, unknown key-record id). Typed readers degrade to cleartext-only rows on it — it is the expected condition for a filenode-v2 broker, not a failure.

View Source
var ErrOwnerUnknown = errors.New("payloads: owner class unknown; payloads id not resolvable yet")

ErrOwnerUnknown: the payloads id for this owner is not locally resolvable yet. The id depends on the owner's class (signed vs derived — see DerivedOwnerSeed), and nothing local discloses it: no head entry for the owner, and no payloads tree of either shape. A signed owner that hasn't synced and a derived owner that never grew a payloads tree are locally indistinguishable, so resolving here would be a guess whose answer flips once the owner arrives. Resolvable after sync delivers the owner (or its payloads tree).

Functions

func DeriveEncKey

func DeriveEncKey(readKey crypto.SymKey) (crypto.SymKey, error)

DeriveEncKey derives the payloads enc-sealing key from a space read key. Deterministic: every member derives the same sealing key from the same read key, so `kid` can stay the ACL key-record id of the read key itself. Implementations of KeyProvider should cache the result per kid.

func DerivedOwnerSeed

func DerivedOwnerSeed(ownerId string) string

DerivedOwnerSeed is the ChangePayload seed of a payloads object whose owner is a DERIVED object. A derived object cannot be a tree parent (objecttree.ErrDerivedParent), so the payloads object can't take the parented WellKnownDeriveSeed shape signed owners get. It is derived UNPARENTED instead, with the ownerId folded into the seed so each derived owner still resolves to its own deterministic, per-owner payloads id (an unparented tree has no ParentId to carry that uniqueness). The two are coupled: unparented + owner-in-seed must move together, or every derived owner would collide on a single id.

func NotWritten added in v0.4.4

func NotWritten(err error) bool

NotWritten reports whether err says registration wrote no row.

func Schema

func Schema() schema.Dataset

Schema declares the payloads dataset. Non-Dynamic: undeclared fields are rejected by the controller on both the local and the inbound route, so the cleartext surface can never silently grow.

func SealEnc

func SealEnc(key crypto.SymKey, p EncPayload) ([]byte, error)

SealEnc serialises p and seals it with key (AES-256-GCM, nonce prepended by crypto.AESKey.Encrypt). The result is the `ct` value of the row's `enc` field.

Types

type EncPayload

type EncPayload struct {
	// Key is the file's wrapped symmetric key (raw bytes; the byte
	// layer owns its format). Empty for inline rows if the caller
	// chose to seal the bytes directly.
	Key []byte
	// Name is the user-facing file name.
	Name string
	// SHA256 is the plaintext content hash — the whole-file dedup key.
	SHA256 []byte
	// Mime is the content type hint.
	Mime string
	// Inline holds the file bytes for the inline tier (< InlineMaxSize),
	// which has no rootCid / S3 / networkSign and rides the CRDT.
	Inline []byte
	// Variant tags this row as an alternate representation (e.g.
	// "thumbnail") of VariantOf. Variants are ordinary sibling rows —
	// own tier, own durability, own refcount; the relationship lives
	// only here in the sealed meta, so the network sees N independent
	// files.
	Variant string
	// VariantOf is the fileId of the original this row is a variant of.
	VariantOf string
}

EncPayload is the member-only plaintext grouped behind the sealed `enc` field. One sealed value on purpose: a single decrypt per row and better wire compression than per-field sealing. The wrapped per-file key stays inside (history is never re-sealed on ACL rotation, so a separate field would buy nothing).

Wire shape (anyenc object inside the ciphertext): {key, name, sha256, mime, inline?} — unknown keys are ignored on read, so richer metadata (e.g. image dimensions) can be added without breaking older readers.

func UnsealEnc

func UnsealEnc(key crypto.SymKey, ct []byte) (EncPayload, error)

UnsealEnc opens a sealed `ct` value and parses the EncPayload.

type Handler

type Handler struct {
	crdt.DefaultHandler
}

Handler is the payloads dataset behavior.

Two validation layers, split along the local/inbound line like the other built-ins:

  • PreValidate (local writes, before the DAG): STRICT by shape. The dataset is SDK-written only (the public Modify API fences it off), so exactly three change shapes exist — register a file, record a networkSign, delete a row — and anything else rejects the whole write.
  • BeforeCreate / BeforeModify (inbound + replay): tolerant per-op drop. BeforeCreate stamps the derived `author`; BeforeModify enforces immutability (rootCid / size / enc are create-time facts) and the networkSign-requires-rootCid rule, dropping only the offending op.

func (Handler) BeforeCreate

func (Handler) BeforeCreate(ctx *crdt.ChangeCtx, _ *crdt.RecordChange, sink *crdt.Sink) error

BeforeCreate stamps the derived `author` from the creating change's signer (the per-change Creator, not the tree-root author — for collective durability another member may register rows in the same payloads object). Tolerates an empty Creator (hand-built changes in tests, pre-bind drains) by simply not stamping.

func (Handler) BeforeModify

func (Handler) BeforeModify(ctx *crdt.ChangeCtx, _ *crdt.RecordChange, op *crdt.Op, _ *crdt.Sink) error

BeforeModify is the inbound-tolerant guard on existing rows: a returned error drops just this op, the rest of the record still applies.

func (Handler) Init

func (Handler) Init(_ context.Context) error

func (Handler) PreValidateMulti

func (Handler) PreValidateMulti(ch *crdt.Change, get crdt.RecordGetter) error

PreValidateMulti implements crdt.LocalPreValidatorMulti — the strict local shape gate, batch-first: bulk flows (a pasted doc with hundreds of images) register and sign files in as few changes as possible.

Batches are homogeneous: a change is either N register-file creates (empty-id records; their fileIds resolve to the derived seed with a digit suffix per extra record — seed, seed:1, …), N networkSign writes (per-record pre-state via the getter), or N row deletes.

type KeyProvider

type KeyProvider interface {
	CurrentKey(ctx context.Context) (kid string, key crypto.SymKey, err error)
	KeyById(ctx context.Context, kid string) (crypto.SymKey, error)
}

KeyProvider resolves the enc-sealing keys (already domain-derived, see DeriveEncKey) by ACL key-record id. CurrentKey serves the seal path; KeyById serves unseal — including rows sealed under an older read key after an ACL rotation (any-sync retains historical read keys in AclState().Keys()). Both return ErrNoKey when the caller has no read access.

type NotWrittenError added in v0.4.4

type NotWrittenError struct{ Err error }

NotWrittenError wraps a registration failure from before the row write: no row exists, so nothing references what the caller staged for it.

func (*NotWrittenError) Error added in v0.4.4

func (e *NotWrittenError) Error() string

func (*NotWrittenError) Unwrap added in v0.4.4

func (e *NotWrittenError) Unwrap() error

type Row

type Row struct {
	// Id is the fileId — derived from the creating change
	// (crdt.DeriveRecordId of its ChangeId), deterministic and never
	// reused.
	Id string
	// RootCid is the UnixFS root of the encrypted file. Empty for
	// inline rows.
	RootCid string
	// Size is the plaintext byte size (cleartext hint; quota uses the
	// node's own measurement).
	Size int64
	// NetworkSign is the node's durable-custody receipt. Empty until
	// the file is durable; always empty for inline rows.
	NetworkSign string
	// Author is the account that registered the file (derived from the
	// creating change's signer).
	Author string
	// ObjectId is the object the file is bound to (cleartext parent
	// reference — the derived payloads-object id stays internal).
	ObjectId string
	// EncKid is the ACL key-record id the `enc` blob was sealed under
	// (cleartext — needed to pick the unseal key after ACL rotation).
	EncKid string

	// Enc holds the unsealed member-only secrets after Unseal.
	Enc EncPayload
	// Sealed is true until Unseal succeeds. A keyless reader keeps
	// Sealed rows — that's the broker view, not an error.
	Sealed bool
	// UnsealErr records why a KEYED unseal failed (GCM rejection —
	// the writer stamped a kid that doesn't match the sealing key —
	// or a malformed enc). The row stays Sealed. Kept per-row so one
	// poisoned row (any write-permitted member can produce one)
	// degrades to an unreadable row instead of failing every listing.
	UnsealErr error
	// contains filtered or unexported fields
}

Row is the typed view of one payloads record. The cleartext fields are always populated from the materialized row; Enc is populated only after a successful Unseal (Sealed reports whether the secrets are still closed).

func RowFromValue

func RowFromValue(v *anyenc.Value) (Row, error)

RowFromValue decodes the typed Row from a materialized payloads record. Byte slices are copied out, so the Row does not retain the caller's buffer.

func (*Row) Inline

func (r *Row) Inline() bool

Inline reports whether the row is an inline-tier file (no rootCid, bytes inside enc).

func (*Row) Unseal

func (r *Row) Unseal(ctx context.Context, kp KeyProvider) error

Unseal opens the row's enc blob via the provider. On ErrNoKey the row simply stays Sealed (the keyless-reader view) and no error is returned. A row whose blob cannot be opened WITH a key — GCM rejects the ciphertext, the enc shape is malformed — stays Sealed with UnsealErr set and also returns nil: the row data is poisoned, not the read (mirrors any-sync tolerating unreadable changes). Only environment failures (the provider failing to resolve keys) propagate as errors.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL