sync

package
v0.1.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 5, 2026 License: MIT Imports: 24 Imported by: 0

Documentation

Overview

ack.go is the wire format for signed per-device sync acknowledgements (P4-SYNC-06, P6-HUB-04 completion). After a fully-clean sync cycle a device publishes a small signed marker to workspaces/<ws>/meta/acks/<device_id>.json stating: the per-origin-device transport cursor it has consumed, the push watermark its own stream has reached, and the HLC watermark at or below which it has applied every event from every other device. A compactor reads the acks of all approved peers and takes the MINIMUM HLC watermark as the safe floor for tombstone garbage collection (GCTombstones).

The safety argument (the "tombstone-safety clock"): an ack is written only after a clean cycle in which the push watermark reached this device's local max Seq, so every event this device mints LATER carries an HLC strictly above its acked watermark. A tombstone whose delete-HLC is below the MINIMUM acked watermark can therefore never be resurrected — no device can still produce an add below that HLC, and every device has already consumed the delete (a clean cycle consumes the whole hub log). A later add above the floor is a legitimate restore, not a resurrection.

A withheld or stale ack can only DELAY tombstone GC (an availability effect, never an integrity one): a missing peer ack skips GC entirely, and a stale ack only lowers the min. Ack forgery requires an approved device's private signing key — a revoked/lost/unknown device's ack is ignored, so it can neither pin nor advance the floor.

Package sync implements the local-first event-log hub protocol, HLC ordering, event apply/dedup, and the EncryptedHub envelope-encryption decorator (P4-SEC-02/SEC-07).

It proves the local-first spine the rest of the product will build on:

  • Hybrid Logical Clock ordering with a device-id tiebreaker (hlc.go),
  • an append-only, content-hashed, idempotent event log (events.go),
  • deterministic replay and order-independent same-path/different-remote conflict DETECTION across two local roots,
  • HLC-gated tombstones, rename, and a clock-skew quarantine guard,
  • envelope encryption of the event log at the hub boundary (encryptedhub.go/eventcrypt.go): XChaCha20-Poly1305 under a per-epoch Workspace Content Key, wrapped with age to approved device recipients, so the hub stores only ciphertext carriers (P4-SEC-02/SEC-07).

Scope and assumptions:

  • Namespace state is treated as single-writer-per-path most of the time; the path/remote conflict class is surfaced for the user and never auto-merged. The safe-automatic cases defined in spec/03 (duplicate skeleton creation, heartbeat latest-wins, recreate-missing-skeleton) may still be resolved without prompting.
  • The on-wire hub protocol is NOT defined here. The device_sig and prev_event_hash chain columns are written and validated locally as a deliberate, accepted divergence from the original "defer until the hub" plan (see docs/audits/AUDIT_RECOMMENDATIONS.md ARCH-2 / spec/07); the chain FORMAT should still be re-reviewed before a production hub freezes it.
  • FileHub (hub.go) is a file-backed TEST hub only.

Before building a bespoke devstraphub, re-evaluate whether a hidden manifest git repo (spec/01, spec/04) is a faster transport than a new service.

snapshot.go is the wire format for full-state snapshot exchange (P4-SYNC-02 / P4-HUB-11): a sealed, content-addressed snapshot object plus a signed retention manifest that names it and carries the hub's per-device retention floors (P6-HUB-04).

Trust model: the snapshot OBJECT is not signed directly — one Ed25519 signature on the retention MANIFEST covers both, because the manifest names the snapshot object by sha256 and the object's plaintext is bound to its carrier fields by the AEAD additional data. An importer therefore verifies, in order: (1) the manifest signature against a locally pinned approved device (fail-closed, no pre-enrollment window — a snapshot import is wholesale state replacement, so unlike event verification there is no bootstrap acceptance path); (2) the fetched object's sha256 against the manifest; (3) the AEAD open with the carrier-derived AAD. A malicious hub can withhold or garble either object (forced refusal, a DoS) but can never inject state.

Encryption deliberately mirrors eventcrypt.go's enc.v2 (XChaCha20-Poly1305 under the per-epoch Workspace Content Key) rather than the age blob plane: the snapshot has exactly the event log's sensitivity and audience (every WCK-granted device), WCK grants already solve group access with no per-device re-wrap on enrollment, and sealing under the CURRENT epoch makes each compaction a natural retirement boundary for old-epoch ciphertext (a fresh joiner never needs a retired epoch's key).

snapshot_build.go assembles the plaintext snapshot document from store reads (P4-HUB-11, producer half). It is the symmetric counterpart to snapshot_import.go: import derives local state from a snapshot; BuildSnapshot derives a snapshot from local state. V/Epoch/KID are left zero here — they are stamped by SealSnapshot so the document and its envelope can never disagree.

snapshot_import.go applies a verified full-state snapshot to the local store (P4-SYNC-02, consumer half). Import is a pure last-writer-wins merge on each row's source-event coordinates — it writes derived namespace state directly, emitting NO synthetic events and fabricating no history. That makes it idempotent and order-independent with respect to event replay: import-then- replay and replay-then-import converge to the same state, because both paths resolve every path by the same (HLC, device_id, event_id) coordinate order.

Index

Constants

View Source
const (
	SkipReasonUnknownVersion = "unknown-envelope-version"
	SkipReasonRetiredV1      = "retired-enc-v1"
	SkipReasonPlaintext      = "plaintext-anti-downgrade"
)

Skip reasons persisted through the NoteSkipped seam (P6-SYNC-02).

View Source
const (
	EventProjectAdded         = "project.added"
	EventProjectUpdated       = "project.updated"
	EventProjectDeleted       = "project.deleted"
	EventProjectRenamed       = "project.renamed"
	EventConflictCreated      = "conflict.created"
	EventConflictResolved     = "conflict.resolved"      // PROD-06
	EventDraftSnapshotCreated = "draft.snapshot.created" // DRAFT-02
	EventDeviceKeyGranted     = "device.key.granted"     // P4-SEC-07: age-wrapped WCK epoch grant
)
View Source
const (
	ConflictSamePathDifferentRemote = "same_path_different_remote"
	ConflictPendingDelete           = "pending_delete_conflict"
	ConflictRenameTargetExists      = "rename_target_exists"
	ConflictEventVerification       = "event_verification_failure"
	ConflictUntrustworthyTime       = "untrustworthy_remote_time"
	ConflictEventHashChain          = "event_hash_chain_break"
)

Conflict type identifiers, exported so the CLI resolver can branch on them (P5-SYNC-04) without duplicating string literals.

View Source
const (
	EventConflictKindVerification  = "verification"
	EventConflictKindDivergent     = "divergent"
	EventConflictKindUndecryptable = "undecryptable"
)

Machine-readable failure kinds for event_verification_failure conflicts. "verification" failures (signature/trust/content-hash) become applicable once the source device is approved, so `devices approve` replays them. "divergent" failures are data-integrity conflicts with an already-stored event of the same ID and must NEVER be auto-resolved by approval. "undecryptable" failures are enc.v2 carriers that failed AEAD authentication on every held key (corruption, forgery, or a hub-side carrier mutation — P6-SYNC-04): permanent, never applied, and never replayed by approval (a replay would fail authentication identically).

View Source
const (
	// RetentionSignatureDomain signs the retention manifest (P6-HUB-04).
	RetentionSignatureDomain = "devstrap:retention:v1"
	// AckSignatureDomain signs per-device sync ack markers (P4-SYNC-06; the
	// marker struct ships with the tombstone-GC work).
	AckSignatureDomain = "devstrap:ack:v1"
	// SnapshotSignatureDomainReserved is reserved for a future directly-signed
	// snapshot object. Do not reuse for anything else.
	SnapshotSignatureDomainReserved = "devstrap:snapshot:v1"
)

Signature domains for the snapshot-exchange plane. devstrap:snapshot:v1 is RESERVED and unused in v1 — snapshot-object integrity comes from the sha256-from-signed-manifest chain plus the AEAD — and is named here so the domain string can never be silently reused for something else.

View Source
const EventEncryptedV1 = "enc.v1"

EventEncryptedV1 is the RETIRED enc.v1 sentinel. enc.v1 bound only (ID, epoch) into the AAD, leaving DeviceID/Seq/HLC hub-mutable; the enc.v2 break (P6-SYNC-04) was taken while every existing hub was a disposable spike, so there is no decrypt path for v1 — EncryptedHub.Pull skips v1 envelopes with a loud "re-found the hub" warning. The constant survives only to recognize and reject that legacy traffic.

View Source
const EventEncryptedV2 = "enc.v2"

EventEncryptedV2 is the sentinel event Type for an encrypted namespace-map envelope (P4-SEC-02 / P4-SEC-07 / P6-SYNC-04). An enc.v2 event preserves the carrier fields (ID, DeviceID, Seq, HLC, DeviceSig) so hub ordering, dedup, and signature verification are byte-for-byte unchanged, and seals the content tuple {Type, PayloadJSON, ContentHash, PrevEventHash} under the Workspace Content Key (WCK) for the event's epoch. The hub therefore stores only ciphertext for the namespace map; ContentHash/PrevEventHash/CreatedAt are cleared and re-stamped on the receiving device after decryption. Unlike enc.v1, the AEAD additional data binds the FULL carrier tuple (ID, DeviceID, Seq, HLC, kid, epoch), so a hub-side mutation of any carrier field is an authentication failure at decrypt time instead of leaking into apply-path semantics (P6-SYNC-04).

Variables

View Source
var ErrBlobNotFound = errors.New("blob not found")

ErrBlobNotFound signals that a requested content-addressed blob is not present on the hub. It wraps os.ErrNotExist so callers can test with errors.Is(err, os.ErrNotExist).

View Source
var ErrInvalidBlobKey = errors.New("invalid blob key")

ErrInvalidBlobKey signals that a blob key (sha256 hex digest) is malformed.

View Source
var ErrMissingWorkspaceKey = errors.New("missing workspace content key for epoch")

ErrMissingWorkspaceKey signals that a pulled enc.v2 event references a key epoch this device does not hold (after all in-batch grants have been ingested). It is returned before the pull cursor advances so the next sync retries cleanly once the missing grant arrives.

View Source
var ErrPlaintextEventFromHub = errors.New("plaintext event from hub (anti-downgrade rejection)")

ErrPlaintextEventFromHub signals that a non-grant event arrived from the hub as plaintext (anti-downgrade). Once envelope encryption is wired, the hub event log must contain only enc.v2 envelopes and device.key.granted grants; any other plaintext type is a downgrade or an unencrypted regression.

View Source
var ErrRetentionConflict = errors.New("retention manifest changed concurrently")

ErrRetentionConflict signals that a conditional PutRetention lost a compare-and-swap race: the manifest changed between the caller's read and its write. The caller must re-read, re-derive floors, and retry or refuse.

View Source
var ErrRetentionNotFound = errors.New("no retention manifest on hub")

ErrRetentionNotFound signals that the hub has no retention manifest (no compaction has ever run). It wraps nothing deliberately — callers treat it as "no floor" on the pull path and "first write" on the compact path.

View Source
var ErrRetentionRollback = errors.New("retention floor rollback")

ErrRetentionRollback signals that the hub served a retention manifest whose floor for some device is LOWER than a floor this device has already verified. Floors are monotonic by protocol; a rollback is a hub attempting to hide (or accidentally losing) its own truncation history.

View Source
var ErrSnapshotRequired = errors.New("full snapshot required")

ErrSnapshotRequired signals that the requested pull cursor has fallen behind the hub's retention horizon and the caller must perform a full-state snapshot exchange (import) before continuing with incremental pulls.

View Source
var ErrSnapshotVerification = errors.New("snapshot verification failed")

ErrSnapshotVerification signals that a retention manifest or snapshot object failed trust verification (unknown/unapproved producer, bad signature, sha256 mismatch, or AEAD failure on every held candidate key). Fail-closed: nothing is imported from an unverified snapshot; the caller keeps its current state and cursors.

View Source
var ErrSweepLockHeld = errors.New("sweep lock already held")

ErrSweepLockHeld signals that a create-only sweep-lock PUT lost to an existing lock object (another sweeper, or a stale lock that must be broken first).

View Source
var ErrSweepLockNotFound = errors.New("sweep lock not found")

ErrSweepLockNotFound signals that no sweep-lock object is present on the hub (nothing to break or read). It wraps the retention/blob not-found family only loosely — callers test it explicitly.

View Source
var ErrUnknownEnvelopeVersion = errors.New("unknown encrypted envelope version")

ErrUnknownEnvelopeVersion signals that an enc.v2 event carries an envelope version this build cannot decrypt. It is a fail-closed anti-downgrade guard: an unknown version is never silently applied.

QuarantineConflictTypes are the conflict types that mean a pulled event was NOT applied (skew quarantine, hash-chain break, verification failure). While any such conflict is open, the local replica's derived state may be missing references other devices still rely on, so mark-and-sweep consumers (hub gc, P6-HUB-01) must refuse to sweep.

Functions

func AckSignaturePayload

func AckSignaturePayload(m AckMarker) []byte

AckSignaturePayload returns the canonical bytes an ack signature covers.

func ApplyEventsWithStats

func ApplyEventsWithStats(ctx context.Context, st *state.Store, events []state.Event, after Cursor) (Cursor, ApplyStats, error)

ApplyEventsWithStats is ApplyEvents plus an ApplyStats report; see ApplyEvents for the cursor semantics. after is the transport cursor the batch was pulled with (nil means "from the beginning", e.g. single-event replays that ignore the returned cursor).

func CreateConflictResolvedEvent

func CreateConflictResolvedEvent(ctx context.Context, st *state.Store, payload ConflictResolvedPayload) (state.Event, error)

CreateConflictResolvedEvent builds and inserts a conflict.resolved event (PROD-06) recording the user's resolution decision so it syncs to every device and the open-conflict count converges.

func CreateConflictResolvedEventTx

func CreateConflictResolvedEventTx(ctx context.Context, st *state.Store, tx *state.Tx, payload ConflictResolvedPayload) (state.Event, error)

func CreateProjectEvent

func CreateProjectEvent(ctx context.Context, st *state.Store, typ string, payload ProjectPayload) (state.Event, error)

func CreateProjectEventTx

func CreateProjectEventTx(ctx context.Context, st *state.Store, tx *state.Tx, typ string, payload ProjectPayload) (state.Event, error)

func DecryptEvent

func DecryptEvent(event state.Event, wck []byte) (state.Event, error)

DecryptEvent restores an enc.v2 carrier event to its original plaintext form using the WCK for the epoch embedded in the envelope. The AEAD additional data (ID, DeviceID, Seq, HLC, KIDForWCK(wck), epoch) is re-derived from the carrier and the CANDIDATE KEY IN HAND — never from the envelope's unauthenticated kid field — so a mutation of any carrier field is an authentication failure, while a relabeled kid hint stays harmless (the true key still authenticates). The restored event keeps the carrier's ID/WorkspaceID/DeviceID/Seq/HLC/DeviceSig and CreatedAt, and restores Type/PayloadJSON/ContentHash/PrevEventHash from the sealed tuple so insertEvent's content-hash re-derivation and signature verification see the exact original bytes.

func EncryptEvent

func EncryptEvent(event state.Event, wck []byte, epoch int64) (state.Event, error)

EncryptEvent seals event's content tuple under the WCK for the given epoch, returning an enc.v2 carrier event. The carrier preserves ID, WorkspaceID, DeviceID, Seq, HLC, and DeviceSig (so hub ordering and signature verification are unchanged) and clears ContentHash, PrevEventHash, and CreatedAt (re-stamped on the receiver after DecryptEvent). The AEAD additional data binds the full carrier tuple — event.ID, DeviceID, Seq, HLC, the sealing key's kid, and the epoch — so mutating ANY of those fields in transit or at rest on the hub is detected as an authentication failure (P6-SYNC-04). The envelope also names the key by kid (KIDForWCK(wck)) so receivers select the exact key when several coexist at the same epoch.

func EnvelopeVersion

func EnvelopeVersion(event state.Event) (int, bool)

EnvelopeVersion extracts the claimed envelope version from an enc.v2-typed event's payload without validating it (P6-SYNC-02): the unknown-version classifier needs the claimed version to log/record it, distinguishing "a newer client wrote v3" (defer — recoverable by upgrading this binary) from malformed junk (quarantine). ok is false when the payload is not parseable envelope JSON at all.

func ImportSnapshot

func ImportSnapshot(ctx context.Context, st *state.Store, snap Snapshot, snapshotSHA256, hubID string) error

ImportSnapshot applies a verified snapshot to the local store as a pure LWW merge in one transaction, then advances the per-device pull cursors to the floor and caches the verified floor for the recovery rollback guard.

The caller (recoverFromSnapshot) has already verified the retention manifest signature against a locally pinned approved device, matched the fetched object's sha256 against the manifest, and unsealed the object under a held WCK, so ImportSnapshot trusts the plaintext. snapshotSHA256 is the content address of that sealed object, recorded with each imported anchor for audit; hubID keys the pull cursors advanced after the merge commits.

func KIDForWCK

func KIDForWCK(wck []byte) string

KIDForWCK derives the key identity for a WCK: hex(sha256(wck)), the full digest as 64 lowercase hex characters (P6-SEC-02). The full digest (not a short prefix) makes crafting a second key with a colliding kid a sha256 collision, so a forged grant can never alias — let alone displace — an existing key's custody slot. The kid names a specific key so two keys minted independently at the same epoch (a joiner's legacy self-mint and the founder's fleet key) coexist in the keyring instead of clobbering, and a grant whose carried kid does not match its unwrapped bytes is rejected. The kid is derived from the secret key but reveals only a one-way hash — it identifies, it does not leak.

func LoadRetentionFloorCache

func LoadRetentionFloorCache(ctx context.Context, st *state.Store, hubID string) (map[string]int64, error)

LoadRetentionFloorCache reads the cached highest verified per-device floor for a hub, or an empty map when none is cached. A garbled cache is treated as empty (rollback detection is a hint, never authoritative).

func MarshalSweepLock

func MarshalSweepLock(l SweepLock) ([]byte, error)

MarshalSweepLock serializes a sweep-lock body.

func NewDeviceKeyGrantEvent

func NewDeviceKeyGrantEvent(typ, payloadJSON string) state.Event

NewDeviceKeyGrantEvent builds an unsigned device.key.granted event from a pre-marshaled DeviceKeyGrant payload (P4-SEC-07). The store stamps HLC, seq, device id, and the device signature on InsertLocalEvent. Grant events are NOT envelope-encrypted (the payload is itself age-wrapped), so the EncryptedHub decorator passes them through unchanged on both Push and Pull.

func NewDraftSnapshotEvent

func NewDraftSnapshotEvent(typ, payloadJSON string) state.Event

NewDraftSnapshotEvent builds an unsigned draft.snapshot.created event from a pre-marshaled payload (DRAFT-02). The store stamps HLC, seq, device id, and the device signature on InsertLocalEvent.

func NewProjectEvent

func NewProjectEvent(deviceID, typ string, hlc int64, payload ProjectPayload) (state.Event, error)

func NewWCK

func NewWCK() ([]byte, error)

NewWCK generates a fresh 32-byte Workspace Content Key for a new epoch.

func ParseEncryptedEnvelope

func ParseEncryptedEnvelope(event state.Event) (encryptedEnvelope, error)

ParseEncryptedEnvelope decodes an enc.v2 event's envelope without decrypting, so the decorator can select the WCK by epoch before calling DecryptEvent. It validates the Type sentinel and the envelope version.

func ParsePendingDeleteConflictPath

func ParsePendingDeleteConflictPath(detailsJSON string) (string, error)

ParsePendingDeleteConflictPath decodes a pending_delete_conflict's path (P5-SYNC-04).

func ParseRetentionFloors

func ParseRetentionFloors(raw []byte) (map[string]int64, error)

ParseRetentionFloors extracts the per-device floors from raw manifest bytes with the same structural fail-closed validation as ParseRetentionManifest: any parse or shape failure is an error so the pull path treats a garbled marker as a hard error rather than "no floor".

func ReplayUndecryptableConflicts

func ReplayUndecryptableConflicts(ctx context.Context, st *state.Store, h EncryptedHub) (int, error)

ReplayUndecryptableConflicts re-attempts every open "undecryptable" quarantine conflict with the keys held now (P6-SYNC-04 review fix, gpt-5.5 Major). Without this, a hostile hub could strip or relabel the untrusted envelope kid on an event whose grant had not arrived yet, steering the defer-vs-quarantine classification into permanent quarantine — and since the cursor advances past quarantined events, a later legitimate grant could never recover it. The quarantine preserves the full carrier, so each sync cycle replays it: once the grant lands, the carrier decrypts, the conflict auto-resolves, and the restored event applies through the normal verified path. Genuinely corrupt carriers keep failing and their conflicts stay open (visible, gc-blocking, deduped — no growth, no wedge).

The replay is DELIBERATELY unconditional — it never skips a conflict as "hopeless" based on what the envelope kid named at quarantine time. Any such classification reads the same attacker-controlled field that caused the misroute: a hub that relabels a not-yet-granted event's kid to a HELD kid would get it marked permanently non-replayable and re-open the exact loss this fix closes (post-#44 dual-review analysis). A hopeless carrier just fails decryption again — cheap, bounded by open conflicts.

The conflict is resolved AFTER a successful apply, so a transient failure (DB error) between decrypt and apply leaves the conflict open and the next cycle retries — the resolve-then-apply window would have been a narrow silent-loss path (post-#44 verify, gpt-5.5). A restored event that fails signature verification still records a FRESH "verification" conflict while the "undecryptable" row is open, because the conflict dedup keys on (event ID, kind). A restored event whose HLC is still beyond the trusted receive skew is left for a later cycle (applying it would land in the transient skew quarantine with no carrier left to retry).

func RetentionFloorMetaKey

func RetentionFloorMetaKey(hubID string) string

RetentionFloorMetaKey is the local_meta key caching the highest verified per-device retention floor for a hub (P4-SYNC-02 rollback guard).

func RetentionSignaturePayload

func RetentionSignaturePayload(m RetentionManifest) []byte

RetentionSignaturePayload returns the canonical bytes a manifest signature covers.

func SealSnapshot

func SealSnapshot(snap Snapshot, wck []byte, epoch int64) (obj []byte, sha256Hex string, err error)

SealSnapshot seals a snapshot document under the WCK for the given epoch and returns the content-addressed object: the envelope bytes plus their sha256 hex (the object key and the manifest's snapshot reference). The snapshot's own V/Epoch/KID are stamped here so document and envelope can never disagree.

func SignAckMarker

func SignAckMarker(m *AckMarker, privateSigningKey string) error

SignAckMarker stamps V and Sig on the marker using the producing device's private signing key (domain devstrap:ack:v1).

func SignRetentionManifest

func SignRetentionManifest(m *RetentionManifest, privateSigningKey string) error

SignRetentionManifest stamps V and Sig on the manifest using the producing device's private signing key (domain devstrap:retention:v1).

func VerifyAckMarker

func VerifyAckMarker(m AckMarker, publicSigningKey string) error

VerifyAckMarker checks an ack's signature against the claimed producer's public signing key. The caller is responsible for the TRUST decision (the key must belong to a locally approved device and the DeviceID must match the object key) — this only proves the marker bytes were signed by that key.

func VerifyRetentionManifest

func VerifyRetentionManifest(m RetentionManifest, publicSigningKey string) error

VerifyRetentionManifest checks a manifest's signature against the claimed producer's public signing key. The caller is responsible for the TRUST decision (the key must belong to a locally pinned, approved device) — this only proves the manifest bytes were signed by that key.

Types

type AckMarker

type AckMarker struct {
	// Cursor is the per-origin-device transport cursor this device has pulled
	// AND consumed (device_id -> highest contiguous Seq). Pull rows only; the
	// push watermark is carried separately by PushedThroughSeq.
	Cursor map[string]int64 `json:"cursor"`
	// DeviceID is the acking device; it must equal the object key's device id.
	DeviceID string `json:"device_id"`
	// HLCWatermark is this device's current HLC clock (state.CurrentHLC): after a
	// clean cycle it is at or above every event HLC the device has applied, so
	// everything at or below it from other devices has been consumed here. It is
	// the value a compactor mins over to floor tombstone GC.
	HLCWatermark int64 `json:"hlc_watermark"`
	// ProducedAt is the HLC at which this marker was produced. In v1 it equals
	// HLCWatermark (both are the current non-minting HLC read); it is kept
	// distinct so a future minted-timestamp variant does not change the wire tag.
	ProducedAt int64 `json:"produced_at_hlc"`
	// PushedThroughSeq is this device's push watermark: the highest local Seq it
	// has published to the hub. A clean-cycle ack has this equal to local max
	// Seq, which is what makes the HLCWatermark a sound tombstone clock.
	PushedThroughSeq int64 `json:"pushed_through_seq"`
	// Sig is the Ed25519 signature over the canonical payload (all fields except
	// Sig), domain AckSignatureDomain.
	Sig string `json:"sig"`
	// V is the marker version.
	V int `json:"v"`
	// WorkspaceID binds the marker to its workspace.
	WorkspaceID string `json:"workspace_id"`
}

AckMarker is one device's signed sync acknowledgement (P4-SYNC-06). Fields are declared in alphabetical json-tag order so the marshaled document matches the canonical signature payload field-for-field (Go marshals structs in declaration order and maps with sorted keys), mirroring RetentionManifest.

func ParseAckMarker

func ParseAckMarker(raw []byte) (AckMarker, error)

ParseAckMarker decodes raw ack bytes without verifying the signature.

type ApplyStats

type ApplyStats struct {
	// Quarantined counts events recorded as conflicts instead of applied
	// (skew quarantine, hash-chain break, verification/divergence failure).
	Quarantined int
	// CursorHeld is true when at least one device's safe cursor stopped short
	// of that device's highest batch Seq — either a transiently-held event
	// (skew/hash-chain, re-delivered next pull) or a Seq gap on the hub.
	CursorHeld bool
}

ApplyStats reports what a single ApplyEvents batch did NOT apply. A non-zero Quarantined or a true CursorHeld means the local replica's derived state may be missing references other devices still rely on, so mark-and-sweep consumers (hub gc, P6-HUB-01) must refuse to sweep this cycle.

type BlobInfo

type BlobInfo struct {
	Key          string
	LastModified time.Time
}

BlobInfo describes one blob on the hub: its sha256 hex key and the hub-reported creation/modification time (zero when the backend cannot provide one). P6-HUB-01: GC uses LastModified for an age grace window.

type ChainAnchor

type ChainAnchor struct {
	DeviceID    string `json:"device_id"`
	Seq         int64  `json:"seq"`
	ContentHash string `json:"content_hash"`
	HLC         int64  `json:"hlc"`
}

ChainAnchor is one origin device's hash-chain anchor: the content hash of the LAST event covered by the snapshot for that device (seq = floor-1). A snapshot-bootstrapped device has no event rows below the floor, so the prev-hash verification of the first post-floor event needs this anchor as its fallback predecessor.

type ConflictRecord

type ConflictRecord struct {
	NamespaceID string
	Type        string
	Details     string // details_json
}

ConflictRecord is a conflict Decide wants recorded, as plain data. The impure step maps it to tx.InsertConflict; the pure fold ignores conflicts (they do not change the namespace projection).

type ConflictResolvedPayload

type ConflictResolvedPayload struct {
	ConflictID  string `json:"conflict_id"`
	NamespaceID string `json:"namespace_id,omitempty"`
	Type        string `json:"type"`
	DetailsJSON string `json:"details_json"`
	Action      string `json:"action"` // keep-local | keep-remote | keep-both
}

ConflictResolvedPayload carries a conflict.resolved event (PROD-06): the user's local resolution decision is audited and synced so every device sees the same outcome and the open-conflict count converges. ConflictID and NamespaceID are the origin device's local row ids (per-device, NOT stable across devices), retained only for display/audit; the apply handler matches on the stable (type, details_json) fingerprint instead (P5-SYNC-02).

type Cursor

type Cursor map[string]int64

Cursor is the sync transport cursor (P5-SYNC-01): for each origin device on the hub, the highest per-device sequence number such that every event with Seq <= it has been pulled AND consumed (applied, deduped, or permanently quarantined). It is deliberately decoupled from the HLC, which remains the apply-ordering key only: every device's own event stream is gapless in Seq (UNIQUE events(device_id, seq), assigned in the same transaction as the HLC), so a per-device Seq cursor can never skip an event no matter how late it lands on the hub — the "offline device forgot to push, syncs late" scenario an HLC watermark permanently stranded. A device absent from the map has cursor 0 (pull from the beginning).

func ApplyEvents

func ApplyEvents(ctx context.Context, st *state.Store, events []state.Event) (Cursor, error)

ApplyEvents sorts and applies a batch of remote events. It returns the safe per-device transport cursor (P5-SYNC-01): for each origin device, the highest Seq such that every slot from after[dev]+1 up to it was CONSUMED by this batch. The cursor must never advance past an event that was transiently skipped within this batch, otherwise that event is never re-delivered and the gap is permanent.

Per origin device, the safe cursor is the end of the contiguous consumed run starting at after[dev]+1. An event is CONSUMED when it was applied, deduped (already inserted — deduplication is consumption, so a device re-pulling its own events advances past them instead of re-fetching forever), or permanently quarantined (implausible HLC, verification/divergence failure, undecryptable enc.v2 carrier): re-delivery of those would fail identically forever. Only TRANSIENTLY-skipped events HOLD a device's cursor: skew-ahead quarantine (valid once local time catches up) and hash-chain breaks (a re-delivery may carry the correct prev_event_hash). The hold is scoped to the offending origin device — other devices' cursors keep advancing (per-device fault isolation the old global HLC low-water mark never had). A Seq gap in the batch (an object missing from the hub) also stops the run, loudly: advancing over it would permanently strand the missing event.

Events with Seq <= 0 (pre-sequence legacy) apply or quarantine as normal but never touch the cursor; they are re-delivered each pull and dedup by ID.

func (Cursor) After

func (c Cursor) After(deviceID string) int64

After returns the cursor position for one origin device (0 when unknown).

type Decision

type Decision struct {
	Mutations []Mutation
	Conflicts []ConflictRecord
}

Decision is the pure result of reconciling one event against a Projection: the intended namespace-map mutations plus the conflicts to record. No database access, no I/O, no *state.Tx.

func Decide

func Decide(proj Projection, event state.Event) (Decision, error)

Decide reconciles one namespace-convergence event (project.added / project.updated / project.deleted) against the projection and returns the intended effects. It is PURE. Events outside the convergence core return an empty Decision (they are dispatched inline in applyEventTx, never here).

type DeviceKeyGrant

type DeviceKeyGrant struct {
	Epoch      int64  `json:"epoch"`
	KID        string `json:"kid,omitempty"` // KIDForWCK of the wrapped key (P6-SEC-02); "" on legacy grants
	Recipient  string `json:"recipient"`     // age X25519 recipient the WCK is wrapped to
	WrappedKey string `json:"wrapped_key"`   // base64(age.Encrypt(wck, recipient))
}

DeviceKeyGrant carries a device.key.granted event (P4-SEC-07): a Workspace Content Key for an epoch, age-wrapped to a single approved device's X25519 recipient. Grant events ride the hub event log as PLAINTEXT (the decorator passes them through unencrypted) because their payload is already asymmetrically wrapped — the hub cannot decrypt the WCK without the recipient's private key. A newly-approved device ingests grants for every held epoch on its first pull so it can decrypt the entire namespace-map history.

type DraftSnapshotPayload

type DraftSnapshotPayload struct {
	Path      string `json:"path"`
	BlobRef   string `json:"blob_ref"`
	ByteSize  int64  `json:"byte_size"`
	FileCount int64  `json:"file_count"`
}

DraftSnapshotPayload carries a draft.snapshot.created event's content-addressed blob reference and limits (DRAFT-02).

type EncryptedHub

type EncryptedHub struct {
	Hub     Hub
	Keyring WorkspaceKeyring
	// Verify checks a grant carrier event's signature/trust before its WCK is
	// ingested (P6-SEC-01). nil disables the check (used by unit tests that
	// exercise decryption only). hubFromOptions wires it to
	// (*state.Store).VerifyRemoteEvent so the trust regime is identical to the
	// apply path.
	Verify func(ctx context.Context, ev state.Event) error
	// Stats, when non-nil, is populated by Pull with observability about the
	// raw batch (P6-SEC-02). RawSeen is the number of objects the backend
	// returned before any decrypt/skip/truncate — the founder/join gate uses
	// it to distinguish a genuinely empty hub (found here) from a populated
	// hub whose events this device cannot yet decrypt (a joiner awaiting its
	// grant, which must NOT self-found). It is also the seam later cursor and
	// GC work (P6-HUB-01/SEC-03/SYNC-02) will read.
	Stats *PullStats
	// MissingKeyWait records that an event sealed under (epoch, kid) was found
	// with no held key, and returns the STABLE first-seen time of that missing
	// key — the start of its grace window (P6-SEC-03). hubFromOptions wires it
	// to (*state.Store).NoteMissingKeyGrant. nil disables grace expiry
	// entirely: a missing key truncates forever (the pre-P6-SEC-03 behavior,
	// kept for unit tests that exercise the truncate contract in isolation).
	MissingKeyWait func(ctx context.Context, epoch int64, kid string) (time.Time, error)
	// GraceWindow bounds how long a missing (epoch, kid) may keep truncating
	// the pull before its events are handed to the undecryptable quarantine so
	// the cursor can advance (P6-SEC-03). Within the window the grant is
	// presumed in flight (truncate = retry next sync); past it the wedge is
	// treated as permanent-until-regrant: the still-encrypted carrier is
	// forwarded, ApplyEvents quarantines it, and a later grant recovers it via
	// ReplayUndecryptableConflicts. Zero means expire immediately (quarantine
	// on first sight); only meaningful when MissingKeyWait is non-nil. The
	// same window bounds the unknown-envelope-version defer (P6-SYNC-02),
	// clocked by NoteSkipped's stable first-seen.
	GraceWindow time.Duration
	// NoteSkipped durably records an event Pull drops from the batch
	// (P6-SYNC-02: unknown envelope version, retired enc.v1, anti-downgrade
	// plaintext) and returns the STABLE first-seen time of the (event,
	// reason) — the grace clock for the recoverable unknown-version class.
	// hubFromOptions wires it to (*state.Store).NoteSkippedEvent; the rows
	// surface in status/doctor, gate hub gc, and clear when the event
	// finally applies. nil keeps skips log-only (unit tests).
	NoteSkipped func(ctx context.Context, ev state.Event, reason string) (time.Time, error)
}

EncryptedHub is a Hub decorator that envelope-encrypts the namespace-map event log at the hub boundary (P4-SEC-02 / P4-SEC-07). It wraps a backend Hub (FileHub or R2Hub) so the backend stores only ciphertext for the event log; the local SQLite keeps plaintext PayloadJSON, and the existing Ed25519 signature and content/prev-hash verification run unchanged on the decrypted events (the carrier preserves ID/DeviceID/Seq/HLC/DeviceSig, and decryption restores Type/PayloadJSON/ContentHash/PrevEventHash before ApplyEvents). Blob operations pass through unchanged — blobs are already age-encrypted by the bundle layer, so the blob plane is already ciphertext.

Grant events (device.key.granted) ride the hub as PLAINTEXT on both Push and Pull because their payload is itself age-wrapped (the hub cannot decrypt the WCK without the recipient's private key). On Pull, grants are ingested into the keyring in HLC order before the rest of the batch is decrypted, so a newly-approved device obtains its WCKs before decrypting history. If Verify is set, Pull verifies each grant's carrier event before ingesting its WCK so an untrusted hub cannot inject attacker-known workspace keys.

func (EncryptedHub) CompactEventsBelow

func (h EncryptedHub) CompactEventsBelow(ctx context.Context, floors Cursor) (int, error)

func (EncryptedHub) DeleteAck

func (h EncryptedHub) DeleteAck(ctx context.Context, deviceID string) error

func (EncryptedHub) DeleteBlob

func (h EncryptedHub) DeleteBlob(ctx context.Context, sha256Hex string) error

func (EncryptedHub) DeleteDeviceStream

func (h EncryptedHub) DeleteDeviceStream(ctx context.Context, deviceID string) (int, error)

func (EncryptedHub) DeleteSnapshotObject

func (h EncryptedHub) DeleteSnapshotObject(ctx context.Context, sha256Hex string) error

func (EncryptedHub) DeleteSweepLock

func (h EncryptedHub) DeleteSweepLock(ctx context.Context) error

func (EncryptedHub) GetBlob

func (h EncryptedHub) GetBlob(ctx context.Context, sha256Hex string) (io.ReadCloser, error)

func (EncryptedHub) GetRetention

func (h EncryptedHub) GetRetention(ctx context.Context) ([]byte, string, error)

func (EncryptedHub) GetSnapshotObject

func (h EncryptedHub) GetSnapshotObject(ctx context.Context, sha256Hex string) ([]byte, error)

func (EncryptedHub) GetSweepLock

func (h EncryptedHub) GetSweepLock(ctx context.Context) ([]byte, time.Time, error)

func (EncryptedHub) ListAcks

func (h EncryptedHub) ListAcks(ctx context.Context) (map[string][]byte, error)

func (EncryptedHub) ListBlobs

func (h EncryptedHub) ListBlobs(ctx context.Context) ([]BlobInfo, error)

func (EncryptedHub) ListSnapshotObjects

func (h EncryptedHub) ListSnapshotObjects(ctx context.Context) ([]BlobInfo, error)

func (EncryptedHub) MigrateLegacyEvents

func (h EncryptedHub) MigrateLegacyEvents(ctx context.Context, dryRun bool) (int, int, error)

func (EncryptedHub) Pull

func (h EncryptedHub) Pull(ctx context.Context, after Cursor) ([]state.Event, error)

Pull fetches events from the backend, primes the keyring, verifies grant carrier events when a verifier is configured, ingests verified in-batch grants in HLC order, then decrypts enc.v2 envelopes back to plaintext.

The hub is untrusted (zero-knowledge), so a single non-conforming object must never be able to wedge sync. Pull therefore degrades instead of aborting the whole batch:

  • Missing (epoch, kid) key: the grant for this event's key has not propagated yet (a missing epoch, or a kid at a held epoch that this device does not hold — the P6-SEC-02 collision case). Pull DEFERS the origin device here (P5-SYNC-01): this and every later event from the same origin device in the batch are dropped, so that device's per-device Seq cursor holds at the last consumed slot and the next sync retries from there once the grant arrives — while OTHER devices' events keep flowing (the old whole-batch truncate held the entire fleet behind one ungranted stream). Deferring (not skipping) is required so a legitimately-decryptable-later event is never permanently stranded by the cursor jumping over it; dropping the deferred device's TAIL (not just the one event) avoids churning hash-chain-break conflicts on its successors, which could not chain-apply anyway.
  • Held-epoch AEAD failure (corruption, forgery, or a hub-side carrier mutation — P6-SYNC-04): the event can never be decrypted by this device, but silently skipping it would be silent permanent loss with no operator signal. Pull FORWARDS the still-encrypted carrier so ApplyEvents records a permanent undecryptable quarantine conflict (surfaced by `conflicts list`, blocking `hub gc`) and advances the cursor past it — visible refusal without a wedge.
  • A malformed/unknown envelope, retired enc.v1 traffic, or a non-grant plaintext event (a downgrade attempt or a pre-envelope legacy event): Pull SKIPS it with a loud warning and continues (P6-SYNC-02 tracks promoting these skip classes to first-class signals). The event is never applied — no unauthenticated data enters the log — but one bad object cannot brick the device.

func (EncryptedHub) Push

func (h EncryptedHub) Push(ctx context.Context, events []state.Event) error

Push envelope-encrypts every non-grant event under the current epoch's WCK and forwards the carrier events to the backend. Grant events pass through unchanged. The carrier preserves ID/DeviceID/Seq/HLC/DeviceSig so hub ordering, dedup, and signature verification are byte-for-byte unchanged.

func (EncryptedHub) PutAck

func (h EncryptedHub) PutAck(ctx context.Context, deviceID string, raw []byte) error

func (EncryptedHub) PutBlob

func (h EncryptedHub) PutBlob(ctx context.Context, sha256Hex string, r io.Reader) error

func (EncryptedHub) PutRetention

func (h EncryptedHub) PutRetention(ctx context.Context, raw []byte, ifMatchETag string) error

func (EncryptedHub) PutSnapshotObject

func (h EncryptedHub) PutSnapshotObject(ctx context.Context, sha256Hex string, body []byte) error

func (EncryptedHub) PutSweepLock

func (h EncryptedHub) PutSweepLock(ctx context.Context, raw []byte) error

func (EncryptedHub) StatBlob

func (h EncryptedHub) StatBlob(ctx context.Context, sha256Hex string) (BlobInfo, error)

func (EncryptedHub) TryDecrypt

func (h EncryptedHub) TryDecrypt(ctx context.Context, event state.Event) (state.Event, error)

TryDecrypt attempts to restore a single enc.v2 carrier with the keys held NOW — the exact-kid candidate first, then every held key at the epoch (the same candidate policy as Pull). It primes the keyring but never truncates, skips, or touches Stats: it exists for the undecryptable-conflict replay path (P6-SYNC-04 review fix), which re-attempts quarantined carriers after later grants arrive.

type FileHub

type FileHub struct {
	Path string
	// RetentionSeqs is the hub's per-device retention horizon (P5-HUB-03,
	// re-based on the Seq transport cursor): for each origin device, the
	// minimum Seq still retained. A Pull whose cursor would leave a gap below
	// a device's floor (after[dev]+1 < RetentionSeqs[dev]) returns
	// ErrSnapshotRequired. Empty means "no compaction yet" (everything
	// retained). Test-only plumbing until snapshot exchange lands
	// (P4-SYNC-02/P4-HUB-11).
	RetentionSeqs map[string]int64
}

FileHub is a file-backed test Hub (HUB-01). The event log is a single JSON array file; blobs are stored in a sibling directory keyed by sha256 hex. It is retained ONLY for tests and the --hub-file spike; the production backend is the R2/S3 implementation (HUB-02).

func (FileHub) CompactEventsBelow

func (h FileHub) CompactEventsBelow(ctx context.Context, floors Cursor) (int, error)

CompactEventsBelow deletes events strictly below each device's floor (Seq > 0 && Seq < floors[dev]). Pre-sequence events (Seq <= 0) are never compacted — they cannot be covered by a Seq floor. The caller must have durably published the superseding snapshot + manifest first.

func (FileHub) DeleteAck

func (h FileHub) DeleteAck(_ context.Context, deviceID string) error

DeleteAck removes a device's sync-ack marker (idempotent).

func (FileHub) DeleteBlob

func (h FileHub) DeleteBlob(_ context.Context, sha256Hex string) error

DeleteBlob removes a content-addressed blob (SEC-01/HUB-12). A missing blob is not an error (idempotent delete), so revoke/GC can call it unconditionally for superseded ciphertext.

func (FileHub) DeleteDeviceStream

func (h FileHub) DeleteDeviceStream(_ context.Context, deviceID string) (int, error)

DeleteDeviceStream removes an entire origin device's events from the log (idempotent). FileHub stores every event in one array, so this filters the device's rows out; it returns the count removed.

func (FileHub) DeleteSnapshotObject

func (h FileHub) DeleteSnapshotObject(_ context.Context, sha256Hex string) error

DeleteSnapshotObject removes a superseded snapshot object (idempotent).

func (FileHub) DeleteSweepLock

func (h FileHub) DeleteSweepLock(_ context.Context) error

DeleteSweepLock removes the sweep-lock object (idempotent).

func (FileHub) GetBlob

func (h FileHub) GetBlob(_ context.Context, sha256Hex string) (io.ReadCloser, error)

GetBlob returns an encrypted blob as a stream. The caller must close the reader. A missing blob returns an error wrapping os.ErrNotExist.

func (FileHub) GetRetention

func (h FileHub) GetRetention(_ context.Context) ([]byte, string, error)

GetRetention returns the raw retention-manifest bytes plus an etag (the sha256 hex of the bytes — FileHub has no HTTP etags).

func (FileHub) GetSnapshotObject

func (h FileHub) GetSnapshotObject(_ context.Context, sha256Hex string) ([]byte, error)

GetSnapshotObject returns a sealed snapshot object. Missing objects wrap ErrBlobNotFound.

func (FileHub) GetSweepLock

func (h FileHub) GetSweepLock(_ context.Context) ([]byte, time.Time, error)

GetSweepLock reads the advisory sweep-lock object and its file mtime (P4-HUB-12). A missing lock is ErrSweepLockNotFound.

func (FileHub) HasEvents

func (h FileHub) HasEvents(ctx context.Context) (bool, error)

HasEvents reports whether any event has ever been recorded on this hub (P4-SEC-07 doctor mismatch check).

func (FileHub) ListAcks

func (h FileHub) ListAcks(_ context.Context) (map[string][]byte, error)

ListAcks returns every device's sync-ack marker keyed by device id.

func (FileHub) ListBlobs

func (h FileHub) ListBlobs(_ context.Context) ([]BlobInfo, error)

ListBlobs returns metadata for every blob in the hub's blob directory (P5-HUB-02).

func (FileHub) ListSnapshotObjects

func (h FileHub) ListSnapshotObjects(_ context.Context) ([]BlobInfo, error)

ListSnapshotObjects returns metadata for every snapshot object on the hub (compaction prunes superseded ones by age, keeping the newest N).

func (FileHub) MigrateLegacyEvents

func (h FileHub) MigrateLegacyEvents(_ context.Context, _ bool) (int, int, error)

MigrateLegacyEvents is a no-op for FileHub: the file-backed test hub stores every event in one JSON array and never used the retired HLC-keyed R2 layout, so there is nothing to re-key (P4-HUB-12). The dryRun flag is irrelevant.

func (FileHub) Pull

func (h FileHub) Pull(ctx context.Context, after Cursor) ([]state.Event, error)

func (FileHub) Push

func (h FileHub) Push(ctx context.Context, events []state.Event) error

func (FileHub) PutAck

func (h FileHub) PutAck(_ context.Context, deviceID string, raw []byte) error

PutAck writes a device's signed sync-ack marker (P4-SYNC-06). Last-writer-wins: each device writes only its own ack, so an unconditional overwrite is correct.

func (FileHub) PutBlob

func (h FileHub) PutBlob(ctx context.Context, sha256Hex string, r io.Reader) error

PutBlob stores an encrypted blob keyed by its sha256 hex digest. The blob is content-addressed: writing the same digest twice is a no-op.

func (FileHub) PutRetention

func (h FileHub) PutRetention(_ context.Context, raw []byte, ifMatchETag string) error

PutRetention writes the retention manifest with compare-and-swap semantics: ifMatchETag "" requires that no manifest exists yet; otherwise the current bytes must still hash to ifMatchETag. A lost race is ErrRetentionConflict. The read-check-write section is serialized by an O_EXCL lock file so two concurrent writers — in one process or across `--hub-file` processes — can never both pass the etag check and silently overwrite each other (post-#65 Codex review, P2).

func (FileHub) PutSnapshotObject

func (h FileHub) PutSnapshotObject(_ context.Context, sha256Hex string, body []byte) error

PutSnapshotObject stores a sealed snapshot object keyed by the sha256 of its bytes. Content-addressed: an existing object is a no-op.

func (FileHub) PutSweepLock

func (h FileHub) PutSweepLock(_ context.Context, raw []byte) error

PutSweepLock writes the sweep-lock object create-only (O_EXCL): an existing lock is ErrSweepLockHeld, mirroring R2's If-None-Match:* conditional put.

func (FileHub) StatBlob

func (h FileHub) StatBlob(_ context.Context, sha256Hex string) (BlobInfo, error)

StatBlob returns one blob's current metadata (P4-HUB-12). A missing blob returns an error wrapping os.ErrNotExist so gc can treat it as already gone.

type HLC

type HLC struct {
	Last    int64
	Now     func() time.Time
	MaxSkew time.Duration
	// contains filtered or unexported fields
}

func (*HLC) Receive

func (h *HLC) Receive(remote int64) (int64, error)

func (*HLC) Send

func (h *HLC) Send() int64

type Hub

type Hub interface {
	Push(ctx context.Context, events []state.Event) error
	Pull(ctx context.Context, after Cursor) ([]state.Event, error)
	PutBlob(ctx context.Context, sha256Hex string, r io.Reader) error
	GetBlob(ctx context.Context, sha256Hex string) (io.ReadCloser, error)
	DeleteBlob(ctx context.Context, sha256Hex string) error
	// ListBlobs returns metadata for every blob currently on the hub
	// (P5-HUB-02). It is the enumeration primitive for mark-and-sweep hub GC:
	// list everything, delete what no current binding/snapshot references.
	ListBlobs(ctx context.Context) ([]BlobInfo, error)
	// StatBlob returns one blob's current metadata (P4-HUB-12). It is the
	// pre-delete revalidation primitive for hub GC: the LastModified in a
	// ListBlobs snapshot goes stale the instant a concurrent sync dedup-re-puts
	// (and refreshes) the object, so gc re-stats each candidate immediately
	// before deleting it and keeps a blob whose fresh mtime shows it was just
	// re-referenced. A missing blob returns an error wrapping os.ErrNotExist.
	StatBlob(ctx context.Context, sha256Hex string) (BlobInfo, error)
	GetRetention(ctx context.Context) (raw []byte, etag string, err error)
	PutRetention(ctx context.Context, raw []byte, ifMatchETag string) error
	PutSnapshotObject(ctx context.Context, sha256Hex string, body []byte) error
	GetSnapshotObject(ctx context.Context, sha256Hex string) ([]byte, error)
	ListSnapshotObjects(ctx context.Context) ([]BlobInfo, error)
	DeleteSnapshotObject(ctx context.Context, sha256Hex string) error
	CompactEventsBelow(ctx context.Context, floors Cursor) (deleted int, err error)
	PutAck(ctx context.Context, deviceID string, raw []byte) error
	ListAcks(ctx context.Context) (map[string][]byte, error)
	DeleteAck(ctx context.Context, deviceID string) error
	DeleteDeviceStream(ctx context.Context, deviceID string) (deleted int, err error)
	MigrateLegacyEvents(ctx context.Context, dryRun bool) (migrated, kept int, err error)
	GetSweepLock(ctx context.Context) (raw []byte, lastModified time.Time, err error)
	PutSweepLock(ctx context.Context, raw []byte) error
	DeleteSweepLock(ctx context.Context) error
}

Hub is the two-plane zero-knowledge sync backend (HUB-01): (a) the signed HLC-ordered namespace-map event log and (b) the content-addressed encrypted blob store. The hub sees only ciphertext plus a signed carrier map. When wrapped by EncryptedHub (P4-SEC-02/SEC-07), event-log payloads are envelope-encrypted (XChaCha20-Poly1305 under a per-epoch Workspace Content Key) so the hub never stores plaintext Type/PayloadJSON/ContentHash; the carrier (ID/DeviceID/Seq/HLC/DeviceSig) remains plaintext so hub ordering, dedup, and Ed25519 signature verification are unchanged. Implementations must be safe for concurrent use.

Event plane:

  • Push appends locally-originated events. Duplicate event IDs are ignored (idempotent), so re-pushing already-delivered events is safe.
  • Pull returns, for EVERY origin device present on the hub (device discovery is the hub's job, so a brand-new device's stream is picked up with no cursor entry), every event with Seq > after[DeviceID], in deterministic apply order (HLC, device_id, id). The per-device Seq boundary is exact — Seq is unique per device — so the old inclusive-HLC boundary re-delivery (HUB-13) is retired; no boundary overlap exists. Events with Seq <= 0 (pre-sequence legacy objects) are always returned: they cannot be cursored and rely on event-ID dedup. If any device's cursor has fallen behind the hub's retention horizon (after[dev]+1 < minRetainedSeq[dev]), Pull returns ErrSnapshotRequired so the caller performs a full-state snapshot exchange before resuming incremental pulls.

Blob plane:

  • PutBlob stores a content-addressed encrypted blob keyed by its sha256 hex digest. Writes are idempotent: a blob already present is a no-op EXCEPT that the dedup hit refreshes the object's LastModified (an unconditional same-bytes re-put on R2, an mtime bump on FileHub). Content addressing makes the re-write byte-safe, and the refresh is load-bearing: `hub gc` keeps blobs younger than its grace window, so a blob re-referenced by a late recovery sync must look freshly written or the sweep would delete a live blob (P4-HUB-12 / the P6-HUB-01 grace-window residual). Its partner on the read side is StatBlob: gc re-stats each candidate immediately before deleting, so a refresh that lands AFTER gc's ListBlobs snapshot is still honored and the just-re-referenced blob survives.
  • GetBlob returns the blob as a stream the caller must close. A missing blob returns an error wrapping os.ErrNotExist.
  • DeleteBlob removes a content-addressed blob. It is the reclamation primitive that makes blob/event GC possible (HUB-12) and lets device revoke delete superseded ciphertext so a revoked key can no longer fetch it (SEC-01). A missing blob is not an error (idempotent delete).

The object-key contract is immutable: events and blobs are addressed by content-derived, collision-resistant identifiers and are never overwritten in place (HUB-06). The single exception is the retention manifest, which is a mutable head object guarded by compare-and-swap (PutRetention).

Retention/snapshot plane (P4-SYNC-02 / P4-HUB-11 / P6-HUB-04):

  • GetRetention returns the raw signed retention-manifest bytes plus an opaque etag for CAS. Absent manifest (no compaction ever) returns ErrRetentionNotFound.
  • PutRetention writes the manifest conditionally: ifMatchETag "" means create-only (the manifest must not exist yet); otherwise the write succeeds only if the current object still matches the etag. A lost race returns ErrRetentionConflict. The hub cannot verify the signature — the manifest is verified fail-closed by importers (see snapshot.go).
  • Snapshot objects are content-addressed by the sha256 of their sealed bytes (concurrent compactors can never clobber each other) and immutable; DeleteSnapshotObject exists only so compaction can prune superseded snapshots.
  • CompactEventsBelow deletes event objects strictly below each device's floor (Seq < floors[dev]). Callers must have durably published a superseding snapshot + manifest FIRST — the hub does not enforce the ordering; the compactor's confirm-before-delete protocol does.

Ack plane (P4-SYNC-06):

  • PutAck writes one device's signed sync-ack marker at meta/acks/<device_id>.json. It is single-writer-per-key (only the owning device writes its own ack) and last-writer-wins (unconditional overwrite).
  • ListAcks returns every ack currently on the hub keyed by device id.
  • DeleteAck removes one device's ack (idempotent) — used on device revoke.
  • DeleteDeviceStream removes an entire origin device's event-log prefix (idempotent), reclaiming a revoked device's stream after a compaction has folded its state into the snapshot. It returns the object count deleted.

Maintenance plane (P4-HUB-12):

  • MigrateLegacyEvents re-keys the retired HLC-keyed legacy layout into the per-device seq layout and deletes the migrated legacy objects. It is idempotent, resumable, and FAILS OPEN: an object whose key does not parse or whose body does not decode as a state.Event with matching (device, seq) is reported and KEPT (never deleted), mirroring the dual-read's fail-open posture — a parse bug must never delete an event it cannot account for. Each object is verified by read-back on the new key before the legacy object is deleted. FileHub has no legacy layout and returns (0, 0, nil).
  • GetSweepLock / PutSweepLock / DeleteSweepLock are the raw sweep-lock ops the advisory sweep mutex is built on (see SweepLock): a create-only PutSweepLock (returns ErrSweepLockHeld on conflict), a GetSweepLock that returns the lock bytes plus the object's backend LastModified for TTL judgment (ErrSweepLockNotFound when absent), and an idempotent DeleteSweepLock to release or break it. The lock is ADVISORY: it serializes cooperating clients only, not a hostile writer (spec/15).

type Mutation

type Mutation struct {
	Kind      MutationKind
	Upsert    state.UpsertProjectParams // Kind == MutationUpsert
	Tombstone TombstoneMutation         // Kind == MutationTombstone
}

Mutation is one namespace-map effect expressed as plain data. The impure persistence step maps MutationUpsert -> tx.UpsertProject and MutationTombstone -> tx.TombstoneProject; the pure Projection.Apply folds them in memory.

type MutationKind

type MutationKind int

MutationKind enumerates the namespace-map effects Decide can request.

const (
	// MutationUpsert activates/updates the row at Upsert.Path with the winner
	// payload and the event's source coordinates.
	MutationUpsert MutationKind = iota
	// MutationTombstone marks the row at Tombstone.Path deleted with a
	// monotonically non-decreasing tombstone HLC.
	MutationTombstone
)

type ProjectPayload

type ProjectPayload struct {
	Path          string `json:"path"`
	Type          string `json:"type"`
	RemoteURL     string `json:"remote_url,omitempty"`
	RemoteKey     string `json:"remote_key,omitempty"`
	DefaultBranch string `json:"default_branch,omitempty"`
}

func ProjectPayloadFromEvent

func ProjectPayloadFromEvent(payloadJSON string) (ProjectPayload, error)

ProjectPayloadFromEvent decodes a project event's payload (P5-SYNC-04), used to recover a losing variant's full remote.

type Projection

type Projection map[string]ProjectionRow

Projection is the namespace map `Decide` reconciles against, keyed by case-folded path_key. In production applyEventTx loads a slice holding just the event's path (0 or 1 entries); the property test maintains the full map and folds `Decide`+`Apply` over event permutations. `Decide` only ever reads the entry for the event's own path, so both usages behave identically.

func (Projection) Apply

func (p Projection) Apply(d Decision) (Projection, error)

Apply folds a Decision's mutations into the projection and returns the next projection, PURELY (the receiver is not modified). It is the in-memory analogue of applyDecisionTx used to property-test convergence; production persists via applyDecisionTx instead. Conflicts do not change the namespace projection.

type ProjectionRow

type ProjectionRow struct {
	NamespaceID         string // stable namespace_entries.id, for conflict attribution
	PathKey             string // case-folded path key
	Path                string // display path
	Type                string
	RemoteURL           string
	RemoteKey           string
	DefaultBranch       string
	Status              string // "active" | "deleted"
	TombstoneHLC        int64  // meaningful only when Status == "deleted"
	SourceEventHLC      int64
	SourceEventDeviceID string
	SourceEventID       string
	DirtyState          string
}

ProjectionRow is the in-memory namespace-entry state `Decide` reads to reconcile a single event. It mirrors the subset of namespace_entries (joined with git_repos / device_project_state) that governs convergence and holds NO database handle. There is at most one row per path_key, whose Status is "active" or "deleted"; a deleted row is canonical (only PathKey/Status/ TombstoneHLC are meaningful — every other field is zero), because a deleted row's sole convergence-relevant input is its tombstone HLC.

type PullStats

type PullStats struct {
	// RawSeen is the count of objects the backend returned for this pull,
	// before decryption, grant ingestion, skipping, or truncation.
	RawSeen int
	// Truncated is the count of raw events deferred because a workspace key
	// grant is not yet held: since P5-SYNC-01 the defer is PER ORIGIN DEVICE —
	// the deferred device's batch tail is dropped (its per-device cursor
	// holds automatically at the last consumed Seq), while other devices'
	// events keep flowing. Truncated sums the dropped per-device tails.
	// Non-zero means this device's view of the log is INCOMPLETE, so
	// consumers deriving a mark set from local state (hub gc, P6-HUB-01) must
	// refuse to sweep.
	Truncated int
	// Skipped is the count of events dropped in the decrypt pass (malformed
	// envelope, retired enc.v1 traffic, anti-downgrade plaintext). Like
	// Truncated, non-zero means the local replica may be missing references
	// other devices still rely on.
	Skipped int
	// Undecryptable is the count of enc.v2 events that failed AEAD
	// authentication on every held candidate key (corruption, forgery, or a
	// hub-side carrier mutation — P6-SYNC-04). These are NOT dropped: the
	// still-encrypted carrier is forwarded in the returned batch so
	// ApplyEvents records a permanent undecryptable quarantine conflict and
	// the cursor advances past it (no silent loss, no wedge).
	Undecryptable int
}

PullStats reports what a single EncryptedHub.Pull observed. Fields are set only when EncryptedHub.Stats is non-nil, and reset at the start of every Pull so a caller always reads the latest cycle.

type RenamePayload

type RenamePayload struct {
	OldPath string `json:"old_path"`
	NewPath string `json:"new_path"`
}

RenamePayload carries a project.renamed event's source and destination paths.

type RetentionManifest

type RetentionManifest struct {
	V           int                  `json:"v"`
	WorkspaceID string               `json:"workspace_id"`
	Floors      map[string]int64     `json:"floors"`
	Snapshot    RetentionSnapshotRef `json:"snapshot"`
	ProducedBy  string               `json:"produced_by"`
	ProducedAt  int64                `json:"produced_at_hlc"`
	PrevSHA256  string               `json:"prev_sha256"`
	Sig         string               `json:"sig"`
}

RetentionManifest is the hub's single signed retention marker (workspaces/<ws>/meta/retention.json, P6-HUB-04): the per-device retention floors plus the snapshot object that covers everything below them. It is written with compare-and-swap so concurrent compactors cannot lose updates, and chained via PrevSHA256 for audit. One manifest with a floor MAP (rather than one marker per device stream) keeps the multi-device floor update atomic.

func ParseRetentionManifest

func ParseRetentionManifest(raw []byte) (RetentionManifest, error)

ParseRetentionManifest decodes raw manifest bytes WITHOUT verifying the signature. Backends use it on the pull path to read floors (they hold no device registry; an unverified floor can only FORCE the snapshot path, where the fail-closed verification lives), and the importer uses it before verification. Structural validation IS performed: a syntactically-valid but hollow object ({} decodes with Floors == nil) must not read as "no floor" — that would let a hub that garbles or truncates its own marker serve a partial post-compaction log as if it were complete.

type RetentionSnapshotRef

type RetentionSnapshotRef struct {
	// Fields are declared in alphabetical json-tag order so the marshaled
	// signature payload is canonical (matches eventSignaturePayloadV2 style).
	Epoch      int64  `json:"epoch"`
	HLC        int64  `json:"hlc"`
	KID        string `json:"kid"`
	ProducedBy string `json:"produced_by"`
	SHA256     string `json:"sha256"`
}

RetentionSnapshotRef names the current snapshot object inside the manifest.

type SamePathConflictInfo

type SamePathConflictInfo struct {
	Path          string
	WinnerKey     string
	WinnerEventID string
	LoserKey      string
	LoserEventID  string
}

SamePathConflictInfo is the exported, parsed view of a same_path_different_remote conflict's details (P5-SYNC-04), so the CLI resolver can recover the competing variants and their origin events.

func ParseSamePathConflictDetails

func ParseSamePathConflictDetails(detailsJSON string) (SamePathConflictInfo, error)

ParseSamePathConflictDetails decodes a same_path_different_remote conflict's details_json (P5-SYNC-04).

type Snapshot

type Snapshot struct {
	V           int                 `json:"v"`
	WorkspaceID string              `json:"workspace_id"`
	ProducedBy  string              `json:"produced_by"`
	HLC         int64               `json:"hlc"`
	Epoch       int64               `json:"epoch"`
	KID         string              `json:"kid"`
	Floor       Cursor              `json:"floor"`
	Anchors     []ChainAnchor       `json:"anchors,omitempty"`
	Entries     []SnapshotEntry     `json:"entries,omitempty"`
	Tombstones  []SnapshotTombstone `json:"tombstones,omitempty"`
}

Snapshot is the plaintext full-state snapshot document (snapshot.v1): the derived namespace map with source-event coordinates for LWW convergence, surviving tombstones, per-device chain anchors, and the per-device floor the producing compactor is about to publish. Device-local state (conflicts, key_grant_waits, sync_skipped_events) is deliberately excluded, and grant events are NOT embedded — device approval re-grants all held epochs as fresh events, which always land above the floor.

func BuildSnapshot

func BuildSnapshot(ctx context.Context, st *state.Store, producedBy string, hlc int64, floors Cursor) (Snapshot, error)

BuildSnapshot reads the derived namespace map, surviving tombstones, and per-device chain anchors from the store and assembles a snapshot document for the given producer, HLC, and floors. The floors map is the exact per-device retention floor the compactor is about to publish; the snapshot's Floor must equal the manifest's Floors (recoverFromSnapshot cross-checks them), so the caller passes the reconciled floors and seals this document under them.

func UnsealSnapshot

func UnsealSnapshot(obj []byte, wck []byte) (Snapshot, error)

UnsealSnapshot opens a sealed snapshot object with one candidate WCK. The AAD is re-derived from the envelope's carrier fields and the CANDIDATE KEY's kid — never the envelope's unauthenticated kid hint — so a mutation of any carrier field, or a wrong candidate key, is an authentication failure. The caller loops held candidates for the envelope's (epoch, kid) exactly like EncryptedHub.Pull does for events.

type SnapshotDraft

type SnapshotDraft struct {
	BlobRef             string `json:"blob_ref"`
	ByteSize            int64  `json:"byte_size"`
	FileCount           int64  `json:"file_count"`
	SourceEventHLC      int64  `json:"source_event_hlc"`
	SourceEventDeviceID string `json:"source_event_device_id"`
	SourceEventID       string `json:"source_event_id"`
}

SnapshotDraft is the LATEST draft-bundle pointer for a draft project, so a bootstrapping device can fetch the age-encrypted bundle blob and hub gc's mark set stays complete after the events that carried it are compacted.

type SnapshotEntry

type SnapshotEntry struct {
	Path                  string         `json:"path"`
	PathKey               string         `json:"path_key"`
	Type                  string         `json:"type"`
	DisplayName           string         `json:"display_name,omitempty"`
	MaterializationPolicy string         `json:"materialization_policy,omitempty"`
	Status                string         `json:"status"`
	SourceEventHLC        int64          `json:"source_event_hlc"`
	SourceEventDeviceID   string         `json:"source_event_device_id"`
	SourceEventID         string         `json:"source_event_id"`
	Git                   *SnapshotGit   `json:"git,omitempty"`
	Draft                 *SnapshotDraft `json:"draft,omitempty"`
}

SnapshotEntry is one active namespace-map row with the source-event coordinates (HLC, device, event id) that make import a pure LWW merge.

type SnapshotEnvelopeInfo

type SnapshotEnvelopeInfo struct {
	WorkspaceID string
	ProducedBy  string
	HLC         int64
	Epoch       int64
	KID         string
}

SnapshotEnvelopeInfo is the parsed plaintext carrier of a sealed snapshot object, exposed so the importer can select WCK candidates by (epoch, kid) before attempting to unseal.

func ParseSnapshotEnvelope

func ParseSnapshotEnvelope(obj []byte) (SnapshotEnvelopeInfo, error)

ParseSnapshotEnvelope decodes a sealed snapshot object's plaintext carrier without decrypting, so the importer can select WCK candidates by (epoch, kid) first. The version check is fail-closed.

type SnapshotGit

type SnapshotGit struct {
	RemoteURL     string `json:"remote_url"`
	RemoteKey     string `json:"remote_key"`
	DefaultBranch string `json:"default_branch,omitempty"`
	CloneFilter   string `json:"clone_filter,omitempty"`
	SparseConfig  string `json:"sparse_config,omitempty"`
	LFSPolicy     string `json:"lfs_policy,omitempty"`
	ForgeKind     string `json:"forge_kind,omitempty"`
}

SnapshotGit mirrors the git_repos row attached to a namespace entry.

type SnapshotTombstone

type SnapshotTombstone struct {
	PathKey             string `json:"path_key"`
	TombstoneHLC        int64  `json:"tombstone_hlc"`
	SourceEventDeviceID string `json:"source_event_device_id,omitempty"`
	SourceEventID       string `json:"source_event_id,omitempty"`
}

SnapshotTombstone is one surviving deleted entry, kept so a stale add cannot resurrect a deleted path on a bootstrapped device.

type SweepLock

type SweepLock struct {
	HolderDevice  string `json:"holder_device"`
	AcquiredAtHLC int64  `json:"acquired_at_hlc"`
	TTLSeconds    int64  `json:"ttl_seconds"`
	// Nonce is a per-acquire random token (crypto/rand hex). Release deletes the
	// lock object only when its bytes still match the exact body this acquire
	// wrote — so a sweeper that overran its TTL and had its lock stale-broken by
	// a successor cannot then delete the SUCCESSOR's lock (the nonce differs).
	Nonce string `json:"nonce"`
}

SweepLock is the advisory sweep-lock body serialized to meta/sweep.lock (P4-HUB-12). It coordinates the destructive hub passes (`hub gc`, `hub compact`, `hub migrate-events`) among COOPERATING clients so they do not interleave; it is not a defense against a hostile writer, which is out of scope (spec/15). Staleness is judged by the object's backend mtime (R2 ListObjectsV2 / FileHub file mtime), never by AcquiredAtHLC — a hostile or clock-skewed self-report must not extend a lock. AcquiredAtHLC is the fallback age source used only when a backend cannot report an object mtime.

func ParseSweepLock

func ParseSweepLock(raw []byte) (SweepLock, error)

ParseSweepLock deserializes a sweep-lock body.

func (SweepLock) AcquiredAt

func (l SweepLock) AcquiredAt() time.Time

AcquiredAt derives a wall-clock acquisition time from the HLC physical component. It is the FALLBACK age source used only when a backend cannot report an object mtime; the primary staleness judgment uses the backend's LastModified.

func (SweepLock) TTL

func (l SweepLock) TTL() time.Duration

TTL returns the lock's configured time-to-live.

type TombstoneMutation

type TombstoneMutation struct {
	Path string
	HLC  int64
}

TombstoneMutation carries a delete/rename-away tombstone.

type WorkspaceKeyring

type WorkspaceKeyring interface {
	// PushKey returns the key new outgoing events encrypt under: the active
	// epoch, its kid (KIDForWCK), and the WCK bytes. epoch == 0 with a nil
	// error means this device holds no workspace key yet (a joiner awaiting
	// its grant). When several keys coexist at the active epoch (P6-SEC-02
	// collision coexistence), the fleet key — grant origin — is preferred over
	// a local self-mint so a legacy joiner converges onto the founder's key.
	// Call Prime first.
	PushKey(ctx context.Context) (epoch int64, kid string, wck []byte, err error)
	// Prime loads every held key's WCK into memory so WCK lookups during a
	// Pull are pure and context-free. EncryptedHub.Pull calls Prime before
	// ingesting in-batch grants and decrypting.
	Prime(ctx context.Context) error
	// WCKCandidates returns the candidate WCKs for decrypting an envelope
	// addressed to (epoch, kid), from the in-memory cache. kid != "" selects
	// the exact key (zero or one candidates); kid == "" (a legacy pre-kid
	// envelope) returns every held key at the epoch, which the caller tries in
	// order — the AEAD authenticates, so a wrong candidate just fails. An
	// empty result means no candidate is held (the grant has not arrived).
	// Call Prime (and ingest in-batch grants) first.
	WCKCandidates(epoch int64, kid string) [][]byte
	// IngestGrant unwraps a device.key.grant addressed to the local device and
	// persists the WCK under its (epoch, kid). Grants for other recipients are
	// a no-op. A grant whose carried kid does not match its unwrapped bytes is
	// rejected, and an already-held (epoch, kid) is never overwritten
	// (P6-SEC-01b/c).
	IngestGrant(ctx context.Context, grant DeviceKeyGrant) error
}

WorkspaceKeyring is the key-epoch abstraction the EncryptedHub decorator uses to encrypt outgoing events and decrypt incoming ones (P4-SEC-07). It is defined here so the decorator depends on the interface, not the concrete keyring, keeping keychain/platform/state dependencies out of internal/sync. The concrete implementation lives in internal/workspacekeys.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL