outbox

package
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 16, 2026 License: MIT Imports: 6 Imported by: 0

Documentation

Overview

Package outbox holds the ordered event streams that peers exchange and the hybrid logical clock that orders their events: the port that stores this instance's outbox, each peer's acknowledged position in it, the receiver's applied positions and the blob fetches that failed while applying, with the values it takes and returns.

Role

platform layer: the ordered event streams of decision 0021 and the clock that orders them. The target layout puts the outbox in the platform, so that a module appends its events in its own transaction and a module's api package may import it. OutboxStore takes and returns the clock and the values, so they are declared beside it and the package imports no module. The platform's store adapters (platform/store/sql and platform/store/inmem) implement OutboxStore, and the metadata store's Outbox returns theirs; no service appends yet. The package imports only the standard library.

Inputs

  • Entries appended inside the transaction of the change they record: origin, stream, clock time, event type, payload and, for a content event, the blob's SHA-256 and size.
  • Peers' acknowledged positions, the receiver's applied positions and the blob fetches that failed while applying.
  • For the clock, a physical time (Next and Merge) or the text form (ParseHLC).

Outputs

  • OutboxStore: Append, Read, LastSeq, Streams and Prune for the entries; PutCursor, Cursor, Cursors and DeleteCursors for peers' positions; Applied and SetApplied for the receiver's; and RecordFailedFetch, FailedFetches and ClearFailedFetch for failed blob fetches.
  • HLC, with Compare, Next, Merge and String, and ParseHLC with its sentinel ErrInvalidHLC.
  • OutboxEntry; PeerCursor, with Behind; AppliedMarker, with Expect; and FailedBlobFetch.

Invariants

  • Seq is gap-free per origin and stream, and concurrent appends to one stream are serialised, never given the same Seq. An append rolled back with its transaction gives its number back, and pruning never resets the numbering.
  • The HLC text form is fixed-width hex, so it sorts as Compare does and can be stored in an indexed column; ParseHLC reads back every value that String writes.
  • The clock never runs behind anything it has seen: Merge takes the largest wall time, and the logical counter orders events that share one.
  • SetApplied runs in the transaction that applies the entry, which makes applying idempotent: a retried batch starts at the next expected sequence, AppliedMarker.Expect.

References

  • Decision 0021: docs/research/decisions/0021-peer-trust-and-sync-protocol.md ("Ordered outbox": streams, sequence numbers, acknowledgements, applied positions and failed fetches; "Conflict rule": the clock time and the origin), with Kulkarni et al., "Logical Physical Clocks", 2014.
  • Decision 0044: docs/research/decisions/0044-modular-monolith-migration.md ("Dependency rules", rule 5: a module's api may import platform/outbox).
  • Research 15: docs/research/15-data-model.md, lines 703 to 706 (the outbox, peer cursor, applied marker and failed fetch tables) and line 763 (the clock's two columns).
  • Target layout: docs/research/architecture-options/5-modular-monolith.md, lines 140 and 141 (platform/outbox, from the store's OutboxStore, generalised in P11), lines 720 to 724 (P11's Event, Append and per-consumer cursors) and lines 341 and 342 (the federation module's core/port, where the target layout keeps the peer types).
  • Plan: docs/implementation/plan.md#p2-stores, #p13-peer-trust-and-federated-repositories and #p9d-modular-monolith-migration, epic PLT (https://github.com/weaveplatform/artifactweave/issues/938).
  • https://github.com/weaveplatform/artifactweave/issues/989

Status

Implemented P2: the clock, the outbox values and OutboxStore, with the metadata store's SQL and in-memory stores (pull request 3). Moved here in P9d. Roadmap: P11 generalises the outbox to Append(ctx, Event) for every consumer, and P13 delivers its streams to peers.

Testing

  • outbox_test.go: TestUnitHLC (Next ahead of and behind physical time, the four Merge cases, Compare and the text form agree on order, ParseHLC reads back what String writes and refuses five malformed values with ErrInvalidHLC) and TestUnitCursorsAndMarkers (Behind and Expect).

The platform's outbox contract (the Outbox, OutboxConcurrent, PeerCursors, AppliedMarkers and FailedFetches subtests of platform/store/storetest: sequences per origin and stream, reads after a sequence and up to a limit, pruning that keeps the numbering, a rolled-back append, concurrent appends, cursor replacement and deletion, applied positions per origin, failed fetch counting and clearing, and cancelled contexts) runs on the platform's in-memory and SQLite stores and on the metadata store's in-memory, SQLite and PostgreSQL stores. The federation model's TestUnitStamp orders stamps through Compare.

Index

Constants

This section is empty.

Variables

View Source
var ErrInvalidHLC = errors.New("invalid hybrid logical clock value")

ErrInvalidHLC reports a hybrid logical clock value that does not parse.

Functions

This section is empty.

Types

type AppliedMarker

type AppliedMarker struct {
	Origin  string
	Stream  string
	LastSeq int64
}

AppliedMarker is the receiver's record of the last sequence applied from one origin's stream, updated in the apply transaction so a retried batch is idempotent.

func (AppliedMarker) Expect

func (m AppliedMarker) Expect() int64

Expect is the sequence the next batch must start at.

type FailedBlobFetch

type FailedBlobFetch struct {
	Stream      string
	SHA256      string
	SHA1        string
	RepoKey     string
	FirstFailed time.Time
	Failures    int
	LastError   string
}

FailedBlobFetch records content a receiver could not fetch from a peer; it backs the federation failed-binary APIs and is retried with back-off.

type HLC

type HLC struct {
	Wall    int64 // Unix milliseconds
	Logical uint32
}

HLC is a hybrid logical clock timestamp (Kulkarni et al., 2014): physical milliseconds plus a logical counter. It orders events across instances whose wall clocks disagree, which last-writer-wins needs (decision 0021).

func ParseHLC

func ParseHLC(s string) (HLC, error)

ParseHLC reads the String form. Text that is not 16 and 8 hex digits around a dot gives an error that wraps ErrInvalidHLC.

func (HLC) Compare

func (h HLC) Compare(o HLC) int

Compare orders two timestamps.

func (HLC) Merge

func (h HLC) Merge(remote HLC, now time.Time) HLC

Merge is the timestamp of receiving remote at physical time now, so the local clock never runs behind anything it has seen.

func (HLC) Next

func (h HLC) Next(now time.Time) HLC

Next is the timestamp of a local event after h at physical time now.

func (HLC) String

func (h HLC) String() string

String renders the timestamp as fixed-width hex, so the text form sorts the same way Compare does and can be stored in an indexed column.

type OutboxEntry

type OutboxEntry struct {
	Origin     string
	Stream     string
	Seq        int64
	Time       HLC
	EventType  string
	Payload    []byte
	BlobSHA256 string
	BlobSize   int64
	Created    time.Time
}

OutboxEntry is one change queued for peers, written in the same transaction as the change it records. (Origin, Stream, Seq) is unique and Seq is gap-free per stream, which lets a receiver detect a missing batch. BlobSHA256 is set for content events; receivers pull the blob.

type OutboxStore

type OutboxStore interface {
	// Append stores e as the next entry of (e.Origin, e.Stream) and returns
	// it with Seq assigned, one more than the stream's last. Call it inside
	// the transaction that makes the change it records; concurrent appends
	// to one stream are serialised, never given the same Seq.
	Append(ctx context.Context, e OutboxEntry) (OutboxEntry, error)
	// Read returns up to limit entries of a stream with Seq above after, in
	// Seq order.
	Read(
		ctx context.Context,
		origin, stream string,
		after int64,
		limit int,
	) ([]OutboxEntry, error)
	// LastSeq returns the highest Seq of a stream, 0 when it has none.
	LastSeq(ctx context.Context, origin, stream string) (int64, error)
	// Streams returns the streams origin has entries or a sequence in,
	// ordered.
	Streams(ctx context.Context, origin string) ([]string, error)
	// Prune removes entries of a stream with Seq at most upTo and returns how
	// many. The stream's sequence is not reset.
	Prune(ctx context.Context, origin, stream string, upTo int64) (int64, error)

	// PutCursor creates or replaces a peer's position in a stream.
	PutCursor(ctx context.Context, c PeerCursor) error
	// Cursor returns a peer's position in a stream; kernel.ErrNotFound when
	// the peer has none there.
	Cursor(ctx context.Context, peer, stream string) (PeerCursor, error)
	// Cursors returns a peer's positions ("" for every peer) ordered by
	// (peer, stream).
	Cursors(ctx context.Context, peer string) ([]PeerCursor, error)
	// DeleteCursors removes every position of a peer; a peer with none is
	// not an error.
	DeleteCursors(ctx context.Context, peer string) error

	// Applied returns the receiver's position in a peer's stream; a zero
	// LastSeq and no error when nothing was applied yet.
	Applied(ctx context.Context, origin, stream string) (AppliedMarker, error)
	// SetApplied records the position, in the transaction that applies the
	// entry, which is what makes applying idempotent.
	SetApplied(ctx context.Context, m AppliedMarker) error

	// RecordFailedFetch stores a failed blob fetch with Failures 1, or
	// increments Failures and replaces LastError of the existing one, keeping
	// its other fields, and returns the record as stored. Failures counts
	// the calls; the value in f is ignored.
	RecordFailedFetch(ctx context.Context, f FailedBlobFetch) (FailedBlobFetch, error)
	// FailedFetches returns a stream's failed fetches ("" for every stream)
	// ordered by (stream, sha256).
	FailedFetches(ctx context.Context, stream string) ([]FailedBlobFetch, error)
	// ClearFailedFetch removes one after a successful retry;
	// kernel.ErrNotFound when there is none.
	ClearFailedFetch(ctx context.Context, stream, sha256 string) error
}

OutboxStore persists the ordered event streams of decision 0021: this instance's outbox, each peer's acknowledged position in it, the receiver's applied positions, and blob fetches that failed while applying.

type PeerCursor

type PeerCursor struct {
	PeerInstanceID string
	Stream         string
	AckedSeq       int64
	LastSent       time.Time
	LastError      string
}

PeerCursor is the highest sequence a peer has acknowledged on a stream.

func (PeerCursor) Behind

func (c PeerCursor) Behind(head int64) int64

Behind is how many entries the peer still has to apply, given the stream's head sequence.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL