journal

package
v0.3.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 29, 2026 License: AGPL-3.0 Imports: 14 Imported by: 0

Documentation

Overview

Package journal is the encoding the durable stores share: P2's journal line (one committed change) and snapshot (the whole store at a version), the kinds they can load and the transcript blob names. The file store (store/filestore) writes them to a journal file and a snapshot file; an object store (P3) writes one line per log object, plus the writer and the schema version, and gzip'd snapshots.

Design: design/workstreams/dispatcher-runtime/LLD/store.md §File store, and §Object store when present; P3.

Index

Constants

View Source
const (
	FileStoreSnapshot    = "snapshot.json"
	FileStoreJournal     = "journal.jsonl"
	FileStoreLock        = "LOCK"
	ObjectStoreLog       = "log"
	ObjectStoreSnapshots = "snapshots"
	TranscriptDir        = "transcripts"
)

The names each durable store keeps at the top of its directory or bucket prefix. Both name transcript blobs TranscriptDir/<TranscriptName>, so neither may open a directory that holds the other's: the file store refuses ObjectStoreLog and ObjectStoreSnapshots, a file:// bucket refuses the file store's files.

View Source
const (
	// TranscriptSuffix ends a transcript blob's name.
	TranscriptSuffix = ".jsonl.gz"

	// MaxTranscriptName keeps names under the 255-byte limit of the file systems we run on.
	MaxTranscriptName = 240
)
View Source
const Format = 1

Format is the snapshot format this package reads and writes.

Variables

BuiltinKinds are the v1alpha1 kinds the store holds.

Functions

func CheckType

func CheckType(types map[string]reflect.Type, op string, obj store.Object, name string) error

CheckType rejects a write of obj, of a kind the store couldn't load back. Writes that are invalid anyway (store.CheckObject) are left to the memory store, so its errors stay the conformant ones. name is the store's, for the message.

func ParseSnapshot

func ParseSnapshot(data []byte, types map[string]reflect.Type, state State) (version uint64, schema int, err error)

ParseSnapshot loads a snapshot's objects into state, which must be empty, and returns its version and schema. A *FormatError is a format this package doesn't read; any other error is a corrupt snapshot, saying where.

func Registry

func Registry(extra map[string]store.Object) (map[string]reflect.Type, error)

Registry returns the Go type of every kind a durable store may hold: the built-in kinds and extra, the internal kinds (a pointer to each one's zero value).

func RunOfTranscript

func RunOfTranscript(name string) (run string, ok bool)

RunOfTranscript is TranscriptName's inverse; ok is false for a name it never returns.

func TranscriptName

func TranscriptName(run string) string

TranscriptName returns run's blob name (P2's file name in transcripts/), or "" when run is too long for one: the run name itself when it is safe, else ~ and its base64url.

func UnknownKind

func UnknownKind(kind string) error

UnknownKind is the error for a stored kind with no registered Go type.

func WriteSnapshot

func WriteSnapshot(w io.Writer, recs []memory.Record, v uint64, schema int) error

WriteSnapshot writes the snapshot of recs at version v to w, one object per line (so it greps), in kind and name order. schema is the writer's schema version; 0 leaves it out (the file store's snapshots have none).

Types

type Entry

type Entry struct {
	V                uint64
	Created, Updated time.Time
	Data             []byte
}

Entry is one object while loading.

type FormatError

type FormatError struct{ Got int }

FormatError is a snapshot whose format this package doesn't read.

func (*FormatError) Error

func (e *FormatError) Error() string
type Header struct {
	Format       int    `json:"format"`
	StoreVersion string `json:"storeVersion"`
	Schema       int    `json:"schema,omitzero"`
}

Header is a snapshot's header.

func ReadHeader

func ReadHeader(data []byte) (Header, error)

ReadHeader parses a snapshot's header: a *FormatError for a format this package doesn't read. The format is decoded alone first, so another format's header fields can't fail it.

type Line

type Line struct {
	V    string          `json:"v"`
	At   time.Time       `json:"at"`
	Op   memory.Op       `json:"op"`
	Kind string          `json:"kind"`
	Name string          `json:"name"`
	Obj  json.RawMessage `json:"obj,omitzero"`
	// W is who wrote the change: the writer's instance id and the write's nonce.
	W *Writer `json:"w,omitzero"`
	// Schema is the writer's store.SchemaVersion.
	Schema int `json:"schema,omitzero"`
}

Line is one committed change: a line of the file store's journal, or one object of the object store's log. W and Schema are the object store's; the file store leaves them out.

func DecodeLine

func DecodeLine(line []byte) (Line, uint64, error)

DecodeLine parses and checks one line, and returns it with its version.

func LineOf

func LineOf(c memory.Change) Line

LineOf returns c as a line.

func (Line) Change

func (l Line) Change(v uint64) memory.Change

Change returns the line as a memory change at version v.

type SnapshotObject

type SnapshotObject struct {
	Kind    string          `json:"kind"`
	Name    string          `json:"name"`
	V       string          `json:"v"`
	Created time.Time       `json:"created"`
	At      time.Time       `json:"at"`
	Obj     json.RawMessage `json:"obj"`
}

SnapshotObject is one object in a snapshot. created, at and v are kept beside obj because kinds without metadata (Event) don't carry them.

type State

type State map[store.Key]*Entry

State is the objects while loading, by key.

func (State) Apply

func (st State) Apply(types map[string]reflect.Type, jl Line, v uint64) error

Apply replays one line, at version v, onto the state. types are the kinds that can be loaded (Registry).

func (State) Records

func (st State) Records(types map[string]reflect.Type) ([]memory.Record, map[string]int)

Records returns the state as memory records (for memory.Restore), and the objects per kind.

type Writer

type Writer struct {
	// ID is the writing store's instance id.
	ID string `json:"id"`
	// Nonce is random per write: a writer that finds its own nonce in the log knows the
	// change is its own.
	Nonce string `json:"nonce"`
}

Writer names the write that made a change.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL