Documentation
¶
Overview ¶
Package journal is the encoding the durable stores share: P2's journal line (one committed change) and snapshot (the whole store at a version), the kinds they can load and the transcript blob names. The file store (store/filestore) writes them to a journal file and a snapshot file; an object store (P3) writes one line per log object, plus the writer and the schema version, and gzip'd snapshots.
Design: design/workstreams/dispatcher-runtime/LLD/store.md §File store, and §Object store when present; P3.
Index ¶
- Constants
- Variables
- func CheckType(types map[string]reflect.Type, op string, obj store.Object, name string) error
- func ParseSnapshot(data []byte, types map[string]reflect.Type, state State) (version uint64, schema int, err error)
- func Registry(extra map[string]store.Object) (map[string]reflect.Type, error)
- func RunOfTranscript(name string) (run string, ok bool)
- func TranscriptName(run string) string
- func UnknownKind(kind string) error
- func WriteSnapshot(w io.Writer, recs []memory.Record, v uint64, schema int) error
- type Entry
- type FormatError
- type Header
- type Line
- type SnapshotObject
- type State
- type Writer
Constants ¶
const ( FileStoreSnapshot = "snapshot.json" FileStoreJournal = "journal.jsonl" FileStoreLock = "LOCK" ObjectStoreLog = "log" ObjectStoreSnapshots = "snapshots" TranscriptDir = "transcripts" )
The names each durable store keeps at the top of its directory or bucket prefix. Both name transcript blobs TranscriptDir/<TranscriptName>, so neither may open a directory that holds the other's: the file store refuses ObjectStoreLog and ObjectStoreSnapshots, a file:// bucket refuses the file store's files.
const ( // TranscriptSuffix ends a transcript blob's name. TranscriptSuffix = ".jsonl.gz" // MaxTranscriptName keeps names under the 255-byte limit of the file systems we run on. MaxTranscriptName = 240 )
const Format = 1
Format is the snapshot format this package reads and writes.
Variables ¶
var BuiltinKinds = []string{ v1alpha1.KindEvent, v1alpha1.KindWorkItem, v1alpha1.KindTask, v1alpha1.KindRun, v1alpha1.KindAgent, v1alpha1.KindEffect, v1alpha1.KindBudgetCounter, v1alpha1.KindApproval, }
BuiltinKinds are the v1alpha1 kinds the store holds.
Functions ¶
func CheckType ¶
CheckType rejects a write of obj, of a kind the store couldn't load back. Writes that are invalid anyway (store.CheckObject) are left to the memory store, so its errors stay the conformant ones. name is the store's, for the message.
func ParseSnapshot ¶
func ParseSnapshot(data []byte, types map[string]reflect.Type, state State) (version uint64, schema int, err error)
ParseSnapshot loads a snapshot's objects into state, which must be empty, and returns its version and schema. A *FormatError is a format this package doesn't read; any other error is a corrupt snapshot, saying where.
func Registry ¶
Registry returns the Go type of every kind a durable store may hold: the built-in kinds and extra, the internal kinds (a pointer to each one's zero value).
func RunOfTranscript ¶
RunOfTranscript is TranscriptName's inverse; ok is false for a name it never returns.
func TranscriptName ¶
TranscriptName returns run's blob name (P2's file name in transcripts/), or "" when run is too long for one: the run name itself when it is safe, else ~ and its base64url.
func UnknownKind ¶
UnknownKind is the error for a stored kind with no registered Go type.
func WriteSnapshot ¶
WriteSnapshot writes the snapshot of recs at version v to w, one object per line (so it greps), in kind and name order. schema is the writer's schema version; 0 leaves it out (the file store's snapshots have none).
Types ¶
type FormatError ¶
type FormatError struct{ Got int }
FormatError is a snapshot whose format this package doesn't read.
func (*FormatError) Error ¶
func (e *FormatError) Error() string
type Header ¶
type Header struct {
Format int `json:"format"`
StoreVersion string `json:"storeVersion"`
Schema int `json:"schema,omitzero"`
}
Header is a snapshot's header.
func ReadHeader ¶
ReadHeader parses a snapshot's header: a *FormatError for a format this package doesn't read. The format is decoded alone first, so another format's header fields can't fail it.
type Line ¶
type Line struct {
V string `json:"v"`
At time.Time `json:"at"`
Op memory.Op `json:"op"`
Kind string `json:"kind"`
Name string `json:"name"`
Obj json.RawMessage `json:"obj,omitzero"`
// W is who wrote the change: the writer's instance id and the write's nonce.
W *Writer `json:"w,omitzero"`
// Schema is the writer's store.SchemaVersion.
Schema int `json:"schema,omitzero"`
}
Line is one committed change: a line of the file store's journal, or one object of the object store's log. W and Schema are the object store's; the file store leaves them out.
func DecodeLine ¶
DecodeLine parses and checks one line, and returns it with its version.
type SnapshotObject ¶
type SnapshotObject struct {
Kind string `json:"kind"`
Name string `json:"name"`
V string `json:"v"`
Created time.Time `json:"created"`
At time.Time `json:"at"`
Obj json.RawMessage `json:"obj"`
}
SnapshotObject is one object in a snapshot. created, at and v are kept beside obj because kinds without metadata (Event) don't carry them.