journal

package
v0.2.7 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 7, 2026 License: MIT Imports: 15 Imported by: 0

Documentation

Overview

Package journal is the durable record of one run, kept so that a run which stops can continue without doing anything twice.

A run is resumed by starting the program again from the top and answering every request it already made from this record. So the record is keyed by the request's own id, which the program assigns, and never by the order requests arrived in: with several requests in flight, arrival order is not the program's order.

It is tiered. The index holds one line for the start and one for the end of each request. A result too large for a line is stored once, by its digest, beside the index. Nothing is ever rewritten or compacted.

PROPOSED, not ruled: the design this implements belongs to. The journal in telara-agents/tap-runtime/journal carries leases, fencing tokens and idempotency keys built for manifest v2's policy store. This package does not replace those; it is what the v3 runner needs to resume. The two are reconciled when that code moves here.

Index

Constants

View Source
const InlineLimit = 8 << 10

InlineLimit is the largest result kept in the index. Anything larger goes to a blob.

View Source
const LeaseTTL = 30 * time.Second

LeaseTTL is how long a lease lasts without being renewed. The holder renews it at a third of that.

Variables

View Source
var ErrChanged = errors.New("the program made a different request under an id it used before")

Functions

func Digest

func Digest(v any) string

Digest identifies the content of a request, whatever order its fields are written in.

func NewRunID

func NewRunID(now time.Time) string

NewRunID is sortable by start time and unique on one machine.

func ReadManifest added in v0.2.2

func ReadManifest(root string, h Header) ([]byte, error)

ReadManifest reads only the saved snapshot, never the mutable package path. A nil result means the journal predates snapshots. Reads and integrity checks are bounded; no symlink in the run-relative blob path is accepted.

func Sweep

func Sweep(root string, days int, now time.Time) int

Sweep removes the records of runs that ended more than days ago. A run that is held is never removed. days of 0 or less keeps everything. It returns how many it removed.

Types

type CallIdentity added in v0.2.2

type CallIdentity struct {
	Alias     string `json:"alias,omitempty"`
	Server    string `json:"server,omitempty"`
	Tool      string `json:"tool,omitempty"`
	Operation string `json:"operation,omitempty"`
	Annotated string `json:"annotated,omitempty"`
	Schema    string `json:"schema,omitempty"`
	Truncated bool   `json:"truncated,omitempty"`
}

CallIdentity names the binding assessed for a request, without its arguments or result. A begin is an attempt, not proof that the provider changed anything.

type Event

type Event struct {
	Phase   string        `json:"phase"`
	ID      string        `json:"id,omitempty"`
	Method  string        `json:"method,omitempty"`
	Effect  string        `json:"effect,omitempty"`
	At      time.Time     `json:"at"`
	Outcome string        `json:"outcome,omitempty"`
	Call    *CallIdentity `json:"call,omitempty"`
}

Event is the safe, displayable part of one journal record. It deliberately excludes request digests, replies, blob paths, and byte counts.

type Header struct {
	RunID           string    `json:"run_id"`
	Package         string    `json:"package"`
	PackageDigest   string    `json:"package_digest"`
	Args            []string  `json:"args"`
	Started         time.Time `json:"started"`
	ManifestDigest  string    `json:"manifest_digest,omitempty"`
	ManifestBytes   int       `json:"manifest_bytes,omitempty"`
	EvidenceVersion int       `json:"evidence_version,omitempty"`
}

Header identifies the run. A run is resumed only against the package it was started with.

type HeldError

type HeldError struct {
	RunID   string
	PID     int
	Expires time.Time
}

HeldError is returned when another process holds the run.

func (*HeldError) Error

func (e *HeldError) Error() string

type InspectState

type InspectState string

InspectState describes whether a journal has reached its terminal record. Unknown means inspection was bounded before the terminal record could be established.

const (
	InspectFinished    InspectState = "finished"
	InspectRunning     InspectState = "running"
	InspectInterrupted InspectState = "interrupted"
	InspectUnknown     InspectState = "unknown"
)

type Journal

type Journal struct {

	// Observed is the clock and random bytes the program was given.
	Observed *Observed
	Header   Header
	// contains filtered or unexported fields
}

func Create

func Create(root string, h Header) (*Journal, error)

Create starts the record of a new run.

func CreateWithManifest added in v0.2.2

func CreateWithManifest(root string, h Header, manifest []byte) (*Journal, error)

CreateWithManifest saves the exact manifest before making the run visible. Resuming never replaces this snapshot with the current package's manifest.

func Open

func Open(root, runID string) (*Journal, error)

Open reads the record of an earlier run so that it can continue. A last line cut short by a crash is ignored: it recorded nothing that finished.

func (*Journal) Begin

func (j *Journal) Begin(id, method, digest, effect string, at time.Time) error

Begin records that a request is about to be acted on.

func (*Journal) BeginCall added in v0.2.2

func (j *Journal) BeginCall(id, method, digest, effect string, call *CallIdentity, at time.Time) error

BeginCall adds resolved identity to the same durable begin used for replay.

func (*Journal) Close

func (j *Journal) Close() error

func (*Journal) Counts

func (j *Journal) Counts() (finished, interrupted int)

Counts reports how many requests were answered and how many were left unanswered.

func (*Journal) Effect

func (j *Journal) Effect(id string) string

Effect is the effect recorded when the request was begun.

func (*Journal) End

func (j *Journal) End(id, outcome string, reply []byte, at time.Time) error

End records the answer to a request.

func (*Journal) Finish

func (j *Journal) Finish(outcome string, at time.Time) error

Finish records that the run ended. A finished run cannot be resumed.

func (*Journal) Lookup

func (j *Journal) Lookup(id, digest string) (State, []byte, error)

Lookup reports what is known of a request. It returns ErrChanged when the id is known and the request is not the one recorded: the program is not following the path it followed before, and replaying its record would hand it answers to questions it did not ask.

type Lease

type Lease struct {
	// contains filtered or unexported fields
}

Lease is a held lease. Release it when the run ends.

func (*Lease) Release

func (l *Lease) Release()

Release gives the lease up. It removes the lease only if it is still this holder's.

type Observed

type Observed struct {
	// contains filtered or unexported fields
}

Observed is what a program saw besides the answers to its requests: the clock and random bytes.

Replayed steps see what they saw before, and steps after the resume point see the real clock and real random bytes. So every reading is recorded as it is given, a resumed program is given the recorded readings in the order it took them, and once they run out it is given the real thing, which is recorded in turn.

A program is one thread, so the order of its readings is its own and is the same when it is started again.

func OpenObserved

func OpenObserved(dir string) (*Observed, error)

OpenObserved opens the record of readings for the run in dir, reading what an earlier run of it took.

func (*Observed) Close

func (o *Observed) Close() error

func (*Observed) Nanotime

func (o *Observed) Nanotime() int64

Nanotime is the monotonic clock a program reads. After the recorded readings run out it carries on from the last of them, so it never goes backwards.

func (*Observed) Read

func (o *Observed) Read(p []byte) (int, error)

Read gives random bytes.

func (*Observed) Replaying

func (o *Observed) Replaying() bool

Replaying reports whether recorded readings remain to be given.

func (*Observed) Sync

func (o *Observed) Sync() error

Sync puts the readings on disk. It is called before a request is recorded as begun, so a request never outlives the readings it was computed from.

func (*Observed) Walltime

func (o *Observed) Walltime() (int64, int32)

Walltime is the clock a program reads.

type Record

type Record struct {
	Phase   string          `json:"phase"` // header, begin, end, finish
	ID      string          `json:"id,omitempty"`
	Method  string          `json:"method,omitempty"`
	Digest  string          `json:"digest,omitempty"` // of the request, so a changed request is noticed
	Effect  string          `json:"effect,omitempty"`
	Reply   json.RawMessage `json:"reply,omitempty"`
	Blob    string          `json:"blob,omitempty"`
	Bytes   int             `json:"bytes,omitempty"`
	At      time.Time       `json:"at"`
	Header  *Header         `json:"header,omitempty"`
	Outcome string          `json:"outcome,omitempty"`
	Call    *CallIdentity   `json:"call,omitempty"`
}

Record is one line of the index.

type Snapshot

type Snapshot struct {
	Header    Header       `json:"header"`
	State     InspectState `json:"state"`
	Outcome   string       `json:"outcome,omitempty"`
	Events    []Event      `json:"events"`
	Truncated bool         `json:"truncated"`
}

Snapshot is a bounded, read-only view of a TAP run.

func Inspect

func Inspect(root, runID string, limit int) (Snapshot, error)

Inspect reads a run without opening it for resume or taking its lease. A cut-short final line is ignored because it cannot describe a durable event. The returned view is bounded; Truncated reports that events were omitted or that the journal exceeded the inspection byte limit.

type State

type State int

State is what is known about one request when a run is resumed.

const (
	Unseen      State = iota // never asked
	Finished                 // asked and answered: reply from the record
	Interrupted              // asked, and the run stopped before an answer was recorded
)

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL