Documentation
¶
Overview ¶
Package journal implements beardrive's append-only operation log.
Every change to a volume is recorded as an Op in a per-device JSONL journal. Journals are append-only and each device only ever writes its own journal, so syncing is conflict-free at the transport level: a sync uploads your journal and downloads everyone else's. The merged view of a volume is a deterministic replay of the union of all ops ordered by (lamport, time, device, seq) — every device converges to the same state.
Index ¶
Constants ¶
const ( KindPut = "put" KindDelete = "delete" )
Variables ¶
This section is empty.
Functions ¶
func Less ¶
Less defines the total order used to replay ops from many devices.
The trailing comparisons are what make it a TOTAL order rather than a pre-order: (lamport, time, device, seq) is forgeable — a peer may push two ops sharing all four — and Sort is only stable, so without them the winner of a tie would be whatever order the caller happened to collect the ops in. Everything Replay reads is compared here, so any two ops that still tie fold to the same state and the invariant holds on the ops themselves.
func Replay ¶
Replay folds a set of ops (from any number of devices) into the resulting volume state. Last writer wins per path under the total order.
func SafePath ¶ added in v0.15.0
SafePath reports whether p is a path an Op may name. It is THE rule, in one place: an op's path is arbitrary JSON off a peer's journal, and it is joined onto a working folder on every device, stored as a metadata row on the hub and rendered as a tree entry in the browser.
It used to be spelled three times — syncer.unsafeRel (device), the core of webapp.cleanUploadPath (browser door) and templates.SafePath (seeding) — and they disagreed: unsafeRel, the rule the /store/* journal door relies on, had no control-character clause, so a NUL-bearing path the browser door answered 400 to was journaled and handed to every device. Three spellings of one rule is how these holes happen; callers add their OWN extra rules (reserved dirs, on-disk boundary) on top of this one, never a second copy of it.
Refused, never normalized: normalizing would land two different journal paths on one file.
func SafeText ¶ added in v0.15.0
SafeText reports whether s is free of the characters that make one rendered row lie about what it says. It is SafePath's character rule, split out because the path is not the only peer-written string the hub serves to a browser and a terminal — an op's Note is rendered next to it in every history row and in `bdrive log`.
Three families, all of which render as nothing or as something else:
- C0 and DEL. Byte-wise on purpose (see lossy): in UTF-8 no continuation byte is < 0x80, so a byte in this range is always a real control character. NUL is also a value the metadata backends disagree about.
- C1 (U+0080..U+009F), which every C0 filter misses and which is CSI and friends to any terminal.
- the bidi format controls (Trojan Source, CVE-2021-42574). "invoicegnp.exe" renders as "invoiceexe.png" in every file listing, tree node, breadcrumb and history row — and downloads as an .exe.
This repo already refuses all three in a project NAME (webapp trimName, "the bidi overrides that reorder a rendered row") and strips them on the way to a terminal (cmd/bdrive safeField). The path and the note, which reach further than either, checked none of them.
Types ¶
type Op ¶
type Op struct {
Seq int64 `json:"seq"` // per-device sequence number, 1-based
Lamport int64 `json:"lamport"` // logical clock for cross-device ordering
Time time.Time `json:"time"`
Device string `json:"device"`
DeviceName string `json:"device_name,omitempty"`
Author string `json:"author,omitempty"` // OS/git identity (offline fallback)
User string `json:"user,omitempty"` // signed-in account email
UserName string `json:"user_name,omitempty"` // signed-in account display name
Kind string `json:"kind"` // "put" or "delete"
Path string `json:"path"` // slash-separated, relative to volume root
Blob string `json:"blob,omitempty"` // sha256 hex of content (put only)
Size int64 `json:"size,omitempty"`
Mode uint32 `json:"mode,omitempty"` // permission bits
Note string `json:"note,omitempty"` // e.g. "conflict copy of <path>"
// Session is the agent session this op was committed during, set ONLY by
// the agent sync hook (`bdrive sync --hook`). Display/join only — never an
// input to Less or Replay, exactly like Mtime below, so replay stays
// deterministic and ops written before this field existed simply carry "".
//
// It exists because Note is user-settable (`bdrive sync --note`): joining
// a run's reads to its writes on the note string would let any member with
// write access forge a note that collides with a teammate's session and
// hang their reads off it. This field is the un-forgeable half of that
// pair, so the join reads it and never the note.
Session string `json:"session,omitempty"`
// Mtime is when the file was last written, as opposed to Time, which is
// when the op was committed. Display only — never an input to Less or
// Replay, since it comes from the filesystem and can be anything.
Mtime time.Time `json:"mtime,omitzero"` // put only
}
Op is a single journaled file operation.
func Parse ¶
Parse decodes a JSONL journal.
A line that does not decode is skipped, never fatal. Append is a plain O_APPEND write (the one state file that cannot be written atomically — it only ever grows), so a crash or a full disk leaves a torn final line; and a peer's journal is bytes someone else chose. All-or-nothing parsing turned either of those into "every op this device ever committed is unreadable", with no recovery path in the CLI. The ops that did decode are still the device's history, and every reader drops the same lines from the same bytes, so replay stays in agreement.
Source Files