Documentation
¶
Overview ¶
Package syncer drives a volume's sync cycle:
re-derive the clock → scan → commit local ops → pull peer journals → preserve conflicts → materialize merged state → push blobs + own journal
The cycle opens by folding every op already in the volume's journal dir through absorbLamport. Nothing else re-derives the clock — a pull absorbs only the new tail it fetched, and a fully-downloaded peer journal yields none — so a device whose stored clock fell below the journals beside it would stay there forever, and every op it wrote would lose replay.
Scanning always happens before pulling, so local edits are committed to the journal (and their content captured in the blob store) before any remote state can overwrite the working folder. Concurrent edits resolve deterministically last-writer-wins; the losing local version is preserved as a "<name>.bdrive-conflict-<device>-<time>" file that syncs like any other.
The exception is a device's FIRST cycle on a volume, which is a join rather than an edit: a local file at a path the project already holds is adopted — the project's version wins everywhere and no conflict copy is made — while the local content is still journaled (below every clock the project can hold) so history keeps it. See step 1b in Cycle.
Index ¶
- Constants
- func CommitTime(op journal.Op) time.Time
- func DisplayTime(op journal.Op) time.Time
- func Drift(folder string, include []string, accepted string, ...) (added, modified, removed int, err error)
- func EscapeIgnore(rel string) string
- func LogEntries(st *store.Store, pathFilter string, limit int) ([]journal.Op, error)
- func Measure(folder string, include []string) (files int, bytes int64, err error)
- func NotSyncedFiles(notSynced []Entry) int
- func SortForDisplay(ops []journal.Op)
- func SyncedFiles(folder string, include []string, accepted string) ([]string, error)
- type Entry
- type Filter
- type Progress
- type Result
- type Session
Constants ¶
const IgnoreFile = ".bdriveignore"
IgnoreFile is the per-folder opt-out list at the mount root. It uses a gitignore-style syntax and, unlike the .bdrive settings file, syncs like any other file so every device shares the same rules.
Variables ¶
This section is empty.
Functions ¶
func CommitTime ¶ added in v0.16.0
CommitTime is when a change entered the project, which is the question `bdrive log` answers. It is not DisplayTime: `mv` preserves mtime, so the put half of a rename carries the original file's write time and sorts away from the delete half of the same rename — a file that appeared seconds ago lands below the fold.
Same clamp as DisplayTime, for the same reason: Op.Time is a peer's JSON, and the one clock a peer does not own is this machine's. An op we cannot date sorts last rather than first, which is the direction that cannot be aimed.
func DisplayTime ¶ added in v0.14.0
DisplayTime is the timestamp to show a human for an op: when the file was written if we know it, otherwise when the op was committed. Ops written before Op.Mtime existed, and deletes (no file left to stat), fall back.
func Drift ¶ added in v0.16.0
func Drift(folder string, include []string, accepted string, cache map[string]store.CachedFile) (added, modified, removed int, err error)
Drift reports what the working folder holds that the state cache does not yet know about: files added, files whose size or mtime moved, and cached paths gone from disk.
It is a pure read, the same contract as Explain and SyncedFiles: no Session, no volume lock, no network, no writes — and it neither stores blobs nor mints ops nor touches the cache it is handed. `bdrive status` is what someone runs when sync is stuck; a version of it that scanned-and-committed would change the thing it was asked to describe.
The comparison is the scan's own cheap change detection (size + mtime against store.CachedFile), reached through walkFolder so the file verdict is the cycle's verdict and not a second copy of it.
accepted is the ignore text this device has accepted (store.SyncState's IgnoreAccepted; "" when there is none), for the reason Explain documents: the scan applies Filter.SkipUp, and a drift count that omitted it would disagree with the very next cycle.
func EscapeIgnore ¶ added in v0.15.0
EscapeIgnore turns a literal project-relative path into a rule line that matches that path and nothing else. It is compile's inverse and lives beside it on purpose: a caller that spells the escaping itself is a second definition of the dialect, which is how `bdrive forget <name>` came to write a glob, a negation or a comment depending on what a teammate had named a file.
func LogEntries ¶
LogEntries returns the volume history, newest first.
func Measure ¶ added in v0.15.0
Measure reports what a first sync of this folder would actually upload: the number of files and their total bytes, after the same filter the cycle uses. It exists so `bdrive init` can warn about a folder nobody meant to sync (a home directory, a video library, a checkout whose .bdriveignore doesn't cover its build output) BEFORE the first push rather than after.
It is deliberately filter-aware: a 40 GB repo whose bulk is node_modules measures as the few MB that really sync, so the warning fires on the cases that are actually expensive and stays quiet on the ones the starter rules already handle.
Unreadable entries are skipped rather than failing — this is advice, and a permission error in one subtree must never block init.
func NotSyncedFiles ¶ added in v0.12.0
NotSyncedFiles is how many files the not-synced list stands for: collapsed directories count their whole subtree, nested mounts count zero because they do sync — through their own project.
func SortForDisplay ¶ added in v0.14.0
SortForDisplay orders ops newest-first by CommitTime — when the change entered the project — so the two halves of a rename sit together and an old file added today sorts by when it arrived. DisplayTime breaks ties and is load-bearing: everything from one scan shares a commit second, and the files' own write times are the only thing that orders it inside that second. Ties beyond that fall back to reversed journal.Less to stay deterministic. This is deliberately NOT the replay order: LogEntries keeps returning causal order because bdrive restore walks it to find a file's previous version.
func SyncedFiles ¶ added in v0.15.0
SyncedFiles lists the mount-relative paths that sync, in walk order. It is the same pure read as Explain — no Session, no volume lock, no network — but without Explain's not-synced accounting, which calls countFiles on every pruned directory: a `bdrive grep` in a repo with node_modules/ would walk node_modules/ in full just to produce a count it throws away.
accepted is the ignore text this device has accepted (store.SyncState's IgnoreAccepted; "" when there is none), for the reason Explain documents: the walk applies Filter.SkipUp, and omitting it would list a file a peer's `!` rule points at that the cycle will not actually send.
Unreadable entries are skipped rather than failing, as everywhere else in this walk.
Types ¶
type Entry ¶ added in v0.12.0
type Entry struct {
Path string
Files int
Nested bool // syncs through its own project — not excluded
}
Entry is one line of the not-synced list. Path ends in "/" when a whole directory collapsed to a single line; Files is how many files it holds.
func Explain ¶ added in v0.12.0
func Explain(folder string, include []string, accepted string) (synced []string, notSynced []Entry, err error)
Explain reports what the sync cycle would and would not send for a folder. It is a pure read: no Session, no volume lock, no network, no writes — the answer comes from the same walk the cycle itself uses, so it cannot drift.
accepted is the ignore text this device has accepted (store.SyncState's IgnoreAccepted; "" when there is none). It is a parameter rather than something loadFilter finds for itself because the answer lives in the volume store and this function deliberately does not open one — but it has to be passed, because the scan applies it (Filter.SkipUp) and an explanation that omits it would report a peer's `!` rule as syncing a file the cycle will not send. That drift is the one thing this function exists not to have.
type Filter ¶
type Filter struct {
// contains filtered or unexported fields
}
Filter decides which paths sync. A path syncs when it is not ignored and, if an include list is set, matches at least one include pattern.
Pattern syntax (a practical gitignore subset): one pattern per line, blank lines and #-comments skipped, `!` re-includes, a trailing `/` matches directories only, a `/` anywhere else anchors the pattern to the mount root (otherwise it matches at any depth), `*` matches within a path segment, `**` across segments, `?` a single character.
func LoadFilter ¶ added in v0.4.0
LoadFilter builds the filter for a folder from its .bdriveignore (if any) plus the include list from the .bdrive settings file — the exact rules the sync cycle applies, for callers outside the cycle (e.g. `bdrive read-log` deciding whether an agent-read path is part of the project).
func (*Filter) AcceptRules ¶ added in v0.15.0
AcceptRules installs the rules this device has accepted as the floor on what may be uploaded. Passing "" installs no floor.
func (*Filter) Negated ¶ added in v0.12.0
Negated reports whether any `!` rule is in play. Scope narrowing is written as negation rules, so this is how callers tell "these rules exclude a few things" from "these rules exclude everything but a few things" — the difference between a safe prune and a destructive one.
func (*Filter) PruneDir ¶
PruneDir reports whether a whole directory can be skipped during the scan walk. Pruning is conservative: never with `!` rules (a child could be re-included) or an include list (a deep child could match).
func (*Filter) SkipUp ¶ added in v0.15.0
SkipUp is Skip for the UPLOAD direction: a path syncs out of this machine only if neither the live rules nor the rules this device has ACCEPTED exclude it.
.bdriveignore syncs, deliberately — scope is team-wide, and round 4 made `sync --prune` refuse when `!` rules are present precisely because of that. That reasoning covered DELETION. Nobody asked what a pulled negation does to the SCAN, and the answer was that one member adding `!.env` to the shared file uploaded every other member's local .env on their next cycle: a file that had never been shared, with no prompt and no local change. The runbook recommends `bdrive init . --only docs,notes` for a repository, which puts the WHOLE repo under the mount with only this synced, teammate-writable file holding the rest of it back.
The asymmetry is the fix, not a ban on shared rules:
- Pulled rules that NARROW apply immediately, both directions. A teammate excluding something takes effect everywhere, as designed.
- Pulled rules that WIDEN apply to materialize (a peer's scope decision still delivers their files down) but not to scan. Widening what leaves your disk is a local decision.
- A device ACCEPTS the current rules the moment it authors them itself — `bdrive init --only`, `bdrive scope add/rm`, or editing the file by hand. That is the acknowledgement, and it is the same act in every case: this machine's user changed the shared rules.
A device joining a project has accepted nothing, so the floor excludes nothing and the pulled rules stand on their own — team-wide scope still works for a new member, which is what a blanket "ignore pulled negations" would have broken.
type Progress ¶ added in v0.3.1
Progress reports upload progress during a cycle's push phase, so the CLI can draw a bar. Total/TotalBytes are set once when the push starts; Done/Bytes climb as blobs finish. Nil OnProgress means no reporting (the daemon).
type Result ¶
type Result struct {
LocalOps int // local changes committed to the journal
PulledOps int // ops received from other devices
Conflicts int // conflict copies created
// Merged counts concurrent edits this cycle resolved by a three-way merge
// instead of forking a conflict copy. Kept apart from Conflicts because
// they mean opposite things to the person reading the line: a merge is
// work the machine finished, a conflict is work it handed back.
Merged int
// Adopted counts paths where this folder's own content gave way to the
// project's on join (step 1b). Not a conflict and not an error — the
// superseded content stays in history — but the user asked for none of it,
// so it is worth a line.
Adopted int
Pruned int // paths removed from the hub by --prune (kept on disk)
Materialized int // files written/removed in the working folder
Pushed bool // own journal/blobs uploaded
// Backfilling reports that this cycle's backfill landed content and left
// more waiting for a later batch, so the next remote pass should come
// soon. It stays false while the backfill lands nothing, which is what
// keeps blobs the hub cannot serve from holding a daemon at full cadence.
Backfilling bool
// Inbound names the paths this cycle materialized on a peer's behalf —
// the same events the inbound spool queues, carried out of the cycle that
// made them so the post_sync hook can hand them to a local command. It
// does NOT replace the spool: see the comment in internal/store/inbound.go
// for why the two coexist.
Inbound []store.InboundEvent
// Offline reports that the remote leg of this cycle had a problem worth
// telling the user about — usually "unreachable", but also "the hub served
// bytes that are not their content address", which is the only signal that
// case ever produces. It is a REPORT, not a gate: a content-level problem
// with one object still lets this device push its own work (Pushed may be
// true alongside it), because otherwise one peer's journal line decides
// whether anyone else's edits ever leave their machine.
// Reverted names paths under a read-only folder whose local edit this
// cycle put back from the hub's version, keeping the user's own copy
// beside it. Distinct from Conflicts: no op was written and no peer will
// ever see the change, so it is not a conflict, it is a refusal.
Reverted []string
Offline bool
OfflineErr error
ReadOnly bool // the hub refused our push: pull-only from here
NoAccess bool // the hub refused our pull: sync paused, nothing touched
AccessErr error
}
Result summarizes one sync cycle.
Offline, ReadOnly, and NoAccess are three different answers and must not be conflated: offline means the hub could not be reached and everything should be retried; ReadOnly means it refused our push (we keep pulling, local ops stay journaled and unpushed); NoAccess means it refused our pull too, so the cycle does nothing at all and leaves the working folder alone. Regaining access self-heals on a later cycle with no manual step.
type Session ¶
type Session struct {
Folder string
MountID string // the stable project mount id from .bdrive/config.json
ExpectedRemote string
Store *store.Store
Device config.Device
// Account is the signed-in user (from `bdrive login`); ops carry it so
// history shows who changed what. Zero on offline/no-auth setups —
// Device.Author remains the fallback identity.
Account config.Settings
// Note, when set, is stamped into every op this session commits — session
// context like "claude-code session <id>". Empty means fall back to the
// store's persisted mount note (store.LoadNoteForMount), which lets a one-shot
// `bdrive sync --note` leave context that the daemon's later scans also
// stamp. Conflict-copy ops keep their own explanatory note.
Note string
// SessionID is the agent session every op this cycle commits is stamped
// with (journal.Op.Session). Set only by `bdrive sync --hook`, and
// deliberately NOT persisted the way Note is (store.SaveNote): the note
// is context that outlives the hook turn, the session id is an identity
// that must not be attached to changes the daemon commits on its own
// later. So a daemon scan after the hook turn carries the note and no
// session — the asymmetry is intended.
SessionID string
// Prune makes this cycle reconcile the hub against the shared ignore
// rules: every path the remote still holds that .bdriveignore (or a
// builtin never-sync rule) now excludes is journaled as a delete, so it
// leaves the hub while staying on disk on every device. Off by default —
// plain `bdrive sync` and the daemon never set it, because pruning must
// be a deliberate act, never a side effect of editing .bdriveignore.
Prune bool
Backend remote.Backend // nil = work offline
// OnProgress, when set, is called during push with upload progress. It may
// be invoked concurrently from upload workers, so it must be safe to call
// from multiple goroutines.
OnProgress func(Progress)
// contains filtered or unexported fields
}
Session ties a working folder to its volume store and (optionally) remote.
func (*Session) Cycle ¶
Cycle runs one full scan/sync/materialize pass under the volume lock, then fires the folder's post_sync hook — after the lock is released, so a hook that itself runs a bdrive command starts working immediately instead of blocking on the flock the cycle that spawned it is still holding (a long push can hold it for a while). Splitting the body out is what makes that ordering a property of the code: no call site has to remember it.
func (*Session) Restore ¶ added in v0.12.0
Restore writes the historical version sha of path back into the working folder as an ordinary local edit. The next Cycle journals it like any other change — nothing here appends to a journal, and no journal is ever rewritten. Restoring is exactly the edit a human could have made by hand, which is why the sync engine needs no new write path for it.
It does not take the volume flock: Cycle does, and holding it here would deadlock the caller that runs both.
Source Files