Documentation
¶
Overview ¶
Package partsync replicates flushed, immutable parts between nodes whose backends are per-node private (shared-nothing cluster mode). Head replication (cluster/replica) protects only the unflushed window; over a shared object store the flushed parts need no replication at all — but with a local-disk backend a peer cannot see them, so a replica instead *mirrors* the owner's backend objects over HTTP: it picks the newest peer copy of the engine's bucket index, copies the part objects it lacks, and installs the index last. The engine then reconciles via its ordinary LoadParts/RefreshReplica path — partsync moves backend objects, never engine state.
Ordering makes a crashed sync harmless: within a part the manifest is copied after the part's other objects, and the bucket index is written after every part, so the local index only ever references fully-copied parts (the same commit-point discipline flush uses). A half-copied part is an unreferenced orphan retried on the next pass.
Objects are content-immutable except the bucket index and the head-identity objects (series.bin / streams.bin), so a plain presence diff drives the copy; the mutable objects are re-fetched whenever the index changed. Every fetched object is verified against the sender's checksum. Local objects the peer no longer has are pruned only after being absent for two consecutive passes, giving in-flight readers a full maintenance cycle to drain (quarantine-by-delay rather than immediate delete).
Index ¶
Constants ¶
const ( // ListPath is the HTTP path serving a node's backend key listing under a prefix. ListPath = "/internal/parts/list" // ObjectPath is the HTTP path serving one backend object verbatim. ObjectPath = "/internal/parts/object" // NotifyPath is the HTTP path an owner POSTs to after a flush/merge so a secondary mirrors // immediately instead of waiting for its next maintenance tick. Advisory and best-effort — // the periodic pull remains the anti-entropy source of truth. NotifyPath = "/internal/parts/notify" )
Variables ¶
var ErrNotExist = errors.New("partsync: object does not exist on peer")
ErrNotExist is returned by Client.Fetch for a key the peer does not have.
Functions ¶
func ListHandler ¶
ListHandler serves the backend keys under the "prefix" query parameter, framed as a uvarint count followed by uvarint-length-prefixed keys.
func ObjectHandler ¶
ObjectHandler serves one backend object (the "key" query parameter) verbatim, with its xxh3 checksum in a response header. A missing key is a 404.
func ValidKey ¶
ValidKey reports whether a remotely-supplied key or prefix is safe to hand to a backend: relative, slash-delimited, and free of traversal or NUL. Backends validate again (the file backend keeps every path under its root); this check is defense-in-depth at every network boundary — the serving handlers reject hostile request parameters, and the syncer rejects hostile key names a compromised peer could return.
Types ¶
type Client ¶
type Client struct {
// HTTP is the client used for peer requests; nil uses [http.DefaultClient]. Pass one with
// timeouts in production (the cluster's tuned client).
HTTP *http.Client
}
Client fetches backend listings and objects from a peer's partsync endpoints.
func (*Client) Fetch ¶
Fetch returns one object from the peer, verified against the sender's checksum. A key the peer lacks returns ErrNotExist.
type KeepFunc ¶
KeepFunc decides whether a peer-listed object key should be mirrored into this node. It lets the caller narrow a pull to a subset of a part's objects — erasure coding passes one that keeps only this node's own shard slot (plus every non-shard object), so a replica stores one shard per part instead of the whole k+m set. A nil KeepFunc keeps everything.
type Stats ¶
type Stats struct {
// Synced is true when a newer peer copy was found and mirrored (Copied may still be zero
// if only the mutable objects changed).
Synced bool
// Copied is the number of objects fetched from the peer.
Copied int
// CopiedBytes is the total size of the fetched objects.
CopiedBytes int64
// Pruned is the number of stale local objects deleted.
Pruned int
}
Stats reports what one Syncer.Sync pass did.
type Syncer ¶
type Syncer struct {
// contains filtered or unexported fields
}
Syncer mirrors engine prefixes of a per-node private backend from cluster peers. Safe for concurrent use across distinct prefixes; per-prefix passes are expected to be serial (the maintenance loop runs one task per engine).
func (*Syncer) Sync ¶
func (s *Syncer) Sync(ctx context.Context, enginePrefix string, peers []string, strict bool, keep KeepFunc) (Stats, error)
Sync mirrors one engine prefix (e.g. "default/metrics") from the newest of peers into the local backend. In strict mode (an owner backfilling before it compacts) the peer copy must be strictly newer than the local one; otherwise (a replica mirroring its owner) any differing peer copy at least as new is installed. Unreachable peers are skipped; having no usable peer index is a no-op, not an error.
type Totals ¶
type Totals struct {
// Passes is every Sync attempt, including no-ops (no usable peer index, nothing newer) —
// the "is the sync loop running?" liveness probe.
Passes int64
// Mirrored is the passes that installed a newer peer copy.
Mirrored int64
// Copied is the objects fetched from peers, CopiedBytes their total size.
Copied int64
CopiedBytes int64
// Pruned is the stale local objects deleted (after the quarantine delay).
Pruned int64
// Errors is the passes that failed part-way (retried by the next maintenance tick).
Errors int64
// LastSyncUnixNano is the wall-clock completion time of the most recent mirroring pass
// (zero until one succeeds) — the "is replication current?" staleness probe.
LastSyncUnixNano int64
}
Totals is a Syncer's cumulative activity across every prefix and pass, for the operator stats surface (storage.StoreStats). Counters only — reading it does no I/O.
Source Files