Documentation
¶
Overview ¶
Package mutationlog is the single source of truth for the stream of graph mutations that flows through Lantern's leaderless full-replica replication design.
The log is an in-memory, append-only ring buffer. Each Entry carries a monotonically increasing [Seq] (assigned atomically by Log.Append), an hlc.Timestamp supplied by the caller, and an opaque MutationOp payload. Keeping the payload as an interface keeps this package proto-free so it can be reused both by internal replication (#181, #184) and by external CDC (#180) without a circular dependency on pb/.
A bounded number of entries are retained for replay. When a subscriber requests a Seq older than Log.FirstSeq the channel is closed and ErrGapped is reported via the cancel func's returned error; the caller is then expected to snapshot and resubscribe (RFC §7). Live subscribers share a bounded outbound channel — slow consumers exert back-pressure on Log.Append rather than allowing the log to drift forward without them.
A WAL hook lets a future durability layer (RFC D1) intercept appends before they fan out. The default NopWAL is a no-op, matching today's in-memory-only behaviour.
This package depends only on the standard library and core/hlc.
Index ¶
- Variables
- type Entry
- type Log
- func (l *Log) Append(op MutationOp, ts hlc.Timestamp) (Entry, error)
- func (l *Log) Cap() int
- func (l *Log) Close() error
- func (l *Log) Evicted() uint64
- func (l *Log) FirstSeq() (uint64, bool)
- func (l *Log) LastSeq() (uint64, bool)
- func (l *Log) Len() int
- func (l *Log) Subscribe(fromSeq uint64) (<-chan Entry, func() error, error)
- type MutationOp
- type NopWAL
- type Options
- type WAL
Constants ¶
This section is empty.
Variables ¶
var ErrClosed = errors.New("mutationlog: log is closed")
ErrClosed is returned by Log.Append and Log.Subscribe after Log.Close has been called.
var ErrGapped = errors.New("mutationlog: requested sequence has been evicted")
ErrGapped is returned to a subscriber when its requested fromSeq has already been evicted from the ring buffer. Callers must snapshot and resubscribe from a fresh sequence number.
Functions ¶
This section is empty.
Types ¶
type Entry ¶
type Entry struct {
// Seq is the log-local position of the entry. It increases by exactly
// one with each successful [Log.Append] call within a single [Log].
Seq uint64
// HLC is the Hybrid Logical Clock stamp supplied by the caller at
// append time. Different origins may produce entries whose HLC ordering
// disagrees with the local Seq ordering; that is by design.
HLC hlc.Timestamp
// Op is the opaque mutation payload.
Op MutationOp
}
Entry is a single durable record in the log.
type Log ¶
type Log struct {
// contains filtered or unexported fields
}
Log is an append-only, bounded, in-memory mutation log.
The zero value is not ready for use; construct with New.
func (*Log) Append ¶
Append assigns the next sequence number to op, persists the entry through the WAL hook, stores it in the ring buffer, and fans it out to every live subscriber. The returned Entry carries the assigned Seq.
Append is safe for concurrent use; calls are serialised so Seq strictly increases by one for each successful return.
func (*Log) Close ¶
Close stops accepting appends and closes every subscriber channel. Calling Close more than once returns nil.
func (*Log) Evicted ¶
Evicted returns the cumulative count of entries dropped from the ring buffer because Append at full capacity displaced the oldest entry. The counter is monotonic for the lifetime of the Log.
func (*Log) FirstSeq ¶
FirstSeq returns the lowest Seq still resident in the ring buffer. It returns 0 (and false) when the log is empty.
func (*Log) LastSeq ¶
LastSeq returns the Seq of the most recently appended entry. It returns 0 (and false) when the log is empty.
func (*Log) Len ¶
Len returns the number of entries currently resident in the ring buffer. 0 <= Len() <= Cap().
func (*Log) Subscribe ¶
Subscribe returns a channel that receives entries starting at fromSeq. Any entries with Seq >= fromSeq still resident in the ring buffer are replayed immediately; subsequent entries are delivered live.
If fromSeq is older than Log.FirstSeq the returned channel is closed immediately and the cancel func reports ErrGapped when invoked.
The returned cancel func unregisters the subscriber and closes the channel. It is safe to call cancel more than once; subsequent calls return nil.
Slow subscribers exert back-pressure: each subscriber has a bounded outbound channel (see Options.SubscriberBuffer). When that channel fills, the subscriber is marked gapped, its channel is closed, and the caller must snapshot and resubscribe from a fresh Seq.
type MutationOp ¶
type MutationOp any
MutationOp is an opaque payload carried by an Entry. The mutationlog package never inspects it; callers (proto encoders, applicators, CDC emitters) own the concrete types.
type Options ¶
type Options struct {
// Capacity is the maximum number of entries retained in the ring buffer
// for replay. Must be > 0; defaults to 1024 when zero.
Capacity int
// SubscriberBuffer is the per-subscriber outbound channel size. Smaller
// values increase back-pressure sensitivity; defaults to 512 when zero.
//
// At sustained write rates a too-small buffer turns transient scheduling
// jitter into permanent gap-closes: the fan-out path uses a
// non-blocking send and closes the subscriber on a full channel
// (see [Log.Append]). 512 gives ~256 ms of headroom at 2k writes/s,
// which is well above typical scheduler stalls on a loaded host.
SubscriberBuffer int
// WAL receives every appended entry before it fans out to subscribers.
// Nil defaults to [NopWAL].
WAL WAL
}
Options configures a Log.
A zero Options is valid: Capacity defaults to 1024 entries and WAL defaults to NopWAL. SubscriberBuffer defaults to 512.
type WAL ¶
WAL is the hook surface for a future write-ahead-log implementation. Implementations must be safe for concurrent use. Log.Append calls WAL.Write while holding the log mutex, so implementations should keep the call cheap (a buffered write is fine; a synchronous fsync is not).
Source Files