Documentation
¶
Overview ¶
Package dlog provides double-buffered write-ahead logging.
Two alternating log files (logA/logB) enable atomic switching during snapshot creation. Patches are appended to the active log.
Related Packages ¶
- github.com/signadot/tony-format/go-tony/system/logd/storage - Storage layer
Index ¶
- type DLog
- func (dl *DLog) AppendEntry(entry *Entry) (logPosition int64, logFile LogFileID, err error)
- func (dl *DLog) AppendToInactive(entry *Entry) (logPosition int64, logFile LogFileID, err error)
- func (dl *DLog) Close() error
- func (dl *DLog) GetActiveLog() LogFileID
- func (dl *DLog) GetInactiveLog() LogFileID
- func (dl *DLog) Iterator() (*DLogIter, error)
- func (dl *DLog) NewSnapshotWriter(commit int64, timestamp string) (*SnapshotWriter, error)
- func (dl *DLog) OpenReaderAt(logFile LogFileID, position int64) (io.ReadSeekCloser, error)
- func (dl *DLog) ReadEntryAt(logFile LogFileID, position int64) (*Entry, error)
- func (dl *DLog) SwitchActive() error
- type DLogFile
- func (dlf *DLogFile) AppendEntry(entry *Entry) (position int64, err error)
- func (dlf *DLogFile) Close() error
- func (dlf *DLogFile) OpenReaderAt(position int64) (io.ReadSeekCloser, error)
- func (dlf *DLogFile) Position() int64
- func (dlf *DLogFile) ReadEntryAt(position int64) (*Entry, error)
- func (dlf *DLogFile) Size() (int64, error)
- type DLogIter
- type Entry
- type LogFileID
- type SnapshotWriter
- func (sw *SnapshotWriter) Abandon()
- func (sw *SnapshotWriter) Close() error
- func (sw *SnapshotWriter) EntryPosition() int64
- func (sw *SnapshotWriter) LogFileID() LogFileID
- func (sw *SnapshotWriter) Seek(offset int64, whence int) (int64, error)
- func (sw *SnapshotWriter) Write(p []byte) (n int, err error)
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type DLog ¶
type DLog struct {
// contains filtered or unexported fields
}
DLog manages the logA/logB double-buffered log files. Follows the double-buffering pattern where one log is active for writes and the other is inactive (being snapshotted or ready for compaction).
func NewDLog ¶
NewDLog creates a new double-buffered log manager. Initializes both logA and logB files, determines active log from state. Defaults to LogA as active if no state exists.
func (*DLog) AppendEntry ¶
AppendEntry appends an Entry to the active log file. The Entry.Commit is provided by the caller (from seq.Seq.NextCommit()). Returns the log position and which log file (A or B) it was written to. Does NOT automatically switch active log - that's handled by the caller when compaction boundaries are reached.
func (*DLog) AppendToInactive ¶ added in v0.0.15
AppendToInactive appends an entry to the inactive log file. Returns the position and log file ID where the entry was written.
func (*DLog) GetActiveLog ¶
GetActiveLog returns the currently active log file ID.
func (*DLog) GetInactiveLog ¶
GetInactiveLog returns the currently inactive log file ID.
func (*DLog) Iterator ¶
Iterator creates an iterator for reading entries from both log files in commit order. Starts at position 0 for both files. Note: Currently uses non-streaming reads. Streaming support can be added later.
func (*DLog) NewSnapshotWriter ¶ added in v0.0.15
func (dl *DLog) NewSnapshotWriter(commit int64, timestamp string) (*SnapshotWriter, error)
NewSnapshotWriter creates a writer for building a snapshot in the inactive log. The caller should create a snap.Builder with this writer, feed events to it, close the builder, then close this writer to finalize the Entry.
func (*DLog) OpenReaderAt ¶ added in v0.0.13
OpenReaderAt opens a reader at the specified position in the log file. This is used to read inline snapshot data stored in the log. All seeks in the returned reader are relative to position (position becomes offset 0). The returned reader must be closed when done. logFile must be "A" or "B".
func (*DLog) ReadEntryAt ¶
ReadEntryAt reads an Entry from the specified log file at the given position. logFile must be "A" or "B".
func (*DLog) SwitchActive ¶
SwitchActive switches the active log (A ↔ B). Called by the caller when compaction boundaries are reached.
type DLogFile ¶
type DLogFile struct {
// contains filtered or unexported fields
}
DLogFile represents a single log file with its operations
func (*DLogFile) AppendEntry ¶
AppendEntry appends an Entry to this log file. Format: [4 bytes: uint32 length (big-endian)][entry data in Tony wire format] Returns the byte position where the entry was written.
func (*DLogFile) OpenReaderAt ¶ added in v0.0.13
func (dlf *DLogFile) OpenReaderAt(position int64) (io.ReadSeekCloser, error)
OpenReaderAt opens a new file handle and returns a reader scoped to the section starting at position. All seeks in the returned reader are relative to position (position becomes offset 0). This is used to read inline snapshot data. The returned reader must be closed when done.
func (*DLogFile) ReadEntryAt ¶
ReadEntryAt reads an Entry from the specified byte position. Reads length prefix, then entry data, deserializes to Entry.
type DLogIter ¶
type DLogIter struct {
// contains filtered or unexported fields
}
DLogIter provides sequential iteration over log entries using streaming reads. Uses streaming parsing to avoid loading entire entries into memory. Iterates over both logA and logB, switching between them based on commit order.
type Entry ¶
type Entry struct {
Commit int64 // Commit number (set when appended to log)
Timestamp string // RFC3339 timestamp
Patch *ir.Node // Root patch/diff (always at root, empty kinded path "")
TxSource *tx.State // Transaction state (for transaction entries)
SnapPos *int64 // Snapshot position (for snapshot entries)
LastCommit *int64 // Last commit before compaction (for compaction entries)
}
Entry represents a log entry written to logA/logB. This structure supports 4 types of entries:
- Plain patch: Patch set, TxSource/SnapPos nil *LastCommit=Commit-1
- Transaction: Patch and TxSource set, SnapPos nil *LastCommit=Commit-1
- Snapshot: SnapPos set to point to log where state exists, TxSource nil, LastCommit nil
- Compaction: Patch set, LastCommit set Commit-*LastCommit > 1, TxSource nil, SnapPos nil
func NewEntry ¶
func NewEntry(state *tx.State, mergedPatch *ir.Node, commit int64, timestamp string, lastCommit int64) *Entry
NewEntry creates a dlog.Entry for a transaction commit. The entry contains the merged patch and transaction state for debugging/dev. Parameters:
- state: The transaction state (will be stored in TxSource)
- mergedPatch: The merged root patch/diff (already merged from all participants)
- commit: The commit number for this entry
- timestamp: RFC3339 timestamp string
- lastCommit: The commit number before this one (typically commit-1)
func (*Entry) FromTony ¶
func (s *Entry) FromTony(data []byte, opts ...gomap.UnmapOption) error
FromTony parses Tony format bytes and populates Entry.
func (*Entry) FromTonyIR ¶
FromTonyIR populates Entry from a Tony IR node.
type SnapshotWriter ¶ added in v0.0.15
type SnapshotWriter struct {
// contains filtered or unexported fields
}
SnapshotWriter is a writer for creating snapshots in the inactive log. It implements io.WriteCloser and io.Seeker for use with snap.Builder. When closed, it writes the snapshot Entry metadata to the log.
func (*SnapshotWriter) Abandon ¶ added in v0.0.15
func (sw *SnapshotWriter) Abandon()
Abandon closes the SnapshotWriter without writing Entry metadata. Used when snapshot creation fails and we need to unlock the log file.
func (*SnapshotWriter) Close ¶ added in v0.0.15
func (sw *SnapshotWriter) Close() error
Close writes the snapshot Entry metadata and unlocks the log file. The Entry is written at the end of the snapshot data (endPos).
func (*SnapshotWriter) EntryPosition ¶ added in v0.0.15
func (sw *SnapshotWriter) EntryPosition() int64
EntryPosition returns the position of the Entry in the log (available after Close).
func (*SnapshotWriter) LogFileID ¶ added in v0.0.15
func (sw *SnapshotWriter) LogFileID() LogFileID
LogFileID returns the log file ID where this snapshot was written.
Source Files