Documentation
¶
Overview ¶
Package dir indexes a directory of Tarantool journal files (xlog / snap / vylog / run / index). It scans by filename signature, validates each file's in-meta vclock against its filename, enforces Lamport-order consistency across the chain (mirrors xdir_index_file at src/box/xlog.c:382), and provides vclock / lsn lookup so a caller can locate the file that contains a given position.
The filename rule is <vclock-signature>.<ext>, with signature equal to the arithmetic sum of per-replica LSNs (vclock_sum, src/lib/vclock/vclock.h:230).
The package keeps no global state. Filesystem concerns (directory listing, file opening for meta) live here, not in `format`.
Index ¶
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var ( // ErrNotFound is returned by LocateVClock / LocateLSN when no entry in // the directory contains the requested position (e.g. the target sits // strictly before the earliest indexed file, or the directory is empty). ErrNotFound = errors.New("dir: no file contains target") // ErrSignatureMismatch is returned by OpenDir when a file's filename // signature (the <digits>.<ext> stem parsed as int64) does not equal // the in-meta VClock.Signature() for that file. Tarantool's // xdir_open_cursor (src/box/xlog.c:451) treats this as fatal; so do we. ErrSignatureMismatch = errors.New("dir: filename signature does not match meta vclock sum") // ErrDuplicate is returned by OpenDir when two indexed files share the // same VClock signature. The directory is structurally ambiguous in // that case — two files claim to start at the same Lamport position. ErrDuplicate = errors.New("dir: two files share the same vclock signature") // ErrIncomparable is returned by OpenDir when adjacent entries (sorted // by signature) have VClocks that are not partially-ordered (i.e. // VClock.Compare returns ok=false). This mirrors xdir_index_file's // strictness — a Tarantool-produced single-instance chain must be // totally ordered on Compare. ErrIncomparable = errors.New("dir: adjacent files have incomparable vclocks") // ErrNoMeta is returned when a journal file's reader yields no Meta // (the file header could not be parsed into a usable meta block). ErrNoMeta = errors.New("has no meta") )
Sentinel errors. Callers are expected to use errors.Is for matching.
Functions ¶
This section is empty.
Types ¶
type Dir ¶
type Dir struct {
// contains filtered or unexported fields
}
Dir is an immutable in-memory index of journal files for one filetype inside one directory. Build via OpenDir; use Files / LocateVClock / LocateLSN to query. Not safe for concurrent mutation — but read-only queries (Files / LocateVClock / LocateLSN) on a constructed Dir are safe to call from multiple goroutines because the underlying slice and maps are never mutated after construction.
func OpenDir ¶
OpenDir scans path for files matching <digits><ext> where ext corresponds to filetype. For each matching file it reads the meta header via reader.Open, validates that the filename signature equals the in-meta VClock.Signature(), and assembles a FileEntry.
Behaviour:
- Files with the literal suffix `.inprogress` are skipped — they are still being written and are not part of the indexed chain.
- Files whose basename does not match `<digits><ext>` are skipped (a directory may legitimately contain README, .git, etc.).
- Signature mismatch → ErrSignatureMismatch wrapping path + values.
- Two indexed files with the same Signature → ErrDuplicate.
- Adjacent entries whose VClock pair is incomparable → ErrIncomparable (mirrors xdir_index_file's per-pair Lamport order assertion).
On success the returned Dir contains the entries sorted ascending by Signature (which is also ascending by VClock.Signature()).
Example ¶
ExampleOpenDir indexes a directory of xlog files. OpenDir reads each file's meta header, checks the filename signature against the in-meta vclock, and returns the entries sorted ascending by signature.
package main
import (
"fmt"
"os"
"path/filepath"
"github.com/google/uuid"
"github.com/tarantool/go-xlog/dir"
"github.com/tarantool/go-xlog/format"
"github.com/tarantool/go-xlog/writer"
)
var exampleInstance = uuid.MustParse("11111111-2222-3333-4444-555555555555")
// writeXlog writes an empty xlog with the given VClock/PrevVClock header into
// baseDir. The filename is <signature>.xlog, and the in-meta VClock signature
// must match that stem — exactly the invariant OpenDir validates.
func writeXlog(baseDir string, vclock, prev format.VClock) {
name := fmt.Sprintf("%d.xlog", vclock.Signature())
meta := &format.Meta{
Filetype: format.FiletypeXLOG,
Version: "example",
InstanceUUID: exampleInstance,
VClock: vclock,
PrevVClock: prev,
}
w, err := writer.Create(filepath.Join(baseDir, name), meta)
if err != nil {
panic(err)
}
if err := w.Close(); err != nil {
panic(err)
}
}
// writeSampleDir populates a fresh temp directory with a two-file xlog chain:
// {1: 5} followed by {1: 10} (whose PrevVClock links back to the first). The
// caller is responsible for removing the returned directory.
func writeSampleDir() string {
baseDir, err := os.MkdirTemp("", "go-xlog-dir-example")
if err != nil {
panic(err)
}
writeXlog(baseDir, format.VClock{1: 5}, nil)
writeXlog(baseDir, format.VClock{1: 10}, format.VClock{1: 5})
return baseDir
}
func main() {
baseDir := writeSampleDir()
defer func() { _ = os.RemoveAll(baseDir) }()
d, err := dir.OpenDir(baseDir, format.FiletypeXLOG)
if err != nil {
panic(err)
}
for _, f := range d.Files() {
fmt.Printf("%s signature=%d vclock=%s\n", filepath.Base(f.Path), f.Signature, f.VClock)
}
}
Output: 5.xlog signature=5 vclock={1: 5} 10.xlog signature=10 vclock={1: 10}
func OpenDirFS ¶
OpenDirFS is OpenDir over an io/fs.FS. Dir is a slash-separated fs path (use "." for the FS root). It enables indexing a journal directory held in an embed.FS, fstest.MapFS, or archive filesystem. Validation is identical to OpenDir (signature match, duplicate-signature and incomparable-vclock rejection); only the listing and opening go through fsys.
func (*Dir) Files ¶
Files returns the indexed entries sorted by Signature ascending. The returned slice aliases the Dir's internal storage; callers must not mutate it. (Dir is shared-read-only by convention.)
func (*Dir) LocateLSN ¶
LocateLSN projects all indexed VClocks onto a single replica axis (replicaID) and returns the entry with the largest VClock[replicaID] such that VClock[replicaID] ≤ lsn. This is the file whose tx stream includes the given LSN for the given replica.
Missing replicaID in an entry's VClock is treated as 0 (consistent with VClock.Signature / Compare semantics).
Returns ErrNotFound when the directory is empty or when lsn is strictly below every indexed VClock[replicaID].
Example ¶
ExampleDir_LocateLSN finds which file holds a given per-replica LSN by projecting every indexed vclock onto that replica's axis.
package main
import (
"fmt"
"os"
"path/filepath"
"github.com/google/uuid"
"github.com/tarantool/go-xlog/dir"
"github.com/tarantool/go-xlog/format"
"github.com/tarantool/go-xlog/writer"
)
var exampleInstance = uuid.MustParse("11111111-2222-3333-4444-555555555555")
// writeXlog writes an empty xlog with the given VClock/PrevVClock header into
// baseDir. The filename is <signature>.xlog, and the in-meta VClock signature
// must match that stem — exactly the invariant OpenDir validates.
func writeXlog(baseDir string, vclock, prev format.VClock) {
name := fmt.Sprintf("%d.xlog", vclock.Signature())
meta := &format.Meta{
Filetype: format.FiletypeXLOG,
Version: "example",
InstanceUUID: exampleInstance,
VClock: vclock,
PrevVClock: prev,
}
w, err := writer.Create(filepath.Join(baseDir, name), meta)
if err != nil {
panic(err)
}
if err := w.Close(); err != nil {
panic(err)
}
}
// writeSampleDir populates a fresh temp directory with a two-file xlog chain:
// {1: 5} followed by {1: 10} (whose PrevVClock links back to the first). The
// caller is responsible for removing the returned directory.
func writeSampleDir() string {
baseDir, err := os.MkdirTemp("", "go-xlog-dir-example")
if err != nil {
panic(err)
}
writeXlog(baseDir, format.VClock{1: 5}, nil)
writeXlog(baseDir, format.VClock{1: 10}, format.VClock{1: 5})
return baseDir
}
func main() {
baseDir := writeSampleDir()
defer func() { _ = os.RemoveAll(baseDir) }()
d, err := dir.OpenDir(baseDir, format.FiletypeXLOG)
if err != nil {
panic(err)
}
for _, lsn := range []int64{7, 10} {
f, err := d.LocateLSN(1, lsn)
if err != nil {
panic(err)
}
fmt.Printf("replica 1 lsn %d is in %s\n", lsn, filepath.Base(f.Path))
}
}
Output: replica 1 lsn 7 is in 5.xlog replica 1 lsn 10 is in 10.xlog
func (*Dir) LocateVClock ¶
LocateVClock returns the entry whose [VClock, NextVClock) interval contains target. "Contains" uses the partial order:
- entry.VClock ≤ target (Compare returns 0 or -1) AND
- next entry's VClock > target (or this is the last entry).
If any per-pair comparison along the way is incomparable (Compare ok=false), we fall back to the signature axis: pick the entry with the largest Signature ≤ target.Signature(). This is correct for files produced by a single Tarantool instance (where the full vclock and its signature carry the same information) but loses precision for diverged vclocks.
Returns ErrNotFound when the directory is empty, or when target is strictly below every indexed entry's VClock.
Example ¶
ExampleDir_LocateVClock finds the file whose [VClock, nextVClock) interval contains a target position under the vector-clock partial order.
package main
import (
"fmt"
"os"
"path/filepath"
"github.com/google/uuid"
"github.com/tarantool/go-xlog/dir"
"github.com/tarantool/go-xlog/format"
"github.com/tarantool/go-xlog/writer"
)
var exampleInstance = uuid.MustParse("11111111-2222-3333-4444-555555555555")
// writeXlog writes an empty xlog with the given VClock/PrevVClock header into
// baseDir. The filename is <signature>.xlog, and the in-meta VClock signature
// must match that stem — exactly the invariant OpenDir validates.
func writeXlog(baseDir string, vclock, prev format.VClock) {
name := fmt.Sprintf("%d.xlog", vclock.Signature())
meta := &format.Meta{
Filetype: format.FiletypeXLOG,
Version: "example",
InstanceUUID: exampleInstance,
VClock: vclock,
PrevVClock: prev,
}
w, err := writer.Create(filepath.Join(baseDir, name), meta)
if err != nil {
panic(err)
}
if err := w.Close(); err != nil {
panic(err)
}
}
// writeSampleDir populates a fresh temp directory with a two-file xlog chain:
// {1: 5} followed by {1: 10} (whose PrevVClock links back to the first). The
// caller is responsible for removing the returned directory.
func writeSampleDir() string {
baseDir, err := os.MkdirTemp("", "go-xlog-dir-example")
if err != nil {
panic(err)
}
writeXlog(baseDir, format.VClock{1: 5}, nil)
writeXlog(baseDir, format.VClock{1: 10}, format.VClock{1: 5})
return baseDir
}
func main() {
baseDir := writeSampleDir()
defer func() { _ = os.RemoveAll(baseDir) }()
d, err := dir.OpenDir(baseDir, format.FiletypeXLOG)
if err != nil {
panic(err)
}
f, err := d.LocateVClock(format.VClock{1: 7})
if err != nil {
panic(err)
}
fmt.Printf("vclock {1: 7} is in %s\n", filepath.Base(f.Path))
}
Output: vclock {1: 7} is in 5.xlog
type FileEntry ¶
type FileEntry struct {
// Path is the absolute or relative path that OpenDir was called with,
// joined with the file's basename. Suitable for passing directly to
// reader.Open.
Path string
// Signature is the integer parsed from the filename's <digits>.<ext>
// stem. It equals VClock.Signature().
Signature int64
// VClock is the file's in-meta VClock (the high-water vector at the
// time the file was opened).
VClock format.VClock
// PrevVClock is the file's in-meta PrevVClock (the chain link to the
// previous file's VClock — empty for SNAP and for the very first file
// in a chain).
PrevVClock format.VClock
// Filetype mirrors the OpenDir filter parameter; redundant with the
// file extension but convenient for callers that pass FileEntry along
// further into the stack.
Filetype format.Filetype
}
FileEntry describes one indexed file inside a Dir. The fields are filled from the file's name (Signature, Path, Filetype-via-extension) and from its parsed meta header (VClock, PrevVClock).
FileEntry is a value type and safe to copy. The VClock / PrevVClock maps are not deep-copied on Files() return — callers must not mutate them.
Source Files