slab

package
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 24, 2026 License: MIT Imports: 13 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	// HeaderSize: CRC(4) + KeyLen(2) + ValueLen(4)
	HeaderSize = 10
)

Variables

View Source 
var (
	// MaxSlabSize is 4GB (hard limit for rotation).
	// Variable to allow testing overrides.
	MaxSlabSize int64 = 4 * 1024 * 1024 * 1024
	// MaxRecordSize bounds a single slab record (header + key + value).
	// Set <= 0 to disable the cap.
	MaxRecordSize int64 = 64 * 1024 * 1024
	// MaxDeadMappings caps the number of old mmaps retained to avoid exhausting
	// vm.max_map_count. Set <= 0 to disable the cap.
	MaxDeadMappings = 64
)
View Source 
var (
	ErrChecksumMismatch = errors.New("slab record checksum mismatch")
	ErrRecordTooLarge   = errors.New("record too large")
	ErrSlabFull         = errors.New("slab file is full")
	ErrReadOnly         = errors.New("slab is read-only")
)

Functions

This section is empty.

Types

type CompressionKind 

type CompressionKind uint8
const (
	CompressionNone CompressionKind = iota
	CompressionZSTD
)

type CompressionOptions 

type CompressionOptions struct {
	Kind            CompressionKind
	MinBytes        int
	MinSavingsBytes int
}

type Options 

type Options struct {
	Compression CompressionOptions
}

type SlabFile 

type SlabFile struct {
	ID   uint32
	Path string
	File *os.File

	RefCount atomic.Int64
	IsZombie atomic.Bool
	Size     int64 // Track size for rotation
	// contains filtered or unexported fields
}

SlabFile represents a single physical .slab file.

func OpenSlab 

func OpenSlab(path string, id uint32) (*SlabFile, error)

OpenSlab opens or creates a slab file.

func OpenSlabLazy 

func OpenSlabLazy(path string, id uint32, size int64) *SlabFile

OpenSlabLazy registers a slab without opening its file descriptor.

func OpenSlabLazyReadOnly 

func OpenSlabLazyReadOnly(path string, id uint32, size int64) *SlabFile

OpenSlabLazyReadOnly registers a slab without opening its file descriptor in read-only mode.

func OpenSlabReadOnly 

func OpenSlabReadOnly(path string, id uint32) (*SlabFile, error)

OpenSlabReadOnly opens an existing slab file in read-only mode.

func (*SlabFile) Close 

func (s *SlabFile) Close() error

func (*SlabFile) Read 

func (s *SlabFile) Read(offset int64, verifyCRC bool) ([]byte, error)

ReadAt reads a record at the given offset (which points to KeyLen, skipping CRC).

func (*SlabFile) ReadUnsafe 

func (s *SlabFile) ReadUnsafe(offset int64, verifyCRC bool) ([]byte, error)

ReadUnsafe returns a zero-copy view of the record. It forces a synchronous remap if the current mapping does not cover the record.

func (*SlabFile) RepairTail 

func (s *SlabFile) RepairTail() error

RepairTail scans the slab file to find the last complete (and checksummed) record and truncates any partial/corrupt tail bytes. This is primarily a crash-recovery helper.

func (*SlabFile) Sync 

func (s *SlabFile) Sync() error

func (*SlabFile) Truncate 

func (s *SlabFile) Truncate(size int64) error

Truncate resizes the slab file. Used for crash recovery.

func (*SlabFile) Write 

func (s *SlabFile) Write(key, value []byte) (int64, error)

Write appends a record to the slab and returns the offset. Thread-safety: This should be called by a single writer (SlabManager mutex).

func (*SlabFile) WriteBatch 

func (s *SlabFile) WriteBatch(buf []byte) (int64, error)

WriteBatch appends a pre-built record stream to the slab and returns the starting file offset. Thread-safety: This should be called by a single writer (SlabManager mutex).

type SlabManager 

type SlabManager struct {
	// contains filtered or unexported fields
}

func NewSlabManager 

func NewSlabManager(dir string) (*SlabManager, error)

func NewSlabManagerReadOnly 

func NewSlabManagerReadOnly(dir string, opts Options) (*SlabManager, error)

func NewSlabManagerWithOptions 

func NewSlabManagerWithOptions(dir string, opts Options) (*SlabManager, error)

func (*SlabManager) AcquireSlabs 

func (sm *SlabManager) AcquireSlabs(set *SlabSet)

AcquireSlabs increments the RefCount for the Set (O(1)).

func (*SlabManager) ActiveSlabID 

func (sm *SlabManager) ActiveSlabID() uint32

func (*SlabManager) ActiveSlabTail 

func (sm *SlabManager) ActiveSlabTail() uint64

func (*SlabManager) Append 

func (sm *SlabManager) Append(key, value []byte) (page.ValuePtr, error)

func (*SlabManager) AppendMany 

func (sm *SlabManager) AppendMany(keys [][]byte, values [][]byte) ([]page.ValuePtr, error)

AppendMany appends multiple records while amortizing system calls. It returns a pointer for each key/value pair (same order as inputs).

Thread-safety: This is serialized by the SlabManager mutex.

func (*SlabManager) Close 

func (sm *SlabManager) Close() error

func (*SlabManager) CurrentSlabSet 

func (sm *SlabManager) CurrentSlabSet() *SlabSet

CurrentSlabSet returns a snapshot of the current slabs.

func (*SlabManager) Flush added in v0.2.0 

func (sm *SlabManager) Flush() error

Flush writes any buffered appends to disk (without fsync).

func (*SlabManager) GetSlabPath 

func (sm *SlabManager) GetSlabPath(id uint32) string

GetSlabPath returns the path for a given slab ID.

func (*SlabManager) MarkZombie 

func (sm *SlabManager) MarkZombie(id uint32) error

MarkZombie removes a slab from the active set so future snapshots stop pinning it. The file is deleted once all snapshots release it.

func (*SlabManager) PruneSlabs 

func (sm *SlabManager) PruneSlabs(maxID uint32) error

func (*SlabManager) Read 

func (sm *SlabManager) Read(ptr page.ValuePtr) ([]byte, error)

Read reads from the slab file identified by ptr.FileID. For Snapshot Isolation, the caller should ensure the file is pinned via a Snapshot. If accessing without snapshot (e.g. during Compaction or internal ops), care must be taken. Current impl uses RLock on the master map, so it's safe against concurrent Close() initiated by Prune/Compaction IF Prune/Compaction removes from map.

func (*SlabManager) ReadUnsafe 

func (sm *SlabManager) ReadUnsafe(ptr page.ValuePtr) ([]byte, error)

func (*SlabManager) ReleaseSlabs 

func (sm *SlabManager) ReleaseSlabs(set *SlabSet) error

ReleaseSlabs decrements the Set RefCount. If 0, unpins files and cleans zombies.

func (*SlabManager) RemapStats 

func (sm *SlabManager) RemapStats() (remaps uint64, deadMappings uint64)

RemapStats returns cumulative mmap remap counts across slabs.

func (*SlabManager) RemoveSlab 

func (sm *SlabManager) RemoveSlab(id uint32) error

RemoveSlab deletes a slab file from disk and unregisters it from the manager. It is only safe to call when the slab is not referenced by any snapshots.

func (*SlabManager) RepairActiveSlabTail 

func (sm *SlabManager) RepairActiveSlabTail() (uint64, error)

RepairActiveSlabTail scans and truncates any partial/corrupt tail records on the active slab (best-effort crash recovery).

func (*SlabManager) Rotate 

func (sm *SlabManager) Rotate() (*SlabFile, error)

Rotate forces creation of a new active slab.

func (*SlabManager) SetActiveSlab 

func (sm *SlabManager) SetActiveSlab(id uint32) error

func (*SlabManager) SetDisableReadChecksum 

func (sm *SlabManager) SetDisableReadChecksum(disable bool)

func (*SlabManager) Sync 

func (sm *SlabManager) Sync() error

func (*SlabManager) TruncateActiveSlab 

func (sm *SlabManager) TruncateActiveSlab(offset uint64) error

func (*SlabManager) ZombieCount 

func (sm *SlabManager) ZombieCount() int

ZombieCount returns the number of zombie slabs.

type SlabSet 

type SlabSet struct {
	Files    map[uint32]*SlabFile
	RefCount atomic.Int64
	// contains filtered or unexported fields
}

SlabSet is an immutable list of SlabFiles active at a specific point in time.

func (*SlabSet) Read 

func (s *SlabSet) Read(ptr page.ValuePtr) ([]byte, error)

func (*SlabSet) ReadUnsafe 

func (s *SlabSet) ReadUnsafe(ptr page.ValuePtr) ([]byte, error)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.