storage

package
v0.0.7 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 27, 2025 License: Apache-2.0 Imports: 17 Imported by: 0

Documentation

Overview

Package storage provides a filesystem based storage layer for tony system api.

Storage manages

  • mappings of virtual document paths to and from the filesyste

  • indexing of virtual document nodes

  • storage of diffs associated with virtual document nodes

  • transactions of multi-participant diffs

  • compaction of diffs

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func FormatLexInt added in v0.0.7 

func FormatLexInt(v int64) string

func ParseLexInt added in v0.0.7 

func ParseLexInt(v string) (int64, error)

Types

type DiffFile 

type DiffFile struct {
	Seq       int64
	Path      string
	Timestamp string
	Diff      *ir.Node
	Pending   bool // true for .pending files, false for .diff files
}

DiffFile represents a diff file on disk.

type FS added in v0.0.7 

type FS struct {
	Root string
}

func (*FS) DeletePending added in v0.0.7 

func (fs *FS) DeletePending(virtualPath string, txSeq int64) error

DeletePending deletes a .pending file.

func (*FS) EnsurePathDir added in v0.0.7 

func (fs *FS) EnsurePathDir(virtualPath string) error

EnsurePathDir ensures that the directory for a given virtual path exists. It creates all necessary parent directories.

func (*FS) FilesystemToPath added in v0.0.7 

func (fs *FS) FilesystemToPath(fsPath string) string

FilesystemToPath converts a filesystem directory path back to a virtual document path. Example: "/logd/paths/children/proc/children/processes" -> "/proc/processes"

func (*FS) FormatLogSegment added in v0.0.7 

func (fs *FS) FormatLogSegment(s *index.LogSegment, pending bool) string

func (*FS) ListChildPaths added in v0.0.7 

func (fs *FS) ListChildPaths(parentPath string) ([]string, error)

ListChildPaths returns all immediate child paths under parentPath. Only returns paths that have data (directories with diff files).

func (*FS) ListLogSegments added in v0.0.7 

func (fs *FS) ListLogSegments(virtualPath string) ([]*index.LogSegment, error)

ListLogSegments lists all diff and pending files for a path, ordered by commit count then tx.

func (*FS) ParseLogSegment added in v0.0.7 

func (fs *FS) ParseLogSegment(p string) (*index.LogSegment, error)

func (*FS) PathToFilesystem added in v0.0.7 

func (fs *FS) PathToFilesystem(virtualPath string) string

PathToFilesystem converts a virtual document path to a filesystem directory path. Example: "/proc/processes" -> "/logd/paths/children/proc/children/processes"

func (*FS) RenamePendingToDiff added in v0.0.7 

func (fs *FS) RenamePendingToDiff(virtualPath string, newCommitCount, txSeq int64) error

type PendingDiff 

type PendingDiff struct {
	Path      string
	DiffFile  string // Full filesystem path to the .pending file
	WrittenAt string // RFC3339 timestamp
}

PendingDiff represents a pending diff in a transaction.

func (*PendingDiff) FromTony added in v0.0.7 

func (s *PendingDiff) FromTony(data []byte, opts ...parse.ParseOption) error

FromTony parses Tony format bytes and populates PendingDiff.

func (*PendingDiff) FromTonyIR added in v0.0.7 

func (s *PendingDiff) FromTonyIR(node *ir.Node, opts ...parse.ParseOption) error

FromTonyIR populates PendingDiff from a Tony IR node.

func (*PendingDiff) ToTony added in v0.0.7 

func (s *PendingDiff) ToTony(opts ...encode.EncodeOption) ([]byte, error)

ToTony converts PendingDiff to Tony format bytes.

func (*PendingDiff) ToTonyIR added in v0.0.7 

func (s *PendingDiff) ToTonyIR(opts ...encode.EncodeOption) (*ir.Node, error)

ToTonyIR converts PendingDiff to a Tony IR node.

type PendingFileRef 

type PendingFileRef struct {
	VirtualPath string
	TxSeq       int64 // Transaction sequence number
}

PendingFileRef references a pending file that needs to be renamed.

func (*PendingFileRef) FromTony added in v0.0.7 

func (s *PendingFileRef) FromTony(data []byte, opts ...parse.ParseOption) error

FromTony parses Tony format bytes and populates PendingFileRef.

func (*PendingFileRef) FromTonyIR added in v0.0.7 

func (s *PendingFileRef) FromTonyIR(node *ir.Node, opts ...parse.ParseOption) error

FromTonyIR populates PendingFileRef from a Tony IR node.

func (*PendingFileRef) ToTony added in v0.0.7 

func (s *PendingFileRef) ToTony(opts ...encode.EncodeOption) ([]byte, error)

ToTony converts PendingFileRef to Tony format bytes.

func (*PendingFileRef) ToTonyIR added in v0.0.7 

func (s *PendingFileRef) ToTonyIR(opts ...encode.EncodeOption) (*ir.Node, error)

ToTonyIR converts PendingFileRef to a Tony IR node.

type Storage 

type Storage struct {
	*seq.Seq

	FS *FS
	// contains filtered or unexported fields
}

Storage provides filesystem-based storage for logd.

func Open 

func Open(root string, umask int, logger *slog.Logger) (*Storage, error)

Open opens or creates a Storage instance with the given root directory. The root directory will be created if it doesn't exist. umask is applied to directory permissions (e.g., 022 for 0755 -> 0755). If logger is nil, slog.Default() will be used.

func (*Storage) AppendTransactionLog 

func (s *Storage) AppendTransactionLog(entry *TransactionLogEntry) error

AppendTransactionLog appends a transaction commit log entry atomically.

func (*Storage) CommitPendingDiff added in v0.0.7 

func (s *Storage) CommitPendingDiff(virtualPath string, txSeq, commitCount int64) error

CommitPendingDiff atomically commits a pending diff by: 1. Renaming .pending to .diff 2. Updating the index

func (*Storage) DeletePendingDiff added in v0.0.7 

func (s *Storage) DeletePendingDiff(virtualPath string, txSeq int64) error

DeletePendingDiff deletes a pending diff file.

func (*Storage) DeleteTransactionState 

func (s *Storage) DeleteTransactionState(transactionID string) error

DeleteTransactionState deletes a transaction state file.

func (*Storage) ListChildPaths added in v0.0.7 

func (s *Storage) ListChildPaths(parentPath string, from, to *int64) ([]string, error)

ListChildPaths returns all immediate child paths under parentPath that have diffs in the commit range [from, to]. Pass nil for unbounded range.

func (*Storage) ListDiffs 

func (s *Storage) ListDiffs(virtualPath string) ([]*index.LogSegment, error)

ListDiffs lists all diff segments for a path, ordered by commit count. The index only contains committed diffs, never pending files. Returns both point segments (individual diffs) and range segments (compacted diffs).

func (*Storage) ReadDiff 

func (s *Storage) ReadDiff(virtualPath string, commitCount, txSeq int64, pending bool) (*DiffFile, error)

ReadDiff reads a diff file from disk. For pending files, commitCount is ignored (can be 0).

func (*Storage) ReadTransactionLog 

func (s *Storage) ReadTransactionLog(minCommitCount *int64) ([]*TransactionLogEntry, error)

ReadTransactionLog reads transaction log entries. If minCommitCount is nil, reads all entries. If minCommitCount is provided, uses binary search to find entries at or after that commit count.

func (*Storage) ReadTransactionState 

func (s *Storage) ReadTransactionState(transactionID string) (*TransactionState, error)

ReadTransactionState reads a transaction state file from disk.

func (*Storage) RecoverTransactions 

func (s *Storage) RecoverTransactions() error

RecoverTransactions replays the transaction log to complete any partial commits.

func (*Storage) UpdateTransactionState 

func (s *Storage) UpdateTransactionState(transactionID string, updateFn func(*TransactionState)) error

UpdateTransactionState updates an existing transaction state file.

func (*Storage) WriteDiff 

func (s *Storage) WriteDiff(virtualPath string, commitCount, txSeq int64, timestamp string, diff *ir.Node, pending bool) error

WriteDiff writes a diff file. For atomic sequence allocation and file writing, use WriteDiffAtomically instead.

func (*Storage) WriteDiffAtomically 

func (s *Storage) WriteDiffAtomically(virtualPath string, timestamp string, diff *ir.Node, pending bool) (commitCount, txSeq int64, err error)

WriteDiffAtomically writes a diff file to disk. commitCount should be 0 for pending files, and the actual commit count for committed files. If pending is true, writes as .pending file; otherwise writes as .diff file. WriteDiffAtomically atomically allocates sequence numbers and writes the diff file. This ensures that files are written in the order that sequence numbers are allocated, preventing race conditions where a later sequence number is written before an earlier one.

func (*Storage) WriteTransactionState 

func (s *Storage) WriteTransactionState(state *TransactionState) error

WriteTransactionState writes a transaction state file to disk.

type TransactionLogEntry 

type TransactionLogEntry struct {
	CommitCount   int64 // Commit count assigned to this transaction
	TransactionID string
	Timestamp     string // RFC3339 timestamp
	PendingFiles  []PendingFileRef
}

TransactionLogEntry represents a transaction commit log entry.

func NewTransactionLogEntry 

func NewTransactionLogEntry(commitCount int64, transactionID string, pendingFiles []PendingFileRef) *TransactionLogEntry

NewTransactionLogEntry creates a new TransactionLogEntry.

func (*TransactionLogEntry) FromTony added in v0.0.7 

func (s *TransactionLogEntry) FromTony(data []byte, opts ...parse.ParseOption) error

FromTony parses Tony format bytes and populates TransactionLogEntry.

func (*TransactionLogEntry) FromTonyIR added in v0.0.7 

func (s *TransactionLogEntry) FromTonyIR(node *ir.Node, opts ...parse.ParseOption) error

FromTonyIR populates TransactionLogEntry from a Tony IR node.

func (*TransactionLogEntry) ToTony added in v0.0.7 

func (s *TransactionLogEntry) ToTony(opts ...encode.EncodeOption) ([]byte, error)

ToTony converts TransactionLogEntry to Tony format bytes.

func (*TransactionLogEntry) ToTonyIR added in v0.0.7 

func (s *TransactionLogEntry) ToTonyIR(opts ...encode.EncodeOption) (*ir.Node, error)

ToTonyIR converts TransactionLogEntry to a Tony IR node.

type TransactionState 

type TransactionState struct {
	TransactionID        string
	ParticipantCount     int
	ParticipantsReceived int
	Status               string // "pending", "committed", "aborted"
	CreatedAt            string // RFC3339 timestamp
	ExpiresAt            string
	Diffs                []PendingDiff
}

TransactionState represents the state of a transaction.

func NewTransactionState 

func NewTransactionState(transactionID string, participantCount int) *TransactionState

NewTransactionState creates a new TransactionState with the given transaction ID and participant count.

func (*TransactionState) FromTony added in v0.0.7 

func (s *TransactionState) FromTony(data []byte, opts ...parse.ParseOption) error

FromTony parses Tony format bytes and populates TransactionState.

func (*TransactionState) FromTonyIR added in v0.0.7 

func (s *TransactionState) FromTonyIR(node *ir.Node, opts ...parse.ParseOption) error

FromTonyIR populates TransactionState from a Tony IR node.

func (*TransactionState) ToTony added in v0.0.7 

func (s *TransactionState) ToTony(opts ...encode.EncodeOption) ([]byte, error)

ToTony converts TransactionState to Tony format bytes.

func (*TransactionState) ToTonyIR added in v0.0.7 

func (s *TransactionState) ToTonyIR(opts ...encode.EncodeOption) (*ir.Node, error)

ToTonyIR converts TransactionState to a Tony IR node.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.