Documentation
¶
Index ¶
- Variables
- type ContentRef
- type ContentResolverFunc
- type Graph
- type MemoryStore
- func (s *MemoryStore) AddNode(n *Node)
- func (s *MemoryStore) AddRef(token, nodeID string) error
- func (s *MemoryStore) AddRoot(n *Node)
- func (s *MemoryStore) Close() error
- func (s *MemoryStore) FlushRefs() error
- func (s *MemoryStore) GetCallers(token string) ([]*Node, error)
- func (s *MemoryStore) GetNode(id string) (*Node, error)
- func (s *MemoryStore) InitRefsDB() error
- func (s *MemoryStore) Invalidate(id string)
- func (s *MemoryStore) ListChildren(id string) ([]string, error)
- func (s *MemoryStore) QueryRefs(query string, args ...any) (*sql.Rows, error)
- func (s *MemoryStore) ReadContent(id string, buf []byte, offset int64) (int, error)
- func (s *MemoryStore) SetResolver(fn ContentResolverFunc)
- type Node
- type SQLiteGraph
- func (g *SQLiteGraph) AddRef(token, nodeID string) error
- func (g *SQLiteGraph) Close() error
- func (g *SQLiteGraph) EagerScan() error
- func (g *SQLiteGraph) FlushRefs() error
- func (g *SQLiteGraph) GetCallers(token string) ([]*Node, error)
- func (g *SQLiteGraph) GetNode(id string) (*Node, error)
- func (g *SQLiteGraph) Invalidate(id string)
- func (g *SQLiteGraph) ListChildren(id string) ([]string, error)
- func (g *SQLiteGraph) QueryRefs(query string, args ...any) (*sql.Rows, error)
- func (g *SQLiteGraph) ReadContent(id string, buf []byte, offset int64) (int, error)
- type SourceOrigin
- type TemplateRenderer
Constants ¶
This section is empty.
Variables ¶
var ErrNotFound = errors.New("node not found")
Functions ¶
This section is empty.
Types ¶
type ContentRef ¶
type ContentRef struct {
DBPath string // Path to the SQLite database
RecordID string // Row ID in the results table
Template string // Content template to re-render
ContentLen int64 // Pre-computed rendered byte length
}
ContentRef is a recipe for lazily resolving file content from a backing store. Instead of storing the full byte content in RAM, we store enough info to re-fetch it on demand.
type ContentResolverFunc ¶
type ContentResolverFunc func(ref *ContentRef) ([]byte, error)
ContentResolverFunc resolves a ContentRef into byte content.
type Graph ¶
type Graph interface {
GetNode(id string) (*Node, error)
ListChildren(id string) ([]string, error)
ReadContent(id string, buf []byte, offset int64) (int, error)
GetCallers(token string) ([]*Node, error)
// Invalidate evicts cached data for a node (size, content).
// Called after write-back to force re-render on next access.
Invalidate(id string)
}
Graph is the interface for the FUSE layer. This allows us to swap the backend later (Memory -> SQLite -> Mmap).
type MemoryStore ¶
type MemoryStore struct {
// contains filtered or unexported fields
}
func NewMemoryStore ¶
func NewMemoryStore() *MemoryStore
func (*MemoryStore) AddNode ¶
func (s *MemoryStore) AddNode(n *Node)
AddNode adds a non-root node to the store.
func (*MemoryStore) AddRef ¶
func (s *MemoryStore) AddRef(token, nodeID string) error
AddRef records a reference from a file (nodeID) to a token.
func (*MemoryStore) AddRoot ¶
func (s *MemoryStore) AddRoot(n *Node)
AddRoot registers a node as a top-level root and adds it to the store. Callers must explicitly declare roots — there is no heuristic.
func (*MemoryStore) Close ¶
func (s *MemoryStore) Close() error
Close closes the refs database and removes the temp file.
func (*MemoryStore) FlushRefs ¶
func (s *MemoryStore) FlushRefs() error
FlushRefs writes all accumulated refs (from AddRef) into the in-memory SQLite sidecar as roaring bitmaps. Guarded by sync.Once — safe to call multiple times; only the first call performs the flush.
func (*MemoryStore) GetCallers ¶
func (s *MemoryStore) GetCallers(token string) ([]*Node, error)
GetCallers implements Graph.
func (*MemoryStore) GetNode ¶
func (s *MemoryStore) GetNode(id string) (*Node, error)
GetNode implements Graph.
func (*MemoryStore) InitRefsDB ¶
func (s *MemoryStore) InitRefsDB() error
InitRefsDB opens an in-memory SQLite database with the same schema as SQLiteGraph's sidecar (node_refs + file_ids + mache_refs vtab). Must be called before FlushRefs. Safe to call multiple times (idempotent).
func (*MemoryStore) Invalidate ¶
func (s *MemoryStore) Invalidate(id string)
Invalidate is a no-op for MemoryStore — nodes are updated in-place.
func (*MemoryStore) ListChildren ¶
func (s *MemoryStore) ListChildren(id string) ([]string, error)
ListChildren implements Graph.
func (*MemoryStore) QueryRefs ¶
QueryRefs executes a SQL query against the in-memory refs database, which includes the mache_refs virtual table.
func (*MemoryStore) ReadContent ¶
ReadContent implements Graph. It handles both inline and lazy content.
func (*MemoryStore) SetResolver ¶
func (s *MemoryStore) SetResolver(fn ContentResolverFunc)
SetResolver configures lazy content resolution for nodes with ContentRef.
type Node ¶
type Node struct {
ID string
Mode fs.FileMode // fs.ModeDir for directories, 0 for regular files
ModTime time.Time // Modification time
Data []byte // Inline content (small files, nil for lazy nodes)
Ref *ContentRef // Lazy content reference (large files, nil for inline nodes)
Properties map[string][]byte // Metadata / extended attributes
Children []string // Child node IDs (directories only)
Origin *SourceOrigin // Source byte range (nil for dirs, JSON, SQLite nodes)
}
Node is the universal primitive. The Mode field explicitly declares whether this is a file or directory.
func (*Node) ContentSize ¶
ContentSize returns the byte length of this node's content, regardless of whether it is inline or lazy.
type SQLiteGraph ¶
type SQLiteGraph struct {
// contains filtered or unexported fields
}
SQLiteGraph implements Graph by querying the source SQLite database directly. No index copy, no ingestion step — the source DB's B+ tree IS the index.
Design: directory structure is derived lazily from schema + DB on first access, then cached in sync.Maps for lock-free concurrent reads from FUSE callbacks.
The scan is single-threaded and streaming: one sequential pass over all records, rendering name templates to build parent→child path relationships. This avoids the deadlock risk and channel overhead of a worker pool — SQLite sequential reads are I/O-bound and template rendering for name fields is cheap.
Memory model after scan:
- dirChildren: sorted []string slices (one per directory), read-only post-scan
- recordIDs: leaf dir path → DB row ID, for on-demand content resolution
- contentCache: FIFO-bounded rendered content (avoids re-fetching hot files)
Content is never loaded during scan — only on FUSE read via resolveContent, which does a primary key lookup + template render + FIFO cache.
Cross-references (token → file bitmap) are stored in a sidecar database (<dbpath>.refs.db) to keep the source DB immutable. Refs are accumulated in-memory during ingestion and flushed once via FlushRefs.
func OpenSQLiteGraph ¶
func OpenSQLiteGraph(dbPath string, schema *api.Topology, render TemplateRenderer) (*SQLiteGraph, error)
OpenSQLiteGraph opens a connection to the source DB and compiles the schema.
func (*SQLiteGraph) AddRef ¶
func (g *SQLiteGraph) AddRef(token, nodeID string) error
AddRef accumulates a reference in-memory. No SQL is issued until FlushRefs. This eliminates the read-modify-write cycle per call — all bitmap mutations happen in RAM, and FlushRefs writes them in a single transaction.
func (*SQLiteGraph) Close ¶
func (g *SQLiteGraph) Close() error
Close closes both the source and sidecar database connections.
func (*SQLiteGraph) EagerScan ¶
func (g *SQLiteGraph) EagerScan() error
EagerScan pre-scans all root nodes so no FUSE callback ever blocks on a scan. Call this before mounting — fuse-t's NFS transport times out if a callback takes >2s.
func (*SQLiteGraph) FlushRefs ¶
func (g *SQLiteGraph) FlushRefs() error
FlushRefs writes all accumulated refs to the sidecar database in a single transaction. Call once after ingestion is complete. This replaces the old per-call AddRef write path, reducing N*M SQL round-trips to exactly len(fileIDMap) + len(pendingRefs) inserts in one transaction.
Guarded by sync.Once — safe to call multiple times; only the first call performs the flush. This prevents the double-call bug where a second flush would reset nextFileID to 0, causing ID collisions in file_ids.
func (*SQLiteGraph) GetCallers ¶
func (g *SQLiteGraph) GetCallers(token string) ([]*Node, error)
GetCallers returns the list of files (nodes) that reference the given token. Reads from the sidecar refs database.
func (*SQLiteGraph) Invalidate ¶
func (g *SQLiteGraph) Invalidate(id string)
Invalidate evicts cached size and content for a node. Must be called after write-back modifies a file's content to prevent stale size/data from being served on the next Getattr or Read.
func (*SQLiteGraph) ListChildren ¶
func (g *SQLiteGraph) ListChildren(id string) ([]string, error)
func (*SQLiteGraph) QueryRefs ¶
QueryRefs executes a SQL query against the refs sidecar database, which includes the mache_refs virtual table.
func (*SQLiteGraph) ReadContent ¶
type SourceOrigin ¶
SourceOrigin tracks the byte range of a construct in its source file. Used by write-back to splice edits into the original source.
Source Files