storage

package
v0.32.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 1, 2026 License: MIT Imports: 7 Imported by: 0

Documentation

Overview

Package storage provides per-plugin SQLite-backed key/value and SQL storage scoped at session, agent, or application levels.

Every plugin can request a Storage handle via PluginContext.Storage(scope). Each (scope, plugin) pair maps to a single .db file under a deterministic path so plugin state is isolated and survives restart. KV methods cover the trivial put/get/list cases; plugins that need joins, transactions, or virtual tables (FTS5) call DB() and own their schema.

The backend is modernc.org/sqlite — pure Go, no CGO, FTS5 included. SQLite is opened in WAL mode with a 5s busy timeout; foreign keys are on by default. Each handle wraps a *sql.DB with a small idle pool.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type CheckpointResult

type CheckpointResult struct {
	// Busy is true when SQLite could not complete the checkpoint because
	// another connection held a read lock.
	Busy bool
	// WALFrames is the number of frames in the WAL, as reported by SQLite.
	WALFrames int
	// Checkpointed is the number of frames successfully moved back into the
	// main database file.
	Checkpointed int
}

CheckpointResult reports what PRAGMA wal_checkpoint(TRUNCATE) did.

SQLite returns three integers. Busy is the one that matters: a busy checkpoint could not reclaim the whole WAL because a reader was still holding frames, which means the main database file is *not* fully self-contained. The VACUUM INTO that follows still produces a correct snapshot in that case, so a busy checkpoint is reported and logged rather than treated as an error.

type Manager

type Manager struct {
	// contains filtered or unexported fields
}

Manager owns the open Storage handles for an engine instance. It resolves scope+pluginID to a deterministic on-disk path and pools handles so two requests for the same (scope, pluginID) share a single *sql.DB.

Manager is constructed once per engine in Engine.newFromConfig and torn down by Engine.Stop. PluginContext.Storage is a thin wrapper around it.

func NewManager

func NewManager(root, agentID string, sessionFn func() string, opts *SQLiteOptions) *Manager

NewManager constructs a storage manager.

  • root is the user data root, typically "~/.nexus" expanded. App and Agent scope paths land beneath it.
  • agentID partitions Agent-scope storage. Empty string collapses Agent to App, which is the common case for CLI / single-agent embedders.
  • sessionFn returns the active session's RootDir. It is called lazily so the manager can be constructed before the session exists. Returning "" causes Open(ScopeSession) to fail with a clear error — Manager does not assume a session is always available.
  • opts are the SQLite tuning parameters; nil falls back to DefaultSQLiteOptions.

func (*Manager) AttachSessionResolver

func (m *Manager) AttachSessionResolver(fn func() string)

AttachSessionResolver replaces the session resolver. The engine wires this after construction so plugins that call Open(ScopeSession) during Init see the live session that was created in Boot.

func (*Manager) Checkpoint

func (m *Manager) Checkpoint(scope Scope) (map[string]CheckpointResult, error)

Checkpoint folds the WAL of every open handle at scope back into its main database file, without producing a copy.

Exported separately from Snapshot because the checkpoint is useful on its own: it bounds the size of a store.db-wal that a long-lived session would otherwise let grow unchecked, and it leaves the *local* database file restorable in place. Snapshot always runs it first, so callers taking a snapshot do not need this as well.

func (*Manager) Close

func (m *Manager) Close() error

Close drains every open handle. Safe to call multiple times; subsequent calls return nil. Errors from individual handles are joined into a single error for visibility but never blockclose of remaining handles.

func (*Manager) Open

func (m *Manager) Open(scope Scope, pluginID string) (Storage, error)

Open returns the storage handle for (scope, pluginID), opening it on first use. Repeated calls with the same arguments share the same *sql.DB.

When scope is ScopeAgent and the manager was constructed without an agent ID, the request collapses to ScopeApp so plugins do not end up with two separate connection pools pointing at the same file.

func (*Manager) Snapshot

func (m *Manager) Snapshot(scope Scope, destDir string) ([]Snapshot, error)

Snapshot writes a self-consistent copy of every open handle at scope into destDir, one per plugin, at <destDir>/<pluginID>/store.db.

Only handles this Manager has actually opened are covered. That is the right set by construction: a store.db in the tree with no handle has no writer in this process, so it is already static and a caller can copy it directly.

A partial failure returns the snapshots taken so far alongside the error, so a caller can clean them up. It deliberately does not press on past a failure: a snapshot missing one plugin's database is not a snapshot.

type Provider

type Provider interface {
	// Open returns the storage handle for (scope, pluginID). Handles are
	// pooled, so repeated calls with the same arguments return the same
	// underlying *sql.DB. Safe for concurrent use.
	Open(scope Scope, pluginID string) (Storage, error)

	// Close drains every open handle. Called by Engine.Stop.
	Close() error
}

Provider opens scoped Storage handles for plugins. The engine constructs a single Provider at boot and exposes per-handle access through the PluginContext.

type SQLiteOptions

type SQLiteOptions struct {
	BusyTimeoutMs int
	CacheSizeKB   int
	PoolMaxIdle   int
	PoolMaxOpen   int
}

SQLiteOptions tunes the per-handle SQLite connection. Defaults are picked for an interactive single-user agent: 5s busy timeout, 2MB cache, small connection pool.

func DefaultSQLiteOptions

func DefaultSQLiteOptions() SQLiteOptions

DefaultSQLiteOptions returns the baseline tuning. Embedders override via the storage config block.

type Scope

type Scope int

Scope determines where a plugin's storage file lands.

Session scope is per-session and disappears when the session is archived. Agent scope persists across sessions for a single agent (used by multi-agent hosts where each agent has its own engine instance). App scope persists across every session and every agent on the machine.

In a single-agent embedder (CLI, oneshot), Agent collapses to App since there is no meaningful per-agent partition.

const (
	// ScopeSession is per-session. Path: <session.RootDir>/plugins/<pluginID>/store.db
	ScopeSession Scope = iota
	// ScopeAgent is per-agent (multi-agent hosts). Path:
	// ~/.nexus/agents/<agentID>/plugins/<pluginID>/store.db. Falls back to
	// ScopeApp when the engine has no AgentID configured.
	ScopeAgent
	// ScopeApp is machine-wide. Path: ~/.nexus/plugins/<pluginID>/store.db
	ScopeApp
)

func (Scope) String

func (s Scope) String() string

String returns a short identifier for the scope, used in logs.

type Snapshot

type Snapshot struct {
	// PluginID owns the database.
	PluginID string
	// Scope is the resolved scope, after ScopeAgent collapses to ScopeApp on
	// an engine with no agent ID.
	Scope Scope
	// LivePath is the store.db the snapshot was taken from. Callers map it
	// back onto a tree-relative path or object key.
	LivePath string
	// Path is the snapshot file. Restorable on its own: no -wal, no -shm.
	Path string
	// Bytes is the snapshot's size on disk.
	Bytes int64
	// Checkpoint records the WAL checkpoint that preceded the snapshot.
	Checkpoint CheckpointResult
	// Duration is the wall time the checkpoint and the copy took together.
	// Surfaced because this cost is O(database size) and is paid on every
	// snapshot — the number an operator needs when a database grows.
	Duration time.Duration
}

Snapshot describes one self-consistent, sidecar-free copy of a plugin's store.db.

type Storage

type Storage interface {
	// DB returns the underlying *sql.DB. Plugins that need raw SQL — joins,
	// transactions, virtual tables (FTS5) — call this and own their schema.
	// The handle is owned by the storage manager; do not Close it.
	DB() *sql.DB

	// Get returns the value stored under key. The bool reports presence;
	// missing keys are not an error.
	Get(key string) ([]byte, bool, error)

	// Put writes value under key. Existing values are replaced.
	Put(key string, value []byte) error

	// Delete removes the key. Missing keys are not an error.
	Delete(key string) error

	// List returns keys with the given prefix in lexicographic order.
	List(prefix string) ([]string, error)

	// Tx runs fn inside a serialized transaction. The transaction is
	// committed when fn returns nil and rolled back on any non-nil error
	// or panic. The kv table is visible inside the transaction.
	Tx(fn func(*sql.Tx) error) error
}

Storage is a per-plugin scoped storage handle.

Both the KV API and DB() share the same underlying *sql.DB and connection pool. Plugins can mix the two freely. The kv table is created lazily on the first KV method call; SQL-only consumers never see it.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL