Documentation
¶
Overview ¶
Package storage's ServerStores bundle is the backend-agnostic surface used by the HTTP server (cmd/server) and the MCP server (cmd/mcp) to obtain a working set of domain stores without compile-time coupling to a specific backend (Postgres pgxpool vs. SQLite database/sql).
Adding a new domain store: extend the interface here, then satisfy it in both internal/storage/factory.go bundles (postgresServerStores and sqliteServerStores). The compile-time `var _ ServerStores = ...` assertions at the bottom of factory.go will keep both backends honest.
Package storage exposes the backend selection plumbing for wayneblacktea.
The personal-OS goal of friend-grade self-hosting drives a pluggable storage layer: the canonical PostgreSQL deployment for Wayne's instance, and a zero-infra SQLite backend (modernc.org/sqlite, pure-Go, no CGo) for friends installing locally with one binary + one .db file.
Both backends are runnable today via NewServerStores in factory.go. EnsureSupported below permits both BackendPostgres and BackendSQLite; the only failure mode it still guards against is an unknown STORAGE_BACKEND value (e.g. "mysql"), which we want to reject at startup rather than halfway through the first request.
Per-domain Go interfaces (`<domain>.StoreIface`) under each domain package — see internal/gtd/iface.go, internal/decision/iface.go, etc. — keep handler / MCP code backend-agnostic; both Postgres- and SQLite-backed Store types satisfy those interfaces via compile-time `var _ Iface = ...` assertions, and ServerStores in server_stores.go bundles the seven of them.
Index ¶
Constants ¶
const ( // AivenAvailableConns is what's actually left for our own pools. AivenAvailableConns = aivenMaxConnections - aivenSuperuserReserved - aivenClientBackends // 15 )
Aiven-measured connection budget, 2026-08. Re-measure against the live instance (`SHOW max_connections;`, Aiven console for reserved/client counts) before changing any of these — see the redeploy-budget test in redeploy_budget_test.go, which fails loudly if this drifts from what the pool-size constants below actually need.
Placement note: these constants must live at package scope (not inside buildPgxPoolConfig below) because redeploy_budget_test.go and 4 CLI/hook call sites in other packages (internal/cli, internal/guard) reference ServerPoolMaxConns / HookPoolMaxConns directly.
const HookPoolMaxConns = 2
HookPoolMaxConns is the pgxpool cap for short-lived CLI/hook processes (wbt-context SessionStart hook, wbt doctor Stop hook, wbt reembed, internal/guard) per backend-security-design.md §5.3. Centralized here so redeploy_budget_test.go can verify the worst-case formula against the real values instead of 4 independently-drifting literals.
const ServerPoolMaxConns = 8
ServerPoolMaxConns is the pgxpool cap for the single long-running cmd/server process. See buildPgxPoolConfig's doc comment below for the fan-out floor (6) and the redeploy-budget ceiling this trades off against.
Variables ¶
var ErrInvalidBackend = errors.New("STORAGE_BACKEND must be 'postgres' or 'sqlite'")
ErrInvalidBackend is returned by BackendFromEnv when the value is set but is neither "postgres" nor "sqlite".
var ErrMissingPGSSLROOTCERT = errors.New("PGSSLROOTCERT required in production")
ErrMissingPGSSLROOTCERT is returned when APP_ENV=production and PGSSLROOTCERT is not set. Fail-fast at boot is safer than a silent unverified connection.
var ErrMissingPostgresDSN = errors.New("postgres backend requires a non-empty DSN")
ErrMissingPostgresDSN signals NewServerStores was asked for a Postgres bundle without a DSN. Callers report it with a fail-fast log.Fatal.
var ErrMissingSQLitePath = errors.New("sqlite backend requires a non-empty file path")
ErrMissingSQLitePath signals NewServerStores was asked for a SQLite bundle without a file path.
Functions ¶
func BuildTLSConfig ¶
BuildTLSConfig constructs a *tls.Config appropriate for the given environment.
- PGSSLROOTCERT starts with "-----BEGIN CERTIFICATE-----" → inline PEM content
- PGSSLROOTCERT set otherwise → file path; read file as PEM
- PGSSLROOTCERT file unreadable OR no valid PEM → error (misconfigured deploy)
- PGSSLROOTCERT not set + APP_ENV=production → ErrMissingPGSSLROOTCERT
- PGSSLROOTCERT not set + APP_ENV != production → nil, nil (system CA pool)
func EnsureSupported ¶
EnsureSupported returns nil when the given backend is one we ship a real implementation for, and ErrInvalidBackend (wrapped) for any unknown value.
As of SQLite v2 cmd dispatch (this commit) both BackendPostgres and BackendSQLite are runnable; the only failure mode is an unknown enum value (e.g. "mysql") that bypassed BackendFromEnv. We keep the function so callers can re-validate after constructing a Backend by hand (tests, future migration tooling) without re-implementing the switch.
func RunMigrations ¶
RunMigrations applies all pending Postgres migrations using the migrations/ directory embedded at compile time. It is a no-op when the environment variable WBT_AUTO_MIGRATE is set to "false".
Fail-fast design: if migrations fail, the returned error causes the server to abort startup. This prevents running against a stale schema.
SQLite backend: the sqlite package manages its own schema via schema.sql; RunMigrations is Postgres-only and must not be called for SQLite.
func SQLitePathFromEnv ¶
func SQLitePathFromEnv() string
SQLitePathFromEnv reads the SQLITE_PATH environment variable and returns it trimmed of surrounding whitespace. Empty input falls back to "./wayneblacktea.db" so the friend-grade install path "just works" when the user only sets STORAGE_BACKEND=sqlite.
Types ¶
type Backend ¶
type Backend string
Backend selects the underlying database engine each domain Store talks to.
const ( // BackendPostgres uses pgxpool against a PostgreSQL server (Aiven, // Railway, local docker, …). The canonical deployment. BackendPostgres Backend = "postgres" // BackendSQLite uses a local file-backed SQLite database. Reserved for // the upcoming friend-grade self-host path; not yet implemented. BackendSQLite Backend = "sqlite" )
func BackendFor ¶
BackendFor resolves the Backend from explicit rawBackend/dsn values, mirroring BackendFromEnv's resolution order without touching process env:
- rawBackend is set (after trimming) → use its value (postgres|sqlite).
- rawBackend is unset and dsn is set → BackendPostgres. This lets Railway / Heroku / Docker deployments work without requiring an extra STORAGE_BACKEND variable when a DSN is already present.
- rawBackend is unset and dsn is unset → BackendSQLite (local-first default for zero-infra installs).
Callers that read from process env directly should prefer BackendFromEnv; BackendFor exists so callers with an explicit DSN (e.g. a fallback-file value that was never written to os.Environ) can resolve the same way without mutating env first. See backend-security-design.md §4.2.
func BackendFromEnv ¶
BackendFromEnv reads the STORAGE_BACKEND and DATABASE_URL environment variables and returns the resolved Backend. See BackendFor for the resolution order; this is a thin env-reading wrapper around it.
func ResolveFromEnv ¶
ResolveFromEnv combines BackendFromEnv + EnsureSupported into a single call for entry-point binaries. Returns the resolved backend or a wrapped error suitable for log.Fatal at startup.
type FactoryConfig ¶
type FactoryConfig struct {
// Backend selects the storage engine. Defaults to BackendPostgres when
// the zero value is passed.
Backend Backend
// PostgresDSN is the libpq-style connection string for the Postgres
// backend. Required when Backend == BackendPostgres.
PostgresDSN string
// SQLitePath is the file path the SQLite backend opens (e.g.
// "./wayneblacktea.db" or ":memory:" for tests). Required when
// Backend == BackendSQLite.
SQLitePath string
// PGSSLRootCert is the file path to a PEM-encoded CA certificate bundle
// used to verify the Postgres server's TLS certificate. When empty and
// AppEnv is "production", NewServerStores returns ErrMissingPGSSLROOTCERT.
// When empty and AppEnv is not "production", the system CA pool is used.
PGSSLRootCert string
// AppEnv is the deployment environment (e.g. "production", "staging").
// Used by BuildTLSConfig to enforce PGSSLROOTCERT in production.
AppEnv string
}
FactoryConfig collects the inputs NewServerStores needs at startup. The fields are intentionally small so cmd/server and cmd/mcp can populate it from env (or from flags during tests) without dragging in framework state.
type ServerStores ¶
type ServerStores interface {
io.Closer
GTD() gtd.StoreIface
Workspace() workspace.StoreIface
Decision() decision.StoreIface
Session() session.StoreIface
Knowledge() knowledge.StoreIface
Learning() learning.StoreIface
Proposal() proposal.StoreIface
Arch() arch.StoreIface
WorkSession() worksession.StoreIface
Vision() vision.StoreIface
Playbook() playbook.StoreIface
Procedural() procedural.StoreIface
Atom() atom.StoreIface
Outcome() outcome.StoreIface
Skill() skill.StoreIface
Discipline() discipline.Store
Reflection() reflection.StoreIface
BehaviorRule() behaviorrule.StoreIface
DisciplineEventStore() watchdog.DisciplineEventStoreIface
// KnowledgePruner / LearningPruner return decay.PrunerStore for the
// active backend's knowledge / concepts stores. Both the Postgres and
// SQLite backends implement decay.PrunerStore via SoftPruneDecayed; the
// type assertion lives here (backend-internal detail) so callers such as
// cmd/server's buildPruner never need to type-assert a backend-specific
// type themselves. Returns nil when the concrete store does not
// implement the interface (unexpected backend) — the daily prune job for
// that table is then skipped gracefully, matching prior behavior.
KnowledgePruner() decay.PrunerStore
LearningPruner() decay.PrunerStore
// WorkspaceID returns the workspace UUID configured at startup, or nil
// when operating in legacy single-workspace mode. Used by MCP tools that
// need to scope writes (e.g. snapshot) without a raw pgxpool reference.
WorkspaceID() *uuid.UUID
// PgxPool returns the underlying pgx pool when this bundle is the
// Postgres backend, or nil for any other backend. Used only by code
// paths that legitimately need a pgx-typed transaction.
PgxPool() *pgxpool.Pool
// PgGTD / PgProposal / PgLearning / PgDecision / PgKnowledge / PgPlaybook
// return concrete *Store handles only when the bundle is Postgres-backed
// (so callers can WithTx(tx) on a pgx.Tx). They return nil on the SQLite
// bundle. See the type doc for the future migration path. PgKnowledge /
// PgPlaybook back internal/proposal's pgAcceptAdapter (the ADR 0003
// accept-seam contract).
PgGTD() *gtd.Store
PgProposal() *proposal.Store
PgLearning() *learning.Store
PgDecision() *decision.Store
PgKnowledge() *knowledge.Store
PgPlaybook() *playbook.Store
// SqliteGTD / SqliteProposal / SqliteLearning / SqliteDecision /
// SqliteKnowledge / SqlitePlaybook return concrete *Store handles only
// when the bundle is SQLite-backed, so callers can use the *Tx
// transactional helpers for atomic cross-store writes (e.g. the
// confirm_proposal accept path for type='decision'). They return nil on
// the Postgres bundle. SqliteKnowledge / SqlitePlaybook back
// internal/storage/sqlite's sqliteAcceptAdapter (the ADR 0003 accept-seam
// contract).
SqliteGTD() *wbtsqlite.GTDStore
SqliteProposal() *wbtsqlite.ProposalStore
SqliteLearning() *wbtsqlite.LearningStore
SqliteDecision() *wbtsqlite.DecisionStore
SqliteKnowledge() *wbtsqlite.KnowledgeStore
SqlitePlaybook() *wbtsqlite.PlaybookStore
// SqliteDB returns the underlying *wbtsqlite.DB when the bundle is
// SQLite-backed, or nil for the Postgres bundle. Used by domain stores
// (e.g. completioncandidate) that need sql.Rows-based list queries
// through the same connection pool as all other stores.
SqliteDB() *wbtsqlite.DB
}
ServerStores is the backend-agnostic store bundle that cmd/server and cmd/mcp consume. It exposes the domain Store interfaces plus a Close hook for the underlying connection (pgx pool or SQLite *sql.DB).
PgxPool returns the live *pgxpool.Pool when the bundle is Postgres-backed, or nil when the bundle is SQLite-backed. Callers that absolutely require a pgx transaction (currently only the MCP proposal-acceptance flow) MUST guard with `if pool := stores.PgxPool(); pool != nil { ... }` and provide a non-tx fallback for the SQLite path.
Concrete pg Store accessors (PgGTD / PgProposal / PgLearning / PgKnowledge / PgPlaybook) return the concrete *Store on the Postgres bundle so that the few code paths that need pgx-typed transactions (proposal materialisation across gtd / learning / proposal / knowledge / playbook) can call WithTx(tx) or the tx-scoped WriteItemTx/CreateTx-style methods; they return nil on the SQLite bundle. New backend-agnostic transactional code SHOULD NOT add more such accessors — the longer-term direction is a TxCoordinator inside the storage package, not pgx leaking into MCP. PgKnowledge / PgPlaybook / SqliteKnowledge / SqlitePlaybook (added for internal/proposal's accept-seam adapters) are a bounded exception under ADR 0003 (docs/adr/0003-dual-backend-orchestration-seam-principle.md), not a reversal of this guidance.
func BuildServerStores ¶
func BuildServerStores(ctx context.Context, backend Backend) (ServerStores, error)
BuildServerStores is the single env-reading entry point for cmd binaries. It reads DATABASE_URL / SQLITE_PATH / PGSSLROOTCERT / APP_ENV from the environment and calls NewServerStores so both cmd/server and `wbt mcp` (via internal/mcprunner) always use the same env variables and defaults without duplicating the switch.
func NewServerStores ¶
func NewServerStores(ctx context.Context, cfg FactoryConfig) (ServerStores, error)
NewServerStores returns a fully wired ServerStores bundle for the requested backend. It is the single entry point both cmd/server and cmd/mcp call so they stay free of backend-specific imports.
Caller MUST defer stores.Close() to release the underlying pool / DB.
Directories
¶
| Path | Synopsis |
|---|---|
|
Package sqlite is the SQLite-backed implementation of the wayneblacktea storage interfaces, intended for friend-grade self-hosting (one binary + one .db file, no Postgres server).
|
Package sqlite is the SQLite-backed implementation of the wayneblacktea storage interfaces, intended for friend-grade self-hosting (one binary + one .db file, no Postgres server). |