Documentation
¶
Overview ¶
Per-principal key derivation and envelope wrapping.
ENVELOPE MODEL (read before changing — this is what makes master-key rotation non-destructive):
- Each database gets its OWN random 256-bit Data Encryption Key (DEK), generated once at creation by NewDEK(). SQLCipher encrypts the file pages with this DEK and only this DEK — it never changes for the life of the file, so the ciphertext pages are never rewritten.
- The DEK is wrapped (AES-256-GCM) under a Key Encryption Key (KEK) derived from the KMS master key: KEK = HKDF-SHA256(masterKey, info = lp(principalType) || lp(id)) The wrapped DEK is stored beside the database (a small sidecar blob); the raw DEK is never written to disk.
- Opening: unwrap the stored DEK with the KEK, open SQLCipher with the DEK.
- Master-key ROTATION: unwrap the DEK with the OLD KEK, rewrap it with the NEW KEK, atomically replace the sidecar. The DEK is unchanged, so the encrypted pages are untouched — rotation is O(1) per database and cannot brick a file. (The legacy "derive the page key directly from the master" scheme made rotation impossible: a new master changed the page key and SQLCipher's HMAC rejected every existing file. Envelope dissolves that.)
`lp(x)` is the length-prefixed encoding of x (uvarint length || bytes). It is injective, so two principals can never collide across the type/id boundary (e.g. type "org"+id "a:b" and type "org:a"+id "b" hash differently — the old "%s:%s" form did not guarantee this).
Package sqlite provides an at-rest-encrypted SQLite driver for Hanzo.
ONE WAY, ALWAYS ENCRYPTED. A keyed store is encrypted on EVERY build — there is no build tag, environment switch, or missing C library under which it falls back to plaintext. EncryptionAvailable() is always true. The SQLCipher 4 page format is the single at-rest format, produced two byte-compatible ways:
- The pure-Go hanzoai/sqlcipher codec (the default, works everywhere): a keyed open decrypts the file to a RAM-backed plaintext copy, the engine reads and writes that copy, and Close/Checkpoint re-encrypts it to the real path (envelope.go). This needs no C toolchain and no libsqlcipher, so native Go (CGO_ENABLED=0) ALWAYS encrypts.
- The live libsqlcipher codec (an optional acceleration on cgo builds that link it): page-level AES-256 keyed inside sqlite3_open_v2, with per-commit durability. A cgo build links it with: CGO_ENABLED=1 \ CGO_CFLAGS="-DSQLITE_HAS_CODEC -DSQLITE_USE_URI=1 -I<sqlcipher>/include/sqlcipher" \ CGO_LDFLAGS="-lsqlcipher" \ go build -tags "libsqlite3 sqlite_fts5"
On a cgo build openDB uses the live codec when a runtime probe proves it encrypts (CodecLinked), and otherwise FALLS BACK to the pure-Go codec envelope — so a mis-linked libsqlcipher degrades to the pure-Go codec, never to plaintext. On the pure-Go build a key always routes to the envelope. Either way the file on disk is SQLCipher ciphertext, readable by the C library and by this package alike.
The database/sql driver name "sqlite" is registered under both build tags (hanzoai/csqlite on cgo, modernc.org/sqlite on pure-Go), so any code doing `_ "github.com/hanzoai/sqlite"` + `sql.Open("sqlite", dsn)` compiles and runs on both — one import, one driver name, one encryption format.
OPT-OUT BUILD TAG `sqlite_purego`: forces the pure-Go (modernc) backend even when CGO_ENABLED=1. It exists for one reason: a binary that links this fork AND another package importing modernc.org/sqlite directly would, under a plain CGO_ENABLED=1 build, register the "sqlite" driver TWICE and panic at init. Such a service builds with `-tags sqlite_purego` so the whole binary registers "sqlite" exactly once. It does NOT change the encryption guarantee: the pure-Go backend also always encrypts a keyed store, via the same codec.
The encryption key, replication mode, threshold config, and per-principal CEK derivation (cek.go, threshold.go) and the codec envelope (envelope.go) are pure Go and tag-neutral; only the driver registration and the DSN pragma syntax differ by build tag (driver_cgo.go / driver_nocgo.go).
Threshold write attestation for multi-party SQLite.
Each party runs a node with a full replica. Write operations require t-of-n parties to sign the write before it's committed. Reads are local.
Use case: MPC wallet shard storage, DEX trade approvals, multi-sig transaction authorization in trading platforms.
Index ¶
- Variables
- func Checkpoint(db *sql.DB) error
- func CodecLinked() bool
- func DSN(path string, rawKey []byte) string
- func DeriveChildKey(parentKey []byte, principalType PrincipalType, principalID string) ([]byte, error)
- func DeriveKey(masterKey []byte, principalType PrincipalType, principalID string) ([]byte, error)
- func EncryptionAvailable() bool
- func IsConstraintForeignKey(err error) bool
- func IsConstraintPrimaryKey(err error) bool
- func IsConstraintUnique(err error) bool
- func NewDEK() ([]byte, error)
- func OpenDB(path string, rawKey []byte) (*sql.DB, error)
- func OpenPragma(dsn string, pragmas []Pragma) (*sql.DB, error)
- func PragmaDSN(path string, pragmas []Pragma) string
- func PrincipalAAD(principalType PrincipalType, principalID string) []byte
- func SetPersistWAL(rawConn any, on bool) error
- func UnwrapDEK(kek, blob, aad []byte) ([]byte, error)
- func WrapDEK(kek, dek, aad []byte) ([]byte, error)
- type CommitHookFn
- type Config
- type DB
- type HookRegisterer
- type Mode
- type Option
- func WithCRDT(nodeID, listen string, peers []string) Option
- func WithKey(passphrase string) Option
- func WithPeers(peers []string) Option
- func WithPrincipalKey(masterKey []byte, principalType PrincipalType, principalID string) Option
- func WithRaft(nodeID, listen string, peers []string) Option
- func WithRawKey(key []byte) Option
- func WithThreshold(t, n int, signingKey ed25519.PrivateKey) Option
- type Pragma
- type PrincipalType
- type ThresholdManager
- func (tm *ThresholdManager) Attest(proposalID [32]byte, nodeID string, signature []byte) error
- func (tm *ThresholdManager) CleanExpired() int
- func (tm *ThresholdManager) Pending() int
- func (tm *ThresholdManager) Propose(sql string, params []any) ([32]byte, error)
- func (tm *ThresholdManager) RegisterPeer(nodeID string, pubKey ed25519.PublicKey)
- func (tm *ThresholdManager) SetCommitFunc(fn func(string, []any) error)
- type WriteProposal
Constants ¶
This section is empty.
Variables ¶
var ( ErrThresholdNotMet = errors.New("sqlite: threshold attestations not met") ErrDuplicateSigner = errors.New("sqlite: duplicate signer") ErrInvalidSig = errors.New("sqlite: invalid attestation signature") ErrSessionExpired = errors.New("sqlite: attestation session expired") )
var DefaultPragmas = []Pragma{
{Name: "busy_timeout", Value: "10000"},
{Name: "journal_mode", Value: "WAL"},
{Name: "journal_size_limit", Value: "200000000"},
{Name: "synchronous", Value: "NORMAL"},
{Name: "foreign_keys", Value: "ON"},
{Name: "temp_store", Value: "MEMORY"},
{Name: "cache_size", Value: "-32000"},
}
DefaultPragmas is the canonical Hanzo embedded-SQLite connection tuning. Order matters: busy_timeout MUST lead so a connection blocks on a busy database before journal_mode=WAL is set (WAL cannot be enabled while another connection holds the database). This is the single source of truth for the unencrypted default-connect DSN used across Hanzo Base (core + multitenant store) — treat it as read-only.
Functions ¶
func Checkpoint ¶ added in v0.4.0
Checkpoint re-encrypts an open envelope-backed database to its encrypted file WITHOUT closing it, so the ciphertext on disk reflects committed writes up to this point. It bounds the crash-loss window between the coarse encrypt-on-close events. On a database that is not envelope-backed (the live-libsqlcipher or unencrypted paths, which persist per-commit) it is a successful no-op.
func CodecLinked ¶ added in v0.1.3
func CodecLinked() bool
CodecLinked reports whether the SQLCipher C codec is linked and ACTUALLY encrypting, proven by a cached one-time runtime probe — NOT a compile-time constant.
The distinction is the whole point. A cgo build only has a live C codec if the SQLite it links is SQLCipher. A cgo build that links plain sqlite (the vendored amalgamation, or libsqlcipher absent) SILENTLY no-ops PRAGMA key and writes PLAINTEXT. A compile-time `return true` could not tell those apart, so it once reported the codec linked for a build that ships plaintext.
This probes the linked engine at runtime (codecEncrypts): open a throwaway keyed database, write a sentinel, and confirm the bytes on disk are real ciphertext that round-trips under the key. The result is cached. When it is true, openDB uses the live libsqlcipher codec (per-commit durability); when it is FALSE, openDB falls back to the pure-Go SQLCipher codec envelope (envelope.go), which encrypts without libsqlcipher — so a keyed store is encrypted either way, never plaintext.
func DSN ¶ added in v0.1.2
DSN builds a canonical IAM SQLite DSN for the active (CGO/csqlite/SQLCipher) backend.
When rawKey is non-nil it is emitted as SQLCipher's native `key=x'HEX'` URI parameter, which SQLCipher applies inside sqlite3_open_v2 — before mattn's pragma battery — making the connection reopen-safe. The DSN consequently CONTAINS the key; callers MUST NOT log it.
rawKey must be exactly 32 bytes when non-nil (validated by Open; OpenDB trusts its caller). A nil key yields an unencrypted DSN for the global/dev/test engine.
The path is percent-escaped (escapeDBPath): a path containing '?' or '#' would otherwise prematurely terminate the file portion of the DSN and DROP the trailing query params — including `key=` — which would silently open the database UNENCRYPTED. Escaping makes that impossible regardless of caller input.
func DeriveChildKey ¶ added in v0.1.3
func DeriveChildKey(parentKey []byte, principalType PrincipalType, principalID string) ([]byte, error)
DeriveChildKey derives a child KEK from a PARENT key (not the master), establishing a key hierarchy: master → per-org KEK → per-user KEK. A per-user database's DEK is therefore wrapped under a KEK that is itself bound to the org, so compromising the master alone is not sufficient without traversing the hierarchy, and an org's user keys are cryptographically contained within that org.
parentKey: 32-byte parent key (e.g. the org KEK from DeriveKey) principalType: child principal type (typically PrincipalUser) principalID: child identifier (user ID)
func DeriveKey ¶
func DeriveKey(masterKey []byte, principalType PrincipalType, principalID string) ([]byte, error)
DeriveKey derives a 256-bit key for a principal from a master key using HKDF-SHA256 with a length-prefixed, injective `info`.
In the envelope model this is the KEK (it WRAPS a per-database DEK); it is no longer used as a SQLCipher page key directly. Callers that need a page key use NewDEK + WrapDEK/UnwrapDEK.
masterKey: 32-byte master encryption key (from KMS) principalType: "global", "org", or "user" principalID: unique identifier (org slug, user ID, "iam" for global)
func EncryptionAvailable ¶ added in v0.1.2
func EncryptionAvailable() bool
EncryptionAvailable reports whether this build can encrypt a keyed store at rest. It is ALWAYS true: the pure-Go hanzoai/sqlcipher codec is compiled into every build, so a keyed open always encrypts — through the live libsqlcipher codec when it is linked and proven (CodecLinked), otherwise through the pure-Go codec envelope (envelope.go). There is no build, and no build-tag or environment switch, under which a keyed store falls back to plaintext.
It is a value, not a capability probe: "can a keyed store be encrypted here?" is unconditionally yes. Which mechanism does it — live C codec vs pure-Go envelope — is CodecLinked's concern, and a broken libsqlcipher is caught by that runtime probe (never silent plaintext), which is orthogonal to whether encryption is available at all.
func IsConstraintForeignKey ¶ added in v0.2.2
IsConstraintForeignKey reports whether err is a FOREIGN KEY-constraint violation from the active SQLite backend.
func IsConstraintPrimaryKey ¶ added in v0.2.2
IsConstraintPrimaryKey reports whether err is a PRIMARY KEY-constraint violation from the active SQLite backend.
func IsConstraintUnique ¶ added in v0.2.2
IsConstraintUnique reports whether err is a UNIQUE-constraint violation from the active SQLite backend. It unwraps via errors.As, so a wrapped error is still classified. Returns false for a nil or non-sqlite error.
func NewDEK ¶ added in v0.1.3
NewDEK generates a fresh random 256-bit Data Encryption Key. Each database is created with exactly one DEK, which never changes for the life of the file.
func OpenDB ¶ added in v0.1.2
OpenDB opens *path* as a database/sql DB on the CGO/SQLCipher backend with the given raw 256-bit key. It is the entry point for callers that need a *sql.DB (e.g. to hand to xorm.NewEngineWithDB). A nil key opens unencrypted. The returned DB is NOT pinged here; the caller (or Open) pings.
func OpenPragma ¶ added in v0.2.3
OpenPragma opens a *sql.DB on the registered "sqlite" backend where EVERY pooled connection has the given pragmas applied, in order, via PRAGMA statements run at connect time.
Use it for pragmas that must hold on every connection but that the two backends do NOT accept uniformly in the DSN. The load-bearing case is wal_autocheckpoint: modernc honors `?_pragma=wal_autocheckpoint(0)` in the DSN, but mattn silently DROPS it (mattn's DSN grammar has no such param), so a binary built CGO=1 would re-enable auto-checkpoint and truncate the WAL out from under a WAL-shipping replicator — losing committed frames. Running the pragma per connection via ExecContext is backend-neutral (both mattn and modernc conns implement driver.ExecerContext), so it takes effect either way.
Order matters: put busy_timeout first so a connection blocks on a busy database before journal_mode=WAL is attempted (WAL can't be set while another connection holds the db).
dsn is a plain DSN — typically "file:"+path, or DSN(path, key) when the file is SQLCipher-encrypted (the key must ride the DSN; pragmas can't carry it). Do NOT also put these pragmas in the DSN; they're applied here.
func PragmaDSN ¶ added in v0.2.1
PragmaDSN builds an unencrypted `file:` DSN for path that applies pragmas in mattn's `_NAME=VALUE` syntax. mattn recognises _busy_timeout, _journal_mode, _synchronous, _foreign_keys and _cache_size and silently ignores any other parameter — so pragmas it cannot express as a DSN param (journal_size_limit, temp_store) are inert here but harmless, while the correctness-critical ones (busy_timeout, WAL) DO apply. The path is percent-escaped (escapeDBPath) so a '?'/'#' in it cannot strip the query. No key is added — keyed/encrypted opens go through OpenDB/DSN.
func PrincipalAAD ¶ added in v0.1.4
func PrincipalAAD(principalType PrincipalType, principalID string) []byte
PrincipalAAD returns the injective binding context for a principal, suitable as the AES-256-GCM additional-authenticated-data when wrapping that principal's DEK (WrapDEK/UnwrapDEK). It is the SAME length-prefixed, injective encoding used for the KEK's HKDF info — one encoding, two uses (DRY):
- HKDF info → domain-separates the KEK per principal.
- GCM AAD → binds the wrapped blob to its principal, so a sidecar lifted from one principal can never be unwrapped under another even if a KEK derivation ever collided. Defense-in-depth atop the KEK separation.
Pass the value to WrapDEK/UnwrapDEK; the same (type,id) MUST be used to wrap and to unwrap, or the GCM tag check fails.
func SetPersistWAL ¶ added in v0.2.3
SetPersistWAL sets SQLite's SQLITE_FCNTL_PERSIST_WAL file control on rawConn — the raw driver connection handed to the callback of (*sql.Conn).Raw. When on, the -wal file is kept across the last connection close (SQLite otherwise deletes it), which WAL-shipping replication (hanzoai/replicate) requires so a crash mid-sync doesn't lose committed frames.
It is the one backend-neutral way to set PERSIST_WAL; consumers never touch the csqlite/modernc conn type directly. Returns an error if rawConn is not this build's SQLite connection type.
CGO backend: csqlite exposes SetFileControlInt; the op constant is csqlite's own SQLITE_FCNTL_PERSIST_WAL (= 10).
func UnwrapDEK ¶ added in v0.1.3
UnwrapDEK opens a blob produced by WrapDEK under the same KEK and the same aad. A wrong KEK, wrong aad (e.g. a sidecar from a different principal), truncated blob, or tampered ciphertext fails the GCM tag and returns an error — never a partial/garbage key.
func WrapDEK ¶ added in v0.1.3
WrapDEK seals a DEK under a KEK with AES-256-GCM and returns the storable blob: version(1) || nonce(12) || ciphertext||tag. The KEK must be 32 bytes (as produced by DeriveKey / DeriveChildKey). The blob is safe to store next to the database; it reveals nothing about the DEK without the KEK.
aad is additional binding context authenticated (but not encrypted) by GCM — pass PrincipalAAD(type,id) so the blob is cryptographically bound to its principal (a sidecar moved to another principal fails the tag, defense-in-depth atop the per-principal KEK). The same aad MUST be supplied to UnwrapDEK. Pass nil for an unbound blob (e.g. the standalone Open() helper). The on-disk AAD is version-byte || aad, so a downgrade is also unforgeable.
Types ¶
type CommitHookFn ¶ added in v0.2.1
type CommitHookFn func() int32
CommitHookFn is a commit-hook callback. Returning a non-zero value ABORTS the commit (SQLite rolls back and reports SQLITE_BUSY); returning zero lets it proceed. This is the one callback type consumers use regardless of backend — the build-tagged adapter converts it to the backend's native signature.
type Config ¶
type Config struct {
// Encryption
RawKey []byte // raw 256-bit key (skips KDF); nil means unencrypted
// Replication
Mode Mode
NodeID string
Listen string // bind address for replication
Peers []string // peer addresses
// Threshold mode
Threshold int // t value (signatures required)
Parties int // n value (total parties)
SigningKey ed25519.PrivateKey // this node's signing key
// contains filtered or unexported fields
}
Config for opening a database.
type DB ¶
DB wraps sql.DB with replication and encryption.
func Open ¶
Open opens an at-rest-encrypted (when keyed), optionally distributed SQLite database. Passing an encryption key (WithKey/WithRawKey/WithPrincipalKey) always encrypts, on every build: the SQLCipher codec (live libsqlcipher when linked, otherwise the pure-Go codec envelope) keys the file. It never writes a keyed store as plaintext.
type HookRegisterer ¶ added in v0.2.1
type HookRegisterer interface {
RegisterCommitHook(CommitHookFn)
}
HookRegisterer installs a commit hook on a driver connection. Values satisfying it are produced by CommitHookRegisterer, never by asserting a raw backend conn directly (see the note above). Passing a nil CommitHookFn clears any installed hook.
func CommitHookRegisterer ¶ added in v0.2.1
func CommitHookRegisterer(driverConn any) (HookRegisterer, bool)
CommitHookRegisterer adapts a raw csqlite driver connection (obtained from (*sql.Conn).Raw) to HookRegisterer. csqlite's *SQLiteConn commit-hook type is `func() int` (not int32), so the adapter widens our CommitHookFn's int32 result to csqlite's int. This is the SAME engine driver_cgo.go registers as the "sqlite" driver name. Returns (nil, false) if driverConn is not a csqlite connection.
type Option ¶
type Option func(*Config)
Option configures a database.
func WithKey ¶
WithKey derives the raw 256-bit page key from a passphrase with a SINGLE, UNSALTED SHA-256 — NOT a slow, salted KDF. It is therefore safe ONLY for a high-entropy passphrase (e.g. a base64-encoded random secret); a human-memorable passphrase is brute-forceable because there is no PBKDF2 stretching and no salt.
A slow KDF is deliberately not applied here: the SQLCipher raw-key form takes the 32 bytes directly, and the file salt (which would seed a KDF) is not known when an Option is constructed. Production keys come from KMS already uniformly random — use WithRawKey / OpenDB(path, key) with those. For a low-entropy passphrase, derive a key out of band with a salted KDF (argon2id / PBKDF2) and pass it via WithRawKey.
func WithPrincipalKey ¶
func WithPrincipalKey(masterKey []byte, principalType PrincipalType, principalID string) Option
WithPrincipalKey derives a KEK, generates/uses a DEK, and configures the database to use the page key. Retained for the standalone Open() helper path; the envelope sidecar lifecycle (which is what production IAM uses) is driven by the consumer (object/orgdb.go) via NewDEK + WrapDEK/UnwrapDEK + OpenDB.
Deprecated for the lifecycle path: this derives a page key directly and so is NOT rotation-safe. Use the envelope helpers for any persistent database.
func WithRawKey ¶
WithRawKey sets a raw 256-bit encryption key (skips KDF).
func WithThreshold ¶
func WithThreshold(t, n int, signingKey ed25519.PrivateKey) Option
WithThreshold enables multi-party threshold attestation for writes.
type Pragma ¶ added in v0.2.1
type Pragma struct{ Name, Value string }
Pragma is one SQLite PRAGMA (name + value) applied at connection-open time via the DSN.
type PrincipalType ¶
type PrincipalType string
PrincipalType identifies the type of principal for key derivation. It forms the domain-separation tag of the derived KEK.
const ( // PrincipalGlobal is the cross-org global/platform database (certs, // providers, the admin org catalog). Distinct from any org named "global". PrincipalGlobal PrincipalType = "global" PrincipalOrg PrincipalType = "org" PrincipalUser PrincipalType = "user" )
type ThresholdManager ¶
type ThresholdManager struct {
// contains filtered or unexported fields
}
ThresholdManager coordinates multi-party write attestation.
func NewThresholdManager ¶
func NewThresholdManager(threshold, parties int, nodeID string, signingKey ed25519.PrivateKey) *ThresholdManager
NewThresholdManager creates a threshold write coordinator.
func (*ThresholdManager) Attest ¶
func (tm *ThresholdManager) Attest(proposalID [32]byte, nodeID string, signature []byte) error
Attest adds a peer's attestation to a pending proposal.
func (*ThresholdManager) CleanExpired ¶
func (tm *ThresholdManager) CleanExpired() int
CleanExpired removes expired proposals.
func (*ThresholdManager) Pending ¶
func (tm *ThresholdManager) Pending() int
Pending returns the number of pending proposals.
func (*ThresholdManager) Propose ¶
func (tm *ThresholdManager) Propose(sql string, params []any) ([32]byte, error)
Propose creates a new write proposal. Returns the proposal ID.
func (*ThresholdManager) RegisterPeer ¶
func (tm *ThresholdManager) RegisterPeer(nodeID string, pubKey ed25519.PublicKey)
RegisterPeer adds a peer's public key for attestation verification.
func (*ThresholdManager) SetCommitFunc ¶
func (tm *ThresholdManager) SetCommitFunc(fn func(string, []any) error)
SetCommitFunc sets the function called when threshold is met.
type WriteProposal ¶
type WriteProposal struct {
ID [32]byte // SHA-256 of the SQL + params
SQL string
Params []any
Proposer string // node ID of proposer
CreatedAt time.Time
ExpiresAt time.Time
// contains filtered or unexported fields
}
WriteProposal is a proposed write that needs t-of-n attestations.