Documentation
¶
Overview ¶
Per-principal key derivation and envelope wrapping.
ENVELOPE MODEL (read before changing — this is what makes master-key rotation non-destructive):
- Each database gets its OWN random 256-bit Data Encryption Key (DEK), generated once at creation by NewDEK(). SQLCipher encrypts the file pages with this DEK and only this DEK — it never changes for the life of the file, so the ciphertext pages are never rewritten.
- The DEK is wrapped (AES-256-GCM) under a Key Encryption Key (KEK) derived from the KMS master key: KEK = HKDF-SHA256(masterKey, info = lp(principalType) || lp(id)) The wrapped DEK is stored beside the database (a small sidecar blob); the raw DEK is never written to disk.
- Opening: unwrap the stored DEK with the KEK, open SQLCipher with the DEK.
- Master-key ROTATION: unwrap the DEK with the OLD KEK, rewrap it with the NEW KEK, atomically replace the sidecar. The DEK is unchanged, so the encrypted pages are untouched — rotation is O(1) per database and cannot brick a file. (The legacy "derive the page key directly from the master" scheme made rotation impossible: a new master changed the page key and SQLCipher's HMAC rejected every existing file. Envelope dissolves that.)
`lp(x)` is the length-prefixed encoding of x (uvarint length || bytes). It is injective, so two principals can never collide across the type/id boundary (e.g. type "org"+id "a:b" and type "org:a"+id "b" hash differently — the old "%s:%s" form did not guarantee this).
Package sqlite provides an encrypted SQLite driver for Hanzo.
Both backends encrypt at rest, in the SAME SQLCipher-4 on-disk format, and a database written by one opens under the other. The driver registers the database/sql driver name "sqlite" under BOTH build configurations, exposing the same public API either way:
- !CGO (//go:build !cgo) → the vendored pure-Go SQLite engine (internal/engine) plus the hanzoai/sqlcipher codec VFS: page-level AES-256 encryption at rest, with NO cgo and NO external C library. This is what CGO-off CI, tests, lint, and pure-Go deployments run. Nothing to link, no amalgamation to build — it always encrypts.
- CGO (//go:build cgo) → hanzoai/csqlite + libsqlcipher: the same page-level AES-256 format via the C codec, for builds that want the C engine's speed. Build with: CGO_ENABLED=1 \ CGO_CFLAGS="-DSQLITE_HAS_CODEC -DSQLITE_USE_URI=1 -I<sqlcipher>/include/sqlcipher" \ CGO_LDFLAGS="-lsqlcipher" \ go build -tags "libsqlite3 sqlite_fts5" A cgo build that forgets to link libsqlcipher silently writes plaintext; CodecLinked() proves the codec at runtime so such a build fails CI.
Because the "sqlite" driver name is registered under both tags, any code doing `_ "github.com/hanzoai/sqlite"` + `sql.Open("sqlite", dsn)` (e.g. Hanzo IAM's xorm engine) compiles and runs encrypted under both — one import, one driver name, two backends, one format.
The encryption key, replication mode, threshold config, and per-principal CEK derivation (cek.go, threshold.go) are pure Go and live in tag-neutral files; only the database/sql driver registration and the connect-time keying differ by build tag (driver_cgo.go / driver_nocgo.go).
Threshold write attestation for multi-party SQLite.
Each party runs a node with a full replica. Write operations require t-of-n parties to sign the write before it's committed. Reads are local.
Use case: MPC wallet shard storage, DEX trade approvals, multi-sig transaction authorization in trading platforms.
Index ¶
- Variables
- func CodecLinked() bool
- func DSN(path string, rawKey []byte) string
- func DeriveChildKey(parentKey []byte, principalType PrincipalType, principalID string) ([]byte, error)
- func DeriveKey(masterKey []byte, principalType PrincipalType, principalID string) ([]byte, error)
- func EncryptionAvailable() bool
- func IsConstraintForeignKey(err error) bool
- func IsConstraintPrimaryKey(err error) bool
- func IsConstraintUnique(err error) bool
- func NewDEK() ([]byte, error)
- func OpenDB(path string, rawKey []byte) (*sql.DB, error)
- func OpenPragma(dsn string, pragmas []Pragma) (*sql.DB, error)
- func PragmaDSN(path string, pragmas []Pragma) string
- func PrincipalAAD(principalType PrincipalType, principalID string) []byte
- func SetPersistWAL(rawConn any, on bool) error
- func UnwrapDEK(kek, blob, aad []byte) ([]byte, error)
- func WrapDEK(kek, dek, aad []byte) ([]byte, error)
- type CommitHookFn
- type Config
- type DB
- type HookRegisterer
- type Mode
- type Option
- func WithCRDT(nodeID, listen string, peers []string) Option
- func WithKey(passphrase string) Option
- func WithPeers(peers []string) Option
- func WithPrincipalKey(masterKey []byte, principalType PrincipalType, principalID string) Option
- func WithRaft(nodeID, listen string, peers []string) Option
- func WithRawKey(key []byte) Option
- func WithThreshold(t, n int, signingKey ed25519.PrivateKey) Option
- type Pragma
- type PrincipalType
- type ThresholdManager
- func (tm *ThresholdManager) Attest(proposalID [32]byte, nodeID string, signature []byte) error
- func (tm *ThresholdManager) CleanExpired() int
- func (tm *ThresholdManager) Pending() int
- func (tm *ThresholdManager) Propose(sql string, params []any) ([32]byte, error)
- func (tm *ThresholdManager) RegisterPeer(nodeID string, pubKey ed25519.PublicKey)
- func (tm *ThresholdManager) SetCommitFunc(fn func(string, []any) error)
- type WriteProposal
Constants ¶
This section is empty.
Variables ¶
var ( ErrThresholdNotMet = errors.New("sqlite: threshold attestations not met") ErrDuplicateSigner = errors.New("sqlite: duplicate signer") ErrInvalidSig = errors.New("sqlite: invalid attestation signature") ErrSessionExpired = errors.New("sqlite: attestation session expired") )
var DefaultPragmas = []Pragma{
{Name: "busy_timeout", Value: "10000"},
{Name: "journal_mode", Value: "WAL"},
{Name: "journal_size_limit", Value: "200000000"},
{Name: "synchronous", Value: "NORMAL"},
{Name: "foreign_keys", Value: "ON"},
{Name: "temp_store", Value: "MEMORY"},
{Name: "cache_size", Value: "-32000"},
}
DefaultPragmas is the canonical Hanzo embedded-SQLite connection tuning. Order matters: busy_timeout MUST lead so a connection blocks on a busy database before journal_mode=WAL is set (WAL cannot be enabled while another connection holds the database). This is the single source of truth for the unencrypted default-connect DSN used across Hanzo Base — treat it as read-only.
ErrEncryptionUnavailable is retained for API symmetry with the pure-Go backend. It is never returned by the CGO backend, which encrypts through libsqlcipher.
Functions ¶
func CodecLinked ¶ added in v0.1.3
func CodecLinked() bool
CodecLinked reports whether at-rest encryption is ACTUALLY working in this process — not merely advertised. EncryptionAvailable() is a backend-capability flag (true under both build tags), but a CGO build that forgot to link libsqlcipher silently writes PLAINTEXT. CodecLinked proves the codec at runtime: it opens a temp DB under a key, writes a marker, and checks the on-disk bytes are real ciphertext (no plaintext "SQLite format 3" header, marker absent).
The pure-Go backend always returns true — its codec VFS is compiled in, nothing to mis-link. On the CGO backend it returns true only when libsqlcipher is actually linked. Use it to gate encryption assertions in tests: the pure-Go build and a correctly-linked CGO build run them; a CGO-without-codec build SKIPS instead of failing on plaintext (the Dockerfile build is the hard gate there).
func DSN ¶ added in v0.1.2
DSN builds a canonical IAM SQLite DSN for the active (CGO/csqlite/SQLCipher) backend.
When rawKey is non-nil it is emitted as SQLCipher's native `key=x'HEX'` URI parameter, which SQLCipher applies inside sqlite3_open_v2 — before any pragma — making the connection reopen-safe. The DSN consequently CONTAINS the key; callers MUST NOT log it.
rawKey must be exactly 32 bytes when non-nil (validated by Open; OpenDB trusts its caller). A nil key yields an unencrypted DSN for the global/dev/test engine.
The path is percent-escaped (escapeDBPath): a path containing '?' or '#' would otherwise prematurely terminate the file portion of the DSN and DROP the trailing query params — including `key=` — which would silently open the database UNENCRYPTED. Escaping makes that impossible regardless of caller input.
func DeriveChildKey ¶ added in v0.1.3
func DeriveChildKey(parentKey []byte, principalType PrincipalType, principalID string) ([]byte, error)
DeriveChildKey derives a child KEK from a PARENT key (not the master), establishing a key hierarchy: master → per-org KEK → per-user KEK. A per-user database's DEK is therefore wrapped under a KEK that is itself bound to the org, so compromising the master alone is not sufficient without traversing the hierarchy, and an org's user keys are cryptographically contained within that org.
parentKey: 32-byte parent key (e.g. the org KEK from DeriveKey) principalType: child principal type (typically PrincipalUser) principalID: child identifier (user ID)
func DeriveKey ¶
func DeriveKey(masterKey []byte, principalType PrincipalType, principalID string) ([]byte, error)
DeriveKey derives a 256-bit key for a principal from a master key using HKDF-SHA256 with a length-prefixed, injective `info`.
In the envelope model this is the KEK (it WRAPS a per-database DEK); it is no longer used as a SQLCipher page key directly. Callers that need a page key use NewDEK + WrapDEK/UnwrapDEK.
masterKey: 32-byte master encryption key (from KMS) principalType: "global", "org", or "user" principalID: unique identifier (org slug, user ID, "iam" for global)
func EncryptionAvailable ¶ added in v0.1.2
func EncryptionAvailable() bool
EncryptionAvailable reports that this build's backend can encrypt at rest.
It returns true for the CGO backend, which is a BACKEND-capability flag, not a proof that libsqlcipher is actually linked: a CGO build WITHOUT the codec links plain sqlite and silently no-ops the key (writes plaintext). The production Dockerfile links libsqlcipher; the package's encryption-proof test verifies real ciphertext at runtime (CodecLinked) so a mis-linked build fails CI instead of shipping plaintext.
func IsConstraintForeignKey ¶ added in v0.2.2
IsConstraintForeignKey reports whether err is a FOREIGN KEY-constraint violation from the active SQLite backend.
func IsConstraintPrimaryKey ¶ added in v0.2.2
IsConstraintPrimaryKey reports whether err is a PRIMARY KEY-constraint violation from the active SQLite backend.
func IsConstraintUnique ¶ added in v0.2.2
IsConstraintUnique reports whether err is a UNIQUE-constraint violation from the active SQLite backend. It unwraps via errors.As, so a wrapped error is still classified. Returns false for a nil or non-sqlite error.
func NewDEK ¶ added in v0.1.3
NewDEK generates a fresh random 256-bit Data Encryption Key. Each database is created with exactly one DEK, which never changes for the life of the file.
func OpenDB ¶ added in v0.1.2
OpenDB opens *path* as a database/sql DB on the CGO/SQLCipher backend with the given raw 256-bit key. It is the entry point for callers that need a *sql.DB (e.g. to hand to xorm.NewEngineWithDB). A nil key opens unencrypted. The returned DB is NOT pinged here; the caller (or Open) pings.
func OpenPragma ¶ added in v0.2.3
OpenPragma opens a *sql.DB on the registered "sqlite" backend where EVERY pooled connection has the given pragmas applied, in order, via PRAGMA statements run at connect time.
Use it for pragmas that must hold on every connection but that the two backends do NOT accept uniformly in the DSN. The load-bearing case is wal_autocheckpoint: the pure-Go engine honors `?_pragma=wal_autocheckpoint(0)` in the DSN, but csqlite silently DROPS it (its DSN grammar has no such param), so a binary built CGO=1 would re-enable auto-checkpoint and truncate the WAL out from under a WAL-shipping replicator — losing committed frames. Running the pragma per connection via ExecContext is backend-neutral (both conns implement driver.ExecerContext), so it takes effect either way.
Order matters: put busy_timeout first so a connection blocks on a busy database before journal_mode=WAL is attempted (WAL can't be set while another connection holds the db).
dsn is a plain (unencrypted) DSN — typically "file:"+path. Encrypted opens go through OpenDB (the key binds to the codec VFS, not the DSN). Do NOT also put these pragmas in the DSN; they're applied here.
func PragmaDSN ¶ added in v0.2.1
PragmaDSN builds an unencrypted `file:` DSN for path that applies pragmas in csqlite's `_NAME=VALUE` syntax. csqlite recognises _busy_timeout, _journal_mode, _synchronous, _foreign_keys and _cache_size and silently ignores any other parameter — so pragmas it cannot express as a DSN param (journal_size_limit, temp_store) are inert here but harmless, while the correctness-critical ones (busy_timeout, WAL) DO apply. The path is percent-escaped (escapeDBPath) so a '?'/'#' in it cannot strip the query. No key is added — keyed/encrypted opens go through OpenDB/DSN.
func PrincipalAAD ¶ added in v0.1.4
func PrincipalAAD(principalType PrincipalType, principalID string) []byte
PrincipalAAD returns the injective binding context for a principal, suitable as the AES-256-GCM additional-authenticated-data when wrapping that principal's DEK (WrapDEK/UnwrapDEK). It is the SAME length-prefixed, injective encoding used for the KEK's HKDF info — one encoding, two uses (DRY):
- HKDF info → domain-separates the KEK per principal.
- GCM AAD → binds the wrapped blob to its principal, so a sidecar lifted from one principal can never be unwrapped under another even if a KEK derivation ever collided. Defense-in-depth atop the KEK separation.
Pass the value to WrapDEK/UnwrapDEK; the same (type,id) MUST be used to wrap and to unwrap, or the GCM tag check fails.
func SetPersistWAL ¶ added in v0.2.3
SetPersistWAL sets SQLite's SQLITE_FCNTL_PERSIST_WAL file control on rawConn — the raw driver connection handed to the callback of (*sql.Conn).Raw. When on, the -wal file is kept across the last connection close (SQLite otherwise deletes it), which WAL-shipping replication (hanzoai/replicate) requires so a crash mid-sync doesn't lose committed frames.
It is the one backend-neutral way to set PERSIST_WAL; consumers never touch the concrete conn type directly. Returns an error if rawConn is not this build's SQLite connection type.
CGO backend: csqlite exposes SetFileControlInt; the op constant is csqlite's own SQLITE_FCNTL_PERSIST_WAL (= 10).
func UnwrapDEK ¶ added in v0.1.3
UnwrapDEK opens a blob produced by WrapDEK under the same KEK and the same aad. A wrong KEK, wrong aad (e.g. a sidecar from a different principal), truncated blob, or tampered ciphertext fails the GCM tag and returns an error — never a partial/garbage key.
func WrapDEK ¶ added in v0.1.3
WrapDEK seals a DEK under a KEK with AES-256-GCM and returns the storable blob: version(1) || nonce(12) || ciphertext||tag. The KEK must be 32 bytes (as produced by DeriveKey / DeriveChildKey). The blob is safe to store next to the database; it reveals nothing about the DEK without the KEK.
aad is additional binding context authenticated (but not encrypted) by GCM — pass PrincipalAAD(type,id) so the blob is cryptographically bound to its principal (a sidecar moved to another principal fails the tag, defense-in-depth atop the per-principal KEK). The same aad MUST be supplied to UnwrapDEK. Pass nil for an unbound blob (e.g. the standalone Open() helper). The on-disk AAD is version-byte || aad, so a downgrade is also unforgeable.
Types ¶
type CommitHookFn ¶ added in v0.2.1
type CommitHookFn func() int32
CommitHookFn is the commit-hook callback under the CGO backend: a distinct func() int32 (csqlite has no matching exported type), adapted to csqlite's native func() int by CommitHookRegisterer. Consumers cannot assert the raw csqlite conn to HookRegisterer under cgo; they must use CommitHookRegisterer.
type Config ¶
type Config struct {
// Encryption
RawKey []byte // raw 256-bit key (skips KDF); nil means unencrypted
// Replication
Mode Mode
NodeID string
Listen string // bind address for replication
Peers []string // peer addresses
// Threshold mode
Threshold int // t value (signatures required)
Parties int // n value (total parties)
SigningKey ed25519.PrivateKey // this node's signing key
// contains filtered or unexported fields
}
Config for opening a database.
type DB ¶
DB wraps sql.DB with replication and encryption.
func Open ¶
Open opens an encrypted, optionally distributed SQLite database.
A key (WithKey/WithRawKey/WithPrincipalKey) encrypts at rest under BOTH backends in the same SQLCipher-4 format: the pure-Go engine keys through the hanzoai/sqlcipher codec VFS, the CGO engine through libsqlcipher. A 32-byte key is required; EncryptionAvailable() is true either way, so the only reason Open rejects a key is a wrong length (never "no encryption on this backend").
type HookRegisterer ¶ added in v0.2.1
type HookRegisterer interface {
RegisterCommitHook(CommitHookFn)
}
HookRegisterer installs a commit hook on a driver connection. Passing a nil CommitHookFn clears any installed hook. Under !cgo the raw engine conn satisfies this directly; under cgo, obtain one via CommitHookRegisterer.
func CommitHookRegisterer ¶ added in v0.2.1
func CommitHookRegisterer(driverConn any) (HookRegisterer, bool)
CommitHookRegisterer adapts a raw csqlite driver connection (obtained from (*sql.Conn).Raw) to HookRegisterer. csqlite's *SQLiteConn commit-hook type is `func() int` (not int32), so the adapter widens our CommitHookFn's int32 result to csqlite's int. This is the SAME engine driver_cgo.go registers as the "sqlite" driver name. Returns (nil, false) if driverConn is not a csqlite connection.
type Option ¶
type Option func(*Config)
Option configures a database.
func WithKey ¶
WithKey derives a raw 256-bit key from a passphrase via SHA-256 and configures sqlcipher to use it directly (skipping KDF).
func WithPrincipalKey ¶
func WithPrincipalKey(masterKey []byte, principalType PrincipalType, principalID string) Option
WithPrincipalKey derives a KEK, generates/uses a DEK, and configures the database to use the page key. Retained for the standalone Open() helper path; the envelope sidecar lifecycle (which is what production IAM uses) is driven by the consumer (object/orgdb.go) via NewDEK + WrapDEK/UnwrapDEK + OpenDB.
Deprecated for the lifecycle path: this derives a page key directly and so is NOT rotation-safe. Use the envelope helpers for any persistent database.
func WithRawKey ¶
WithRawKey sets a raw 256-bit encryption key (skips KDF).
func WithThreshold ¶
func WithThreshold(t, n int, signingKey ed25519.PrivateKey) Option
WithThreshold enables multi-party threshold attestation for writes.
type Pragma ¶ added in v0.2.1
type Pragma struct{ Name, Value string }
Pragma is one SQLite PRAGMA (name + value) applied at connection-open time.
type PrincipalType ¶
type PrincipalType string
PrincipalType identifies the type of principal for key derivation. It forms the domain-separation tag of the derived KEK.
const ( // PrincipalGlobal is the cross-org global/platform database (certs, // providers, the admin org catalog). Distinct from any org named "global". PrincipalGlobal PrincipalType = "global" PrincipalOrg PrincipalType = "org" PrincipalUser PrincipalType = "user" )
type ThresholdManager ¶
type ThresholdManager struct {
// contains filtered or unexported fields
}
ThresholdManager coordinates multi-party write attestation.
func NewThresholdManager ¶
func NewThresholdManager(threshold, parties int, nodeID string, signingKey ed25519.PrivateKey) *ThresholdManager
NewThresholdManager creates a threshold write coordinator.
func (*ThresholdManager) Attest ¶
func (tm *ThresholdManager) Attest(proposalID [32]byte, nodeID string, signature []byte) error
Attest adds a peer's attestation to a pending proposal.
func (*ThresholdManager) CleanExpired ¶
func (tm *ThresholdManager) CleanExpired() int
CleanExpired removes expired proposals.
func (*ThresholdManager) Pending ¶
func (tm *ThresholdManager) Pending() int
Pending returns the number of pending proposals.
func (*ThresholdManager) Propose ¶
func (tm *ThresholdManager) Propose(sql string, params []any) ([32]byte, error)
Propose creates a new write proposal. Returns the proposal ID.
func (*ThresholdManager) RegisterPeer ¶
func (tm *ThresholdManager) RegisterPeer(nodeID string, pubKey ed25519.PublicKey)
RegisterPeer adds a peer's public key for attestation verification.
func (*ThresholdManager) SetCommitFunc ¶
func (tm *ThresholdManager) SetCommitFunc(fn func(string, []any) error)
SetCommitFunc sets the function called when threshold is met.
type WriteProposal ¶
type WriteProposal struct {
ID [32]byte // SHA-256 of the SQL + params
SQL string
Params []any
Proposer string // node ID of proposer
CreatedAt time.Time
ExpiresAt time.Time
// contains filtered or unexported fields
}
WriteProposal is a proposed write that needs t-of-n attestations.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
internal
|
|
|
engine
Package sqlite is a sql/database driver using a CGo-free port of the C SQLite3 library.
|
Package sqlite is a sql/database driver using a CGo-free port of the C SQLite3 library. |
|
engine/vfs
codec.go — a read-write, os-backed SQLite VFS that encrypts every page with the SQLCipher-4 page format (github.com/hanzoai/sqlcipher), giving the pure-Go engine real at-rest encryption byte-compatible with C SQLCipher.
|
codec.go — a read-write, os-backed SQLite VFS that encrypts every page with the SQLCipher-4 page format (github.com/hanzoai/sqlcipher), giving the pure-Go engine real at-rest encryption byte-compatible with C SQLCipher. |
|
engine/vtab
Package vtab defines a Go-facing API for implementing SQLite virtual table modules on top of the github.com/hanzoai/sqlite/internal/engine driver.
|
Package vtab defines a Go-facing API for implementing SQLite virtual table modules on top of the github.com/hanzoai/sqlite/internal/engine driver. |
|
libc
Package libc is a partial reimplementation of C libc in pure Go.
|
Package libc is a partial reimplementation of C libc in pure Go. |
|
libc/honnef.co/go/netdb
Package netdb provides a Go interface for the protoent and servent structures as defined in netdb.h
|
Package netdb provides a Go interface for the protoent and servent structures as defined in netdb.h |
|
mathutil
Package mathutil provides utilities supplementing the standard 'math' and 'math/rand' packages.
|
Package mathutil provides utilities supplementing the standard 'math' and 'math/rand' packages. |
|
memory
Package memory implements a memory allocator.
|
Package memory implements a memory allocator. |