sqlite

package module
v0.1.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 23, 2026 License: BSD-3-Clause Imports: 19 Imported by: 0

README

Hanzo SQLite

Dual-backend SQLite driver for the Hanzo ecosystem. Registers the database/sql driver name sqlite under both build configurations and exposes the same API either way:

Build Backend Encryption Use
CGO_ENABLED=1 + -tags libsqlite3 + libsqlcipher mattn/go-sqlite3 → SQLCipher AES-256 page-level, at rest production
CGO_ENABLED=0 modernc.org/sqlite (pure Go) none CI tests / lint / local dev

One import, one driver name, two backends:

import _ "github.com/hanzoai/sqlite" // registers "sqlite" under both tags

db, _ := sql.Open("sqlite", dsn)     // mattn+SQLCipher (cgo) or modernc (!cgo)

The pure-Go backend cannot encrypt. Demanding a key on it (Open(path, WithKey(...)), OpenDB(path, key)) returns ErrEncryptionUnavailable and writes nothing — it never silently persists plaintext.

Building the encrypted (production) backend — READ THIS

mainline mattn/go-sqlite3 has no sqlcipher build tag and no sqlite3_key() binding. SQLCipher works only when you:

  1. link the system sqlite (the libsqlite3 tag) against libsqlcipher, and
  2. enable the codec + URI keying via CGO flags, and
  3. supply the key as SQLCipher's native URI key parameter so it is applied inside sqlite3_open_v2 — before mattn's pragma battery runs.
CGO_ENABLED=1 \
CGO_CFLAGS="-DSQLITE_HAS_CODEC -DSQLITE_USE_URI=1 -I<sqlcipher>/include/sqlcipher" \
CGO_LDFLAGS="-L<sqlcipher>/lib -lsqlcipher" \
go build -tags "libsqlite3 sqlite_fts5" ./...

Alpine: apk add gcc musl-dev sqlcipher-dev pkgconfig.

Why not -tags sqlcipher + PRAGMA key in a ConnectHook?

Both are traps that ship plaintext:

  • -tags sqlcipher is inert in mainline mattn (no such tag) → links plain sqlite → PRAGMA key is a silent no-op.
  • mattn runs PRAGMA busy_timeout/journal_mode/foreign_keys/... via sqlite3_exec before the ConnectHook fires. On an existing encrypted file that touches the header before the key is set → file is not a database on reopen. So a ConnectHook can create but never reopen a SQLCipher DB.

The URI key parameter sidesteps both: SQLCipher's VFS keys the connection at open time. TestEncryptionProof asserts real ciphertext on disk and a working keyed reopen, so a mis-linked build fails CI instead of shipping plaintext.

The key rides the DSN (file:PATH?...&key=x'HEX'). Never log the DSN. IAM keeps showSql=false and does not log it.

Encryption

  • Algorithm: AES-256 (SQLCipher 4 defaults: 4096-byte pages, PBKDF2-HMAC-SHA512, 256000 iters, per-page HMAC-SHA512).
  • Key: raw 256-bit (no passphrase KDF) via WithRawKey / the key=x'HEX' DSN param, or derived per principal:
// CEK = HKDF-SHA256(masterKey, "{org|user}:{id}")
db, _ := sqlite.Open("data.db", sqlite.WithPrincipalKey(masterKey, sqlite.PrincipalOrg, "acme"))

// Or get a *sql.DB to hand to xorm.NewEngineWithDB:
cek, _ := sqlite.DeriveKey(masterKey, sqlite.PrincipalOrg, "acme")
sqldb, _ := sqlite.OpenDB(dbPath, cek)
eng, _ := xorm.NewEngineWithDB("sqlite", "", core.FromDB(sqldb))

Different orgs/users get different CEKs (domain-separated info); destroying the master key renders every derived CEK irrecoverable.

Per-principal CEK

DeriveKey(masterKey, principalType, principalID) → 32-byte CEK via HKDF-SHA256. Master key is 32 bytes, sourced from KMS. Used for per-org and per-user database isolation in IAM, KMS, and other Hanzo services.

Threshold write attestation

ThresholdManager coordinates t-of-n Ed25519 attestations for writes (MPC shard storage, multi-sig). Pure Go; available under both backends.

License

Apache-2.0

Documentation

Overview

Per-principal key derivation and envelope wrapping.

ENVELOPE MODEL (read before changing — this is what makes master-key rotation non-destructive):

  1. Each database gets its OWN random 256-bit Data Encryption Key (DEK), generated once at creation by NewDEK(). SQLCipher encrypts the file pages with this DEK and only this DEK — it never changes for the life of the file, so the ciphertext pages are never rewritten.
  2. The DEK is wrapped (AES-256-GCM) under a Key Encryption Key (KEK) derived from the KMS master key: KEK = HKDF-SHA256(masterKey, info = lp(principalType) || lp(id)) The wrapped DEK is stored beside the database (a small sidecar blob); the raw DEK is never written to disk.
  3. Opening: unwrap the stored DEK with the KEK, open SQLCipher with the DEK.
  4. Master-key ROTATION: unwrap the DEK with the OLD KEK, rewrap it with the NEW KEK, atomically replace the sidecar. The DEK is unchanged, so the encrypted pages are untouched — rotation is O(1) per database and cannot brick a file. (The legacy "derive the page key directly from the master" scheme made rotation impossible: a new master changed the page key and SQLCipher's HMAC rejected every existing file. Envelope dissolves that.)

`lp(x)` is the length-prefixed encoding of x (uvarint length || bytes). It is injective, so two principals can never collide across the type/id boundary (e.g. type "org"+id "a:b" and type "org:a"+id "b" hash differently — the old "%s:%s" form did not guarantee this).

Package sqlite provides an encrypted SQLite driver for Hanzo.

It is dual-backend and registers the database/sql driver name "sqlite" under BOTH build configurations, exposing the same public API either way:

  • CGO (//go:build cgo) → mattn/go-sqlite3 + SQLCipher: page-level AES-256 encryption at rest. This is the production engine. Build with: CGO_ENABLED=1 \ CGO_CFLAGS="-DSQLITE_HAS_CODEC -DSQLITE_USE_URI=1 -I<sqlcipher>/include/sqlcipher" \ CGO_LDFLAGS="-lsqlcipher" \ go build -tags "libsqlite3 sqlite_fts5" NOTE: the `sqlcipher` tag is INERT in mainline mattn (ships PLAINTEXT); the real recipe is the `libsqlite3` tag linked against libsqlcipher with the codec + URI flags above.
  • !CGO (//go:build !cgo) → modernc.org/sqlite (pure Go): NO encryption. Used only for CGO-off CI (test + lint) and local dev. Demanding a key on this backend is a hard error — it never silently stores plaintext.

Because the "sqlite" driver name is registered under both tags, any code doing `_ "github.com/hanzoai/sqlite"` + `sql.Open("sqlite", dsn)` (e.g. Hanzo IAM's xorm engine) compiles and runs in CGO-off CI and runs encrypted in the CGO production build — one import, one driver name, two backends.

The encryption key, replication mode, threshold config, and per-principal CEK derivation (cek.go, threshold.go) are pure Go and live in tag-neutral files; only the database/sql driver registration and the connect-time pragma application differ by build tag (driver_cgo.go / driver_nocgo.go).

Threshold write attestation for multi-party SQLite.

Each party runs a node with a full replica. Write operations require t-of-n parties to sign the write before it's committed. Reads are local.

Use case: MPC wallet shard storage, DEX trade approvals, multi-sig transaction authorization in trading platforms.

Index

Constants

This section is empty.

Variables

View Source 
var (
	ErrThresholdNotMet = errors.New("sqlite: threshold attestations not met")
	ErrDuplicateSigner = errors.New("sqlite: duplicate signer")
	ErrInvalidSig      = errors.New("sqlite: invalid attestation signature")
	ErrSessionExpired  = errors.New("sqlite: attestation session expired")
)
View Source 
var ErrEncryptionUnavailable = errors.New("sqlite: encryption requested but this build has no SQLCipher backend (build with CGO_ENABLED=1 -tags libsqlite3 linked against libsqlcipher)")

ErrEncryptionUnavailable is returned when an encryption key is supplied to a backend that cannot encrypt. Never returned by the CGO backend.

Functions

func CodecLinked added in v0.1.3 

func CodecLinked() bool

CodecLinked reports whether at-rest encryption is ACTUALLY working in this process — not merely advertised. EncryptionAvailable() is a backend-capability flag (true for any CGO build), but a CGO build that forgot to link libsqlcipher silently writes PLAINTEXT. CodecLinked proves the codec at runtime: it opens a temp DB under a key, writes a marker, and checks the on-disk bytes are real ciphertext (no plaintext "SQLite format 3" header, marker absent).

Use it to gate encryption assertions in tests: a properly-linked build runs them; a CGO-without-codec build SKIPS instead of failing on plaintext (the Dockerfile build, which links libsqlcipher, is where the hard ciphertext gate runs). Returns false on the pure-Go backend.

func DSN added in v0.1.2 

func DSN(path string, rawKey []byte) string

DSN builds a canonical IAM SQLite DSN for the active (CGO/mattn/SQLCipher) backend.

When rawKey is non-nil it is emitted as SQLCipher's native `key=x'HEX'` URI parameter, which SQLCipher applies inside sqlite3_open_v2 — before mattn's pragma battery — making the connection reopen-safe. The DSN consequently CONTAINS the key; callers MUST NOT log it.

rawKey must be exactly 32 bytes when non-nil (validated by Open; OpenDB trusts its caller). A nil key yields an unencrypted DSN for the global/dev/test engine.

The path is percent-escaped (escapeDBPath): a path containing '?' or '#' would otherwise prematurely terminate the file portion of the DSN and DROP the trailing query params — including `key=` — which would silently open the database UNENCRYPTED. Escaping makes that impossible regardless of caller input.

func DeriveChildKey added in v0.1.3 

func DeriveChildKey(parentKey []byte, principalType PrincipalType, principalID string) ([]byte, error)

DeriveChildKey derives a child KEK from a PARENT key (not the master), establishing a key hierarchy: master → per-org KEK → per-user KEK. A per-user database's DEK is therefore wrapped under a KEK that is itself bound to the org, so compromising the master alone is not sufficient without traversing the hierarchy, and an org's user keys are cryptographically contained within that org. 

parentKey:     32-byte parent key (e.g. the org KEK from DeriveKey)
principalType: child principal type (typically PrincipalUser)
principalID:   child identifier (user ID)

func DeriveKey 

func DeriveKey(masterKey []byte, principalType PrincipalType, principalID string) ([]byte, error)

DeriveKey derives a 256-bit key for a principal from a master key using HKDF-SHA256 with a length-prefixed, injective `info`.

In the envelope model this is the KEK (it WRAPS a per-database DEK); it is no longer used as a SQLCipher page key directly. Callers that need a page key use NewDEK + WrapDEK/UnwrapDEK. 

masterKey:     32-byte master encryption key (from KMS)
principalType: "global", "org", or "user"
principalID:   unique identifier (org slug, user ID, "iam" for global)

func EncryptionAvailable added in v0.1.2 

func EncryptionAvailable() bool

EncryptionAvailable reports that this build's backend can encrypt at rest.

It returns true for the CGO backend, which is a BACKEND-capability flag, not a proof that libsqlcipher is actually linked: a CGO build WITHOUT the codec links plain sqlite and silently no-ops the key (writes plaintext). The production Dockerfile links libsqlcipher; the package's encryption-proof test verifies real ciphertext at runtime so a mis-linked build fails CI instead of shipping plaintext.

func NewDEK added in v0.1.3 

func NewDEK() ([]byte, error)

NewDEK generates a fresh random 256-bit Data Encryption Key. Each database is created with exactly one DEK, which never changes for the life of the file.

func OpenDB added in v0.1.2 

func OpenDB(path string, rawKey []byte) (*sql.DB, error)

OpenDB opens *path* as a database/sql DB on the CGO/SQLCipher backend with the given raw 256-bit key. It is the entry point for callers that need a *sql.DB (e.g. to hand to xorm.NewEngineWithDB). A nil key opens unencrypted. The returned DB is NOT pinged here; the caller (or Open) pings.

func PrincipalAAD added in v0.1.4 

func PrincipalAAD(principalType PrincipalType, principalID string) []byte

PrincipalAAD returns the injective binding context for a principal, suitable as the AES-256-GCM additional-authenticated-data when wrapping that principal's DEK (WrapDEK/UnwrapDEK). It is the SAME length-prefixed, injective encoding used for the KEK's HKDF info — one encoding, two uses (DRY):

  • HKDF info → domain-separates the KEK per principal.
  • GCM AAD → binds the wrapped blob to its principal, so a sidecar lifted from one principal can never be unwrapped under another even if a KEK derivation ever collided. Defense-in-depth atop the KEK separation.

Pass the value to WrapDEK/UnwrapDEK; the same (type,id) MUST be used to wrap and to unwrap, or the GCM tag check fails.

func UnwrapDEK added in v0.1.3 

func UnwrapDEK(kek, blob, aad []byte) ([]byte, error)

UnwrapDEK opens a blob produced by WrapDEK under the same KEK and the same aad. A wrong KEK, wrong aad (e.g. a sidecar from a different principal), truncated blob, or tampered ciphertext fails the GCM tag and returns an error — never a partial/garbage key.

func WrapDEK added in v0.1.3 

func WrapDEK(kek, dek, aad []byte) ([]byte, error)

WrapDEK seals a DEK under a KEK with AES-256-GCM and returns the storable blob: version(1) || nonce(12) || ciphertext||tag. The KEK must be 32 bytes (as produced by DeriveKey / DeriveChildKey). The blob is safe to store next to the database; it reveals nothing about the DEK without the KEK.

aad is additional binding context authenticated (but not encrypted) by GCM — pass PrincipalAAD(type,id) so the blob is cryptographically bound to its principal (a sidecar moved to another principal fails the tag, defense-in-depth atop the per-principal KEK). The same aad MUST be supplied to UnwrapDEK. Pass nil for an unbound blob (e.g. the standalone Open() helper). The on-disk AAD is version-byte || aad, so a downgrade is also unforgeable.

Types

type Config 

type Config struct {
	// Encryption
	RawKey []byte // raw 256-bit key (skips KDF); nil means unencrypted

	// Replication
	Mode   Mode
	NodeID string
	Listen string   // bind address for replication
	Peers  []string // peer addresses

	// Threshold mode
	Threshold  int                // t value (signatures required)
	Parties    int                // n value (total parties)
	SigningKey ed25519.PrivateKey // this node's signing key
	// contains filtered or unexported fields
}

Config for opening a database.

type DB 

type DB struct {
	*sql.DB
	// contains filtered or unexported fields
}

DB wraps sql.DB with replication and encryption.

func Open 

func Open(path string, opts ...Option) (*DB, error)

Open opens an encrypted, optionally distributed SQLite database.

Under the !cgo backend, passing an encryption key (WithKey/WithRawKey/ WithPrincipalKey) is a hard error: the pure-Go engine cannot encrypt and we refuse to silently persist plaintext when a caller asked for encryption.

func (*DB) Encrypted added in v0.1.2 

func (db *DB) Encrypted() bool

Encrypted reports whether this DB is backed by an at-rest-encrypted engine.

type Mode 

type Mode string

Mode determines the replication strategy. 

const (
	ModeSingle    Mode = "single"    // Local only
	ModeRaft      Mode = "raft"      // Strong consistency (leader writes)
	ModeCRDT      Mode = "crdt"      // Eventual consistency (all write)
	ModeThreshold Mode = "threshold" // t-of-n attestation for writes
)

type Option 

type Option func(*Config)

Option configures a database.

func WithCRDT 

func WithCRDT(nodeID, listen string, peers []string) Option

WithCRDT enables CRDT eventual consistency replication.

func WithKey 

func WithKey(passphrase string) Option

WithKey derives a raw 256-bit key from a passphrase via SHA-256 and configures sqlcipher to use it directly (skipping KDF).

func WithPeers 

func WithPeers(peers []string) Option

WithPeers sets replication peers.

func WithPrincipalKey 

func WithPrincipalKey(masterKey []byte, principalType PrincipalType, principalID string) Option

WithPrincipalKey derives a KEK, generates/uses a DEK, and configures the database to use the page key. Retained for the standalone Open() helper path; the envelope sidecar lifecycle (which is what production IAM uses) is driven by the consumer (object/orgdb.go) via NewDEK + WrapDEK/UnwrapDEK + OpenDB.

Deprecated for the lifecycle path: this derives a page key directly and so is NOT rotation-safe. Use the envelope helpers for any persistent database.

func WithRaft 

func WithRaft(nodeID, listen string, peers []string) Option

WithRaft enables Raft consensus replication.

func WithRawKey 

func WithRawKey(key []byte) Option

WithRawKey sets a raw 256-bit encryption key (skips KDF).

func WithThreshold 

func WithThreshold(t, n int, signingKey ed25519.PrivateKey) Option

WithThreshold enables multi-party threshold attestation for writes.

type PrincipalType 

type PrincipalType string

PrincipalType identifies the type of principal for key derivation. It forms the domain-separation tag of the derived KEK. 

const (
	// PrincipalGlobal is the cross-org global/platform database (certs,
	// providers, the admin org catalog). Distinct from any org named "global".
	PrincipalGlobal PrincipalType = "global"
	PrincipalOrg    PrincipalType = "org"
	PrincipalUser   PrincipalType = "user"
)

type ThresholdManager 

type ThresholdManager struct {
	// contains filtered or unexported fields
}

ThresholdManager coordinates multi-party write attestation.

func NewThresholdManager 

func NewThresholdManager(threshold, parties int, nodeID string, signingKey ed25519.PrivateKey) *ThresholdManager

NewThresholdManager creates a threshold write coordinator.

func (*ThresholdManager) Attest 

func (tm *ThresholdManager) Attest(proposalID [32]byte, nodeID string, signature []byte) error

Attest adds a peer's attestation to a pending proposal.

func (*ThresholdManager) CleanExpired 

func (tm *ThresholdManager) CleanExpired() int

CleanExpired removes expired proposals.

func (*ThresholdManager) Pending 

func (tm *ThresholdManager) Pending() int

Pending returns the number of pending proposals.

func (*ThresholdManager) Propose 

func (tm *ThresholdManager) Propose(sql string, params []any) ([32]byte, error)

Propose creates a new write proposal. Returns the proposal ID.

func (*ThresholdManager) RegisterPeer 

func (tm *ThresholdManager) RegisterPeer(nodeID string, pubKey ed25519.PublicKey)

RegisterPeer adds a peer's public key for attestation verification.

func (*ThresholdManager) SetCommitFunc 

func (tm *ThresholdManager) SetCommitFunc(fn func(string, []any) error)

SetCommitFunc sets the function called when threshold is met.

type WriteProposal 

type WriteProposal struct {
	ID        [32]byte // SHA-256 of the SQL + params
	SQL       string
	Params    []any
	Proposer  string // node ID of proposer
	CreatedAt time.Time
	ExpiresAt time.Time
	// contains filtered or unexported fields
}

WriteProposal is a proposed write that needs t-of-n attestations.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.