sqlite

package module
v0.3.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 18, 2026 License: BSD-3-Clause Imports: 19 Imported by: 0

README

sqlite

Hanzo SQLite

Dual-backend SQLite driver for the Hanzo ecosystem. Registers the database/sql driver name sqlite under both build configurations and exposes the same API either way. Both backends encrypt at rest, in the same SQLCipher-4 format — a database written by one opens under the other.

Build Backend Encryption Use
CGO_ENABLED=0 (default) vendored pure-Go engine + hanzoai/sqlcipher codec VFS AES-256 page-level, at rest default — CI, tests, pure-Go deploys
CGO_ENABLED=1 + -tags libsqlite3 + libsqlcipher hanzoai/csqlite → SQLCipher AES-256 page-level, at rest the C engine, for speed

One import, one driver name, two backends, one format:

import _ "github.com/hanzoai/sqlite" // registers "sqlite" under both tags

db, _ := sql.Open("sqlite", dsn)     // pure-Go codec VFS (!cgo) or csqlite+SQLCipher (cgo)

The pure-Go backend always encrypts — no cgo, no external C library, nothing to link, and go list -m all shows zero modernc.org/* (the engine is vendored in-tree). A keyed database is single-writer, single-process (WAL on an in-process wal-index).

Building the CGO backend (optional, for the C engine's speed)

The default CGO_ENABLED=0 build already encrypts and needs no flags. The CGO backend uses hanzoai/csqlite linked against libsqlcipher, with the key supplied as SQLCipher's native URI key parameter (applied inside sqlite3_open_v2, so create and reopen both work):

CGO_ENABLED=1 \
CGO_CFLAGS="-DSQLITE_HAS_CODEC -DSQLITE_USE_URI=1 -I<sqlcipher>/include/sqlcipher" \
CGO_LDFLAGS="-L<sqlcipher>/lib -lsqlcipher" \
go build -tags "libsqlite3 sqlite_fts5" ./...

Alpine: apk add gcc musl-dev sqlcipher-dev pkgconfig. A cgo build that forgets to link libsqlcipher silently writes plaintext; CodecLinked() proves the codec at runtime and TestEncryptionProof fails such a build.

On the CGO backend the key rides the DSN (file:PATH?...&key=x'HEX'). Never log the DSN. On the pure-Go backend the key never touches the DSN — it binds to the codec VFS.

Encryption

  • Algorithm: AES-256 (SQLCipher 4 defaults: 4096-byte pages, PBKDF2-HMAC-SHA512, 256000 iters, per-page HMAC-SHA512).
  • Key: raw 256-bit (no passphrase KDF) via WithRawKey / the key=x'HEX' DSN param, or derived per principal:
// CEK = HKDF-SHA256(masterKey, "{org|user}:{id}")
db, _ := sqlite.Open("data.db", sqlite.WithPrincipalKey(masterKey, sqlite.PrincipalOrg, "acme"))

// Or get a *sql.DB to hand to xorm.NewEngineWithDB:
cek, _ := sqlite.DeriveKey(masterKey, sqlite.PrincipalOrg, "acme")
sqldb, _ := sqlite.OpenDB(dbPath, cek)
eng, _ := xorm.NewEngineWithDB("sqlite", "", core.FromDB(sqldb))

Different orgs/users get different CEKs (domain-separated info); destroying the master key renders every derived CEK irrecoverable.

Per-principal CEK

DeriveKey(masterKey, principalType, principalID) → 32-byte CEK via HKDF-SHA256. Master key is 32 bytes, sourced from KMS. Used for per-org and per-user database isolation in IAM, KMS, and other Hanzo services.

Threshold write attestation

ThresholdManager coordinates t-of-n Ed25519 attestations for writes (MPC shard storage, multi-sig). Pure Go; available under both backends.

License

Apache-2.0

Documentation

Overview

Per-principal key derivation and envelope wrapping.

ENVELOPE MODEL (read before changing — this is what makes master-key rotation non-destructive):

  1. Each database gets its OWN random 256-bit Data Encryption Key (DEK), generated once at creation by NewDEK(). SQLCipher encrypts the file pages with this DEK and only this DEK — it never changes for the life of the file, so the ciphertext pages are never rewritten.
  2. The DEK is wrapped (AES-256-GCM) under a Key Encryption Key (KEK) derived from the KMS master key: KEK = HKDF-SHA256(masterKey, info = lp(principalType) || lp(id)) The wrapped DEK is stored beside the database (a small sidecar blob); the raw DEK is never written to disk.
  3. Opening: unwrap the stored DEK with the KEK, open SQLCipher with the DEK.
  4. Master-key ROTATION: unwrap the DEK with the OLD KEK, rewrap it with the NEW KEK, atomically replace the sidecar. The DEK is unchanged, so the encrypted pages are untouched — rotation is O(1) per database and cannot brick a file. (The legacy "derive the page key directly from the master" scheme made rotation impossible: a new master changed the page key and SQLCipher's HMAC rejected every existing file. Envelope dissolves that.)

`lp(x)` is the length-prefixed encoding of x (uvarint length || bytes). It is injective, so two principals can never collide across the type/id boundary (e.g. type "org"+id "a:b" and type "org:a"+id "b" hash differently — the old "%s:%s" form did not guarantee this).

Package sqlite provides an encrypted SQLite driver for Hanzo.

Both backends encrypt at rest, in the SAME SQLCipher-4 on-disk format, and a database written by one opens under the other. The driver registers the database/sql driver name "sqlite" under BOTH build configurations, exposing the same public API either way:

  • !CGO (//go:build !cgo) → the vendored pure-Go SQLite engine (internal/engine) plus the hanzoai/sqlcipher codec VFS: page-level AES-256 encryption at rest, with NO cgo and NO external C library. This is what CGO-off CI, tests, lint, and pure-Go deployments run. Nothing to link, no amalgamation to build — it always encrypts.
  • CGO (//go:build cgo) → hanzoai/csqlite + libsqlcipher: the same page-level AES-256 format via the C codec, for builds that want the C engine's speed. Build with: CGO_ENABLED=1 \ CGO_CFLAGS="-DSQLITE_HAS_CODEC -DSQLITE_USE_URI=1 -I<sqlcipher>/include/sqlcipher" \ CGO_LDFLAGS="-lsqlcipher" \ go build -tags "libsqlite3 sqlite_fts5" A cgo build that forgets to link libsqlcipher silently writes plaintext; CodecLinked() proves the codec at runtime so such a build fails CI.

Because the "sqlite" driver name is registered under both tags, any code doing `_ "github.com/hanzoai/sqlite"` + `sql.Open("sqlite", dsn)` (e.g. Hanzo IAM's xorm engine) compiles and runs encrypted under both — one import, one driver name, two backends, one format.

The encryption key, replication mode, threshold config, and per-principal CEK derivation (cek.go, threshold.go) are pure Go and live in tag-neutral files; only the database/sql driver registration and the connect-time keying differ by build tag (driver_cgo.go / driver_nocgo.go).

Threshold write attestation for multi-party SQLite.

Each party runs a node with a full replica. Write operations require t-of-n parties to sign the write before it's committed. Reads are local.

Use case: MPC wallet shard storage, DEX trade approvals, multi-sig transaction authorization in trading platforms.

Index

Constants

This section is empty.

Variables

View Source 
var (
	ErrThresholdNotMet = errors.New("sqlite: threshold attestations not met")
	ErrDuplicateSigner = errors.New("sqlite: duplicate signer")
	ErrInvalidSig      = errors.New("sqlite: invalid attestation signature")
	ErrSessionExpired  = errors.New("sqlite: attestation session expired")
)
View Source 
var ErrEncryptionUnavailable = errors.New("sqlite: encryption unavailable")

ErrEncryptionUnavailable is retained for API symmetry with the pure-Go backend. It is never returned by the CGO backend, which encrypts through libsqlcipher.

Functions

func CodecLinked added in v0.1.3 

func CodecLinked() bool

CodecLinked reports whether at-rest encryption is ACTUALLY working in this process — not merely advertised. EncryptionAvailable() is a backend-capability flag (true under both build tags), but a CGO build that forgot to link libsqlcipher silently writes PLAINTEXT. CodecLinked proves the codec at runtime: it opens a temp DB under a key, writes a marker, and checks the on-disk bytes are real ciphertext (no plaintext "SQLite format 3" header, marker absent).

The pure-Go backend always returns true — its codec VFS is compiled in, nothing to mis-link. On the CGO backend it returns true only when libsqlcipher is actually linked. Use it to gate encryption assertions in tests: the pure-Go build and a correctly-linked CGO build run them; a CGO-without-codec build SKIPS instead of failing on plaintext (the Dockerfile build is the hard gate there).

func DSN added in v0.1.2 

func DSN(path string, rawKey []byte) string

DSN builds a canonical IAM SQLite DSN for the active (CGO/csqlite/SQLCipher) backend.

When rawKey is non-nil it is emitted as SQLCipher's native `key=x'HEX'` URI parameter, which SQLCipher applies inside sqlite3_open_v2 — before any pragma — making the connection reopen-safe. The DSN consequently CONTAINS the key; callers MUST NOT log it.

rawKey must be exactly 32 bytes when non-nil (validated by Open; OpenDB trusts its caller). A nil key yields an unencrypted DSN for the global/dev/test engine.

The path is percent-escaped (escapeDBPath): a path containing '?' or '#' would otherwise prematurely terminate the file portion of the DSN and DROP the trailing query params — including `key=` — which would silently open the database UNENCRYPTED. Escaping makes that impossible regardless of caller input.

func DeriveChildKey added in v0.1.3 

func DeriveChildKey(parentKey []byte, principalType PrincipalType, principalID string) ([]byte, error)

DeriveChildKey derives a child KEK from a PARENT key (not the master), establishing a key hierarchy: master → per-org KEK → per-user KEK. A per-user database's DEK is therefore wrapped under a KEK that is itself bound to the org, so compromising the master alone is not sufficient without traversing the hierarchy, and an org's user keys are cryptographically contained within that org. 

parentKey:     32-byte parent key (e.g. the org KEK from DeriveKey)
principalType: child principal type (typically PrincipalUser)
principalID:   child identifier (user ID)

func DeriveKey 

func DeriveKey(masterKey []byte, principalType PrincipalType, principalID string) ([]byte, error)

DeriveKey derives a 256-bit key for a principal from a master key using HKDF-SHA256 with a length-prefixed, injective `info`.

In the envelope model this is the KEK (it WRAPS a per-database DEK); it is no longer used as a SQLCipher page key directly. Callers that need a page key use NewDEK + WrapDEK/UnwrapDEK. 

masterKey:     32-byte master encryption key (from KMS)
principalType: "global", "org", or "user"
principalID:   unique identifier (org slug, user ID, "iam" for global)

func EncryptionAvailable added in v0.1.2 

func EncryptionAvailable() bool

EncryptionAvailable reports that this build's backend can encrypt at rest.

It returns true for the CGO backend, which is a BACKEND-capability flag, not a proof that libsqlcipher is actually linked: a CGO build WITHOUT the codec links plain sqlite and silently no-ops the key (writes plaintext). The production Dockerfile links libsqlcipher; the package's encryption-proof test verifies real ciphertext at runtime (CodecLinked) so a mis-linked build fails CI instead of shipping plaintext.

func NewDEK added in v0.1.3 

func NewDEK() ([]byte, error)

NewDEK generates a fresh random 256-bit Data Encryption Key. Each database is created with exactly one DEK, which never changes for the life of the file.

func OpenDB added in v0.1.2 

func OpenDB(path string, rawKey []byte) (*sql.DB, error)

OpenDB opens *path* as a database/sql DB on the CGO/SQLCipher backend with the given raw 256-bit key. It is the entry point for callers that need a *sql.DB (e.g. to hand to xorm.NewEngineWithDB). A nil key opens unencrypted. The returned DB is NOT pinged here; the caller (or Open) pings.

func PrincipalAAD added in v0.1.4 

func PrincipalAAD(principalType PrincipalType, principalID string) []byte

PrincipalAAD returns the injective binding context for a principal, suitable as the AES-256-GCM additional-authenticated-data when wrapping that principal's DEK (WrapDEK/UnwrapDEK). It is the SAME length-prefixed, injective encoding used for the KEK's HKDF info — one encoding, two uses (DRY):

  • HKDF info → domain-separates the KEK per principal.
  • GCM AAD → binds the wrapped blob to its principal, so a sidecar lifted from one principal can never be unwrapped under another even if a KEK derivation ever collided. Defense-in-depth atop the KEK separation.

Pass the value to WrapDEK/UnwrapDEK; the same (type,id) MUST be used to wrap and to unwrap, or the GCM tag check fails.

func UnwrapDEK added in v0.1.3 

func UnwrapDEK(kek, blob, aad []byte) ([]byte, error)

UnwrapDEK opens a blob produced by WrapDEK under the same KEK and the same aad. A wrong KEK, wrong aad (e.g. a sidecar from a different principal), truncated blob, or tampered ciphertext fails the GCM tag and returns an error — never a partial/garbage key.

func WrapDEK added in v0.1.3 

func WrapDEK(kek, dek, aad []byte) ([]byte, error)

WrapDEK seals a DEK under a KEK with AES-256-GCM and returns the storable blob: version(1) || nonce(12) || ciphertext||tag. The KEK must be 32 bytes (as produced by DeriveKey / DeriveChildKey). The blob is safe to store next to the database; it reveals nothing about the DEK without the KEK.

aad is additional binding context authenticated (but not encrypted) by GCM — pass PrincipalAAD(type,id) so the blob is cryptographically bound to its principal (a sidecar moved to another principal fails the tag, defense-in-depth atop the per-principal KEK). The same aad MUST be supplied to UnwrapDEK. Pass nil for an unbound blob (e.g. the standalone Open() helper). The on-disk AAD is version-byte || aad, so a downgrade is also unforgeable.

Types

type Config 

type Config struct {
	// Encryption
	RawKey []byte // raw 256-bit key (skips KDF); nil means unencrypted

	// Replication
	Mode   Mode
	NodeID string
	Listen string   // bind address for replication
	Peers  []string // peer addresses

	// Threshold mode
	Threshold  int                // t value (signatures required)
	Parties    int                // n value (total parties)
	SigningKey ed25519.PrivateKey // this node's signing key
	// contains filtered or unexported fields
}

Config for opening a database.

type DB 

type DB struct {
	*sql.DB
	// contains filtered or unexported fields
}

DB wraps sql.DB with replication and encryption.

func Open 

func Open(path string, opts ...Option) (*DB, error)

Open opens an encrypted, optionally distributed SQLite database.

Under the !cgo backend, passing an encryption key (WithKey/WithRawKey/ WithPrincipalKey) is a hard error: the pure-Go engine cannot encrypt and we refuse to silently persist plaintext when a caller asked for encryption.

func (*DB) Encrypted added in v0.1.2 

func (db *DB) Encrypted() bool

Encrypted reports whether this DB is backed by an at-rest-encrypted engine.

type Mode 

type Mode string

Mode determines the replication strategy. 

const (
	ModeSingle    Mode = "single"    // Local only
	ModeRaft      Mode = "raft"      // Strong consistency (leader writes)
	ModeCRDT      Mode = "crdt"      // Eventual consistency (all write)
	ModeThreshold Mode = "threshold" // t-of-n attestation for writes
)

type Option 

type Option func(*Config)

Option configures a database.

func WithCRDT 

func WithCRDT(nodeID, listen string, peers []string) Option

WithCRDT enables CRDT eventual consistency replication.

func WithKey 

func WithKey(passphrase string) Option

WithKey derives a raw 256-bit key from a passphrase via SHA-256 and configures sqlcipher to use it directly (skipping KDF).

func WithPeers 

func WithPeers(peers []string) Option

WithPeers sets replication peers.

func WithPrincipalKey 

func WithPrincipalKey(masterKey []byte, principalType PrincipalType, principalID string) Option

WithPrincipalKey derives a KEK, generates/uses a DEK, and configures the database to use the page key. Retained for the standalone Open() helper path; the envelope sidecar lifecycle (which is what production IAM uses) is driven by the consumer (object/orgdb.go) via NewDEK + WrapDEK/UnwrapDEK + OpenDB.

Deprecated for the lifecycle path: this derives a page key directly and so is NOT rotation-safe. Use the envelope helpers for any persistent database.

func WithRaft 

func WithRaft(nodeID, listen string, peers []string) Option

WithRaft enables Raft consensus replication.

func WithRawKey 

func WithRawKey(key []byte) Option

WithRawKey sets a raw 256-bit encryption key (skips KDF).

func WithThreshold 

func WithThreshold(t, n int, signingKey ed25519.PrivateKey) Option

WithThreshold enables multi-party threshold attestation for writes.

type PrincipalType 

type PrincipalType string

PrincipalType identifies the type of principal for key derivation. It forms the domain-separation tag of the derived KEK. 

const (
	// PrincipalGlobal is the cross-org global/platform database (certs,
	// providers, the admin org catalog). Distinct from any org named "global".
	PrincipalGlobal PrincipalType = "global"
	PrincipalOrg    PrincipalType = "org"
	PrincipalUser   PrincipalType = "user"
)

type ThresholdManager 

type ThresholdManager struct {
	// contains filtered or unexported fields
}

ThresholdManager coordinates multi-party write attestation.

func NewThresholdManager 

func NewThresholdManager(threshold, parties int, nodeID string, signingKey ed25519.PrivateKey) *ThresholdManager

NewThresholdManager creates a threshold write coordinator.

func (*ThresholdManager) Attest 

func (tm *ThresholdManager) Attest(proposalID [32]byte, nodeID string, signature []byte) error

Attest adds a peer's attestation to a pending proposal.

func (*ThresholdManager) CleanExpired 

func (tm *ThresholdManager) CleanExpired() int

CleanExpired removes expired proposals.

func (*ThresholdManager) Pending 

func (tm *ThresholdManager) Pending() int

Pending returns the number of pending proposals.

func (*ThresholdManager) Propose 

func (tm *ThresholdManager) Propose(sql string, params []any) ([32]byte, error)

Propose creates a new write proposal. Returns the proposal ID.

func (*ThresholdManager) RegisterPeer 

func (tm *ThresholdManager) RegisterPeer(nodeID string, pubKey ed25519.PublicKey)

RegisterPeer adds a peer's public key for attestation verification.

func (*ThresholdManager) SetCommitFunc 

func (tm *ThresholdManager) SetCommitFunc(fn func(string, []any) error)

SetCommitFunc sets the function called when threshold is met.

type WriteProposal 

type WriteProposal struct {
	ID        [32]byte // SHA-256 of the SQL + params
	SQL       string
	Params    []any
	Proposer  string // node ID of proposer
	CreatedAt time.Time
	ExpiresAt time.Time
	// contains filtered or unexported fields
}

WriteProposal is a proposed write that needs t-of-n attestations.

Source Files

View all Source files

Directories

Path Synopsis
internal
engine
Package sqlite is a sql/database driver using a CGo-free port of the C SQLite3 library.
 Package sqlite is a sql/database driver using a CGo-free port of the C SQLite3 library.
engine/lib
engine/vfs
codec.go — a read-write, os-backed SQLite VFS that encrypts every page with the SQLCipher-4 page format (github.com/hanzoai/sqlcipher), giving the pure-Go engine real at-rest encryption byte-compatible with C SQLCipher.
 codec.go — a read-write, os-backed SQLite VFS that encrypts every page with the SQLCipher-4 page format (github.com/hanzoai/sqlcipher), giving the pure-Go engine real at-rest encryption byte-compatible with C SQLCipher.
engine/vtab
Package vtab defines a Go-facing API for implementing SQLite virtual table modules on top of the github.com/hanzoai/sqlite/internal/engine driver.
 Package vtab defines a Go-facing API for implementing SQLite virtual table modules on top of the github.com/hanzoai/sqlite/internal/engine driver.
libc
Package libc is a partial reimplementation of C libc in pure Go.
 Package libc is a partial reimplementation of C libc in pure Go.
libc/errno
libc/fcntl
libc/fts
libc/grp
libc/honnef.co/go/netdb
Package netdb provides a Go interface for the protoent and servent structures as defined in netdb.h
 Package netdb provides a Go interface for the protoent and servent structures as defined in netdb.h
libc/langinfo
libc/limits
libc/netdb
libc/netinet/in
libc/poll
libc/pthread
libc/pwd
libc/signal
libc/stdio
libc/stdlib
libc/sys/socket
libc/sys/stat
libc/sys/types
libc/termios
libc/time
libc/unistd
libc/utime
libc/uuid/uuid
libc/wctype
mathutil
Package mathutil provides utilities supplementing the standard 'math' and 'math/rand' packages.
 Package mathutil provides utilities supplementing the standard 'math' and 'math/rand' packages.
memory
Package memory implements a memory allocator.
 Package memory implements a memory allocator.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.