README
¶
Hanzo SQLCipher
The SQLCipher 4 on-disk page format, in pure Go, with zero dependencies.
A port of the SQLCipher codec (sqlcipher/sqlcipher
v4.5.6, BSD-3, © ZETETIC LLC — see NOTICE). It is the format and only the
format: it knows nothing about database/sql, drivers or engines, so a driver, a
backup or replication path, a migration or a forensic tool can all read and write
SQLCipher files without dragging in a SQL engine.
It is byte-compatible with the C library in both directions — it reads
databases libsqlcipher wrote, and libsqlcipher reads databases it writes. No
migration, no new format.
// Decrypt a SQLCipher database into a plaintext SQLite database.
in, _ := os.Open("encrypted.db")
out, _ := os.Create("plain.db")
err := sqlcipher.DecryptFile(out, in, sqlcipher.RawKey(key), sqlcipher.Params{})
// Or work a page at a time.
salt, _ := sqlcipher.FileSalt(page1)
c, _ := sqlcipher.NewCodec(sqlcipher.RawKey(key), salt, sqlcipher.Params{})
plain, err := c.Decrypt(pgno, page) // ErrKey on a wrong key — never garbage
The format
page 1 on disk: [ salt (16) | ciphertext | IV (16) | HMAC-SHA512 (64) ]
page N on disk: [ ciphertext | IV (16) | HMAC-SHA512 (64) ]
Pages are independent — no cross-page state, no chaining. The salt is the first
16 bytes of page 1 and is the only plaintext in the file; page 1 therefore
encrypts only the bytes after it, and decrypting restores SQLite's
SQLite format 3\0 magic over it.
The IV and tag live in SQLite's per-page reserve, which SQLite records in byte 20 of the database header — so an encrypted database carries its own reserve size and needs no out-of-band configuration to be read.
| Cipher | AES-256-CBC, no padding |
| Authentication | HMAC-SHA512 over ciphertext ‖ IV ‖ pgno_le32 |
| Passphrase KDF | PBKDF2-HMAC-SHA512, 256000 iterations |
| Page-auth key | PBKDF2-HMAC-SHA512 of the page key over salt ⊕ 0x3a, 2 iterations |
| Page size | 4096 (default) |
| Reserve | 80 = IV(16) + HMAC(64) |
Every constant was taken from the C source and confirmed against a live
libsqlcipher. The format is unchanged between 4.5.6 and 4.6.1 (4.6.1 merged
crypto.h/crypto.c/crypto_impl.c into sqlcipher.c — a refactor, not a
format change).
Failing closed
Decrypt authenticates before it decrypts. A wrong key, a wrong salt, a
flipped bit in the ciphertext, the IV or the tag, or a valid page replayed at
another page number all return ErrKey with no data. It never returns garbage
plaintext, and it never silently persists plaintext.
Proof
go test ./... — 28 tests, no C toolchain needed. Golden vectors pin the KDF, the
page-auth key and the exact bytes of a page under a fixed IV, so a refactor cannot
silently change the format. Vectors alone would only pin the port to itself, so
both keying paths are cross-verified against databases the C library actually
wrote, committed as fixtures:
testdata/c-4.5.6.db— raw key, 7 pages, overflow + index + random/zero blobstestdata/c-4.5.6-passphrase.db— passphrase, proving PBKDF2 at 256000 iterations agrees
cd parity && go test ./... — the full cross-engine gate, needing libsqlcipher
and a C compiler: the C library writes → pure Go reads → modernc.org/sqlite
writes into the decrypted database → pure Go encrypts → the C library reads back
what Go wrote.
Scope
This is the format, not an engine. It gives you offline read/write of SQLCipher
files. It does not turn a pure-Go SQLite into an encrypting one: SQLCipher's
codec is a hook inside SQLite's pager, above the VFS, and it encrypts WAL frames
and rollback-journal records too — whose checksums SQLite computes over the
ciphertext. A VFS-level codec sits below the pager and cannot reproduce that.
See LLM.md for the evidence.
License
BSD-3-Clause. Ported from SQLCipher; ZETETIC LLC's copyright and the full BSD-3 notice are retained in LICENSE. SQLCipher is a trademark of ZETETIC LLC; this project is not affiliated with or endorsed by them.
Documentation
¶
Overview ¶
Package sqlcipher implements the SQLCipher 4 on-disk page format in pure Go.
It is a port of the SQLCipher codec (https://github.com/sqlcipher/sqlcipher, v4.5.6, BSD-3, Copyright (c) ZETETIC LLC) — see NOTICE. It knows nothing about database/sql or SQLite drivers: it is the format, and only the format. Anything that must read or write SQLCipher files — a driver, a backup or replication path, a migration or a forensic tool — uses this package directly.
The format ¶
A SQLCipher database is a SQLite database whose pages are individually encrypted, with per-page reserve bytes carrying the IV and authentication tag:
page N on disk: [ ciphertext | IV (16) | HMAC-SHA512 (64) ] page 1 on disk: [ salt (16) | ciphertext | IV (16) | HMAC-SHA512 (64) ]
Pages are independent: no cross-page state, no chaining. The salt is the first SaltSize bytes of page 1 and is stored in the clear — it is the only part of the file that is not ciphertext. Because page 1's first SaltSize bytes hold the salt instead of SQLite's "SQLite format 3\x00" magic, page 1 encrypts only the bytes after that offset; Decrypt restores the magic, which is what SQLite's pager expects to see.
The ciphertext is AES-256-CBC with no padding, so its length is always a multiple of the AES block size. The IV is fresh on every page write. The HMAC covers ciphertext || IV || page number (little-endian uint32), which authenticates the page contents, the IV, and the page's position in the file — so pages cannot be reordered, and a modified IV is detected.
Keying ¶
A Key is either a raw 32-byte key (no KDF — SQLCipher's x'HEX' form) or a passphrase (PBKDF2-HMAC-SHA512, 256000 iterations by default). Either way the page-authentication key is a second, distinct key derived from the page encryption key using the salt masked with 0x3a and 2 PBKDF2 iterations.
Failing closed ¶
Decrypt authenticates before it decrypts and returns ErrKey on any HMAC mismatch. A wrong key errors; it never returns garbage plaintext.
Index ¶
Constants ¶
const ( SaltSize = 16 // KDF salt: the first bytes of page 1, stored in the clear KeySize = 32 // AES-256 IVSize = 16 // AES block size HMACSize = 64 // SHA-512 digest // Reserve is the per-page trailer SQLite must leave free at the end of every // page: IV || HMAC, rounded up to a multiple of the AES block size (it is // already a multiple, so 16+64=80). SQLite records it in byte 20 of the // database header, so a database carries its own reserve size. Reserve = IVSize + HMACSize // DefaultPageSize and DefaultIter are SQLCipher 4's defaults. DefaultPageSize = 4096 DefaultIter = 256000 )
Format sizes, all fixed by SQLCipher 4.
Variables ¶
var ErrKey = errors.New("sqlcipher: wrong key or corrupted page")
ErrKey reports that a page did not authenticate: the key is wrong, or the page was corrupted or tampered with. The two are deliberately indistinguishable.
Functions ¶
func DecryptFile ¶
DecryptFile decrypts a whole database, writing a plaintext SQLite database that any SQLite build can open without a key. The salt is read from the source's page 1.
The result keeps the source's per-page Reserve trailer — SQLite records the reserve in byte 20 of the header, so the plaintext database describes itself and stays a lossless round-trip back through EncryptFile.
func EmptyPlaintext ¶ added in v0.1.1
EmptyPlaintext returns the bytes of a minimal empty SQLite database — one page, an empty sqlite_master, no tables — that reserves Reserve (80) bytes per page.
It is the plaintext seed a NEW encrypted database is built from. EncryptFile requires its source to already reserve Reserve bytes per page (header byte 20), because the per-page IV+HMAC trailer is written into that reserve; a database that does not reserve it cannot be encrypted without losing data. A plaintext SQLite engine cannot be asked to create a database with a non-zero reserve through SQL (the reserve is set via SQLITE_FCNTL_RESERVE_BYTES, not a PRAGMA), so a pure-Go writer that wants a fresh encrypted database starts from this seed: open it, write the schema (SQLite honours the reserve recorded in the header, so every page it writes leaves room for the trailer), then EncryptFile the result.
The layout follows the SQLite file format (https://www.sqlite.org/fileformat.html §1.3 "Database Header", §1.6 "B-tree Pages"): a single leaf table b-tree page holding an empty sqlite_master, with byte 20 set to Reserve and the cell-content-area pointer at the usable-size boundary (pageSize − Reserve). The in-header page count is 1 and the file is exactly one page, so an engine trusts the header size (the change counter equals the version-valid-for field). p.PageSize selects the page size (0 ⇒ DefaultPageSize); p.Iter is unused here.
func EncryptFile ¶
EncryptFile encrypts a whole plaintext SQLite database.
The source must already reserve Reserve bytes per page (header byte 20), which is what a database DecryptFile produced does; otherwise there is no room for the IV and tag and this returns an error rather than truncating data. A nil salt draws a fresh random one, which is what a new database wants; pass a salt only to rewrite a database under its existing one.
Types ¶
type Codec ¶
type Codec struct {
// contains filtered or unexported fields
}
Codec encrypts and decrypts the pages of one database. It is immutable and safe for concurrent use.
func NewCodec ¶
NewCodec binds key material to a database salt. salt is SaltSize bytes, read from the first bytes of page 1 of an existing database (see FileSalt), or freshly random for a new one.
Deriving from a passphrase runs Params.Iter PBKDF2 iterations and is deliberately slow; do it once per database, not once per page.
func (*Codec) Decrypt ¶
Decrypt decrypts one on-disk page and returns its plaintext. page must be exactly PageSize bytes; the result is a fresh slice of the same length.
The page is authenticated before it is decrypted, so a wrong key returns ErrKey rather than plausible-looking garbage. pgno is 1-based, as SQLite numbers pages.
Decrypting page 1 restores SQLite's "SQLite format 3\x00" magic over the salt, which is what SQLite's pager expects to read. The reserve trailer is carried through unchanged; SQLite ignores it.
func (*Codec) Encrypt ¶
Encrypt encrypts one plaintext page and returns the bytes to store on disk. page must be exactly PageSize bytes, of which only the first PageSize-Reserve carry data — SQLite guarantees that by reserving Reserve bytes per page.
Every call draws a fresh IV, so encrypting the same page twice yields different ciphertext. Encrypting page 1 writes the database salt over SQLite's magic.
type Key ¶
type Key struct {
// contains filtered or unexported fields
}
Key is database key material: a raw key or a passphrase. It is inert until bound to a database salt by NewCodec.
func Passphrase ¶
Passphrase derives the page encryption key from pass via PBKDF2-HMAC-SHA512.
type Params ¶
type Params struct {
PageSize int // cipher_page_size; 0 means DefaultPageSize
Iter int // kdf_iter, passphrase keying only; 0 means DefaultIter
}
Params are the format parameters that vary between databases. The zero value means SQLCipher 4 defaults; set a field only to interoperate with a database that was written with a non-default PRAGMA.
Source Files