Documentation
¶
Overview ¶
Package sqlitedb opens SQLite the way a server needs it: one connection for writes and several for reads.
The reason is a failure mode that only appears once a deployment has been running for a while. SQLite in WAL mode lets readers run alongside a writer, but Go's connection pool sits above SQLite, so a pool of one serialises everything regardless of what the database would allow. A message browser search scanning the payload column then holds the only connection, and every arriving message waits behind it.
Measured on this machine, a write takes 104µs with the store idle and 2.6ms while a search scans four thousand messages: twenty-five times slower. That scales with the table, so at a few hundred thousand stored messages a report turns into seconds of delivery latency, and a write is on the acknowledgement path. The sender sees a timeout, resends, and somebody spends a morning looking for a problem in the network.
Splitting the pools costs nothing and removes the interaction: reports get their own connections, and the writer keeps one to itself because SQLite permits exactly one writer anyway.
Index ¶
- func IsPostgres(db *sql.DB) bool
- func IsPostgresURL(target string) bool
- func IsUniqueViolation(err error) bool
- func OpenSQL(target string) (*sql.DB, error)
- func RedactURL(target string) string
- func TestPostgres() bool
- func TestURL(name string) string
- func Translate(query string) string
- type DB
- type Options
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func IsPostgres ¶ added in v0.1.17
IsPostgres reports whether db is a PostgreSQL pool opened here.
func IsPostgresURL ¶ added in v0.1.17
IsPostgresURL reports whether a database target is a PostgreSQL URL rather than a file.
func IsUniqueViolation ¶ added in v0.1.17
IsUniqueViolation reports whether err is a unique constraint refusing a row, from either database.
func OpenSQL ¶ added in v0.1.17
OpenSQL opens one pool, for a caller that wants a single *sql.DB: an in-memory or file SQLite database limited to one connection, as the stores expect, or PostgreSQL for a URL (and, in a test binary with PERFUSE_TEST_POSTGRES set, for every target: see testSchema).
func RedactURL ¶ added in v0.1.17
RedactURL hides the password in a database URL, for logs and error messages.
func TestPostgres ¶ added in v0.1.17
func TestPostgres() bool
TestPostgres reports whether this is a test binary running the suite against PostgreSQL.
Types ¶
type DB ¶
type DB struct {
// Write is limited to a single connection, because SQLite takes one writer at
// a time and queuing beats "database is locked".
Write *sql.DB
// Read carries several connections. WAL lets them run while a write is in
// progress, so a report does not stall delivery.
Read *sql.DB
// Path is the file, or ":memory:".
Path string
// contains filtered or unexported fields
}
DB holds the two pools. Both point at the same file.
func Open ¶
Open prepares both pools for the database at path.
":memory:" is supported for tests, and is deliberately given a single shared connection: an in-memory database belongs to its connection, so a second one would silently be a different, empty database. That is the kind of thing that makes a test pass for the wrong reason.
A postgres:// or postgresql:// URL opens PostgreSQL instead, for servers that share one store (see postgres.go).
type Options ¶
type Options struct {
// Readers is the size of the read pool. Defaults to the number of CPUs,
// bounded to a sensible range: below two there is no point splitting, and far
// above that a burst of expensive reports would compete for memory rather
// than for the disk.
Readers int
// BusyTimeoutMS is how long a statement waits for a lock before failing.
// Defaults to 5000. A brief lock should be a wait, not an error.
BusyTimeoutMS int
// ForeignKeys enables enforcement. Defaults to on.
ForeignKeys *bool
}
Options tune the pools. The defaults suit a server.
Source Files