gormdb

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 9, 2026 License: LGPL-3.0 Imports: 22 Imported by: 0

Documentation

Overview

Package gormdb opens a GORM connection to the database selected by configuration. Business modules take a *gorm.DB and get the same API and the same error sentinels (gorm.ErrDuplicatedKey, gorm.ErrForeignKeyViolated, ...) on SQLite and on Postgres.

SQLite adds rules of its own, as its writer is a single connection (see Open):

  • inside db.Transaction or db.Connection, use only tx: a statement on db waits for the connection the callback holds, forever;
  • SQL that depends on the connection (last_insert_rowid(), changes(), temp tables) needs a transaction or db.Connection;
  • db.DB() is the writer pool and must not be tuned;
  • WITH ... queries, even read-only ones, go to the single writer.

Code that keeps to these rules runs on both databases; test it on gormdbtest.Sqlite, a temporary file with the same writer/reader setup.

The core package imports no database driver. The application picks the drivers it wants (see the driver subpackages) and passes them to Open, so the binary only links the databases it actually uses.

Index

Constants

View Source
const (
	DriverSqlite   = "sqlite"
	DriverPostgres = "postgres"
)

Supported drivers.

View Source
const (
	Read  = dbresolver.Read
	Write = dbresolver.Write
)

Read and Write force a statement onto the read pool or the writer of a SQLite file database: db.Clauses(gormdb.Read).Raw(...) or db.Clauses(gormdb.Write).Find(...). They are dbresolver's clauses, so apps need not import it; on Postgres and ":memory:" they change nothing.

Variables

This section is empty.

Functions

func Close

func Close(db *gorm.DB) error

Close releases the database's connections, the read-only pool included. It is safe to call more than once, and from several goroutines. Always release through Close: db.DB() is the writer pool only, so db.DB().Close() leaves the reader pool open; its limits must not be changed either (see Open).

For a SQLite file database Close closes the readers first and waits for reads still running before it closes the writer, so the writer can checkpoint and remove the -wal file. It waits up to 5s: a read still running then (rows never closed) is reported as an error, and the -wal file is left next to the database, which still holds every write.

func Open

func Open(cfg Cfg, drivers Drivers) (*gorm.DB, error)

Open validates cfg and opens the configured database with the matching driver from drivers; the driver for the other database may be nil.

GORM's TranslateError is on, so callers check errors.Is(err, gorm.ErrDuplicatedKey) and errors.Is(err, gorm.ErrForeignKeyViolated) regardless of the database. Release the database with Close.

SQLite runs a single write connection plus a pool of read-only ones (see SqliteCfg.Readers), in WAL mode. Writers queue in Go instead of exhausting SQLite's busy timeout; a read does not wait for a write, only for a free reader (more concurrent reads than Readers queue in Go). db.DB() returns the writer pool: do not change its limits (SetMaxOpenConns above 1 brings SQLITE_BUSY back), and statements sent through it all go to the writer.

Routing, the contract: a plain SELECT goes to the read pool, everything else to the writer. That holds for the queries GORM builds (Table, Select, Find, ...) and for raw SQL alike, from Raw or Exec: routing goes by the SQL text, not by the method that sends it. db.Clauses(gormdb.Read) forces a statement onto the read pool, db.Clauses(gormdb.Write) onto the writer. Inside db.Transaction and db.Connection every statement on tx stays on the connection they hold, the writer, reads included.

The routing is GORM's dbresolver; how it tells a plain SELECT apart is its behaviour at the version gormdb is tested with, not a gormdb promise: raw SQL is a read when it begins with SELECT, is longer than 10 characters and does not end in FOR UPDATE, and a built query with a FOR clause (FOR UPDATE) goes to the writer. WITH ..., PRAGMA and other raw SQL go to the writer, the single connection, also when they only read. Force a statement with gormdb.Read or gormdb.Write where the guess matters.

SQL that depends on the connection (SELECT last_insert_rowid(), changes(), temp tables) must run in a transaction or in db.Connection, because a bare SELECT lands on another, read-only connection; db.Clauses(gormdb.Write) reaches the writer too, but another goroutine may write in between. For a new row's ID prefer Create, which reads it back, or INSERT ... RETURNING.

Inside db.Transaction or db.Connection, use only tx: the single writer connection is held by the callback, so a statement on db that reaches the writer waits for it forever (a deadlock).

On a SQLite file database the dbresolver plugin slot (gorm:db_resolver) is taken by the read pool, so an app's own db.Use(dbresolver.Register(...)) returns gorm.ErrRegistered. Open checks that its handling of db.Connection is in effect with the linked GORM and dbresolver versions, and fails with an error naming them if not.

":memory:" uses one connection for everything and has no read pool, so it behaves differently from a file database: routing is never exercised, any statement on db inside a transaction deadlocks (reads included), and so does a write while rows from a query are still open; connection-dependent SQL works where it would fail on a file. Test against a file in a temporary directory (gormdbtest.Sqlite) rather than ":memory:".

WAL does not work on network filesystems: keep the database on a local disk. Copying a database file also needs its -wal file, or the latest writes are lost; close the database first for a self-contained file.

Postgres gets PostgresCfg's pool settings, and a connect timeout so Open fails fast against an unreachable host.

func Ping

func Ping(ctx context.Context, db *gorm.DB) error

Ping checks that the database is reachable: the writer pool (db.DB()) and, for a SQLite file database, the read pool, one connection of each. The SQLite writer is a single connection, so Ping waits, within ctx, for a transaction in progress.

func ReaderDB

func ReaderDB(db *gorm.DB) (readers *sql.DB, ok bool)

ReaderDB returns the read-only pool of a SQLite file database opened by Open, for its DBStats; ok is false for Postgres and ":memory:", which have none. Do not change its limits or close it: Close releases it.

Types

type Cfg

type Cfg struct {
	// Driver is DriverSqlite or DriverPostgres.
	Driver   string
	Sqlite   SqliteCfg
	Postgres PostgresCfg
	// Logger receives GORM's log output; nil keeps GORM's default logger.
	Logger gormlogger.Interface
	// NamingStrategy maps structs and fields to table and column names; nil
	// keeps GORM's default (snake_case, plural table names).
	NamingStrategy schema.Namer
	// NowFunc returns the time GORM writes to CreatedAt and UpdatedAt; nil
	// keeps GORM's default, the local time.
	NowFunc func() time.Time
	// SkipDefaultTransaction stops GORM from wrapping each Create, Update and
	// Delete in a transaction of its own.
	SkipDefaultTransaction bool
}

Cfg selects and configures the database. Only the section matching Driver is used.

Logger, NamingStrategy, NowFunc and SkipDefaultTransaction are set in code, not loaded from a config file; they are passed to gorm.Config as they are. gorm.Config's PrepareStmt is deliberately not offered: it swaps the connection pools for prepared-statement wrappers, which interacts with dbresolver's own prepared-statement store and with the wrapper that keeps a db.Connection on its connection, a combination gormdb does not test.

func (Cfg) Validate

func (c Cfg) Validate() error

Validate reports a configuration Open cannot work with. Call it while loading the config to fail before anything starts.

type Drivers

type Drivers struct {
	Sqlite   SqliteDriver
	Postgres PostgresDriver
}

Drivers holds the database drivers an app compiles in. Only the one matching Cfg.Driver is used; the other may be nil.

type PostgresCfg

type PostgresCfg struct {
	// Host is a host name, an IP address or the directory of a Unix socket,
	// without a port (see Port) and, for IPv6, without brackets.
	Host string
	Port int // zero means 5432
	// User may be empty: libpq's defaults (PGUSER, the OS user) then apply,
	// and Password, if set, is still sent.
	User     string
	Password string
	DbName   string
	// SslMode is libpq's sslmode: disable, allow, prefer, require, verify-ca
	// or verify-full; empty means "require".
	SslMode string
	// ConnectTimeout bounds how long opening a connection may take, so Open
	// fails fast against an unreachable host; zero means 10s. It is sent in
	// whole seconds, rounded up.
	ConnectTimeout time.Duration
	// MaxOpenConns, MaxIdleConns and ConnMaxLifetime tune the connection pool
	// (see database/sql); zero keeps database/sql's default: no limit on open
	// connections, 2 idle ones, no maximum lifetime.
	MaxOpenConns    int
	MaxIdleConns    int
	ConnMaxLifetime time.Duration
	// Params are extra connection parameters, such as application_name,
	// search_path or default_query_exec_mode. They must not repeat one of the
	// fields above (host, port, user, password, dbname, sslmode,
	// connect_timeout).
	Params map[string]string
}

PostgresCfg configures a Postgres connection. Printing it with fmt, or logging it with slog, shows Password and every Params value as "REDACTED" (the Params keys stay visible): a parameter such as sslpassword can hold a secret.

func (PostgresCfg) DSN

func (c PostgresCfg) DSN() string

DSN returns the connection URL. User, password, database name and Params are escaped, so any character is safe in them.

func (PostgresCfg) LogValue

func (c PostgresCfg) LogValue() slog.Value

LogValue shows the configuration in slog with Password and the Params values redacted.

func (PostgresCfg) String

func (c PostgresCfg) String() string

String shows the configuration with Password and the Params values redacted.

type PostgresDriver

type PostgresDriver interface {
	// OpenDB returns a connection pool for cfg, already validated. The
	// shipped driver connects with cfg.DSN(); another may read the fields
	// and build the pool its own way. Like sql.Open it must not connect yet.
	// Open sets MaxOpenConns, MaxIdleConns and ConnMaxLifetime on the result.
	OpenDB(cfg PostgresCfg) (*sql.DB, error)
	// Dialector wraps an open connection pool. Translate must keep the
	// original error.
	Dialector(conn gorm.ConnPool) gorm.Dialector
}

PostgresDriver adapts a Postgres driver package to gormdb. As with SQLite, the driver opens the pool and Open owns it: Open applies the pool settings, hands the pool to Dialector, and closes it if opening fails.

An app that must build the pool itself (credentials that rotate, a tracing wrapper) embeds the shipped gormdb/postgres Driver and replaces OpenDB.

type SqliteCfg

type SqliteCfg struct {
	// Path of the database file, or ":memory:" for an in-memory database (one
	// connection, no read pool). It is a filesystem path, never a URI: a path
	// starting with "file:" is rejected. Its directory is created when missing; keep
	// the file in a directory of its own, SQLite writes -wal and -shm files
	// next to it.
	Path string
	// BusyTimeout is how long a connection waits for a lock held by another
	// one before failing with SQLITE_BUSY; zero means 5s. SQLite counts in
	// whole milliseconds: it is rounded down, to at least 1ms and at most
	// 2^31-1 ms (about 24 days). It must not be negative.
	BusyTimeout time.Duration
	// Readers is the size of the read-only connection pool; zero means 10.
	// ":memory:" has no read pool, so it must stay zero there.
	Readers int
	// Pragmas are extra pragmas, name to value, such as "synchronous":
	// "NORMAL" or "cache_size": "-20000". Every connection runs them, the
	// writer and each reader, as most pragmas are per-connection settings; a
	// db.Exec("PRAGMA ...") would reach a single pooled connection. A name is
	// a plain identifier and a value an identifier or an integer, so nothing
	// else can reach the DSN. busy_timeout, foreign_keys, journal_mode and
	// locking_mode are rejected: Open and the drivers set them. A pragma that
	// writes to the database file (user_version, application_id) fails on
	// the read-only readers; run it once with db.Exec.
	Pragmas map[string]string
}

SqliteCfg configures a SQLite database file.

type SqliteDriver

type SqliteDriver interface {
	// OpenDB returns a connection pool for dsn, a file: URI or
	// ":memory:?...", whose query holds the parameters below. Like sql.Open
	// it must not connect yet.
	OpenDB(dsn string) (*sql.DB, error)
	// WriterParams returns the writer's DSN query parameters in the driver's
	// own format: busy timeout, foreign keys, WAL and an immediate
	// transaction lock. ReaderParams returns a reader's: busy timeout and
	// foreign keys only. Each call returns a map the caller may keep.
	WriterParams(busyTimeout time.Duration) url.Values
	ReaderParams(busyTimeout time.Duration) url.Values
	// PragmaParams returns the DSN query parameters that make every
	// connection opened by OpenDB run the extra pragmas (SqliteCfg.Pragmas,
	// already validated). Open adds them to the writer's and the readers'
	// parameters. Each call returns a map the caller may keep.
	PragmaParams(pragmas map[string]string) url.Values
	// Dialector wraps an open connection pool. Translate must keep the
	// original error.
	Dialector(conn gorm.ConnPool) gorm.Dialector
}

SqliteDriver adapts a SQLite driver package to gormdb. The two adapters shipped with gormdb, gormdb/sqlite and gormdb/sqlitecgo, are the supported implementations: they pass the behaviour suite in gormdb's internal sqlitetest package, which is not importable from outside the module.

Each driver decides how its DSN sets the pragmas; Open verifies on the writer, after it opened it, that journal_mode is WAL (file databases) and foreign_keys is on, and fails naming this contract if not.

Directories

Path Synopsis
Package gormdbtest opens databases for tests.
Package gormdbtest opens databases for tests.
internal
sqliteerr
Package sqliteerr maps SQLite constraint errors to gorm's sentinel errors.
Package sqliteerr maps SQLite constraint errors to gorm's sentinel errors.
sqlitetest
Package sqlitetest is the behaviour suite every SQLite driver adapter must pass; each driver package runs it from its own tests.
Package sqlitetest is the behaviour suite every SQLite driver adapter must pass; each driver package runs it from its own tests.
Package postgres adapts the gorm.io/driver/postgres driver (pgx) to gormdb.
Package postgres adapts the gorm.io/driver/postgres driver (pgx) to gormdb.
Package sqlite adapts the pure-Go glebarez SQLite driver to gormdb, so the binary builds without CGO.
Package sqlite adapts the pure-Go glebarez SQLite driver to gormdb, so the binary builds without CGO.
Package sqlitecgo adapts the CGO mattn/go-sqlite3 driver to gormdb.
Package sqlitecgo adapts the CGO mattn/go-sqlite3 driver to gormdb.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL