Documentation
¶
Overview ¶
Package pgtest provides the postgres testcontainer setup that every postgres-backed suite in this repo would otherwise hand-roll: start the container with the shared retry policy and wait strategy, open a pgx-backed *sql.DB against it, ping it, and tear all of it down afterwards.
Callers describe the shape they want with Options and receive a live Instance inside a closure, so a test body says what it does with postgres and nothing about how postgres is stood up or torn down.
One server, one isolated database per test ¶
A container per test gives perfect isolation at a price that stops scaling once a package has a few dozen: every test pays a container start plus a full migration replay, and a package running its tests in parallel asks the Docker daemon for that many postgres instances at once. Past a certain width the daemon stops answering and containers fail their readiness wait — not because anything is wrong with the test, but because nothing was rationing the daemon.
Isolation does not require a container, though. Start one Instance per test binary and hand each test its own corner of it:
- Instance.Schema creates a private schema, injects it through the DSN's search_path, and drops it on cleanup. Cheap, and available on managed postgres where CREATE DATABASE is restricted.
- Instance.Template migrates one database, and Template.Clone copies it per test with CREATE DATABASE ... TEMPLATE — a file copy rather than a replay of every migration. Stronger isolation, since it covers extensions and everything else schema-scoped rules do not, and no search_path to inject.
Neither is strictly better. Schemas are cheaper and portable; clones isolate more and skip per-test migration entirely.
The lock key that makes schemas parallel ¶
Schema-isolated tests migrate concurrently, and they must not serialize on one advisory lock while doing it. Pass migrate.WithSchemaScopedLockKey() to the Migrator you hand to WithMigration: it derives the lock ID from the connection's current schema, so deployments on the default schema still share one lock and test schemas never contend with each other. Without it, parallel setup becomes a queue.
Clones need no such thing. Postgres advisory locks are per-database, and migrations run once into the template before any test starts.
Per-binary setup from TestMain ¶
Run is one of the two per-binary shapes; TestMain is the other, and it is the one a suite arrives with when it already had per-package fixtures. A *testing.M is not a testing.TB and cannot be adapted into one — the interface has an unexported method for exactly that reason — so Start and Instance.NewTemplate are Run and Instance.Template with the two testing.TB decisions handed back instead of taken: teardown is returned rather than registered, and an unavailable postgres is ErrNoPostgres rather than a skip.
var template *pgtest.Template
func TestMain(m *testing.M) { os.Exit(run(m)) }
func run(m *testing.M) int {
// testing.Short() panics before flag.Parse, so a TestMain that gates on
// -short parses first. Without the gate a -short run starts a container
// and then skips every test that would have queried it.
flag.Parse()
if testing.Short() {
return m.Run()
}
pg, teardown, err := pgtest.Start(context.Background())
if err != nil {
// ErrNoPostgres means nothing was started, which is this suite's
// cue to let its tests skip themselves.
return m.Run()
}
defer func() { _ = teardown() }()
tmpl, dropTemplate, err := pg.NewTemplate(context.Background(), pgtest.WithMigration(migrator.Migrate))
if err != nil {
return 1
}
defer func() { _ = dropTemplate() }()
template = tmpl
return m.Run()
}
The body is a function returning a code rather than TestMain itself because os.Exit does not run deferred functions: teardown has to happen before the exit, and the only way to have both is to put the exit outside.
Both teardowns return an error, because draining a pool and terminating a container can each fail and there is nothing here to log it to on their behalf. A suite that wants to hear about it logs it; the discard above is the other legitimate answer, and it is at least written down as one.
Instance.Schema and Template.Clone keep their testing.TB and need no companion — by the time either is called there is a test in hand.
Index ¶
- Constants
- Variables
- func Run(tb testing.TB, fn func(ctx context.Context, pg *Instance), opts ...Option)
- type Instance
- func (i *Instance) ConnectionStringFor(tb testing.TB, database, username, password string) string
- func (i *Instance) NewTemplate(ctx context.Context, opts ...IsolationOption) (*Template, func() error, error)
- func (i *Instance) Open(tb testing.TB, connectionString string) *sql.DB
- func (i *Instance) Schema(tb testing.TB, opts ...IsolationOption) *Isolated
- func (i *Instance) Template(tb testing.TB, opts ...IsolationOption) *Template
- type Isolated
- type IsolationOption
- type MigrateFunc
- type Option
- func WithCredentials(database, username, password string) Option
- func WithCustomizers(customizers ...testcontainers.ContainerCustomizer) Option
- func WithDSNFromEnv(name string) Option
- func WithImage(image string) Option
- func WithMaxConnections(n int) Option
- func WithMaxOpenConns(n int) Option
- func WithRequiredPostgres() Option
- type Template
Constants ¶
const ( // DefaultIsolatedMaxOpenConns and DefaultIsolatedMaxIdleConns size the pool // Schema and Clone hand back. They are deliberately tiny: the connection // ceiling belongs to the whole run, not to one test, and a pool sized for a // service that owns its database is the wrong shape for a few dozen suites // sharing one server. What over-sizing looks like downstream is "too many // clients already" from whichever test connects last. DefaultIsolatedMaxOpenConns = 4 DefaultIsolatedMaxIdleConns = 2 )
const ( // DefaultImage is the postgres image Run launches when no override is given. DefaultImage = "postgres:17-alpine" // DriverName is the database/sql driver Instance.DB and Instance.Open use. DriverName = "pgx" // DefaultMaxConnections is the server-wide connection ceiling Run provisions // the container with, replacing postgres' default of 100. // // One container now serves every test in a binary — Schema and Clone hand out // pools against one server rather than one server each — so the ceiling is // spent by the whole run at once instead of per test. At the default it is // whichever test connects last that fails, with "too many clients already", // which reads as flake rather than as the budget it is. DefaultMaxConnections = 200 )
Variables ¶
var ErrNoPostgres = platformerrors.New("pgtest: no postgres available")
ErrNoPostgres reports that no postgres was available and none was started: the RUN_CONTAINER_TESTS gate is closed and WithRequiredPostgres was not given. Run turns that situation into a skip, which is a testing.TB's move and therefore not one Start can make — so Start names it instead, and the caller decides. A TestMain that wants the suite to skip its way through ignores this sentinel; one that wants a hard failure returns the error.
Functions ¶
func Run ¶
Run resolves a postgres, opens a pool against it, and hands both to fn as an Instance. It is Start with the postgres-shaped setup — image, credentials, readiness wait, sql.Open, ping — already applied and the lifecycle owned, so the closure starts from a database it can query and ends without tidying up.
The resolution ladder, in order:
- the DSN in the environment variable named by WithDSNFromEnv, if that option was given and the variable is set. No container is started.
- -short, which skips.
- a container. Whether an unavailable one skips or fails is the suite's call — see WithRequiredPostgres — and by default it skips, along with the RUN_CONTAINER_TESTS gate.
Startup failures fail the test, and teardown of both the pool and the container is registered with tb.Cleanup — so fn is free to spawn parallel subtests against the Instance and return before they run.
One Run per test binary is the shape this is built for. Give each test its own schema with Instance.Schema, or its own database with Instance.Template and Template.Clone, rather than a container each. A binary whose per-binary setup lives in TestMain wants Start instead; Run is that function with a testing.TB's skip and cleanup applied on top.
Types ¶
type Instance ¶
type Instance struct {
// DB is an open, pinged pool against Database as Username.
DB *sql.DB
// Container is the underlying testcontainer, for the rare test that needs
// Exec or a snapshot. Its lifecycle is not yours to manage. It is nil when
// WithDSNFromEnv resolved the server, since there is no container then;
// Host and Port are populated either way.
Container *postgrescontainer.PostgresContainer
// ConnectionString is the DSN DB was opened with.
ConnectionString string
// Host and Port locate the server DB is connected to, whether that is the
// container or the server named by WithDSNFromEnv.
Host string
Port string
// Database, Username and Password are the credentials the server was
// reached with, exposed so tests can reconnect or grant against them.
Database string
Username string
Password string
}
Instance is the live postgres handed to a Run closure. DB covers the common case; the remaining fields are there for the tests that need a second connection, a different role, or the container API itself.
func Start ¶
Start resolves a postgres and opens a pool against it for a caller with no testing.TB — a TestMain, most often. It is Run's body with the two testing.TB-shaped decisions handed back instead of taken: teardown is returned rather than registered, and an unavailable postgres is ErrNoPostgres rather than a skip.
The resolution ladder is Run's, minus the rung that needs a test:
- the DSN in the environment variable named by WithDSNFromEnv, if that option was given and the variable is set. No container is started.
- a container, unless the RUN_CONTAINER_TESTS gate is closed and WithRequiredPostgres was not given, which is ErrNoPostgres.
-short is not consulted here, because Start cannot know whether its caller has parsed flags yet: testing.Short() before flag.Parse panics rather than reporting false, and a library entry point is the wrong place to find that out. A TestMain that wants -short honored parses first and gates itself, which costs one line and saves starting a container the run will not use:
func run(m *testing.M) int {
flag.Parse()
if testing.Short() {
return m.Run() // nothing started; the tests skip themselves
}
...
}
Individual tests can skip through containers.SkipIfNotRunning instead, which reads -short at a point in the binary's life where it has been parsed.
The returned teardown closes the pool and terminates the container, in that order, and running it is the caller's job. Running it *before* os.Exit is the part that is easy to get wrong, since os.Exit does not run deferred functions — see the package documentation for the shape that gets it right.
func (*Instance) ConnectionStringFor ¶
ConnectionStringFor builds a DSN for this server under different credentials, for suites that connect as a role they created rather than as the provisioning superuser.
func (*Instance) NewTemplate ¶
func (i *Instance) NewTemplate(ctx context.Context, opts ...IsolationOption) (*Template, func() error, error)
NewTemplate is Instance.Template for a caller with no testing.TB — a TestMain, most often, which is the shape a per-binary template already belongs to. It is Template's body with the testing.TB-shaped decisions handed back instead of taken: the drop is returned as a teardown rather than registered, and a failure anywhere in there is an error rather than a fatal.
See Start, which is where a caller in that position gets its Instance, and which documents when the returned teardown has to run.
Name it with WithLabel if the databases want to be identifiable; without a test to borrow a name from the template is tmpl_<random>.
func (*Instance) Open ¶
Open opens and pings an additional pool against this container and closes it when the test ends. Use it alongside ConnectionStringFor to connect as another role; for the provisioning role, DB is already open.
func (*Instance) Schema ¶
func (i *Instance) Schema(tb testing.TB, opts ...IsolationOption) *Isolated
Schema creates a private schema on this instance, opens a pool whose search_path points at it, runs WithMigration if one was given, and drops the schema when tb ends.
Everything unqualified — the tables the migrations create, the rows the test writes, and goose's own version table — lands inside the schema, so two tests running in parallel against one server never see each other's data. Give each test its own Schema; sharing one puts them back in the same database they were trying to get out of.
The migration must not serialize with its peers. See WithMigration.
func (*Instance) Template ¶
func (i *Instance) Template(tb testing.TB, opts ...IsolationOption) *Template
Template creates a database, runs WithMigration against it once, and returns a handle that Clone copies per test. The database is dropped when tb ends.
The migration pool is closed before Template returns, and that is load-bearing rather than tidy: CREATE DATABASE ... TEMPLATE refuses to run while any session is attached to the template, so a pool left open would fail the first clone instead of this call.
type Isolated ¶
type Isolated struct {
// DB is an open, pinged, deliberately small pool. For a schema its
// search_path names Name, so unqualified DDL and DML land inside it.
DB *sql.DB
// Name is the schema or database name, unique within the run. Tests that
// need to reconnect, or to assert on catalog rows, need it.
Name string
// ConnectionString is the DSN DB was opened with.
ConnectionString string
}
Isolated is one test's private corner of a shared server: a schema from Instance.Schema, or a database from Template.Clone. Either way DB is a live, migrated pool and the underlying object is dropped when the test ends.
type IsolationOption ¶
type IsolationOption func(*isolationOptions)
IsolationOption configures Instance.Schema, Instance.Template and Template.Clone.
func WithLabel ¶
func WithLabel(label string) IsolationOption
WithLabel names the schema or database for the human reading a failure. It is sanitized and trimmed like a test's name, and what actually keeps two of them apart is the random suffix either way.
Schema, Template and Clone default to the name of the test that asked, and need this only when that name is not the useful one. NewTemplate has no test to take a name from, so without it a per-binary template is tmpl_<random> — unique, but anonymous in a `\l` listing or a stuck-session query.
func WithMigration ¶
func WithMigration(fn MigrateFunc) IsolationOption
WithMigration supplies the migration to run against the new schema or database. Absent, the schema or clone is handed back empty, which is what a test that creates its own tables wants.
For a schema, build the Migrator with migrate.WithSchemaScopedLockKey() or parallel setup serializes on one advisory lock. For a template it does not matter: the migration runs once, before any clone exists.
func WithPoolSize ¶
func WithPoolSize(maxOpen, maxIdle int) IsolationOption
WithPoolSize overrides DefaultIsolatedMaxOpenConns and DefaultIsolatedMaxIdleConns for this schema or clone. Non-positive values leave database/sql's own unlimited defaults in place, which for a suite sharing one server is rarely what you want.
type MigrateFunc ¶
MigrateFunc applies a schema to a freshly created schema or database. It is exactly the shape of database.Migrator's Migrate method, so a *migrate.Migrator satisfies it as a method value:
m, err := migrate.New(dialect.Postgres, migrations, migrate.WithSchemaScopedLockKey()) must.NoError(t, err) schema := pg.Schema(t, pgtest.WithMigration(m.Migrate))
It is a parameter rather than an import because database/migrate's own tests use this package, and importing it back would close the cycle.
type Option ¶
type Option func(*options)
Option configures Run and Start.
func WithCredentials ¶
WithCredentials overrides the database name, superuser and password the container is provisioned with. Tests that create or drop roles want distinct credentials so they cannot collide with the identifiers under test.
func WithCustomizers ¶
func WithCustomizers(customizers ...testcontainers.ContainerCustomizer) Option
WithCustomizers appends testcontainers customizers to the ones Run already applies. They run after the defaults, so they can override the wait strategy.
func WithDSNFromEnv ¶
WithDSNFromEnv names an environment variable holding a postgres DSN. When it is set and non-empty, Run connects to that server and starts no container at all — the first rung of the resolution ladder, ahead of -short and ahead of starting anything.
It is how a suite runs against a postgres that CI already provides, and how a developer points the whole binary at a local server. The container-only fields of Instance are absent on this path: Container is nil, and Database, Username and Password are read out of the DSN rather than from WithCredentials.
func WithImage ¶
WithImage overrides DefaultImage. Use it for postgres derivatives that the rest of this setup still applies to, e.g. "pgvector/pgvector:pg17".
func WithMaxConnections ¶
WithMaxConnections overrides DefaultMaxConnections, the server-wide ceiling the container is started with. Raise it for a binary whose tests are both numerous and parallel; the budget is one number shared by every pool Run, Schema and Clone hand out, so it has to cover the widest moment of the run rather than the widest single test. Zero leaves the image's own default.
func WithMaxOpenConns ¶
WithMaxOpenConns caps Instance.DB's pool. Set it well above the number of concurrent subtests sharing an Instance, otherwise they starve each other. Zero (the default) leaves database/sql's unlimited default in place.
func WithRequiredPostgres ¶
func WithRequiredPostgres() Option
WithRequiredPostgres makes an unavailable postgres a test failure instead of a skip, by way of containers.Required.
The default gate is right for this module and wrong for a service: a library whose consumers may have no Docker daemon should skip, while a service whose postgres backend is only ever exercised here should fail loudly, because a skip is indistinguishable from a pass and a backend can reach zero coverage that way without anyone noticing. -short still skips either way.
type Template ¶
type Template struct {
// Name is the template database's name.
Name string
// contains filtered or unexported fields
}
Template is a migrated database that Clone copies per test. Build one per binary, from Instance.Template.
func (*Template) Clone ¶
func (t *Template) Clone(tb testing.TB, opts ...IsolationOption) *Isolated
Clone copies the template into a fresh database and hands back a pool over it, dropped when tb ends. The copy is a file copy rather than a replay of every migration, which is what makes per-test isolation affordable.
WithMigration is honored here too, for the occasional test that needs a migration the template does not carry; most callers migrate the template and pass nothing.
Source Files