Documentation
¶
Overview ¶
Package dockerdb manages the throwaway Postgres rig runs migrations against.
The container is deliberately long-lived: it is started on the first command that needs it and left running, so the next command finds it warm. Starting and stopping a database on every invocation would make the inner loop several seconds slower for no benefit — nothing in it is precious, and `rig db reset` throws it away when the schema needs to start over.
Commands are run by shelling out to docker or podman rather than by linking a client library. Four operations — inspect, run, start, remove — do not justify the dependency, and shelling out is what makes podman work for free.
Index ¶
- Constants
- Variables
- func Create(ctx context.Context, rt Runtime, log io.Writer, name string, args ...string) error
- func ElectricURL(port int) string
- func HostPort(port int) int
- func Isolated() bool
- func Migrate(ctx context.Context, opt MigrateOptions) (applied int, err error)
- func PortOf(ctx context.Context, runtimeBin, name string) (int, error)
- func Publish(bind string, port, containerPort int) string
- func Qualify(name string) string
- type CLIRuntime
- type Config
- type DB
- type Electric
- type ElectricConfig
- type MigrateOptions
- type Runtime
Constants ¶
const ( // PortDefault is what `rig init` writes into a new project, so it is the one // number here a real project sees. Not a suite's, and not to be taken by one. PortDefault = 55432 // PortDefaultElectric is the sync service's default beside it, when // `database.electric` is enabled. internal/project/defaults.go carries the // same number; being listed here is what keeps a suite off it. PortDefaultElectric = 55433 // The examples, whose databases `make examples` brings up. Declared here, // defined in each example's own configuration. PortExampleTodo = 55440 PortExampleFantasyFootball = 55441 PortExampleAuth = 55442 PortExampleAuthOAuth = 55443 PortExampleLinearlite = 55444 // PortExampleLinearliteElectric is that example's sync service — an HTTP // port rather than a database one, in the list because it is the same host // and the same collision. PortExampleLinearliteElectric = 55445 // PortFiles is internal/filestest: uploads, the tenancy around them, and the // sweeper. PortFiles = 55494 // PortS3MinIO is the MinIO container rigs3's suite runs against, and it is // the second exception to this file's rule: rigs3 is a published module and // cannot import this one, so the number lives in its test and is only // declared here. Listed for the reason the examples are — so that every port // a suite in this repository takes is allocated from one list. PortS3MinIO = 55446 // PortRigTest is the database rigtest's own suite runs against, and it is // the third exception, for the same reason as the second: rigtest is // published and cannot import this package, so the number lives in // rigtest/postgres_docker_test.go and is only declared here. PortRigTest = 55447 // PortMigrate is the database the migrate module's own suite runs against, // and it is the fourth exception. The reason is the third's and one more: // migrate is published, and this package imports it, so an import the other // way would be a cycle as well as a dependency on the CLI. The number lives // in migrate/postgres_docker_test.go and is only declared here. PortMigrate = 55448 // PortIdempotency is internal/idemtest, where two transactions contend for // one key. Its own container because the contention is the test: a suite // sharing a database with one that holds locks of its own would fail as a // timeout nobody could place. PortIdempotency = 55495 // PortElectricSync is the ElectricSQL container in internal/electrictest, // which is an HTTP port rather than a database one. It is in the same list // because it is the same host and the same collision. PortElectricSync = 55490 // PortIntrospect is internal/introspect, reading a real catalogue. PortIntrospect = 55491 // PortElectricDB is the database internal/electrictest syncs from. PortElectricDB = 55492 // PortCLIEndToEnd is internal/cli's end-to-end loop. PortCLIEndToEnd = 55493 // PortCLISetup and PortCLISetupOwn are internal/cli's setup-project tests. // Two, because one of them takes the schema over and the other does not, and // they run in parallel. PortCLISetup = 55496 PortCLISetupOwn = 55497 // PortAuth is internal/authtest. PortAuth = 55498 // PortCLIElectricDB and PortCLIElectricSync are internal/cli's test of // `rig db up` managing the sync service beside the database. PortCLIElectricDB = 55499 PortCLIElectricSync = 55489 // PortPresence is internal/presencetest: the heartbeat that is an upsert, the // sweep that two replicas may run at once, and the DDL actually applying. PortPresence = 55488 // PortThrottle is internal/throttletest: the one statement that both spends // a slot and reports the total, and what it does when several replicas run // it at once on one row. PortThrottle = 55487 // PortCache is internal/cachetest: whether a notification really is // delivered on commit and discarded on rollback, and what a listener that // lost its connection knows when it comes back. Its own container because // one of those tests kills a backend, and a suite sharing the database would // see it as its own connection dying. PortCache = 55486 // PortElectricRoleDB and PortElectricRoleSync are internal/electrictest's // least-privilege case: the sync service connecting as the `electric` role // rig's foundation creates rather than as the superuser `rig db up` hands it. // // Its own pair rather than a case in the suite next door, because the // container's DATABASE_URL is what the test is about — it has to be a // different sync service, and one whose Postgres has been through the role // migration. PortElectricRoleDB = 55485 PortElectricRoleSync = 55484 )
The ports the Docker suites pin, in one place.
Every suite starts a container on a fixed host port so a warm one can be reused between runs, and until this file existed each of them chose that port by grepping for a number nobody else seemed to be using. That works until two suites pick the same one, and then it fails as a connection to somebody else's database with somebody else's schema in it — which looks like a bug in the code under test rather than a collision.
Adding a suite means adding a constant here first. TestPortsAreUnique is what makes that more than an instruction.
These numbers keep two suites in one checkout apart. They do nothing about two checkouts, which is a different problem with a different answer — see isolate.go, where a port from this list becomes a request rather than a requirement.
They also do nothing about a process that is not rig's. The whole block sits inside the ephemeral range Linux hands to outbound connections (32768–60999), so a number here can be taken before a suite asks for it — which is why create.go retries a container the engine could not publish, on this path as much as on the isolated one.
The examples are the exception and are listed rather than used: each one's port lives in its own rig.yaml and main.go, in a module that cannot import this one. They are named here so the numbers are still allocated from one list, and TestPortsAreUnique holds them to the same rule. PortS3MinIO is there for the same reason from the other direction — rigs3 is published, and a published module that imported this one would require the CLI to build.
const ElectricSyncPort = 3000
ElectricSyncPort is the port the sync service listens on inside its container. Fixed by the image; the host side is ElectricConfig.Port.
const IsolateEnv = "RIG_DB_ISOLATE"
IsolateEnv is the environment variable that names this checkout.
Its value is opaque and only has to differ between checkouts; a path is the obvious thing to use and is what rig's Makefile passes. Unset or empty means one checkout, which is every project rig generates and every machine with one copy of it.
Variables ¶
var ErrNoRuntime = errors.New("no container engine found")
ErrNoRuntime is returned when no container engine is usable.
var ErrPortInUse = errors.New("the container engine could not take a host port")
ErrPortInUse is what survives every attempt: the engine could not take a host port for the container.
Worth matching with errors.Is rather than reading, because under isolation there is no port in the request to report — the one that was refused is the engine's own choice, and only its message names it.
Functions ¶
func Create ¶ added in v0.6.0
Create starts a detached container, retrying while the engine cannot take a host port for it.
args are everything after `run --detach --name <name>`. It is exported for the suites that start a sidecar of their own rather than going through Start or StartElectric — they publish under the same isolation and lose the same race. log may be nil.
func ElectricURL ¶ added in v0.3.0
ElectricURL is where a sync service on this port answers.
Spelled once, because two callers build it: this handle, which knows the port a container actually published, and `rig db url --electric`, which answers from rig.yaml without starting anything.
func HostPort ¶
HostPort is the port to publish a container on, given the one configured.
Zero means "whatever is free", which is the only answer that cannot collide. A digest would have been the same trick used for the name, and it does not work here: a port is sixteen bits, rig's own suites want a block of them, and with a dozen checkouts on one machine the birthday collision is likelier than not. The kernel already allocates ports without collisions, so under isolation it does the allocating and DB.URL reports what it chose.
func Isolated ¶
func Isolated() bool
Isolated reports whether this checkout keeps its containers to itself.
func Migrate ¶
func Migrate(ctx context.Context, opt MigrateOptions) (applied int, err error)
Migrate applies every pending migration to the development database.
The work is rig/migrate, the same package an application embeds to migrate itself in production. One implementation is the point: `rig db up` here and the deployment there have to agree about what the schema is, and two readers of the same files would eventually disagree about one of them.
func PortOf ¶
PortOf reports the host port a container publishes, for one this package did not start. Under isolation that is the only way to know it.
func Publish ¶
Publish is the --publish argument for a container this package does not manage, so that a test starting a sidecar by hand gets the same treatment its database got.
func Qualify ¶
Qualify is the container name to use for one whose configured name is name.
Unchanged when this is the only checkout, so `docker ps` still shows todo-db and `rig db psql` is still the way in. Suffixed with a digest of the isolation token when it is not — long enough that two checkouts colliding is not a thing that happens, short enough to still read as the name it came from.
Types ¶
type CLIRuntime ¶
type CLIRuntime struct{ Bin string }
CLIRuntime drives a container engine through its command-line tool.
func (CLIRuntime) Available ¶
func (r CLIRuntime) Available(ctx context.Context) error
Available implements Runtime. It asks the daemon for its version rather than only checking that the binary exists, because an installed docker with no daemon behind it is the common failure and deserves a clear message.
type Config ¶
type Config struct {
Image string
Name string
Port int
Database string
User string
Password string
Runtime string // "" picks docker, then podman
Log io.Writer
StartWait time.Duration
// Settings are extra server parameters, without the -c. Live sync needs
// wal_level=logical, which cannot be set after the server has started.
Settings []string
// Bind is the host address the port is published on. Empty means
// 127.0.0.1, which is the right answer for a database nobody else should
// reach. A test that puts another container in front of this one needs
// 0.0.0.0 instead: on Linux, a sibling container arrives over the bridge,
// and a loopback-only publish refuses it.
Bind string
}
Config describes the throwaway database.
func (Config) URL ¶
URL is the connection string for the container.
The port is the configured one, so this answers for a Config and not for a running container: under isolation the port is zero here and the kernel picks the real one. DB.URL is what reports that, and is what a caller that has started something should use.
TimeZone is pinned rather than inherited. A timestamptz holds an instant and no zone, so nothing about what is *stored* depends on this — but `::date`, `date_trunc`, and an input string with no offset are all read in the session's zone, so a daily aggregate silently buckets differently on a machine set to Stockholm than on one set to UTC. Introspection and every generated repository therefore agree on one answer, and it is the one that does not move twice a year. pgx passes an unrecognised parameter through as a startup setting, so this is all it takes.
type DB ¶
type DB struct {
// contains filtered or unexported fields
}
DB is a running throwaway database.
func Attach ¶
Attach builds a handle to a container without starting it, for commands that stop or remove one.
func Start ¶
Start brings the database up and waits until it accepts connections.
An existing container is reused when it is running the same image on the same port. That is the common case — the second and every later command in a session — and it is what keeps `rig generate` fast. A container that does not match is replaced rather than adapted: it holds no state worth keeping.
type Electric ¶
type Electric struct {
// contains filtered or unexported fields
}
Electric is a running sync service.
func AttachElectric ¶
func AttachElectric(ctx context.Context, cfg ElectricConfig) (*Electric, error)
AttachElectric builds a handle without starting anything, for commands that stop or remove the container.
func StartElectric ¶
func StartElectric(ctx context.Context, cfg ElectricConfig) (*Electric, error)
StartElectric brings the sync service up and waits until it reports its replication stream active.
An existing container is reused when it is running the same image on the same port against the same database URL. The last condition is the one specific to this container: a database that was recreated — a tmpfs restart, an isolated checkout on a new kernel-chosen port — leaves a reused sync service holding a replication slot into nothing, so anything stale is replaced rather than adapted. The caller that knows the database is fresh removes this container first; the URL check catches what it cannot see.
type ElectricConfig ¶
type ElectricConfig struct {
Image string
Name string
// Port is the host port to publish on. Zero means the kernel picks, which
// is what isolation asks for.
Port int
// DBPort is the host port the database really publishes on — [DB.Port],
// not the configured number, because under isolation they differ. The sync
// service reaches it through the container runtime's host gateway, which is
// why the host port is the right one to hand over.
DBPort int
DBName string
DBUser string
DBPassword string
Runtime string // "" picks docker, then podman
Log io.Writer
StartWait time.Duration
}
ElectricConfig describes the sync-service container.
type MigrateOptions ¶
type MigrateOptions struct {
// Dir holds the project's own migration files.
Dir string
// Table is the bookkeeping table. Naming it per project keeps two rig
// projects sharing a database from fighting over one another's history.
Table string
// Foundation are rig's own migration sets, applied before the project's.
//
// Empty is the ordinary case, and it is what `migrations.foundation: vendored`
// produces: those migrations are already files under Dir, so applying them
// from here as well would be applying them twice under two histories.
//
// Under `embedded` they are the modules' own sets and this is the only place
// they come from. They go first because rig's DDL never references a project's
// table while a project's routinely references rig's — a join table pointing
// at rig_notification, a file column pointing at rig_file. A schema read
// without them is a schema rig would then generate the wrong code from, and
// quietly: the API-key write guard and every notifiable resource are decided
// by whether those tables were there.
Foundation []migrate.Source
// URL is the database to migrate.
URL string
// Log receives progress, if set.
Log io.Writer
}
MigrateOptions describe a migration run.
type Runtime ¶
type Runtime interface {
// Name is the executable, for messages.
Name() string
// Run executes a command and returns its standard output.
Run(ctx context.Context, args ...string) (string, error)
// Available reports whether the engine can be reached.
Available(ctx context.Context) error
}
Runtime is a container engine.