sqlstore

package
v0.7.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 25, 2026 License: BSD-2-Clause Imports: 24 Imported by: 0

Documentation

Overview

Package sqlstore provides the shared SQL plumbing (dialect differences, connection setup, schema migrations) used by every service's relational storage implementation. Repository code lives in each service's own db package (next to its Mongo sibling), not here — this package only holds what's genuinely shared: the small set of behaviors that differ between Postgres and MariaDB, and connecting/migrating a database handle.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ApplySchema

func ApplySchema(ctx context.Context, db *sqlx.DB, dialect Dialect, cfg *SQL) error

ApplySchema runs any pending schema migrations for the given dialect against db. Safe to call on every service startup: golang-migrate tracks applied versions in a schema_migrations table in the target database and is a no-op once the schema is already current, mirroring how Mongo index creation already happens idempotently at startup.

cfg is the same config Connect built db from; ApplySchema needs it directly for MariaDB (see the "mariadb" case below) rather than only the already-opened db.

ctx bounds connection acquisition and the driver setup phase (both respect ctx via PingContext/QueryRowContext internally) -- e.g. a caller can time out or cancel startup if the database is unreachable, rather than hanging indefinitely. golang-migrate's Up() call itself has no context support, so ctx cannot interrupt schema application once it's actually running; in practice a bad network/DB fails during connection setup, before that point.

func CloseBackend

func CloseBackend(sqlDB *sqlx.DB, closeMongo func() error) error

CloseBackend closes whichever backend connection is actually active: sqlDB if non-nil (the SQL backend), otherwise closeMongo - the caller's own Mongo disconnect call, passed as a thunk so this package doesn't need to import the mongo driver just for this one alternative path.

func ProbeStatus

func ProbeStatus(ctx context.Context, cache *ProbeCache, ping func(context.Context) error) *apiv1_status.StatusProbe

ProbeStatus implements the shared caching health-probe pattern every db.Service.Status (apigw, verifier, ...) uses regardless of which backend (Mongo or SQL) is active: ping at most once per probeCacheTTL, returning the cached apiv1_status.StatusProbe otherwise. ping is the caller's own backend-specific ping call (e.g. *sqlx.DB.PingContext, or a closure over *mongo.Client.Ping) - this package deliberately doesn't import the mongo driver just to express "the other branch". The caller is expected to have already opened its own tracing span around ctx (this package can't import pkg/trace itself: pkg/model imports pkg/sqlstore, and pkg/trace imports pkg/model, so sqlstore -> trace would be a cycle).

Types

type Dialect

type Dialect interface {
	// Name identifies the dialect: "postgres" or "mariadb".
	Name() string
	// DriverName is the database/sql driver name to pass to sql.Open/sqlx.Open.
	DriverName() string
	// Rebind rewrites a query written with "?" placeholders into this
	// dialect's native placeholder style: a no-op for MariaDB, $1/$2/...
	// for Postgres.
	Rebind(query string) string
	// JSONColumnType is the column type used for JSON-valued columns:
	// "JSONB" for Postgres, "JSON" for MariaDB. The actual column types
	// live in the migration files; this is informational, for tests and
	// tooling that want to assert a dialect-appropriate schema.
	JSONColumnType() string
	// UpsertClause returns the dialect-native SQL fragment to append to an
	// "INSERT INTO table (...) VALUES (...)" statement to make it an
	// upsert-by-natural-key. conflictCols are the natural-key columns that
	// define the conflict target; updateCols are the columns to overwrite
	// when a conflict occurs. If updateCols is empty, the upsert becomes a
	// no-op-on-conflict insert (insert-if-absent).
	UpsertClause(conflictCols, updateCols []string) string
	// JSONContains returns a boolean SQL expression testing whether the JSON
	// value in column contains all the keys/values of the JSON document
	// bound to the next "?" placeholder (combine with Rebind as usual).
	JSONContains(column string) string
	// JSONTextExtract returns a SQL expression extracting the text value at
	// the given top-level key of a JSON column.
	JSONTextExtract(column, key string) string
	// CaseInsensitiveLike returns a boolean SQL expression doing a
	// case-insensitive substring match of column against the next "?"
	// placeholder. On MariaDB this relies on the column's collation being
	// case-insensitive (the default for this schema's text columns); it is
	// not otherwise enforced at the SQL level the way Postgres's ILIKE is.
	CaseInsensitiveLike(column string) string
}

Dialect captures the handful of things that genuinely differ between the two supported relational backends. Every repository method is written once, using a Dialect to produce backend-correct SQL text, so Postgres and MariaDB share one code path instead of two.

var MariaDBDialect Dialect = mariaDBDialect{}

MariaDBDialect is the Dialect implementation for MariaDB/MySQL.

var PostgresDialect Dialect = postgresDialect{}

PostgresDialect is the Dialect implementation for PostgreSQL.

func Connect

func Connect(ctx context.Context, cfg *SQL) (*sqlx.DB, Dialect, error)

Connect opens a connection pool for the backend selected by cfg.Backend and returns it alongside the matching Dialect. Pings the database before returning so connection/auth failures surface at startup rather than on first query.

func ConnectAndApplySchema

func ConnectAndApplySchema(ctx context.Context, cfg *SQL) (*sqlx.DB, Dialect, error)

ConnectAndApplySchema connects (via Connect) and immediately runs schema migrations (via ApplySchema) against the new connection pool, closing that pool if migrations fail. Connect itself already closes on a ping failure; this closes the other failure path a bare Connect+ApplySchema call pair would otherwise leak the pool on, so callers get one call that's clean on every failure path instead of needing to remember cleanup themselves.

func ForName

func ForName(name string) (Dialect, error)

ForName returns the Dialect for the given backend name ("postgres" or "mariadb"), as used by Common.SQL.Backend.

type JSON

type JSON[T any] struct {
	V T
}

JSON is a generic sql.Scanner/driver.Valuer wrapper for marshaling a Go value to/from a JSON/JSONB column. Neither database/sql nor sqlx do this automatically for struct fields, so repository row structs use this to mark which fields need JSON (de)serialization, e.g.:

type row struct {
    UUID       string                       `db:"uuid"`
    Parameters JSON[SomeParametersType]      `db:"params"`
}

func (*JSON[T]) Scan

func (j *JSON[T]) Scan(src any) error

Scan implements sql.Scanner.

func (JSON[T]) Value

func (j JSON[T]) Value() (driver.Value, error)

Value implements driver.Valuer.

type MariaDBConfig

type MariaDBConfig struct {
	// Host is the MariaDB server hostname. Required when Common.SQL.Backend
	// is "mariadb"; enforced by a SQL-level struct validation rather than a
	// plain "required_if" tag here, since "Backend" lives on the parent SQL
	// struct, not on MariaDBConfig, and required_if can only reference
	// sibling fields.
	Host string `yaml:"host" validate:"omitempty" doc_example:"\"mariadb\""`
	// Port is the MariaDB server port
	Port int `yaml:"port" default:"3306"`
	// User is the MariaDB connection user. Required when Common.SQL.Backend
	// is "mariadb" (see Host doc comment for why this isn't a required_if tag).
	User string `yaml:"user" validate:"omitempty"`
	// Password is the MariaDB connection password. May also be set via secrets.yaml
	// (Common.SQL.MariaDB.Password), following the same split as Mongo.URI.
	Password string `yaml:"password,omitempty"`
	// Database is the MariaDB database name
	Database string `yaml:"database" default:"vc" doc_example:"\"vc\""`
	// TLS enables TLS for the MariaDB connection.
	TLS bool `yaml:"tls" default:"false"`
	// CAFilePath is the path to a PEM-encoded CA certificate used to verify the server's certificate.
	CAFilePath string `yaml:"ca_file_path,omitempty"`
	// CertFilePath is the path to a PEM-encoded client certificate for mutual TLS (mTLS).
	CertFilePath string `yaml:"cert_file_path,omitempty" validate:"required_with=KeyFilePath"`
	// KeyFilePath is the path to a PEM-encoded client private key for mutual TLS (mTLS).
	KeyFilePath string `yaml:"key_file_path,omitempty" validate:"required_with=CertFilePath"`
	// MaxOpenConns is the maximum number of open connections to the database.
	MaxOpenConns int `yaml:"max_open_conns" default:"25"`
	// MaxIdleConns is the maximum number of idle connections in the pool.
	MaxIdleConns int `yaml:"max_idle_conns" default:"5"`
}

MariaDBConfig holds MariaDB/MySQL connection settings. Kept as a separate struct from PostgresConfig (rather than shared) since default port and TLS parameter semantics differ enough between the two drivers to want independent validation tags.

func (*MariaDBConfig) DSN

func (m *MariaDBConfig) DSN() (string, error)

DSN returns a go-sql-driver/mysql connection string for this MariaDB configuration, for the application's own long-lived connection pool. When CA/client certificate paths are set, this also registers a named TLS config with the mysql driver (mysql.RegisterTLSConfig) and references it in the returned DSN; the caller does not need to register anything itself.

Does not enable MultiStatements: see MigrationDSN for why that's kept to a separate, migration-only connection.

func (*MariaDBConfig) MigrationDSN

func (m *MariaDBConfig) MigrationDSN() (string, error)

MigrationDSN returns a connection string identical to DSN, except with MultiStatements enabled. Schema migration files contain more than one SQL statement per file (e.g. a CREATE TABLE followed by CREATE INDEX statements), which requires the go-sql-driver/mysql driver's MultiStatements option to execute more than one statement per Exec call.

Kept as a separate DSN/connection from the application's own pool (DSN above) rather than enabling MultiStatements there too: MySQL's wire protocol requires an explicit opt-in (CLIENT_MULTI_STATEMENTS) for multiple semicolon-separated statements in one query, and every ordinary request-path query in this codebase is a single statement built via sqlx with bound parameters -- it never needs this capability. Enabling it repo-wide on the shared pool would only widen the blast radius of any future SQL-injection bug (stacked queries) for no benefit, so it's scoped to sqlstore.ApplySchema's dedicated migration connection instead.

type PostgresConfig

type PostgresConfig struct {
	// Host is the Postgres server hostname. Required when Common.SQL.Backend
	// is "postgres"; enforced by a SQL-level struct validation rather than a
	// plain "required_if" tag here, since "Backend" lives on the parent SQL
	// struct, not on PostgresConfig, and required_if can only reference
	// sibling fields.
	Host string `yaml:"host" validate:"omitempty" doc_example:"\"postgres\""`
	// Port is the Postgres server port
	Port int `yaml:"port" default:"5432"`
	// User is the Postgres connection user. Required when Common.SQL.Backend
	// is "postgres" (see Host doc comment for why this isn't a required_if tag).
	User string `yaml:"user" validate:"omitempty"`
	// Password is the Postgres connection password. May also be set via secrets.yaml
	// (Common.SQL.Postgres.Password), following the same split as Mongo.URI.
	Password string `yaml:"password,omitempty"`
	// Database is the Postgres database name
	Database string `yaml:"database" default:"vc" doc_example:"\"vc\""`
	// SSLMode is the Postgres SSL mode: disable, require, verify-ca, or verify-full
	SSLMode string `yaml:"ssl_mode" default:"disable" validate:"omitempty,oneof=disable require verify-ca verify-full"`
	// CAFilePath is the path to a PEM-encoded CA certificate used to verify the server's certificate.
	CAFilePath string `yaml:"ca_file_path,omitempty"`
	// CertFilePath is the path to a PEM-encoded client certificate for mutual TLS (mTLS).
	CertFilePath string `yaml:"cert_file_path,omitempty" validate:"required_with=KeyFilePath"`
	// KeyFilePath is the path to a PEM-encoded client private key for mutual TLS (mTLS).
	KeyFilePath string `yaml:"key_file_path,omitempty" validate:"required_with=CertFilePath"`
	// MaxOpenConns is the maximum number of open connections to the database.
	MaxOpenConns int `yaml:"max_open_conns" default:"25"`
	// MaxIdleConns is the maximum number of idle connections in the pool.
	MaxIdleConns int `yaml:"max_idle_conns" default:"5"`
}

PostgresConfig holds PostgreSQL connection settings.

func (*PostgresConfig) DSN

func (p *PostgresConfig) DSN() string

DSN returns a libpq keyword/value connection string for this Postgres configuration, understood directly by pgx (both sslmode and the sslrootcert/sslcert/sslkey file-path parameters are native libpq connection parameters, so no separate *tls.Config needs to be built here).

type ProbeCache

type ProbeCache struct {
	// contains filtered or unexported fields
}

ProbeCache guards a cached apiv1_status.StatusProbe against concurrent Status() calls - a health-check/readiness endpoint is commonly hit by multiple requests in flight at once, and the plain read-then-write this replaces was a genuine data race under that load. Only one goroutine at a time ever runs ping (see ProbeStatus): a call arriving while the cache is stale but another goroutine's ping is already in flight simply waits for the lock, then re-checks NextCheck and returns that fresh result rather than pinging again.

type SQL

type SQL struct {
	// Backend selects the storage backend for services that support relational
	// storage. "mongo" (default, current behavior) keeps existing Mongo-backed
	// behavior unchanged; "postgres" and "mariadb" select the corresponding
	// relational backend.
	Backend string `yaml:"backend" default:"mongo" validate:"omitempty,oneof=mongo postgres mariadb"`
	// Postgres holds Postgres-specific connection settings, used when Backend is "postgres".
	Postgres *PostgresConfig `yaml:"postgres,omitempty" validate:"required_if=Backend postgres"`
	// MariaDB holds MariaDB/MySQL-specific connection settings, used when Backend is "mariadb".
	MariaDB *MariaDBConfig `yaml:"mariadb,omitempty" validate:"required_if=Backend mariadb"`
}

SQL holds relational database configuration, used by services that support a relational storage backend as an alternative to MongoDB. Backend selection is config-time only: a running service uses exactly one backend for its whole lifetime.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL