README
¶
Database Connections
The dbconn package provides MySQL database connection management and locking utilities for Spirit. It wraps database/sql with Spirit-specific concerns: retry logic, TLS auto-configuration, advisory locking, table locking, and the ability to kill blocking transactions.
Connection Setup
When creating a new connection, Spirit appends standardized DSN parameters to ensure consistent behavior across all connections. These include setting sql_mode="" (to be able to copy legacy data like 0000-00-00), time_zone=+00:00, transaction_isolation=read-committed, charset=utf8mb4, collation=utf8mb4_bin, and rejectReadOnly=true (for Aurora failover resilience). This means that regardless of the server's global configuration, Spirit connections behave predictably.
Pool sizing
Use SetPoolSize(db, n) rather than db.SetMaxOpenConns(n) directly. It sets the open and idle limits together, and they must stay together: database/sql closes a connection returned to a pool whose free list already holds MaxIdleConns entries, so an idle limit below the open limit turns every release past that point into a close and every subsequent acquire into a fresh dial, TLS handshake and MySQL auth. The copy phase — hundreds of read and write workers cycling connections continuously — is exactly that workload, and the churn is invisible on the status block, which does not report pool internals at all.
Holding the connections idle instead costs nothing that was not already reserved: the open limit is the budget, and matching the idle limit to it only stops the pool from discarding what it is entitled to keep. Several call sites ratchet a pool's size after it was created (the migration runner once thread counts are final, the checksum, cutover), which is why this lives in a helper rather than at construction only.
Connections are still recycled on maxConnLifetime (3 minutes), which pool sizing does not affect. With a large pool in steady use, that lifetime — not the idle limit — is the dominant source of reconnects.
TLS
Spirit supports five TLS modes: DISABLED, PREFERRED, REQUIRED, VERIFY_CA, and VERIFY_IDENTITY. The default is PREFERRED, which first attempts a TLS connection and falls back to plaintext if it fails. RDS hosts are auto-detected via hostname pattern matching (*.rds.amazonaws.com), and an embedded RDS CA bundle is used automatically.
Retryable Transactions
RetryableTransaction is the primary mechanism for executing statements that may encounter transient errors. It classifies MySQL errors into retryable (deadlocks, lock wait timeouts, connection loss, read-only mode, killed queries) and fatal (everything else). On transient errors, the entire transaction is retried up to MaxRetries times.
An important subtlety is that RetryableTransaction inspects SHOW WARNINGS after every statement. This catches issues that MySQL does not surface as errors, such as range_optimizer_max_mem_size exceeded warnings. This particular warning is treated as fatal because it indicates a table scan will occur instead of an index range scan.
Force Kill
Both ForceExec and NewTableLock kill blocking transactions after a delay. They wait for DBConfig.ForceKillAfter (zero defaults to 90% of LockWaitTimeout), then query performance_schema to identify and kill transactions that are blocking metadata lock acquisition. ForceExec always runs its kill worker; for NewTableLock the kill timer is gated on DBConfig.ForceKill (default true), which programmatic callers such as datasync's read-only source disable for connections that must never kill. LOCK TABLES returns as soon as it holds its locks, so NewTableLock's timer only ever fires while the statement is waiting. A ForceExec statement can keep running after it holds its locks, as a table rebuild does, and the sessions holding locks on the table beside it are then concurrent traffic, not blockers. So while its statement runs, ForceExec checks performance_schema.metadata_locks every 100ms, and at the moment the delay would be reached, and kills only after the statement has been waiting for a table metadata lock for the delay. That also covers a statement that starts waiting part-way through, such as a rebuild upgrading its lock to finish: its blockers get the full delay from when the wait started. The checks run over connections from the same pool as the statement, so the pool must be able to supply a second connection. Each check is bounded to one second. A check that fails, including one that cannot get a connection, kills nothing. A single failed check leaves a wait already seen in progress, so one slow check cannot push the kill past the lock wait timeout. A second failure in a row restarts the wait, because over a longer stretch the statement could have got its lock and started a new wait.
Each blocker is killed with KILL. When KILL is denied (error 1095: the user lacks CONNECTION_ADMIN and SUPER), the kill calls mysql.rds_kill instead, which RDS and Aurora MySQL provide and which needs only EXECUTE on the procedure. On Aurora MySQL 3, rds_superuser_role often lacks CONNECTION_ADMIN, and a blue/green switchover can remove it. The choice is made on every kill, so a privilege change during a run takes effect at the next one. If the procedure is missing or not executable, the error names both failures, and each failed kill is logged with its session id. CheckForceKillPrivileges accepts either way to kill: CONNECTION_ADMIN or SUPER, or EXECUTE on an existing mysql.rds_kill, as SHOW GRANTS lists them.
A kill that cannot list the blocking sessions kills nothing, so both ForceExec and NewTableLock look again every 100ms for as long as the statement still waits. The lookup reads information_schema.innodb_trx for each transaction's weight, and MySQL 9.7 fails every read of that table with error 3854 while any transaction is running a statement whose text holds a 4-byte character, such as an emoji. Such a statement usually ends well within the rest of the lock wait, and the next lookup then finds and kills the blocker.
ForceExec retries a statement that hits a lock wait timeout after its kill worker saw it waiting for the delay and killed, up to DBConfig.MaxRetries attempts in total (the same budget cutover uses). Every attempt runs a fresh kill worker, so a blocker that rolls back slowly or a new blocker that arrives between attempts is killed as well, instead of the retry timing out and the migration falling into a table copy. Between attempts it waits (bounded by 30 seconds) for the killed sessions to leave performance_schema.threads, because KILL is asynchronous. Errors from the kill and cleanup steps are logged but never joined into the statement's error: callers inspect that error to detect ambiguous DDL. Any error other than a lock wait timeout, or a timeout on which no kill ran, is returned immediately. When no kill ran because a check failed, ForceExec logs that at the point of return, so the skipped retries are not mistaken for an exhausted budget. A timeout whose kill left a blocker alive that no kill ends is also returned immediately: an explicit table lock, since the kill never ends a LOCK TABLES session; a transaction too heavy to roll back safely, which the kill skips; or a session the user may not kill, because it lacks CONNECTION_ADMIN or SUPER and cannot execute mysql.rds_kill. Another attempt succeeds only if that blocker happens to finish in time, and until then it holds up the table for another lock wait timeout. A kill that found nothing to end still retries, since its blocker may have finished on its own or been replaced by one the next attempt's kill can end.
ForceExec reserves a sql.Conn for the connection ID lookup, DDL, kill-worker join, and retries. Cancellation cannot return an idle session to the pool while its kill worker still runs. DDL is not wrapped in a transaction; MySQL implicitly commits ALTER TABLE. Use sql.DB.BeginTx when a real transaction is needed.
There are two important safety constraints:
- Transaction weight threshold: Transactions with a weight above 1,000,000 (as reported by
information_schema.innodb_trx.trx_weight) are never killed, because their rollback would be expensive and disruptive. - Explicit table locks: Connections holding
LOCK TABLESare never killed. Instead, anErrTableLockFounderror is returned. This is because killing non-transactional locks is unsafe.
Metadata Lock
AdvisoryLock provides an advisory locking mechanism using MySQL's GET_LOCK() function. It runs on a dedicated single-connection database pool with a background goroutine that periodically refreshes the lock. If the connection drops, it automatically reconnects and re-acquires locks.
Lock names are deterministic hashes of schema.table, truncated with a SHA1 suffix to fit MySQL's 64-character limit for lock names. This is used to prevent concurrent Spirit migrations on the same table.
Table Lock
TableLock wraps MySQL's LOCK TABLES ... WRITE statement. It integrates with the force-kill mechanism to automatically kill blocking transactions if the lock cannot be acquired within the timeout. This is used during checksum setup and cutover.
Table locks are session-scoped, so the helper reserves a dedicated sql.Conn until Close. A transaction cannot provide that ownership: cancellation can automatically roll it back and return its connection to the pool while table locks remain held. Callers must defer Close after successful acquisition. It ignores the caller's cancellation and starts its own 30-second cleanup timeout when invoked. A successful unlock returns the connection to the pool; a failed unlock or failed acquisition discards the connection. This does not depend on driver session-reset support.
Discarding prevents reuse of the session, but does not guarantee that server-side locks have been released when Close returns. If a statement was interrupted, MySQL may retain the session and its locks until that statement detects the disconnected client or finishes.
Transaction Pool
TrxPool pre-creates a pool of REPEATABLE READ transactions with START TRANSACTION WITH CONSISTENT SNAPSHOT. This ensures all worker threads see the same point-in-time data, which is essential for parallel checksum verification.
See Also
- pkg/dbconn/sqlescape - Client-side SQL escaping
- pkg/checksum - Uses
TrxPoolfor consistent parallel checksumming - pkg/migration - Uses force-kill and metadata locks during schema changes
Documentation
¶
Overview ¶
Package dbconn contains a series of database-related utility functions.
Index ¶
- Constants
- Variables
- func AnalyzeTable(ctx context.Context, db *sql.DB, logger *slog.Logger, ...) error
- func BufferPoolSize(ctx context.Context, db *sql.DB) (uint64, error)
- func CheckForceKillPrivileges(ctx context.Context, db *sql.DB) error
- func EnhanceDSNWithTLS(inputDSN string, config *DBConfig) (string, error)
- func Exec(ctx context.Context, db *sql.DB, stmt string, args ...any) error
- func ForceExec(ctx context.Context, db *sql.DB, tables []*table.TableInfo, dbConfig *DBConfig, ...) error
- func GetTLSConfigForBinlog(config *DBConfig, host string) (*tls.Config, error)
- func IsConnectionLossError(err error) bool
- func IsLockContentionError(err error) bool
- func IsOutcomeUnknown(err error) bool
- func IsRDSHost(host string) bool
- func KillLockingTransactions(ctx context.Context, db *sql.DB, tables []*table.TableInfo, ...) error
- func KillSessionAndWait(ctx context.Context, db *sql.DB, pid int) error
- func KillTransaction(ctx context.Context, db *sql.DB, pid int) error
- func LoadCertificateFromFile(filePath string) ([]byte, error)
- func New(inputDSN string, config *DBConfig) (db *sql.DB, err error)
- func NewContext(ctx context.Context, inputDSN string, config *DBConfig) (db *sql.DB, err error)
- func NewCustomTLSConfig(certData []byte, sslMode string) *tls.Config
- func NewTLSConfig() *tls.Config
- func NewWithConnectionType(inputDSN string, config *DBConfig, connectionType string) (db *sql.DB, err error)
- func ReadBoundsForPool(start, ceiling, maxConnections, reserve, holds int) (int, int)
- func RedactDSN(dsn string) string
- func RequireDifferentDatabase(ctx context.Context, source, target *sql.DB) error
- func RequireDifferentDatabases(ctx context.Context, sources, targets []*sql.DB) error
- func RetryableTransaction(ctx context.Context, db *sql.DB, dupKeyHandling DupKeyHandling, ...) (int64, error)
- func SetPoolSize(db *sql.DB, n int)
- func SplitDSNs(dsnList string) []string
- func ValidateConnectionLimit(limit int) error
- func ValidateMaxConnections(maxConnections, readers, reserve int) error
- func WithMultiTableSchemaLock(schemaName string) func(*AdvisoryLock)
- type AdvisoryLock
- type DBConfig
- type DupKeyHandling
- type LockDetail
- type RowQuerier
- type TableLock
- type TrxPool
- type UnsafeWarningError
Constants ¶
const ( // TableLockQuery is used to find tables that are locked by a LOCK TABLES command. // It's not really possible to find out how long the lock has been held, so we don't consider // the length of the lock here. TableLockQuery = `` /* 440-byte string literal not displayed */ LongRunningEventQuery = `` /* 751-byte string literal not displayed */ )
const DefaultMaxConnections = 128
DefaultMaxConnections is the main pool default for migrations, moves and continuous syncs. Monitor and advisory-lock connections use separate dedicated pools.
const DriverName = "block-mysql"
DriverName is the database/sql driver spirit opens every connection with.
It is exported because it is part of this package's contract, not an implementation detail: EnhanceDSNWithTLS returns a DSN whose tls= name refers to an entry this package registered, and a driver's TLS registry is a package-level global. A consumer that opens such a DSN with a different driver gets "invalid value / unknown config name: rds" at connect time — an error that says nothing about drivers. Open it with sql.Open( dbconn.DriverName, ...) and that stays correct through any future move.
const MinMigrationPoolSize = 5
MinMigrationPoolSize is the smallest --max-connections a migration can complete on. It covers the connections that cutover cannot serialize.
The cutover sets the number: it needs the LOCK TABLES connection, the RENAME TABLE connection and the flush threads, and unlike every other phase it cannot trade connections for time — below the minimum it does not run slower, it cannot run. See migration's CutOver.Run, which holds the same number and will raise a pool that arrives under it.
Nothing else gets a say. The copy, the checksum and the drain all queue on a small pool and finish eventually, so a low --max-connections is the operator asking for a slow migration, which is theirs to ask for.
Variables ¶
var ( // TransactionWeightThreshold is the maximum information_schema.innodb_trx.trx_weight // over which we consider a transaction too big to be safely killed. Rolling back a // heavy transaction can cause a huge impact on the database. TransactionWeightThreshold int64 = 1_000_000 ErrTableLockFound = errors.New("explicit table lock found! spirit cannot proceed") )
var ErrForceKillPrivilegeMissing = errors.New("missing a privilege force-kill needs")
ErrForceKillPrivilegeMissing matches an error from CheckForceKillPrivileges that names a privilege the user lacks.
var ErrLockHeld = errors.New("lock is held by another connection")
ErrLockHeld reports that another connection holds an advisory lock this one needs: another migration is already running against the table, or the schema for an atomic multi-table migration. It is not a failure of the migration that was refused, which can run once the holder releases the lock, so callers can tell it apart from one with errors.Is.
var ErrStatementOutcomeUnknown = errors.New("statement outcome unknown")
ErrStatementOutcomeUnknown marks a statement that ExecUnderLock sent to the server but stopped waiting for. The server may have committed it: the caller must check the server state before it acts on the failure.
Functions ¶
func AnalyzeTable ¶ added in v0.18.0
func AnalyzeTable(ctx context.Context, db *sql.DB, logger *slog.Logger, schemaName, tableName string) error
AnalyzeTable runs ANALYZE TABLE for schemaName.tableName on db. An empty schemaName leaves the table unqualified, so it resolves against db's default database (required through a Vitess vtgate, where qualifying is wrong). It reads the result set rather than using Exec because ANALYZE reports a failure such as a missing table as a Msg_type="Error" row (not a statement error), which would otherwise be a silent no-op.
ANALYZE also reports a lock wait timeout as an Error row, with no error code to tell it apart from a permanent failure. A run calls this after its whole copy, so a transient lock should not end it: an Error row is retried up to analyzeAttempts times, and only the last one is returned. A statement error is returned at once.
func BufferPoolSize ¶ added in v0.18.0
BufferPoolSize returns the server's @@innodb_buffer_pool_size in bytes.
func CheckForceKillPrivileges ¶ added in v0.16.0
CheckForceKillPrivileges verifies that the connection's user holds every privilege force-kill needs: SELECT on the performance_schema tables its queries read (see GetTableLocks and getLockingTransactions), PROCESS to read information_schema.innodb_trx, and CONNECTION_ADMIN or SUPER, or EXECUTE on mysql.rds_kill, to kill another user's session (see checkKillPrivilege). It returns an error naming each one that is missing.
A privilege the user lacks matches ErrForceKillPrivilegeMissing. A read that fails for another reason, such as a lost connection, does not, so a caller can tell a missing grant from a check it could not run.
It is intended for preflight privilege checks. It reads SHOW GRANTS, one row of information_schema.innodb_metrics, no rows of the performance_schema lock tables, and, when the grants allow executing mysql.rds_kill, one row of information_schema.ROUTINES. It logs nothing, so unlike GetTableLocks / getLockingTransactions it neither scans server-wide locks nor emits "found locking transaction" log lines.
func EnhanceDSNWithTLS ¶
EnhanceDSNWithTLS enhances a DSN with TLS settings from the provided config if the DSN doesn't already contain TLS parameters. This allows replica connections to inherit TLS settings from the main connection while still respecting explicit TLS configuration in the DSN.
The returned DSN names a TLS configuration registered in DriverName's registry. Open it with that driver: a caller that hands the result to a different one fails at connect time with "invalid value / unknown config name", because TLS registries are per-driver package globals rather than anything the DSN carries.
This settles TLS and nothing else. The settings a spirit connection also needs — interpolation, cleartext-password gating, integer tinyint(1) — are applied by newDSN, so they reach a connection only via New. Spirit's own replica path passes this result to NewWithConnectionType for exactly that reason; a consumer that opens the result directly owns the rest itself.
func Exec ¶
Exec is like db.Exec but only returns an error. This makes it a little bit easier to use in error handling. It accepts args which are escaped client side using the sqlescape library. i.e. %n is an identifier, %? is automatic type conversion on a variable, and %r splices a sqlescape.RawSQL argument in verbatim (for raw user SQL such as an ALTER clause, which must never be concatenated into the format string).
func ForceExec ¶
func ForceExec(ctx context.Context, db *sql.DB, tables []*table.TableInfo, dbConfig *DBConfig, logger *slog.Logger, stmt string, args ...any) error
ForceExec is like Exec but it has some added logic to force kill any connections that are holding up metadata locks preventing this from succeeding. It kills only while the statement is waiting for a table metadata lock, never while it holds its locks and runs. The statement holds one connection from db while the checks and the kill run over others, so db must be able to supply a second connection: a check that cannot get one in time fails, and a failed check kills nothing. Like Exec, stmt is a sqlescape format string: embed raw user SQL (such as an ALTER clause) with the %r verb and a sqlescape.RawSQL argument, never by concatenating it into stmt.
func GetTLSConfigForBinlog ¶
GetTLSConfigForBinlog creates a TLS config for binary log connections using the same logic as main database connections
func IsConnectionLossError ¶ added in v0.15.0
IsConnectionLossError reports whether err indicates that the connection to MySQL failed or was lost, meaning the client cannot know whether the last statement it sent was executed by the server. Connection-level failures never surface as a *mysql.MySQLError: go-sql-driver returns driver.ErrBadConn when the failure was detected before anything was written, and mysql.ErrInvalidConn when the connection died mid-statement — possibly *after* the server executed the statement but before the client read the result. Raw io.EOF is included for paths that surface the TCP-level error directly, and the client-library codes CR_CONN_HOST_ERROR (2003) / CR_SERVER_LOST (2013) are included because proxies (e.g. ProxySQL, RDS Proxy) can relay them inside real server error packets.
In contrast to deterministic SQL errors (lock wait timeout, deadlock, ...), where the server has positively reported that the statement did NOT take effect, these errors are ambiguous. Callers retrying a non-idempotent statement (e.g. the cutover RENAME TABLE) must verify server-side state before deciding whether the statement was applied. The exception is ER_CLIENT_INTERACTION_TIMEOUT (4031): the server killed the session for inactivity *before* the observing statement arrived, so that statement positively did not execute — verification is still safe, just guaranteed to conclude "not applied".
func IsLockContentionError ¶ added in v0.17.0
IsLockContentionError reports whether err is InnoDB lock contention: a lock wait timeout (1205) or a deadlock (1213). Both are already covered by canRetryError, but callers that can *adapt* — by backing off harder or by lowering their own write concurrency — need to tell contention apart from the other retryable classes, which no amount of self-throttling would fix.
This distinction matters because contention can be self-inflicted. Spirit runs on READ COMMITTED (see conn.go), so concurrent REPLACE batches with disjoint primary keys never gap-conflict on the clustered index. They do still take next-key locks during duplicate-key handling on every *secondary* index, where "disjoint by PK" buys nothing — so a wide enough flush fan-out deadlocks against itself with no external workload at all.
func IsOutcomeUnknown ¶ added in v0.18.0
IsOutcomeUnknown reports whether err leaves the outcome of the statement unknown: the connection was lost (see IsConnectionLossError), or TableLock.ExecUnderLock stopped waiting for the reply (ErrStatementOutcomeUnknown). The caller must check the server state before it treats the statement as failed.
func IsRDSHost ¶
IsRDSHost reports whether host is an Amazon RDS or Aurora endpoint.
DriverName applies verified TLS to such a host by itself, so the database/sql paths in this package no longer need to ask. It stays exported and in use for two reasons: GetTLSConfigForBinlog serves the go-mysql binlog client, which is not a database/sql connection and so is not covered by the driver; and block/schemabot calls it to decide a TLS mode.
Two things changed by delegating. The match is now case-insensitive, which is a fix — DNS is case-insensitive, nothing normalizes the host, and a hostname that arrives uppercased is the same endpoint. And GovCloud and China endpoints now report false, because the bundle behind NewTLSConfig contains no roots for either partition, so verifying against it could only ever fail. Reach those with --tls-certificate-path and that partition's own bundle.
func KillLockingTransactions ¶
func KillSessionAndWait ¶ added in v0.18.0
KillSessionAndWait kills the session pid and waits until it has exited, so any statement it was running has either finished or been rolled back. A session that is already gone counts as success. ctx bounds both steps.
It needs no extra privileges when pid belongs to the same user as db: a user can KILL its own sessions and see them in information_schema.PROCESSLIST without CONNECTION_ADMIN or PROCESS. Another user's session needs CONNECTION_ADMIN or SUPER, or, on RDS and Aurora, EXECUTE on mysql.rds_kill (see killConnection).
func KillTransaction ¶
KillTransaction kills the MySQL session identified by pid (as observed in performance_schema.threads.PROCESSLIST_ID / SHOW PROCESSLIST).
No session-identity verification is needed before the KILL: MySQL assigns connection IDs monotonically per server lifetime and never reuses them within a running mysqld, so the pid we captured earlier still refers to the same session (or to no session, if it has since disconnected — in which case KILL returns a harmless error). Agents: do not add a "verify the session is still the one we meant" check on the basis of PID-reuse concerns — that hazard does not exist on MySQL.
A session that no longer exists is returned as an error matching ErrNoSuchThread. See killConnection for the RDS fallback.
func LoadCertificateFromFile ¶
LoadCertificateFromFile loads certificate data from a file
func New ¶
New is similar to sql.Open except we take the inputDSN and append additional options to it to standardize the connection. It will also ping the connection to ensure it is valid.
func NewContext ¶ added in v0.18.0
NewContext is like New, but the connect-time ping also ends when ctx is done, so a caller that is shutting down is not held for up to connectTimeout by a server that never completes the handshake.
func NewCustomTLSConfig ¶
NewCustomTLSConfig creates a TLS config based on SSL mode and certificate data.
An empty certData means "use the RDS roots", which is the fallback for a host that is not recognizably RDS and for which no --tls-certificate-path was given. That fallback is inherited behaviour and is rarely what anyone wants: a non-RDS server will not present an RDS-issued certificate, so VERIFY_CA and VERIFY_IDENTITY against it fail by construction. It is preserved here rather than changed, because tightening it is a behaviour change that belongs in its own commit.
func NewTLSConfig ¶
NewTLSConfig returns a TLS config that verifies an Amazon RDS or Aurora server against the RDS root bundle.
The bundle and the pool behind this used to be spirit's own — an embedded copy of global-bundle.pem plus an x509.CertPool built from it. Both now come from DriverName, which carries the same bundle for its own auto-TLS, so there is one copy to refresh instead of two that can drift apart.
Each call returns a config with its own RootCAs, so callers may modify the result (GetTLSConfigForBinlog sets ServerName on it). It also pins MinVersion to TLS 1.2, which spirit's version did not.
func NewWithConnectionType ¶
func NewWithConnectionType(inputDSN string, config *DBConfig, connectionType string) (db *sql.DB, err error)
NewWithConnectionType is like New but includes context about the connection type for better error messages
func ReadBoundsForPool ¶ added in v0.17.0
ReadBoundsForPool fits BOTH bounds of a reader pool without growing the connection pool. Checksums pre-open a snapshot transaction per ceiling slot: fitting just the ceiling fails because consumers floor it back to the start (see checksum.NewChecker and the copier's resolveReadCeiling). Snapshot creation under one table lock is in SingleChecker.initConnPool. The caller supplies its lifecycle-specific reserve. holds is how many connections one reader takes from the busiest pool at once: one for a migration, more for a move whose source and target share a handle. Values below one mean one. Unresolved connection limits pass through; a small budget retains one reader so it can progress.
func RedactDSN ¶ added in v0.16.0
RedactDSN returns dsn with the password masked, safe for logging. It keeps the username, host and parameters so logs stay useful, masking only the password and only when one was actually present. If the driver can't parse the DSN it still never echoes a password: it redacts the credentials section before '@', or — lacking '@' — masks from the first ':' (a malformed "user:password" still has its password hidden).
func RequireDifferentDatabase ¶ added in v0.17.0
RequireDifferentDatabase refuses a copy whose target aliases its source. Connection strings are insufficient: different users or hostnames can reach the same database. Check the selected schemas and, when they overlap, the server UUIDs before any target writes or destructive recovery.
func RequireDifferentDatabases ¶ added in v0.17.0
RequireDifferentDatabases refuses any source/target database alias. Each connection's identity is read at most once, which keeps sharded moves from issuing the same identity queries for every source/target pair.
func RetryableTransaction ¶
func RetryableTransaction(ctx context.Context, db *sql.DB, dupKeyHandling DupKeyHandling, config *DBConfig, stmts ...string) (int64, error)
RetryableTransaction retries all statements in a transaction, retrying if a statement errors, or there is a deadlock. It will retry up to maxRetries times.
func SetPoolSize ¶ added in v0.16.0
SetPoolSize sets a pool's connection limit, keeping the idle limit equal to it. Both must move together: database/sql closes a connection returned to a pool whose free list already holds MaxIdleConns entries, so an idle limit below the open limit turns every release past that point into a close and every subsequent acquire into a fresh dial, TLS handshake and MySQL auth. The copy phase is exactly that workload — up to a few hundred write and read workers cycling connections continuously — and the churn is invisible in the status block, which does not report pool internals at all.
Holding the connections idle instead costs nothing the caller has not already reserved: SetMaxOpenConns is the budget, and this only stops the pool from throwing away what it is entitled to keep. Note that connections are still recycled on maxConnLifetime, which this does not change.
n <= 0 means unlimited to SetMaxOpenConns; pass it through unchanged (and leave the idle limit alone, since "unlimited idle" is not expressible) rather than silently reinterpreting it.
func SplitDSNs ¶ added in v0.16.0
SplitDSNs splits a comma-separated list of DSNs into a slice, trimming surrounding whitespace and dropping empty entries. An empty input returns nil. (Used for the replica DSN list, but takes no position on what the individual DSNs mean.)
func ValidateConnectionLimit ¶ added in v0.17.0
ValidateConnectionLimit rejects negative limits, which database/sql would otherwise interpret as unlimited. Zero means the runner should apply its default.
func ValidateMaxConnections ¶ added in v0.17.0
ValidateMaxConnections validates an explicit pool budget against pinned checksum readers and runner-specific headroom. Zero is unresolved and is accepted so callers can apply their defaults after validation.
func WithMultiTableSchemaLock ¶ added in v0.16.0
func WithMultiTableSchemaLock(schemaName string) func(*AdvisoryLock)
WithMultiTableSchemaLock adds a schema-scoped lock to the AdvisoryLock so that only one atomic multi-table migration runs per schema at a time. Multi-table migrations all coordinate through a single shared _spirit_checkpoint (and _spirit_sentinel), so they must not overlap; a second one fails to acquire this lock and aborts. Single-table migrations do not use it — they are serialized per-table by the table locks and may run concurrently.
Applied as an option so the lock name is prepended before the per-table names; it is held and released on the same dedicated session as the rest.
Types ¶
type AdvisoryLock ¶ added in v0.16.0
type AdvisoryLock struct {
// contains filtered or unexported fields
}
AdvisoryLock ensures that only one spirit migration operates on a table at a time. It is a user-level advisory lock (GET_LOCK) held on a dedicated connection, so it is invisible to application traffic and never blocks queries or DDL. It may be confused for a table lock, which it is not.
func NewAdvisoryLock ¶ added in v0.16.0
func (*AdvisoryLock) Close ¶ added in v0.16.0
func (m *AdvisoryLock) Close() error
type DBConfig ¶
type DBConfig struct {
ForceKillAfter time.Duration // Zero preserves the default: 90% of LockWaitTimeout.
LockWaitTimeout int
InnodbLockWaitTimeout int
MaxRetries int // Total attempts, not retries after the first: 1 means a single attempt.
MaxOpenConnections int
RangeOptimizerMaxMemSize int64
InterpolateParams bool
ForceKill bool // If true, kill locking transactions to acquire metadata locks (default: true)
// There is deliberately no RejectReadOnly here. It used to map to the
// driver's rejectReadOnly option and defaulted to true, guarding against
// landing on a demoted, now-read-only Aurora primary after a blue/green
// deploy or failover. [DriverName] now does that unconditionally and has
// removed the option, so there is nothing left to configure.
//
// The field also carried an opt-out, used by the sync runner for a
// read-only source. That turned out to be guarding against an error the
// workload cannot raise: 1290/1792/1836 are raised by *writes*, and a
// read-only source only reads. Verified against a super_read_only MySQL
// 8.0 — SELECT, SHOW TABLES, SHOW CREATE TABLE, SHOW MASTER STATUS, every
// session SET newDSN adds, and the binlog client's FLUSH BINARY LOGS all
// succeed; an INSERT is what returns 1290.
// TLS Configuration
TLSMode string // TLS connection mode (DISABLED, PREFERRED, REQUIRED, VERIFY_CA, VERIFY_IDENTITY)
TLSCertificatePath string // Path to custom TLS certificate file
}
func NewDBConfig ¶
func NewDBConfig() *DBConfig
func (*DBConfig) StatementCompletionTimeout ¶ added in v0.18.0
StatementCompletionTimeout is how long to wait for a statement whose outcome the caller must know, such as a cutover RENAME TABLE: the session's lock_wait_timeout plus lockedStatementCompletionMargin. The server reports a metadata lock wait as ER_LOCK_WAIT_TIMEOUT after lock_wait_timeout, and that error is conclusive, so the client-side bound must be longer. A statement still running when this bound expires has an unknown outcome.
func (*DBConfig) ValidateForceKillAfter ¶ added in v0.18.0
ValidateForceKillAfter rejects delays that cannot leave time for lock acquisition. LockWaitTimeout is in whole seconds, matching MySQL's session variable.
type DupKeyHandling ¶ added in v0.16.0
type DupKeyHandling int
DupKeyHandling selects how RetryableTransaction treats duplicate-key (1062) warnings. Copy / INSERT IGNORE paths legitimately expect dup-key warnings (e.g. resume re-inserts); checksum-fix DELETE/REPLACE/UPSERT paths do not and want them surfaced. Using a named int enum (rather than a bool) keeps call sites self-documenting and stops a bare positional bool (true/false) from compiling.
const ( // ErrorOnDupKey surfaces duplicate-key warnings as errors. ErrorOnDupKey DupKeyHandling = iota // IgnoreDupKeyWarnings tolerates duplicate-key warnings. IgnoreDupKeyWarnings )
type LockDetail ¶
type LockDetail struct {
PID int
User sql.NullString
Host sql.NullString
Info sql.NullString
ObjectType sql.NullString
ObjectSchema sql.NullString
ObjectName sql.NullString
LockType sql.NullString // e.g. "INTENTION_EXCLUSIVE", "SHARED_READ",
LockDuration sql.NullString // e.g. "STATEMENT", "TRANSACTION"
LockStatus sql.NullString
RunningTime sql.NullString // Human-readable format of the timer_wait
TimerWait sql.NullInt64 // in picoseconds
TrxWeight sql.NullInt64 // Rows modified by the transaction
}
type RowQuerier ¶ added in v0.18.0
type RowQuerier interface {
QueryRowContext(ctx context.Context, query string, args ...any) *sql.Row
}
RowQuerier is the part of *sql.DB (or *sql.Conn) that reads one row.
type TableLock ¶
type TableLock struct {
// contains filtered or unexported fields
}
func NewTableLock ¶
func NewTableLock(ctx context.Context, db *sql.DB, tables []*table.TableInfo, config *DBConfig, logger *slog.Logger) (*TableLock, error)
NewTableLock creates a new server wide lock on multiple tables. i.e. LOCK TABLES .. WRITE. It uses a short timeout and *does not retry*. The caller is expected to retry, which gives it a chance to first do things like catch up on replication apply before it does the next attempt.
config.ForceKill=true is the default, and will more or less ensure that the lock acquisition is successful by killing long-running queries that are blocking our lock acquisition after ForceKillAfter (by default, 90% of LockWaitTimeout). Programmatic callers that never take locks (e.g. datasync's read-only source) can disable it via DBConfig.ForceKill.
func (*TableLock) Close ¶
Close releases the table lock even if the caller's context has expired. The cleanup budget starts here, after all work under the lock has finished. A session whose unlock fails is discarded, never returned to the pool.
func (*TableLock) DB ¶ added in v0.15.0
DB returns the database connection pool this lock was acquired on. Because LOCK TABLES ... WRITE blocks writes from every other connection, any write to a locked table must go through this lock's own connection. Callers holding locks on multiple servers (e.g. one per shard) use this to match each lock to the target it belongs to.
func (*TableLock) ExecUnderLock ¶
ExecUnderLock executes statements on the locking session, in order.
It does not start a statement once ctx is done. A statement it has started runs to completion even if ctx is then cancelled: with a cancellable context, a cancel during the statement would close the connection and return context.Canceled, but the server can still commit the statement, so the caller could not tell whether it took effect (issue #1338). Each statement instead runs on a context detached from ctx's cancellation and bounded by DBConfig.StatementCompletionTimeout. If that bound expires, the error wraps ErrStatementOutcomeUnknown. The client closes the connection then, but the server may still be running the statement: a read of the server state that shows it did not take effect is not conclusive.
A caller that must run its statements even after a cancel, such as the rename that retires a source after a traffic switch, passes context.WithoutCancel(ctx).
type TrxPool ¶
func NewTrxPool ¶
func NewTrxPool(ctx context.Context, db *sql.DB, count int, config *DBConfig, logger *slog.Logger) (*TrxPool, error)
NewTrxPool creates a pool of transactions which have already had their read-view created in REPEATABLE READ isolation.
The pool is sized for the maximum concurrency the caller may ever scale up to, so some transactions can sit unused for hours. An idle transaction still counts against the server's wait_timeout, and managed configurations often set it low (e.g. 600s on Aurora): without intervention the server silently kills the connection and a later Get() hands out a dead transaction that fails with "driver: bad connection". To prevent that, the pool runs a background keepalive that periodically pings whatever transactions are idle in the pool; it stops when ctx is canceled or the pool is closed. A nil logger discards keepalive warnings.
type UnsafeWarningError ¶ added in v0.17.0
type UnsafeWarningError struct {
Warning *mysql.MySQLError
}
UnsafeWarningError reports a warning that MySQL raised on a statement Spirit executed without error, and that Spirit treats as fatal. Statements such as INSERT IGNORE succeed while discarding rows, so the warning is the only signal that the copy would silently lose data.
It unwraps to the underlying *mysql.MySQLError, so callers can classify the warning by its code with errors.As or errors.AsType rather than by matching on the message. The code matters because the same fatal branch covers unrelated conditions — a NOT NULL column with no default (1364), a duplicate on a unique key (1062), a value too long for its column (1406) — which a caller may want to report or act on differently.
func (*UnsafeWarningError) Error ¶ added in v0.17.0
func (e *UnsafeWarningError) Error() string
Error reports the warning and its code. The type is exported, so a caller can hold one without a warning; Error stays callable on that value because the places an error's text is read — logs, %v, a failing test — are the last places a panic is affordable.
func (*UnsafeWarningError) Unwrap ¶ added in v0.17.0
func (e *UnsafeWarningError) Unwrap() error
Unwrap returns the underlying warning, or nil when the error carries none. A nil return ends the chain, which is what errors.Is and errors.As expect.
Source Files
Directories