README
¶
Database
Overview
The Database component provides utilities for working with various database systems, including PostgreSQL, SQLite, and MongoDB. It offers connection management, health checks, transaction handling, and retry mechanisms for transient errors.
Features
- Multi-Database Support: Work with PostgreSQL, SQLite, and MongoDB using a consistent API
- Connection Management: Initialize and configure database connections with sensible defaults
- Health Checks: Verify database connectivity with built-in health check functions
- Transaction Handling: Execute operations within transactions with automatic rollback on errors
- Retry Mechanisms: Automatically retry operations that fail due to transient errors
- Error Handling: Structured error types for better error handling and reporting
Installation
go get github.com/abitofhelp/servicelib/db
Quick Start
See the Connection example for a complete, runnable example of how to use the database component.
API Documentation
Core Types
PostgresConfig
Configuration for a PostgreSQL connection pool.
type PostgresConfig struct {
// URI is the PostgreSQL connection URI
URI string
// Timeout is the timeout for connection operations
Timeout time.Duration
// MaxConns is the maximum number of open connections
MaxConns int32
// MinConns is the minimum number of open connections
MinConns int32
// MaxConnLifetime is the maximum lifetime of a connection
MaxConnLifetime time.Duration
// MaxConnIdleTime is the maximum idle time of a connection
MaxConnIdleTime time.Duration
// HealthCheckPeriod is the period between health checks
HealthCheckPeriod time.Duration
}
RetryConfig
Configuration for retry operations.
type RetryConfig struct {
// MaxRetries is the maximum number of retry attempts
MaxRetries int
// InitialBackoff is the initial backoff duration
InitialBackoff time.Duration
// MaxBackoff is the maximum backoff duration
MaxBackoff time.Duration
// BackoffFactor is the factor by which the backoff increases
BackoffFactor float64
// Logger is the logger to use for logging retries
Logger *logging.ContextLogger
}
Key Methods
InitPostgresPool
Initializes a PostgreSQL connection pool.
func InitPostgresPool(ctx context.Context, config PostgresConfig) (*pgxpool.Pool, error)
InitSQLiteDB
Initializes a SQLite database connection.
func InitSQLiteDB(ctx context.Context, uri string, timeout, connMaxLifetime time.Duration, maxOpenConns, maxIdleConns int) (*sql.DB, error)
InitMongoClient
Initializes a MongoDB client.
func InitMongoClient(ctx context.Context, uri string, timeout time.Duration) (*mongo.Client, error)
ExecutePostgresTransaction
Executes a function within a PostgreSQL transaction.
func ExecutePostgresTransaction(ctx context.Context, pool *pgxpool.Pool, fn func(tx pgx.Tx) error, retryConfig ...RetryConfig) error
ExecuteSQLTransaction
Executes a function within a SQL transaction.
func ExecuteSQLTransaction(ctx context.Context, db *sql.DB, fn func(tx *sql.Tx) error, retryConfig ...RetryConfig) error
CheckPostgresHealth / CheckSQLiteHealth / CheckMongoHealth
Check if a database connection is healthy.
func CheckPostgresHealth(ctx context.Context, pool *pgxpool.Pool) error
func CheckSQLiteHealth(ctx context.Context, db *sql.DB) error
func CheckMongoHealth(ctx context.Context, client *mongo.Client) error
Examples
For complete, runnable examples, see the following directories in the EXAMPLES directory:
- Connection - Shows how to connect to different databases
- Health Check - Shows how to perform health checks on database connections
- Query - Shows how to execute queries against databases
- Transaction - Shows how to use transactions with automatic retries
- Repository - Shows how to implement the repository pattern
- Retry - Shows how to use retry mechanisms for database operations
Best Practices
- Use Connection Pooling: Always use connection pooling for better performance and resource management
- Set Appropriate Timeouts: Configure timeouts based on expected operation durations
- Use Transactions: Wrap related operations in transactions to ensure data consistency
- Handle Transient Errors: Use retry mechanisms for operations that may fail due to transient errors
- Close Connections: Always close database connections when they are no longer needed
Troubleshooting
Common Issues
Connection Failures
If you're experiencing connection failures, check:
- Database server is running and accessible
- Connection string is correct
- Network connectivity between your application and the database
- Firewall rules allow the connection
Transaction Deadlocks
If you're experiencing transaction deadlocks:
- Ensure consistent order of resource access
- Keep transactions short and focused
- Consider using optimistic concurrency control
- Monitor and analyze deadlock logs
Related Components
- Retry - Used for retrying operations that fail due to transient errors
- Errors - Provides structured error types for database operations
- Logging - Used for logging database operations
- Telemetry - Used for tracing database operations
Contributing
Contributions to this component are welcome! Please see the Contributing Guide for more information.
License
This project is licensed under the MIT License - see the LICENSE file for details.
Documentation
¶
Overview ¶
Package db provides utilities for working with databases.
Package db provides utilities for working with various database systems.
This package offers a unified interface for connecting to, querying, and managing different types of databases, including PostgreSQL, MongoDB, and SQLite. It provides functions for initializing database connections, executing transactions, checking database health, and handling common database errors.
The package is designed to work with the context package for proper cancellation and timeout handling, the telemetry package for tracing database operations, and the retry package for handling transient errors.
Key features:
- Database initialization for PostgreSQL, MongoDB, and SQLite
- Transaction management with automatic retries for transient errors
- Health check functions for monitoring database connectivity
- Error handling with classification of transient vs. permanent errors
- Telemetry integration for tracing database operations
- Logging of database operations and errors
Example usage for PostgreSQL:
// Configure PostgreSQL connection
config := db.PostgresConfig{
ConnectionString: "postgres://user:password@localhost:5432/mydb",
MaxConnections: 10,
ConnectTimeout: 5 * time.Second,
MaxConnLifetime: 30 * time.Minute,
MaxConnIdleTime: 5 * time.Minute,
}
// Initialize PostgreSQL connection pool
pool, err := db.InitPostgresPool(ctx, config)
if err != nil {
log.Fatalf("Failed to connect to database: %v", err)
}
defer pool.Close()
// Execute a transaction
err = db.ExecutePostgresTransaction(ctx, pool, func(tx pgx.Tx) error {
// Perform database operations within the transaction
_, err := tx.Exec(ctx, "INSERT INTO users (name, email) VALUES ($1, $2)", "John", "john@example.com")
return err
})
Example usage for MongoDB:
// Initialize MongoDB client
client, err := db.InitMongoClient(ctx, "mongodb://localhost:27017", 5*time.Second)
if err != nil {
log.Fatalf("Failed to connect to MongoDB: %v", err)
}
defer client.Disconnect(ctx)
// Check MongoDB health
if err := db.CheckMongoHealth(ctx, client); err != nil {
log.Printf("MongoDB health check failed: %v", err)
}
Example usage for SQLite:
// Initialize SQLite database
sqliteDB, err := db.InitSQLiteDB(ctx, "file:mydb.sqlite", 5*time.Second, 30*time.Minute, 10, 5)
if err != nil {
log.Fatalf("Failed to connect to SQLite: %v", err)
}
defer sqliteDB.Close()
// Execute a transaction
err = db.ExecuteSQLTransaction(ctx, sqliteDB, func(tx *sql.Tx) error {
// Perform database operations within the transaction
_, err := tx.Exec("INSERT INTO users (name, email) VALUES (?, ?)", "John", "john@example.com")
return err
})
The package is designed to be used throughout the application to provide consistent database access patterns and error handling.
Index ¶
- Constants
- func CheckMongoHealth(ctx context.Context, client *mongo.Client) error
- func CheckPostgresHealth(ctx context.Context, pool *pgxpool.Pool) error
- func CheckSQLiteHealth(ctx context.Context, db *sql.DB) error
- func ExecutePostgresTransaction(ctx context.Context, pool *pgxpool.Pool, fn func(tx pgx.Tx) error, ...) error
- func ExecuteSQLTransaction(ctx context.Context, db *sql.DB, fn func(tx *sql.Tx) error, ...) error
- func InitMongoClient(ctx context.Context, uri string, timeout time.Duration) (*mongo.Client, error)
- func InitPostgresPool(ctx context.Context, config PostgresConfig) (*pgxpool.Pool, error)
- func InitSQLiteDB(ctx context.Context, uri string, timeout, connMaxLifetime time.Duration, ...) (*sql.DB, error)
- func IsTransientError(err error) bool
- func LogDatabaseConnection(ctx context.Context, logger *logging.ContextLogger, dbType string)
- type MongoClientInterface
- type PgxPoolInterface
- type PgxTxInterface
- type PostgresConfig
- type RetryConfig
- type SQLDBInterface
- type SQLTxInterface
Constants ¶
const DefaultTimeout = 30 * time.Second
DefaultTimeout is the default timeout for database operations
Variables ¶
This section is empty.
Functions ¶
func CheckMongoHealth ¶
CheckMongoHealth checks if a MongoDB connection is healthy. Parameters:
- ctx: The context for the operation
- client: The MongoDB client
Returns:
- error: An error if the health check fails
func CheckPostgresHealth ¶
CheckPostgresHealth checks if a PostgreSQL connection is healthy. Parameters:
- ctx: The context for the operation
- pool: The PostgreSQL connection pool
Returns:
- error: An error if the health check fails
func CheckSQLiteHealth ¶
CheckSQLiteHealth checks if a SQLite connection is healthy. Parameters:
- ctx: The context for the operation
- db: The SQLite database connection
Returns:
- error: An error if the health check fails
func ExecutePostgresTransaction ¶
func ExecutePostgresTransaction(ctx context.Context, pool *pgxpool.Pool, fn func(tx pgx.Tx) error, retryConfig ...RetryConfig) error
ExecutePostgresTransaction executes a function within a PostgreSQL transaction. Parameters:
- ctx: The context for the operation
- pool: The PostgreSQL connection pool
- fn: The function to execute within the transaction
- retryConfig: Optional retry configuration for transient errors
Returns:
- error: An error if the transaction fails
func ExecuteSQLTransaction ¶
func ExecuteSQLTransaction(ctx context.Context, db *sql.DB, fn func(tx *sql.Tx) error, retryConfig ...RetryConfig) error
ExecuteSQLTransaction executes a function within a SQL transaction. Parameters:
- ctx: The context for the operation
- db: The SQL database connection
- fn: The function to execute within the transaction
- retryConfig: Optional retry configuration for transient errors
Returns:
- error: An error if the transaction fails
func InitMongoClient ¶
InitMongoClient initializes a MongoDB client. Parameters:
- ctx: The context for the operation
- uri: The MongoDB connection URI
- timeout: The timeout for the connection operation
Returns:
- *mongo.Client: The initialized MongoDB client
- error: An error if initialization fails
func InitPostgresPool ¶
InitPostgresPool initializes a PostgreSQL connection pool. Parameters:
- ctx: The context for the operation
- config: The PostgreSQL connection configuration
Returns:
- *pgxpool.Pool: The initialized PostgreSQL connection pool
- error: An error if initialization fails
func InitSQLiteDB ¶
func InitSQLiteDB(ctx context.Context, uri string, timeout, connMaxLifetime time.Duration, maxOpenConns, maxIdleConns int) (*sql.DB, error)
InitSQLiteDB initializes a SQLite database connection. Parameters:
- ctx: The context for the operation
- uri: The SQLite connection URI
- timeout: The timeout for the connection operation
- maxOpenConns: The maximum number of open connections
- maxIdleConns: The maximum number of idle connections
- connMaxLifetime: The maximum lifetime of a connection
Returns:
- *sql.DB: The initialized SQLite database connection
- error: An error if initialization fails
func IsTransientError ¶ added in v1.2.0
IsTransientError checks if an error is a transient database error that can be retried.
func LogDatabaseConnection ¶
func LogDatabaseConnection(ctx context.Context, logger *logging.ContextLogger, dbType string)
LogDatabaseConnection logs a successful database connection. Parameters:
- ctx: The context for the operation
- logger: The logger to use
- dbType: The type of database (e.g., "MongoDB", "PostgreSQL", "SQLite")
Types ¶
type MongoClientInterface ¶
type MongoClientInterface = interfaces.MongoClientInterface
MongoClientInterface is an alias for interfaces.MongoClientInterface for backward compatibility
type PgxPoolInterface ¶
type PgxPoolInterface = interfaces.PgxPoolInterface
PgxPoolInterface is an alias for interfaces.PgxPoolInterface for backward compatibility
type PgxTxInterface ¶
type PgxTxInterface = interfaces.PgxTxInterface
PgxTxInterface is an alias for interfaces.PgxTxInterface for backward compatibility
type PostgresConfig ¶ added in v1.2.0
type PostgresConfig struct {
// URI is the PostgreSQL connection URI
URI string
// Timeout is the timeout for connection operations
Timeout time.Duration
// MaxOpenConns is the maximum number of open connections
MaxConns int32
// MinOpenConns is the minimum number of open connections
MinConns int32
// MaxConnLifetime is the maximum lifetime of a connection
MaxConnLifetime time.Duration
// MaxConnIdleTime is the maximum idle time of a connection
MaxConnIdleTime time.Duration
// HealthCheckPeriod is the period between health checks
HealthCheckPeriod time.Duration
}
PostgresConfig holds configuration for a PostgreSQL connection pool.
func DefaultPostgresConfig ¶ added in v1.5.0
func DefaultPostgresConfig() PostgresConfig
DefaultPostgresConfig returns a default configuration for a PostgreSQL connection pool.
type RetryConfig ¶ added in v1.2.0
type RetryConfig struct {
// MaxRetries is the maximum number of retry attempts
MaxRetries int
// InitialBackoff is the initial backoff duration
InitialBackoff time.Duration
// MaxBackoff is the maximum backoff duration
MaxBackoff time.Duration
// BackoffFactor is the factor by which the backoff increases
BackoffFactor float64
// Logger is the logger to use for logging retries
Logger *logging.ContextLogger
}
RetryConfig holds configuration for retry operations.
func DefaultRetryConfig ¶ added in v1.2.0
func DefaultRetryConfig() RetryConfig
DefaultRetryConfig returns the default retry configuration.
type SQLDBInterface ¶
type SQLDBInterface = interfaces.SQLDBInterface
SQLDBInterface is an alias for interfaces.SQLDBInterface for backward compatibility
type SQLTxInterface ¶
type SQLTxInterface = interfaces.SQLTxInterface
SQLTxInterface is an alias for interfaces.SQLTxInterface for backward compatibility
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Package interfaces provides database interfaces for the db package.
|
Package interfaces provides database interfaces for the db package. |
|
Package mocks is a generated GoMock package.
|
Package mocks is a generated GoMock package. |