pkg

Go Package Collection

A collection of production-ready Go packages for building web services: database, Redis (standard + cluster), JWT, crypto, GCS, structured logging, HTTP middleware, Prometheus metrics, graceful shutdown, and utilities. Follows clean architecture boundaries — domain, use-case, and infrastructure are separate.

Contents

• Installation
• Packages
  • config
  • database
  • redis
  • logger
  • middlewares
  • handler
  • response
  • jwt
  • crypto
  • gcs
  • types
  • util
  • domain
  • usecase
  • repositories
• Documentation
• Usage
• Environment Variables
• Production Wiring Example
• Testing
• License

Installation

go get github.com/turahe/pkg

Requires Go 1.21+.

Packages

config

Loads all configuration from environment variables (or a .env file via godotenv). All packages read from the global Configuration singleton.

API:

config.Setup(configPath string) error   // load .env; "" uses env vars only
config.GetConfig() *Configuration       // get global config (builds from env if needed)
config.SetConfig(cfg *Configuration)    // override config (testing / manual wiring)

Configuration struct (abbreviated):

type Configuration struct {
    Server      ServerConfiguration
    Cors        CorsConfiguration
    Database    DatabaseConfiguration
    DatabaseSite DatabaseConfiguration  // optional second DB
    Redis       RedisConfiguration
    GCS         GCSConfiguration
    RateLimiter RateLimiterConfiguration
    Timezone    TimezoneConfiguration
}

database

Enterprise-grade GORM database layer. Supports MySQL, Postgres, SQLite, SQL Server, and Google Cloud SQL (Postgres/MySQL) with IAM auth and Private IP. Includes SQL redaction in logs (passwords, tokens, card numbers).

Drivers: mysql · postgres · sqlite · sqlserver · cloudsql-mysql · cloudsql-postgres

New API (recommended — dependency injection):

db, err := database.New(&cfg.Database, database.Options{
    LogLevel: logger.Warn,
})
// or with Cloud SQL options:
db, err := database.New(&cfg.Database, database.Options{
    UseIAM:       true,
    UsePrivateIP: true,
    LogLevel:     logger.Warn,
})

gormDB  := db.DB()                    // *gorm.DB
err     = db.Health(ctx)             // ping with PingTimeout-bounded context
err     = db.Close()                  // close connections + Cloud SQL connector

Options:

High-RPS pool preset:

db, err := database.New(&cfg.Database, database.Options{}, database.WithProductionPoolDefaults)
// sets MaxOpenConns=150, MaxIdleConns=50

Legacy API (global, backward-compatible):

database.Setup()          // init from config env vars
database.GetDB()          // *gorm.DB (panics if not initialized)
database.GetDBSite()      // secondary *gorm.DB (falls back to primary)
database.HealthCheck(ctx) // check both primary and site DB
database.IsAlive()        // bool
database.Cleanup()        // close all connections

redis

Redis client wrapper for both standalone Redis and Redis Cluster (Google Cloud Memorystore). Connection pooling, timeouts, and all common commands included.

Setup:

redis.Setup() error          // reads config; no-op if REDIS_ENABLED=false
redis.Close() error          // close client; call on shutdown
redis.IsAlive() bool         // ping check
redis.GetUniversalClient() redis.Cmdable  // works with both modes
redis.GetRedis() *redis.Client            // standard mode only
redis.GetRedisCluster() *redis.ClusterClient  // cluster mode only

String operations:

redis.Get(key string) (string, error)
redis.Set(key string, value interface{}, expiration time.Duration) error
redis.Delete(keys ...string) error
redis.MGet(keys ...string) ([]interface{}, error)
redis.MSet(pairs ...interface{}) error

Hash operations:

redis.HGet(key, field string) (string, error)
redis.HGetAll(key string) (map[string]string, error)
redis.HSet(key, field string, value interface{}) error
redis.HSetMap(key string, fields map[string]interface{}) error

List operations:

redis.LPush(key string, values ...interface{}) error
redis.RPop(key string) (string, error)
redis.LRange(key string, start, stop int64) ([]string, error)

Set operations:

redis.SAdd(key string, members ...interface{}) error
redis.SMembers(key string) ([]string, error)
redis.SRem(key string, members ...interface{}) error

Distributed lock:

redis.AcquireLock(key string, expiration time.Duration) (bool, error)
redis.ExtendLock(key string, expiration time.Duration) (bool, error)
redis.ReleaseLock(key string) error

Pipeline, Pub/Sub, Scan:

redis.Pipeline(fn func(redis.Pipeliner) error) error
redis.PipelineSet(pairs map[string]interface{}, expiration time.Duration) error
redis.PublishMessage(channel string, message interface{}) error
redis.SubscribeToChannel(channel string) *redis.PubSub
redis.ScanKeys(pattern string, count int64) ([]string, error)  // cluster-aware

logger

Structured logging built on log/slog. Outputs Google Cloud Logging-compatible JSON with severity, time, message, trace_id, correlation_id, sourceLocation, and optional fields. The underlying writer is lazy-initialized on first log write (no I/O at startup).

Plain functions:

logger.Debugf(format string, args ...interface{})
logger.Infof(format string, args ...interface{})
logger.Warnf(format string, args ...interface{})
logger.Errorf(format string, args ...interface{})
logger.Fatalf(format string, args ...interface{})  // calls os.Exit(1)

Structured fields:

logger.Debug(msg string, fields logger.Fields)
logger.Info(msg string, fields logger.Fields)
logger.Warn(msg string, fields logger.Fields)
logger.Error(msg string, fields logger.Fields)

Context-bound (includes trace_id / correlation_id automatically):

// Store IDs in context (done by RequestID / TraceMiddleware):
ctx = logger.WithTraceID(ctx, traceID)
ctx = logger.WithCorrelationID(ctx, correlationID)

// Read IDs from context:
logger.GetTraceID(ctx)
logger.GetCorrelationID(ctx)

// Context-bound logger — all log calls carry the IDs from ctx:
log := logger.WithContext(ctx)
log.Infof("user %s logged in", userID)
log.Errorf("operation failed: %v", err)

// Or use context-aware free functions:
logger.InfofContext(ctx, "processed %d records", n)
logger.ErrorfContext(ctx, "failed: %v", err)

Configuration:

logger.SetLogLevel(slog.LevelDebug)  // default: Info
logger.GetLogger() *slog.Logger
logger.GetWriter() io.Writer         // file + stderr multi-writer

middlewares

Gin middleware collection. Designed to be composed in a specific order for correct behaviour.

Recommended middleware order:

router := gin.New()

// Create JWT verifier (Manager for sign+verify, or Verifier for verify-only)
jwtManager, err := jwt.NewManager(context.Background(), config.GetConfig())
if err != nil {
    log.Fatal(err)
}
// Or for API-only services: jwtVerifier, _ := jwt.NewVerifier(ctx, config.GetConfig())

router.Use(
    middlewares.RecoveryHandler,              // 1. catch panics first
    middlewares.RequestID(),                  // 2. inject trace/correlation IDs
    middlewares.LoggerMiddleware(),            // 3. log with IDs in context
    middlewares.Metrics(),                    // 4. Prometheus instrumentation
    middlewares.RequestTimeout(10*time.Second), // 5. bound all downstream handlers
    middlewares.CORS(),                       // 6. CORS headers
    middlewares.AuthMiddleware(jwtManager),   // 7. JWT auth (pass *Manager or *Verifier)
    middlewares.RateLimiter(),                // 8. rate limit (requires Redis)
)
router.NoMethod(middlewares.NoMethodHandler())
router.NoRoute(middlewares.NoRouteHandler())

Reference:

Prometheus metrics exposed by Metrics():

Register the scrape endpoint separately:

import "github.com/prometheus/client_golang/prometheus/promhttp"
router.GET("/metrics", gin.WrapH(promhttp.Handler()))

handler

Base handler for Gin with binding, pagination, error routing, and role checks.

type BaseHandler struct{}

// Bind request body / query / URI by content-type and method.
func (c *BaseHandler) ValidateReqParams(ctx *gin.Context, params interface{}) error

// 422 validation error response (Laravel-style field map).
func (c *BaseHandler) HandleValidationError(ctx *gin.Context, serviceCode string, err error)

// Clamp page/size to defaults (1, 10) and max (100).
func (c *BaseHandler) NormalizePagination(pageNumber, pageSize int) (int, int)

// Extract and validate URL param; writes 422 and returns false if missing.
func (c *BaseHandler) GetIDFromParam(ctx *gin.Context, paramName, serviceCode string) (string, bool)

// Prefer reqID; fall back to URL param.
func (c *BaseHandler) GetIDFromRequestOrParam(ctx *gin.Context, reqID, paramName, serviceCode string) (string, bool)

// Route domain errors to correct HTTP status. Returns true if handled.
// Checks errors.Is(ErrNotFound) → 404, errors.Is(ErrUnauthorized) → 401,
// then notFoundMessages list, then falls back to 500.
func (c *BaseHandler) HandleServiceError(ctx *gin.Context, serviceCode string, err error, notFoundMessages ...string) bool

// Build SimplePaginationResponse from a slice, page info, and total.
func (c *BaseHandler) BuildPaginationResponse(data []interface{}, pageNumber, pageSize int, total int64) response.SimplePaginationResponse

// Read "user_id" string from Gin context (set by AuthMiddleware).
func (c *BaseHandler) GetCurrentUserID(ctx *gin.Context) (string, bool)

// Return true if any userRole matches any requiredRole.
func (c *BaseHandler) CheckUserHasRole(userRoles, requiredRoles []string) bool

response

Standardised JSON responses with a 7-digit composite code (HTTP(3) + Service(2) + Case(2)).

Success responses:

response.Ok(ctx)
response.OkWithMessage(ctx, message)
response.OkWithData(ctx, data)
response.Created(ctx, data)
response.Updated(ctx, data)
response.Deleted(ctx)

Error responses:

response.FailWithMessage(ctx, message)
response.FailWithDetailed(ctx, httpStatus, serviceCode, caseCode, data, message)
response.ValidationError(ctx, serviceCode, err)          // 422 with field map
response.ValidationErrorSimple(ctx, serviceCode, field, message)
response.NotFoundError(ctx, serviceCode, caseCode, message)   // 404
response.UnauthorizedError(ctx, message)                       // 401
response.ForbiddenError(ctx, message)                          // 403
response.ConflictError(ctx, serviceCode, caseCode, message)    // 409

Pagination responses:

response.SimplePaginated(ctx, data, pageNumber, pageSize, hasNext, hasPrev)
response.CursorPaginated(ctx, data, nextCursor, hasNext)

Service codes: ServiceCodeCommon, ServiceCodeAuth, ServiceCodeTransaction, ServiceCodeWithdrawal, ServiceCodeUser, ServiceCodeAdmin, ServiceCodeMerchant, ServiceCodeSetting, ServiceCodeRole, ServiceCodePermission, ServiceCodeNotification, ServiceCodeIPWhitelist, ServiceCodeApiKey, ServiceCodeDeposit, ServiceCodeWallet, ServiceCodePhone, ServiceCodeEmail, ServiceCodeTwoFactor, ServiceCodeBank, ServiceCodeBankAccount, and more. Case codes cover success, validation, auth, not-found, business logic, server errors, conflicts, email/phone change, 2FA, and bank/bank-account flows (see response/codes.go).

jwt

JWT token generation and validation with HS256, RS256 (default), or ES256. Keys can come from config (env or file paths) or Google Cloud Secret Manager (with a 30s context timeout). No global state: use Manager (sign + verify), Signer (issue tokens only), or Verifier (validate only). Supports issuer, audience, key ID (kid), and token type (access, refresh, impersonation).

All-in-one (single service):

manager, err := jwt.NewManager(ctx, config.GetConfig())
if err != nil {
    log.Fatal(err)
}
router.Use(middlewares.AuthMiddleware(manager))

token, _ := manager.GenerateToken(userID)
claims, _ := manager.ValidateToken(tokenString)

Split services (auth server issues, API server only verifies):

// Auth/login service — needs private key or secret
signer, err := jwt.NewSigner(ctx, config.GetConfig())
token, _ := signer.GenerateToken(userID)
refresh, _ := signer.GenerateRefreshToken(userID)
impersonation, _ := signer.GenerateImpersonationToken(adminID, "admin", targetID, 15*time.Minute)

// API/gateway service — needs only public key or secret
verifier, err := jwt.NewVerifier(ctx, config.GetConfig())
router.Use(middlewares.AuthMiddleware(verifier))  // TokenVerifier interface
claims, _ := verifier.ValidateToken(tokenString)

API summary:

Config / env: Default algorithm is RS256. For HS256 set JWT_SIGNING_ALGORITHM=HS256 and SERVER_SECRET. For RS256/ES256 set JWT_PRIVATE_KEY_PATH and JWT_PUBLIC_KEY_PATH (or use Secret Manager: JWT_SECRET_MANAGER_PROJECT_ID and secret names). Optional: JWT_ISSUER, JWT_AUDIENCE, JWT_KEY_ID.

crypto

bcrypt password hashing.

crypto.HashAndSalt(plainPassword []byte) string
crypto.ComparePassword(hashedPassword string, plainPassword []byte) bool

gcs

Google Cloud Storage wrapper. Uses Application Default Credentials (ADC) or an explicit service account JSON file.

gcs.Setup() error
gcs.GetClient() *storage.Client
gcs.GetBucket() *storage.BucketHandle
gcs.ReadObject(objectName string) ([]byte, error)
gcs.ReadObjectAsReader(objectName string) (io.ReadCloser, error)
gcs.WriteObject(objectName string, data []byte, contentType string) error
gcs.DeleteObject(objectName string) error
gcs.ObjectExists(objectName string) (bool, error)
gcs.ListObjects(prefix string) ([]string, error)
gcs.Close() error

types

Shared types for handlers and repositories (no infrastructure dependencies).

// Conditions is the WHERE clause map passed to repository methods.
type Conditions map[string]interface{}

// PageInfo holds offset-based pagination request (pageNumber, pageSize) with form/json tags.
type PageInfo struct {
    PageNumber int `form:"pageNumber" json:"pageNumber"`
    PageSize   int `form:"pageSize" json:"pageSize"`
}

// TimeRange holds a start/end pair (IANA or RFC3339 strings).
type TimeRange struct {
    Start string `json:"start"`
    End   string `json:"end"`
}

util

util.IsEmpty(value interface{}) bool
util.InAnySlice[T comparable](haystack []T, needle T) bool
util.RemoveDuplicates[T comparable](haystack []T) []T
util.FormatPhoneNumber(phone *string, countryCode *string) *string  // E.164 via nyaruka/phonenumbers

domain

Domain errors and port interfaces. No dependencies on infrastructure.

// Sentinel errors for handlers and use cases (use with errors.Is).
var (
    ErrNotFound     = errors.New("not found")
    ErrUnauthorized = errors.New("unauthorized")
)

domain/port

Port interfaces implemented by infrastructure (e.g. repositories).

type GetByID interface {
    GetByID(ctx context.Context, id string) (interface{}, bool, error)
}

usecase

Application service layer. Use cases depend only on domain and ports.

type Runner interface { Run(ctx context.Context) error }
type Func func(ctx context.Context) error  // adapts func to Runner

// Example: get item by ID; returns domain.ErrNotFound when not found.
func GetItemByID(ctx context.Context, repo port.GetByID, id string) (interface{}, error)

repositories

GORM-based base repository: CRUD, First, Find, Scan, SimplePagination, RawSQL, ExecSQL. Use with dependency injection (NewBaseRepositoryWithDB) or global DB (NewBaseRepository). Implements IBaseRepository; adapt to domain/port in your app.

repo := repositories.NewBaseRepositoryWithDB(db.DB())
repo.Create(ctx, &model)
repo.First(ctx, &out, types.Conditions{"id = ?": id})
repo.SimplePagination(ctx, &model, &out, page, size, conditions, orders, "User", "Items")

Documentation

All packages follow GoDoc conventions:

• Each package has a doc.go describing its role, responsibilities, constraints, and what it must not do.
• Exported symbols have comments that start with the symbol name and describe behavior clearly.
• Repository, adapter, and domain packages document contracts, error behavior, and context use.

Run go doc or view pkg.go.dev for the full API.

Minimal server

package main

import (
    "context"
    "log"
    "net/http"
    "os"
    "os/signal"
    "sync/atomic"
    "syscall"
    "time"

    "github.com/gin-gonic/gin"
    "github.com/prometheus/client_golang/prometheus/promhttp"
    "github.com/turahe/pkg/config"
    "github.com/turahe/pkg/database"
    "github.com/turahe/pkg/middlewares"
    pkgredis "github.com/turahe/pkg/redis"
    "gorm.io/gorm/logger"
)

func main() {
    cfg := config.GetConfig()

    // Database
    db, err := database.New(&cfg.Database, database.Options{LogLevel: logger.Warn})
    if err != nil {
        log.Fatal(err)
    }

    // Redis (optional)
    if cfg.Redis.Enabled {
        if err := pkgredis.Setup(); err != nil {
            log.Printf("redis: %v", err)
        }
    }

    // Readiness gate
    var ready atomic.Bool
    ready.Store(true)

    // Router
    router := gin.New()
    router.Use(
        middlewares.RecoveryHandler,
        middlewares.RequestID(),
        middlewares.LoggerMiddleware(),
        middlewares.Metrics(),
        middlewares.RequestTimeout(10*time.Second),
    )
    router.GET("/metrics", gin.WrapH(promhttp.Handler()))
    router.GET("/live", func(c *gin.Context) {
        c.JSON(http.StatusOK, gin.H{"status": "ok"})
    })
    router.GET("/ready", func(c *gin.Context) {
        if !ready.Load() {
            c.JSON(http.StatusServiceUnavailable, gin.H{"status": "shutting_down"})
            return
        }
        if err := db.Health(c.Request.Context()); err != nil {
            c.JSON(http.StatusServiceUnavailable, gin.H{"status": "degraded", "error": err.Error()})
            return
        }
        c.JSON(http.StatusOK, gin.H{"status": "ok"})
    })

    // Application routes ...
    router.NoMethod(middlewares.NoMethodHandler())
    router.NoRoute(middlewares.NoRouteHandler())

    srv := &http.Server{
        Addr:              ":" + cfg.Server.Port,
        Handler:           router,
        ReadHeaderTimeout: 5 * time.Second,
        ReadTimeout:       10 * time.Second,
        WriteTimeout:      10 * time.Second,
        IdleTimeout:       60 * time.Second,
    }
    go func() {
        if err := srv.ListenAndServe(); err != nil && err != http.ErrServerClosed {
            log.Printf("server: %v", err)
        }
    }()

    // Graceful shutdown (25 s matches terminationGracePeriodSeconds: 30)
    sig := make(chan os.Signal, 1)
    signal.Notify(sig, syscall.SIGINT, syscall.SIGTERM)
    <-sig
    ready.Store(false) // signal readiness probe → 503 immediately

    shutCtx, cancel := context.WithTimeout(context.Background(), 25*time.Second)
    defer cancel()
    srv.Shutdown(shutCtx)

    if cfg.Redis.Enabled {
        pkgredis.Close()
    }
    db.Close()
}

Repository with dependency injection

import (
    "github.com/turahe/pkg/database"
    "github.com/turahe/pkg/repositories"
    "github.com/turahe/pkg/types"
)

db, _ := database.New(&cfg.Database, database.Options{})
repo := repositories.NewBaseRepositoryWithDB(db.DB())

// Create
err := repo.Create(ctx, &myModel)

// Find with conditions
var results []MyModel
err = repo.Find(ctx, &results, types.Conditions{
    "status = ?": "active",
    "tenant_id = ?": tenantID,
})

// Pagination (pass preload names to avoid N+1)
total, err := repo.SimplePagination(ctx, &MyModel{}, &results, page, size,
    types.Conditions{"status = ?": "active"},
    []string{"created_at DESC"},
    "User", "Items",  // preloads
)

Clean architecture wiring

// domain/port/repository.go
type ItemRepository interface {
    GetByID(ctx context.Context, id string) (*Item, bool, error)
}

// infrastructure/repository/item.go
type itemRepo struct{ base repositories.IBaseRepository }

func (r *itemRepo) GetByID(ctx context.Context, id string) (*Item, bool, error) {
    var item Item
    notFound, err := r.base.First(ctx, &item, types.Conditions{"id = ?": id})
    return &item, !notFound, err
}

// usecase/get_item.go
func GetItem(ctx context.Context, repo port.ItemRepository, id string) (*Item, error) {
    item, found, err := repo.GetByID(ctx, id)
    if err != nil { return nil, err }
    if !found { return nil, domain.ErrNotFound }
    return item, nil
}

Environment Variables

Copy .env.example to .env and call config.Setup(""):

cp .env.example .env

Server

JWT

Secret Manager calls use a 30s context timeout.

Database

Redis

Rate Limiter

GCS

Production Wiring Example

See cmd/example/main.go for a complete wiring of:

• Dependency-injected database with health check
• Optional Redis setup and graceful Close()
• gin.New() with explicit middleware stack (recovery → trace → logging → metrics → timeout)
• /live (liveness), /ready (readiness with component checks), /metrics (Prometheus)
• HTTP server with all timeouts set
• Graceful shutdown: readiness gate → srv.Shutdown(25s) → Redis close → DB close

Kubernetes probes

livenessProbe:
  httpGet:
    path: /live
    port: 8080
  initialDelaySeconds: 5
  periodSeconds: 10

readinessProbe:
  httpGet:
    path: /ready
    port: 8080
  initialDelaySeconds: 5
  periodSeconds: 5

terminationGracePeriodSeconds: 30

Docker

docker build -t myapp .
docker run --env-file .env -p 8080:8080 myapp

The multi-stage Dockerfile uses golang:1.25-alpine to build and gcr.io/distroless/base-debian12:nonroot as the runtime image. Runs as UID 65532 (non-root).

Testing

Run all tests:

go test ./...

With race detector (requires CGO):

CGO_ENABLED=1 go test -race ./...

With coverage:

go test -coverprofile=coverage.out ./...
go tool cover -func=coverage.out

Run tests with Docker Compose

Run the full test suite (including integration tests) without local Go or databases:

make test-docker

Or with docker compose directly:

docker compose -f docker-compose.test.yml up --build --abort-on-container-exit --exit-code-from test
docker compose -f docker-compose.test.yml down -v

The test runner waits for Redis, MySQL, and Postgres to be healthy, then runs go test -v -race -count=1 ./.... Coverage is written to coverage.out and a summary is printed.

Integration tests (Redis · MySQL · Postgres) — local

Start services with Docker Compose:

docker compose up -d

Run with environment variables:

REDIS_ENABLED=true REDIS_HOST=127.0.0.1 REDIS_PORT=6379 \
DATABASE_DRIVER=mysql DATABASE_HOST=127.0.0.1 DATABASE_PORT=3306 \
DATABASE_USERNAME=root DATABASE_PASSWORD=root DATABASE_DBNAME=testdb \
go test ./...

Integration tests skip automatically when services are unavailable. CI runs the full matrix (Go 1.21–1.25.4) with these services via GitHub Actions.

Packages with tests: config, crypto, database, gcs, handler, jwt, logger, middlewares, redis, repositories, response, types, util.

License

MIT