go-bricks

GoBricks

Modern building blocks for Go microservices. GoBricks brings together configuration, HTTP, messaging, database, logging and observability primitives that teams need to ship production-grade services fast.

---

Table of Contents

1. Why GoBricks?
2. Developer Resources
3. Feature Overview
4. Quick Start
5. Configuration
6. Error Handling and Diagnostics
7. Modules and Application Structure
8. HTTP Server
9. HTTP Client
10. Messaging
11. Database
12. Cache
13. Scheduler and Outbox
14. KeyStore
15. JOSE (Sign-then-Encrypt Bodies)
16. Multi-Tenant Implementation
17. Observability and Operations
18. Examples
19. Contributing
20. License

---

Why GoBricks?

• Production-ready defaults for the boring-but-essential pieces (server, logging, configuration, tracing).
• Composable module system that keeps HTTP, database, and messaging concerns organized.
• Mission-critical integrations (PostgreSQL, Oracle, RabbitMQ, Flyway) with unified ergonomics.
• Modern Go practices with type safety, performance optimizations, and Go 1.26 features.
• Extensible design that works with modern Go idioms and the wider ecosystem.

---

Developer Resources

For Contributors & Framework Developers:

• CLAUDE.md - Comprehensive development guide with architecture, commands, testing, and workflows
• llms.txt - Quick code examples optimized for LLMs and copy-paste development
• CONTRIBUTING.md - Coding standards, tooling, and contribution workflow

For Application Developers:

• go-bricks-demo-project - Complete working examples and patterns
• MULTI_TENANT.md - Multi-tenant architecture and implementation guide
• Go Reference - Complete API documentation

Quick Commands:

make check              # Pre-commit: fmt + lint + test
make test-integration   # Integration tests (requires Docker)
go test -run TestName   # Run specific test

---

Feature Overview

• Modular architecture with explicit dependencies and lifecycle hooks
• Echo-based HTTP server with typed handlers, standardized response envelopes, and raw response mode for legacy API migration
• Production-ready HTTP client with retries, exponential backoff, W3C trace propagation, and interceptor chains
• AMQP messaging with validate-once, replay-many pattern, auto-scaling consumer concurrency, and panic recovery
• Configuration loader merging defaults, YAML, and environment variables with struct-based injection (config: tags)
• Multi-database support for PostgreSQL and Oracle with struct-based column extraction and type-safe query builders
• Named databases for accessing multiple databases (including mixed vendors) in single-tenant mode
• Redis cache integration with type-safe serialization, multi-tenant isolation, and automatic lifecycle management
• Transactional outbox for reliable at-least-once event publishing (dual-write problem solved)
• Job scheduler with gocron, overlapping prevention, panic recovery, and system APIs
• Named RSA key pair management from DER files or base64 environment variables
• JOSE middleware for nested JWE-of-JWS protection on HTTP request/response bodies (Visa-style integrations)
• Multi-tenant architecture with complete resource isolation and context propagation
• Flyway migration integration for schema evolution
• Observability with W3C trace propagation, custom metrics, dual-mode logs, and health endpoints
• Enterprise-grade quality with comprehensive linting, 80%+ test coverage, and security scanning

---

Quick Start

Requirements

• Go 1.26 required
• Modern Go toolchain with module support
• Docker Desktop or Docker Engine (integration tests only)

Install

go mod init your-service
go get github.com/gaborage/go-bricks@latest

Bootstrap an application

// cmd/main.go
package main

import (
    "log"

    "github.com/gaborage/go-bricks/app"
    "github.com/gaborage/go-bricks/config"
)

func main() {
    cfg, err := config.Load()
    if err != nil {
        log.Fatal(err)
    }

    framework, _, err := app.NewWithConfig(cfg, nil)
    if err != nil {
        log.Fatal(err)
    }

    // Register modules here
    if err := framework.RegisterModule(&users.Module{}); err != nil {
        log.Fatal(err)
    }

    if err := framework.Run(); err != nil {
        log.Fatal(err)
    }
}

Configuration file

app:
  name: "my-service"
  version: "v1.0.0"
  env: "development"
  rate:
    limit: 200

server:
  port: 8080

database:
  type: postgresql
  host: localhost
  port: 5432
  database: mydb
  username: postgres
  password: password

cache:
  enabled: true
  type: redis
  redis:
    host: localhost
    port: 6379
    database: 0
    pool_size: 10

log:
  level: info
  pretty: true

GoBricks loads defaults → config.yaml → config.<env>.yaml → environment variables. app.env controls the environment suffix and defaults to development.

---

Configuration

GoBricks uses Koanf for configuration management with layered loading: defaults → config.yaml → config.<env>.yaml → environment variables.

Access Patterns

cfg, _ := config.Load()

// Simple values with defaults
host := cfg.String("server.host", "0.0.0.0")
port := cfg.Int("server.port", 8080)

// Required values with validation
apiKey, err := cfg.RequiredString("custom.api.key")
if err != nil {
    return fmt.Errorf("missing api key: %w", err)
}

// Structured configuration under custom.*
var custom struct {
    FeatureFlag bool   `koanf:"feature.flag"`
    Endpoint    string `koanf:"api.endpoint"`
}
_ = cfg.Unmarshal("custom", &custom)

Config Injection

Inject configuration directly into structs using config: tags with automatic validation:

type ServiceConfig struct {
    APIKey   string        `config:"custom.api.key" required:"true"`
    Timeout  time.Duration `config:"custom.api.timeout" default:"30s"`
    Retries  int           `config:"custom.api.retries" default:"3"`
}

func (m *Module) Init(deps *ModuleDeps) error {
    var cfg ServiceConfig
    if err := deps.Config.InjectInto(&cfg); err != nil {
        return err // Fails fast if required keys are missing
    }
    m.service = NewService(cfg)
    return nil
}

Supported tags: config:"key.path" (required), required:"true" (fail if missing), default:"value" (fallback). Supported types: string, int, int64, float64, bool, time.Duration.

Environment Variables

Environment variables use uppercase with underscores and automatically map to dot notation:

DATABASE_HOST=prod-db.company.com   # maps to database.host
DATABASE_SERVICE_NAME=FREEPDB1      # maps to database.service.name
CUSTOM_API_TIMEOUT=30s              # maps to custom.api.timeout

See the config-injection example for advanced patterns.

---

Error Handling and Diagnostics

Clear, actionable error messages for configuration and startup failures.

Error Format

config_<category>: <field> <message> <action>

Categories: missing (required field), invalid (bad value), not_configured (optional feature), connection (resource failure)

Examples

config_missing: database.host required set DATABASE_HOST env var or add database.host to config.yaml

config_invalid: database.port invalid port; must be between 1 and 65535 must be one of: 1-65535

config_not_configured: messaging.broker.url (optional) to enable: set MESSAGING_BROKER_URL env var or add messaging.broker.url to config.yaml

Features

• Dual paths: shows both env var and YAML path
• Semantic distinction: differentiates missing, invalid, and optional
• Multi-tenant context: includes tenant ID when applicable
• No error chains: single clear message per issue

---

Modules and Application Structure

Module interface

type Module interface {
    Name() string
    Init(deps *ModuleDeps) error
    Shutdown() error
}

// Optional — implement to register HTTP routes (detected via duck-typing at startup)
type RouteRegisterer interface {
    RegisterRoutes(hr *server.HandlerRegistry, r server.RouteRegistrar)
}

// Optional — implement to declare AMQP exchanges, queues, and consumers
type MessagingDeclarer interface {
    DeclareMessaging(decls *messaging.Declarations)
}

ModuleDeps injects shared services into every module:

type ModuleDeps struct {
    Logger        logger.Logger                                          // Structured logging
    Config        *config.Config                                         // Configuration access
    Tracer        trace.Tracer                                           // Distributed tracing (no-op if disabled)
    MeterProvider metric.MeterProvider                                   // Custom metrics (no-op if disabled)
    Scheduler     JobRegistrar                                           // Job scheduling (nil if no scheduler module)
    Outbox        OutboxPublisher                                        // Transactional event publishing (nil if disabled)
    KeyStore      KeyStore                                               // Named RSA key pairs (nil if not configured)
    DB            func(ctx context.Context) (database.Interface, error)  // Tenant-aware database
    DBByName      func(ctx context.Context, name string) (database.Interface, error) // Named databases
    Messaging     func(ctx context.Context) (messaging.AMQPClient, error) // Tenant-aware messaging
    Cache         func(ctx context.Context) (cache.Cache, error)         // Tenant-aware cache
}

Registering a module

func register(framework *app.App) error {
    return framework.RegisterModule(&users.Module{})
}

Init is called once to capture dependencies and Shutdown releases resources. Route registration and messaging are opt-in: if your module implements RouteRegisterer, the framework calls RegisterRoutes to attach HTTP handlers; if it implements MessagingDeclarer, DeclareMessaging is called to declare AMQP infrastructure (validated once, replayed per-tenant). Modules that don't need HTTP or AMQP simply omit the interface. The framework ensures proper lifecycle ordering and error handling across all module hooks.

---

HTTP Server

Echo v5-based server with typed handlers, standardized response envelopes, and comprehensive middleware (logging, recovery, rate limiting, CORS).

Typed Handlers

func (h *Handler) createUser(req CreateReq, ctx server.HandlerContext) (server.Result[User], server.IAPIError) {
    user := h.svc.Create(req)
    return server.Created(user), nil
}

Request structs use tags for binding/validation (path, query, header, validate). Responses follow consistent {data:…, meta:…} envelope structure. Use server.WithRawResponse() to bypass the envelope for legacy API migration (Strangler Fig pattern).

Routing Configuration

server:
  path:
    base: "/api/v1"     # All routes prefixed
    health: "/health"   # Liveness endpoint
    ready: "/ready"     # Readiness endpoint

Override with environment variables: SERVER_PATH_BASE, SERVER_PATH_HEALTH, SERVER_PATH_READY.

---

HTTP Client

Production-ready HTTP client with built-in observability and resilience.

client := httpclient.NewBuilder(logger).
    WithTimeout(10 * time.Second).
    WithRetries(3, 500 * time.Millisecond).
    WithDefaultHeader("Accept", "application/json").
    WithW3CTrace(true).
    Build()

resp, err := client.Get(ctx, &httpclient.Request{
    URL: "https://api.example.com/users",
})

Features: Builder pattern, W3C trace propagation, exponential backoff with full jitter, request/response interceptor chains, structured logging. Methods: Get, Post, Put, Patch, Delete, Do.

See llms.txt for more examples and go-bricks-demo-project for working demos.

---

Messaging

AMQP/RabbitMQ support with validate-once, replay-many declaration pattern:

• Declarative Infrastructure: Exchanges, queues, bindings, publishers, and consumers declared as data structures, validated upfront
• Multi-Tenant Isolation: Declarations replayed to tenant-specific registries for complete separation
• Auto-Reconnection: Exponential backoff for resilient operations
• Context Propagation: Tenant IDs and trace information flow automatically through messaging
• Consumer Concurrency: Auto-scaling workers (NumCPU * 4) with configurable overrides and resource safeguards

---

Database

Unified interface supporting PostgreSQL and Oracle with vendor-specific optimizations and type-safe query building across all operations.

Type-Safe Filter API

GoBricks provides a composable Filter API that works across SELECT, UPDATE, DELETE, and JOIN operations:

// Build reusable filters
f := qb.Filter()

// SELECT with filters
users := qb.Select("id", "name", "email").
    From("users").
    Where(f.Eq("status", "active")).
    Where(f.Gt("created_at", startDate))

// UPDATE with filters
qb.Update("users").
    Set("status", "inactive").
    Where(f.Lt("last_login", cutoffDate))

// DELETE with filters
qb.Delete("users").
    Where(f.Eq("status", "deleted")).
    Where(f.Lt("deleted_at", retentionDate))

// JOIN with type-safe column comparisons
jf := qb.JoinFilter()
query := qb.Select("u.name", "p.bio").
    From("users u").
    LeftJoinOn("profiles p", jf.EqColumn("u.id", "p.user_id")).
    Where(f.NotNull("p.bio"))

Filter Methods: Eq, NotEq, Lt, Lte, Gt, Gte, In, NotIn, Like, Null, NotNull, Between, Exists, NotExists, InSubquery, And, Or, Not, Raw

JoinFilter Methods: EqColumn, NotEqColumn, LtColumn, LteColumn, GtColumn, GteColumn, Eq, NotEq, Lt, Lte, Gt, Gte, In, NotIn, Between, Like, Null, NotNull, And, Or, Raw

All methods automatically handle:

• Vendor-specific quoting (Oracle reserved words like NUMBER, DATE)
• Placeholder formatting ($1 for PostgreSQL, :1 for Oracle)
• Type safety at compile time with refactor-friendly interfaces

Struct-Based Column Extraction

Eliminate column repetition using db:"column_name" struct tags. Columns are extracted once via reflection and cached forever (~26ns access):

type User struct {
    ID    int64  `db:"id"`
    Name  string `db:"name"`
    Level int    `db:"level"` // Oracle reserved word — auto-quoted
}

cols := qb.Columns(&User{})
f := qb.Filter()

query := qb.Select(cols.Fields("ID", "Name")...).
    From("users").
    Where(f.Eq(cols.Col("Level"), 5))
// Oracle: SELECT "ID", "NAME" FROM users WHERE "LEVEL" = :1
// PostgreSQL: SELECT id, name FROM users WHERE level = $1

Benefits: DRY (define columns once), type-safe (panics on typos), vendor-aware (auto-quotes Oracle reserved words), refactor-friendly, zero overhead after first use. See CLAUDE.md for table aliases, INSERT patterns, and advanced examples.

Database Support

• PostgreSQL: $1, $2 placeholders, pgx driver, advanced features
• Oracle: :1, :2 placeholders, reserved word quoting, service name/SID support, SEQUENCE objects, UDT registration

Features

• Connection pooling with production-safe defaults and health monitoring
• Named databases for multi-database single-tenant setups (deps.DBByName(ctx, "legacy"))
• Flyway integration for schema migrations (see below)
• Performance tracking via OpenTelemetry
• Type-safe query builders prevent runtime SQL errors

Schema Migrations (Flyway)

The migration package wraps the Flyway CLI for repeatable schema versioning. It auto-selects the per-vendor migration scripts and config (flyway/flyway-<vendor>.conf, migrations/<vendor>/) and surfaces Migrate, Info, and Validate commands plus RunMigrationsAtStartup for opt-in startup migrations:

m := migration.NewFlywayMigrator(cfg, logger)
if err := m.Migrate(ctx, nil); err != nil {     // nil → use vendor-aware defaults
    log.Fatal(err)
}

Flyway must be installed and on PATH (the path is validated against a safe-path allowlist). Defaults can be overridden via a *migration.Config struct (FlywayPath, ConfigPath, MigrationPath, Timeout, DryRun).

Multi-Tenant Migrations (CI/CD)

For multi-tenant fleets, GoBricks ships a CLI (go-bricks-migrate in tools/migration/) and a library entry point (migration.MigrateAll) that fan migrations out across every tenant. Tenant IDs come from a control-plane HTTP API conforming to a pre-defined contract that uses the standard go-bricks APIResponse envelope; tenant database credentials are recovered from AWS Secrets Manager at the documented gobricks/migrate/<tenant_id> naming convention.

go-bricks-migrate migrate \
  --source-url https://control-plane.example.com/api \
  --aws-region us-east-1 \
  --secrets-prefix gobricks/migrate/

See wiki/multi_tenant_migration.md for the full HTTP contract, IAM policy, secret payload formats (canonical and AWS RDS rotation fallback), CI/CD recipe, and library usage. Architectural rationale lives in ADR-018.

---

Cache

GoBricks provides Redis-based caching with type-safe serialization, multi-tenant isolation, and automatic lifecycle management through the CacheManager.

Quick Example

import (
    "context"
    "time"

    "github.com/gaborage/go-bricks/cache"
)

type UserService struct {
    getCache func(context.Context) (cache.Cache, error)
}

func (s *UserService) GetUser(ctx context.Context, id int64) (*User, error) {
    // Cache(ctx) resolves the tenant from context automatically
    cache, err := s.getCache(ctx)
    if err != nil {
        return nil, err
    }

    cacheKey := fmt.Sprintf("user:%d", id)

    // Try cache first
    data, err := cache.Get(ctx, cacheKey)
    if err == nil {
        return cache.Unmarshal[User](data)
    }

    // Cache miss - query database
    user, err := s.queryDatabase(ctx, id)
    if err != nil {
        return nil, err
    }

    // Store in cache with TTL
    data, _ = cache.Marshal(user)
    cache.Set(ctx, cacheKey, data, 5*time.Minute)

    return user, nil
}

Key Features

• Type-Safe Serialization: CBOR encoding with compile-time type safety
• Multi-Tenant Isolation: Automatic tenant context resolution
• Lifecycle Management: Lazy initialization, LRU eviction (max 100 tenants), idle cleanup (15m default)
• Atomic Operations: GetOrSet for deduplication, CompareAndSet for distributed locking
• Performance: <1ms latency for Get/Set, 100k ops/sec throughput
• Observability: OpenTelemetry spans, metrics, and health checks

Configuration

cache:
  enabled: true
  type: redis
  manager:
    maxsize: 100              # Max tenant cache instances (LRU-evicted)
    idlettl: 15m              # Close caches idle longer than this
    cleanupinterval: 5m       # How often the cleanup goroutine runs
  redis:
    host: localhost
    port: 6379
    password: ${CACHE_REDIS_PASSWORD}   # From environment
    database: 0
    pool_size: 10              # Defaults to NumCPU * 2 if unset

The manager.* keys tune the per-tenant lifecycle; tune them up if you serve many tenants or want connections to live longer between bursts.

Operations

Operation	Method	Use Case
Basic read	Get(ctx, key)	Query result caching
Basic write	Set(ctx, key, value, ttl)	Store computed data with TTL
Deduplication	GetOrSet(ctx, key, value, ttl)	Idempotency keys, atomic SET NX
Distributed lock	CompareAndSet(ctx, key, expected, new, ttl)	Cross-pod job coordination
Type-safe store	Marshal(v) + Set()	Struct serialization (CBOR)
Type-safe retrieve	Get() + Unmarshal[T](data)	Struct deserialization

For comprehensive examples, see the Cache Operations section in llms.txt.

Testing with Mock Cache

GoBricks provides cache/testing package for unit tests without Redis dependencies:

import cachetest "github.com/gaborage/go-bricks/cache/testing"

func TestMyService(t *testing.T) {
    mockCache := cachetest.NewMockCache()

    // Configure mock behavior
    mockCache.WithGetFailure(cache.ErrNotFound)

    // Inject into service
    deps := &app.ModuleDeps{
        Cache: func(ctx context.Context) (cache.Cache, error) {
            return mockCache, nil
        },
    }
    svc := NewService(deps)

    // Run tests
    result, err := svc.GetUser(ctx, 123)

    // Assert cache operations
    cachetest.AssertOperationCount(t, mockCache, "Get", 1)
}

Features: Fluent configuration API, operation tracking, 20+ assertion helpers, TTL expiration testing, multi-tenant support.

See also: database/testing for similar patterns.

---

Scheduler and Outbox

Job Scheduler

gocron-based job scheduling integrated with the module system. Lazy initialization, overlapping prevention (mutex per job), automatic panic recovery with stack trace logging, and system APIs (GET /_sys/jobs, POST /_sys/job/:jobId).

func (m *Module) Init(deps *app.ModuleDeps) error {
    return deps.Scheduler.DailyAt("cleanup-job", &CleanupJob{}, mustParseTime("03:00"))
}

// Implement the Executor interface
type CleanupJob struct{}
func (j *CleanupJob) Execute(ctx scheduler.JobContext) error {
    // ctx provides: JobID(), TriggerType(), Logger(), DB(), Messaging(), Config()
    return nil
}

Schedule types: FixedRate(duration), DailyAt(time), WeeklyAt(weekday, time), HourlyAt(minute), MonthlyAt(dayOfMonth, time).

Transactional Outbox

Solves the dual-write problem: events are written to an outbox table in the same database transaction as business data, then reliably delivered to the message broker by a background relay. Delivery guarantee: at-least-once (consumers must be idempotent).

func (s *OrderService) CreateOrder(ctx context.Context, req CreateOrderReq) error {
    db, _ := s.getDB(ctx)
    tx, _ := db.Begin(ctx)
    defer tx.Rollback(ctx)

    // 1. Write business data
    _, err := tx.Exec(ctx, "INSERT INTO orders (id, customer_id) VALUES ($1, $2)",
        req.ID, req.CustomerID)
    if err != nil { return err }

    // 2. Write event to outbox (SAME transaction — atomic!)
    // Publish returns the event UUID — propagated as the x-outbox-event-id
    // AMQP header so downstream consumers can deduplicate.
    payload, _ := json.Marshal(OrderCreatedEvent{OrderID: req.ID})
    eventID, err := s.outbox.Publish(ctx, tx, &app.OutboxEvent{
        EventType:   "order.created",
        AggregateID: fmt.Sprintf("order-%d", req.ID),
        Payload:     payload,
        Exchange:    "order.events",
    })
    if err != nil { return err }
    s.logger.Info().Str("event_id", eventID).Msg("order.created enqueued")

    return tx.Commit(ctx) // Event GUARANTEED to reach the broker eventually
}

The relay job polls for pending events every pollinterval (default 5s), publishes them to AMQP with an x-outbox-event-id header for deduplication, and a cleanup job removes published events after retentionperiod (default 72h). See CLAUDE.md for full configuration options.

---

KeyStore

Named RSA key pair management for encryption and signing. Keys are loaded at startup from DER files or base64-encoded environment variables.

func (m *Module) Init(deps *app.ModuleDeps) error {
    // Sign a JWT
    privateKey, err := deps.KeyStore.PrivateKey("signing")
    if err != nil { return err }

    // Verify a signature
    publicKey, err := deps.KeyStore.PublicKey("signing")
    if err != nil { return err }

    return nil
}

Configuration:

keystore:
  keys:
    signing:
      public:
        file: certs/signing_public.der            # Local dev: path to DER file
      private:
        file: certs/signing_private.der
    encryption:
      public:
        value: ${ENCRYPTION_PUBLIC_KEY_BASE64}    # Cloud/EKS: base64-encoded DER via env var
      private:
        value: ${ENCRYPTION_PRIVATE_KEY_BASE64}

Each key pair has a public and private source. For required sources, set exactly one of file: (DER path) or value: (base64-encoded DER, typically referencing an environment variable). For verification-only services, the private entry may be omitted.

See keystore/ package for full API documentation.

---

JOSE (Sign-then-Encrypt Bodies)

⚠️ Production requirement: JOSE protects request/response bodies, not the transport. Routes using JOSE middleware must still be served over HTTPS in production — TLS continues to protect URLs, headers, traffic patterns, and the trust handshake itself.

The jose package adds nested JWE-of-JWS protection on HTTP request and response bodies — designed for Visa Token Services-style integrations and any partner API that requires sign-then-encrypt outbound and decrypt-then-verify inbound on every payload. It's struct-tag opt-in: the framework discovers JOSE-protected routes at registration time, validates every kid against the keystore, and panics fast if anything is misconfigured rather than failing per-request.

type CreateTokenRequest struct {
    _   struct{} `jose:"decrypt=our-signing,verify=visa-vts-verify"`
    PAN string   `json:"pan" validate:"required"`
}

type CreateTokenResponse struct {
    _     struct{} `jose:"sign=our-signing,encrypt=visa-vts-encrypt"`
    Token string   `json:"token"`
}

Key properties:

• Bidirectional symmetry enforced — request and response must both carry tags, or neither (registration-time check).
• Strict algorithm allowlist — RS256/PS256 for signing; RSA-OAEP-256 + A256GCM for encryption. alg=none, HS*, and RSA1_5 are rejected at parse time.
• Hybrid error envelope — pre-trust failures (decrypt failed, signature invalid) emit a plaintext minimal {code,message} envelope so nothing leaks to unauthenticated peers; post-trust handler errors emit the standard APIResponse envelope, encrypted with the route's outbound policy.
• Fail-fast at startup — every kid is resolved against the keystore at RegisterHandler time. Missing keys, asymmetric tags, and WithRawResponse() conflicts panic at startup, never at runtime.
• Observability — spans (jose.decode_request, jose.encode_response), failure counter (jose.failures.total by code/direction), duration histogram (jose.operation.duration).

Wire it by registering keystore.NewModule() before any module that declares JOSE-tagged routes — app/module_registry.go then auto-injects KeyStore, logger, tracer, and meter into the middleware. See CLAUDE.md for the complete failure-mode → IAPIError table and llms.txt for end-to-end examples.

---

Multi-Tenant Implementation

Complete resource isolation with per-tenant database and messaging connections.

Key Features

• Tenant Resolution: Headers, subdomains, or custom strategies with validation
• Resource Isolation: Separate database/messaging connections per tenant
• Context Propagation: Tenant ID flows automatically through all operations
• Declaration Replay: Messaging infrastructure validated once, replayed per-tenant

Quick Setup

multitenant:
  enabled: true
  resolver:
    type: "header"
    header: "X-Tenant-ID"
  tenants:
    tenant1:
      database: { ... }
      messaging: { url: "..." }

Modules access tenant resources via deps.DB(ctx) and deps.Messaging(ctx).

Custom Integration

Implement app.TenantStore to integrate with AWS Secrets Manager, HashiCorp Vault, or custom backends.

See MULTI_TENANT.md for detailed architecture and multitenant-aws example for implementation patterns.

---

Observability and Operations

• OpenTelemetry first: native traces, metrics, and dual-mode logs.
• Structured logging via Zerolog with OTLP export for action + trace streams.
• Tracing propagates W3C traceparent headers.
• Metrics capture HTTP/messaging/database timings plus custom application metrics via deps.MeterProvider.
• Health endpoints: /health (liveness) and /ready (readiness with DB/messaging checks).
• Graceful shutdown coordinates servers, consumers, and background workers.

Configuring OpenTelemetry (Traces + Dual-Mode Logs)

1. Enable observability in configuration

   observability:
     enabled: true
     service:
       name: checkout-api
       version: 1.2.3
       environment: production
     trace:
       enabled: true
       endpoint: otel-collector:4317
       protocol: grpc        # http for OTLP/HTTP
       insecure: true        # disable TLS when talking to a collector inside the VPC
       export:
         timeout: 5s
     logs:
       enabled: true
       endpoint: otel-collector:4317
       protocol: grpc
       insecure: true
       slow_request_threshold: 750ms   # requests slower than this become WARN result_code
       export:
         timeout: 5s
       batch:
         timeout: 3s
       max:
         queue:
           size: 2048
         batch:
           size: 512
   

   • All application logs default to log.type="trace" and only WARN+ severities are exported (≈95 % volume reduction).
   • Middleware writes synthesized request summaries with log.type="action" for every healthy request; they keep INFO-level retention.

2. Initialize the OpenTelemetry provider and hook the logger

   import (
       "github.com/gaborage/go-bricks/logger"
       "github.com/gaborage/go-bricks/observability"
       "github.com/gaborage/go-bricks/server"
   )
   
   func wireLogging(cfg *config.Config, e *echo.Echo) (observability.Provider, logger.Logger, error) {
       obsProvider, err := observability.NewProvider(&cfg.Observability)
       if err != nil {
           return nil, nil, err
       }
   
       appLogger := logger.New(cfg.Log.Level, cfg.Log.Pretty).
           WithOTelProvider(obsProvider)
   
       e.Use(server.LoggerWithConfig(appLogger, server.LoggerConfig{
           HealthPath:           "/health",
           ReadyPath:            "/ready",
           SlowRequestThreshold: cfg.Observability.Logs.SlowRequestThreshold,
       }))
   
       return obsProvider, appLogger, nil
   }
   

   • The bridge automatically enriches Zerolog output with trace_id, span_id, and log.type.
   • server.LoggerWithConfig emits action logs and registers the WARN/ERROR hook so requests with elevated severity stay in the trace stream.

3. Route the two log classes in your backend

   • Grafana Loki: logql query {log.type="action"} for request summaries, {log.type="trace", trace_id="abc"} for correlated traces.
   • Datadog: create indexes @log.type:action (long retention) and @log.type:trace (short retention) to control costs.

4. Local development tip: set observability.trace.endpoint=stdout and observability.logs.endpoint=stdout to pretty-print spans and logs without running a collector.

---

Examples

Complete Working Examples

Explore the go-bricks-demo-project repository:

• http/handlers – typed HTTP handler module
• http/client – fluent HTTP client with retries and interceptors
• oracle – Oracle insert with reserved column quoting
• config-injection – custom configuration namespace demo
• trace-propagation – W3C tracing demonstration
• openapi-demo – OpenAPI specification generation
• multitenant-aws – multi-tenant app with AWS Secrets Manager

Quick Code Reference

See llms.txt for copy-paste-ready code snippets covering:

• Module system patterns
• HTTP handlers (POST, GET, PUT, DELETE) and raw response mode
• HTTP client with retries, interceptors, and W3C traces
• Database queries (SELECT, UPDATE, DELETE, JOINs, subqueries)
• Struct-based column extraction and named databases
• Messaging (AMQP declarations, publishers, consumers)
• Transactional outbox and job scheduler
• Configuration injection with struct tags
• Custom errors and middleware
• KeyStore for RSA key management
• Multi-tenant configuration
• Observability setup and custom metrics

---

Contributing

Issues and pull requests are welcome!

Getting Started:

• CONTRIBUTING.md - Coding standards, tooling, and workflow
• CLAUDE.md - Comprehensive development guide with architecture details, commands, and testing guidelines

Quick Setup:

go test ./...              # Run unit tests
make check                 # Pre-commit checks (fmt, lint, test)
make test-integration      # Integration tests (Docker required)

---

License

MIT © Contributors

---

Built with ❤️ for the Go community.