metrics

Metrics Package

A pluggable metrics collection system with Prometheus implementation.

Quick Start

import "github.com/bitechdev/ResolveSpec/pkg/metrics"

// Initialize Prometheus provider with default config
provider := metrics.NewPrometheusProvider(nil)
metrics.SetProvider(provider)

// Apply middleware to your router
router.Use(provider.Middleware)

// Expose metrics endpoint
http.Handle("/metrics", provider.Handler())

Configuration

You can customize the metrics provider using a configuration struct:

import "github.com/bitechdev/ResolveSpec/pkg/metrics"

// Create custom configuration
config := &metrics.Config{
    Enabled:  true,
    Provider: "prometheus",
    Namespace: "myapp", // Prefix all metrics with "myapp_"
    HTTPRequestBuckets: []float64{0.01, 0.05, 0.1, 0.5, 1, 2, 5},
    DBQueryBuckets: []float64{0.001, 0.01, 0.05, 0.1, 0.5, 1},
}

// Initialize with custom config
provider := metrics.NewPrometheusProvider(config)
metrics.SetProvider(provider)

Configuration Options

HTTP path label: the middleware uses, in order: HTTPPathNormalizer, the matched http.ServeMux pattern (r.Pattern, e.g. /users/{id}), then the raw path with numeric/UUID/hex/opaque-token segments replaced by :id. For routers other than ServeMux, supply HTTPPathNormalizer with your route template. The HTTPMaxPaths cap applies on top.

Default HTTP Request Buckets: [0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10]

Default DB Query Buckets: [0.001, 0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5]

Enabled flag and JSON pull

Config.Enabled is honoured: a disabled provider records nothing, Middleware passes requests straight through, Handler()/JSONHandler() answer 404, and push loops are not started (manual pushes return an error). Note a &metrics.Config{} literal has Enabled: false; use DefaultConfig() or set Enabled: true. NewPrometheusProvider(nil) is enabled.

provider.JSONHandler() serves the same JSON as the push json format on GET/HEAD:

http.Handle("/metrics", provider.Handler())          // Prometheus text
http.Handle("/metrics.json", provider.JSONHandler()) // JSON

Resetting Stats

• provider.Reset() clears counters, histograms and the cache-size gauge (live gauges such as in-flight requests are kept). Package-level metrics.Reset() does the same for the current provider if it implements metrics.Resetter.
• provider.PushAndReset() pushes to the Pushgateway and resets only if the push succeeded (errors if no Pushgateway is configured).
• Config.PushgatewayResetOnPush: true makes the automatic push loop do this on every tick.
• provider.ResetHandler() is a POST-only endpoint (?push=true to push first). It has no auth: mount it on an internal route.

http.Handle("/metrics/reset", provider.ResetHandler())

Note: the normal /metrics scrape is read-only and never clears anything. Observations recorded between a push and its reset are lost. Prometheus handles the counter drop as a reset, but if you reset often, prefer increase()/rate() over raw counter values.

Custom Push Endpoint (Optional)

POST metrics to your own server, optionally clearing local stats after a 2xx reply:

provider := metrics.NewPrometheusProvider(&metrics.Config{
    PushEndpointURL:            "https://collector.example.com/metrics",
    PushEndpointFormat:         "json", // or "text" (Prometheus exposition, default)
    PushEndpointHeaders:        map[string]string{"Authorization": "Bearer token"},
    PushEndpointInterval:       30,     // seconds; 0 = manual only
    PushEndpointTimeout:        10,     // seconds (default 10)
    PushEndpointResetOnSuccess: true,   // clear local stats after a 2xx
})

err := provider.PushToEndpoint(ctx) // manual push; also honours ResetOnSuccess
provider.StopAutoPush()             // stops the Pushgateway and endpoint loops

The json body is a list of {name, help, type, metrics:[{labels, value | count, sum, buckets}]}. Failures (non-2xx, network, timeout) are logged and never reset stats, so the next tick retries with the accumulated data. The payload covers everything in the default Prometheus registry, including Go runtime metrics.

Pushgateway Configuration (Optional)

For batch jobs, cron tasks, or short-lived processes, you can push metrics to Prometheus Pushgateway:

config := &metrics.Config{
    PushgatewayURL:      "http://pushgateway:9091",
    PushgatewayJobName:  "batch-job",
    PushgatewayInterval: 30, // Push every 30 seconds
}

Provider Interface

The package uses a provider interface, allowing you to plug in different metric systems:

type Provider interface {
    RecordHTTPRequest(method, path, status string, duration time.Duration)
    IncRequestsInFlight()
    DecRequestsInFlight()
    RecordDBQuery(operation, table string, duration time.Duration, err error)
    RecordCacheHit(provider string)
    RecordCacheMiss(provider string)
    UpdateCacheSize(provider string, size int64)
    Handler() http.Handler
}

Recording Metrics

HTTP Metrics (Automatic)

When using the middleware, HTTP metrics are recorded automatically:

router.Use(provider.Middleware)

Collected:

• Request duration (histogram)
• Request count by method, path, and status
• Requests in flight (gauge)

Database Metrics

start := time.Now()
rows, err := db.Query("SELECT * FROM users WHERE id = ?", userID)
duration := time.Since(start)

metrics.GetProvider().RecordDBQuery("SELECT", "users", duration, err)

Cache Metrics

// Record cache hit
metrics.GetProvider().RecordCacheHit("memory")

// Record cache miss
metrics.GetProvider().RecordCacheMiss("memory")

// Update cache size
metrics.GetProvider().UpdateCacheSize("memory", 1024)

Prometheus Metrics

When using PrometheusProvider, the following metrics are available:

Note: If a custom Namespace is configured, all metric names will be prefixed with {namespace}_.

Prometheus Queries

HTTP Request Rate

rate(http_requests_total[5m])

HTTP Request Duration (95th percentile)

histogram_quantile(0.95, rate(http_request_duration_seconds_bucket[5m]))

Database Query Error Rate

rate(db_queries_total{status="error"}[5m])

Cache Hit Rate

rate(cache_hits_total[5m]) / (rate(cache_hits_total[5m]) + rate(cache_misses_total[5m]))

No-Op Provider

If metrics are disabled:

// No provider set - uses no-op provider automatically
metrics.GetProvider().RecordHTTPRequest(...) // Does nothing

Custom Provider

Implement your own metrics provider:

type CustomProvider struct{}

func (c *CustomProvider) RecordHTTPRequest(method, path, status string, duration time.Duration) {
    // Send to your metrics system
}

// Implement other Provider interface methods...

func (c *CustomProvider) Handler() http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        // Return your metrics format
    })
}

// Use it
metrics.SetProvider(&CustomProvider{})

Pushgateway Usage

Automatic Push (Batch Jobs)

For jobs that run periodically, use automatic pushing:

package main

import (
    "time"

    "github.com/bitechdev/ResolveSpec/pkg/metrics"
)

func main() {
    // Configure with automatic pushing every 30 seconds
    config := &metrics.Config{
        Enabled:             true,
        Provider:            "prometheus",
        Namespace:           "batch_job",
        PushgatewayURL:      "http://pushgateway:9091",
        PushgatewayJobName:  "data-processor",
        PushgatewayInterval: 30, // Push every 30 seconds
    }

    provider := metrics.NewPrometheusProvider(config)
    metrics.SetProvider(provider)

    // Ensure cleanup on exit
    defer provider.StopAutoPush()

    // Your batch job logic here
    processBatchData()
}

Manual Push (Short-lived Processes)

For one-time jobs or when you want manual control:

package main

import (
    "log"

    "github.com/bitechdev/ResolveSpec/pkg/metrics"
)

func main() {
    // Configure without automatic pushing
    config := &metrics.Config{
        Enabled:            true,
        Provider:           "prometheus",
        PushgatewayURL:     "http://pushgateway:9091",
        PushgatewayJobName: "migration-job",
        // PushgatewayInterval: 0 (default - no auto-push)
    }

    provider := metrics.NewPrometheusProvider(config)
    metrics.SetProvider(provider)

    // Run your job
    err := runMigration()

    // Push metrics at the end
    if pushErr := provider.Push(); pushErr != nil {
        log.Printf("Failed to push metrics: %v", pushErr)
    }

    if err != nil {
        log.Fatal(err)
    }
}

Docker Compose with Pushgateway

version: '3'
services:
  batch-job:
    build: .
    environment:
      PUSHGATEWAY_URL: "http://pushgateway:9091"

  pushgateway:
    image: prom/pushgateway
    ports:
      - "9091:9091"

  prometheus:
    image: prom/prometheus
    ports:
      - "9090:9090"
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml
    command:
      - '--config.file=/etc/prometheus/prometheus.yml'

prometheus.yml for Pushgateway:

global:
  scrape_interval: 15s

scrape_configs:
  # Scrape the pushgateway
  - job_name: 'pushgateway'
    honor_labels: true  # Important: preserve job labels from pushed metrics
    static_configs:
      - targets: ['pushgateway:9091']

Complete Example

Basic Usage

package main

import (
    "database/sql"
    "log"
    "net/http"
    "time"

    "github.com/bitechdev/ResolveSpec/pkg/metrics"
    "github.com/gorilla/mux"
)

func main() {
    // Initialize metrics with default config
    provider := metrics.NewPrometheusProvider(nil)
    metrics.SetProvider(provider)

    // Create router
    router := mux.NewRouter()

    // Apply metrics middleware
    router.Use(provider.Middleware)

    // Expose metrics endpoint
    router.Handle("/metrics", provider.Handler())

    // Your API routes
    router.HandleFunc("/api/users", getUsersHandler)

    log.Fatal(http.ListenAndServe(":8080", router))
}

func getUsersHandler(w http.ResponseWriter, r *http.Request) {
    // Record database query
    start := time.Now()
    users, err := fetchUsers()
    duration := time.Since(start)

    metrics.GetProvider().RecordDBQuery("SELECT", "users", duration, err)

    if err != nil {
        http.Error(w, "Internal Server Error", 500)
        return
    }

    // Return users...
}

With Custom Configuration

package main

import (
    "log"
    "net/http"

    "github.com/bitechdev/ResolveSpec/pkg/metrics"
    "github.com/gorilla/mux"
)

func main() {
    // Custom metrics configuration
    metricsConfig := &metrics.Config{
        Enabled:  true,
        Provider: "prometheus",
        Namespace: "myapp",
        // Custom buckets optimized for your application
        HTTPRequestBuckets: []float64{0.01, 0.05, 0.1, 0.5, 1, 2, 5, 10},
        DBQueryBuckets: []float64{0.001, 0.005, 0.01, 0.05, 0.1, 0.5, 1},
    }

    // Initialize with custom config
    provider := metrics.NewPrometheusProvider(metricsConfig)
    metrics.SetProvider(provider)

    router := mux.NewRouter()
    router.Use(provider.Middleware)
    router.Handle("/metrics", provider.Handler())

    log.Fatal(http.ListenAndServe(":8080", router))
}

Docker Compose Example

version: '3'
services:
  app:
    build: .
    ports:
      - "8080:8080"

  prometheus:
    image: prom/prometheus
    ports:
      - "9090:9090"
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml
    command:
      - '--config.file=/etc/prometheus/prometheus.yml'

  grafana:
    image: grafana/grafana
    ports:
      - "3000:3000"
    depends_on:
      - prometheus

prometheus.yml:

global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'resolvespec'
    static_configs:
      - targets: ['app:8080']

Best Practices

1. Label Cardinality: Keep labels low-cardinality

   • ✅ Good: method, status_code
   • ❌ Bad: user_id, timestamp

2. Path Normalization: Done automatically for the path label (see Configuration Options)

   // /api/users/123 is recorded as /api/users/:id
   

3. Metric Naming: Follow Prometheus conventions

   • Use _total suffix for counters
   • Use _seconds suffix for durations
   • Use base units (seconds, not milliseconds)

4. Performance: Metrics collection is lock-free and highly performant

   • Safe for high-throughput applications
   • Minimal overhead (<1% in most cases)

5. Pull vs Push:

   • Use Pull (default): Long-running services, web servers, microservices
   • Use Push (Pushgateway): Batch jobs, cron tasks, short-lived processes, serverless functions
   • Pull is preferred for most applications as it allows Prometheus to detect if your service is down