log

Log Package

The log package acts as a helpful abstraction around the slog Logger that is built into the standard library in Go.

Engineers in GitLab must use this package in order to instantiate loggers for their application to ensure that there is consistency across all of our services in how we emit logs from our systems.

Usage

// returns a new JSON logger that outputs logs to the stdout
logger := log.New()

// A standard Info line
// Note: We strongly encourage that you use the *Context methods
// for observability purposes to ensure you'll be benefiting from
// field enrichment.
logger.InfoContext(ctx, "some info")

Environment variables

The logger can be configured via environment variables, which is the recommended approach for cloud-native deployments.

Values are case-insensitive. If you set an unrecognised value, the logger prints an error message to stderr and falls back to the default.

For example, in a Kubernetes deployment you might set:

env:
  - name: GITLAB_LOG_FORMAT
    value: json
  - name: GITLAB_LOG_LEVEL
    value: WARN

Or locally, when debugging:

GITLAB_LOG_FORMAT=text GITLAB_LOG_LEVEL=DEBUG ./my-service

Programmatic configuration

If you need finer-grained control, you can configure the logger in code. Values set here take precedence over the corresponding environment variables.

level := slog.LevelError
logger := log.NewWithConfig(&log.Config{
    // log.WithWriter - allows you to pass in a custom io.Writer
    // should you wish.
    Writer: os.Stderr,
    // allows you to define the minimum logging level for said logger.
    // When set, this overrides GITLAB_LOG_LEVEL.
    LogLevel: &level,
    // allows you to set the output to text format
    UseTextFormat: true,
})

Timestamp behavior

By default, all timestamps emitted by the logger are in UTC to ensure consistency across GitLab services running in different timezones. This is a deliberate design choice that makes correlation and analysis of logs across distributed systems straightforward.

// Timestamps are automatically in UTC
logger := log.New()
logger.Info("message")  // timestamp will be in UTC, e.g., "2026-04-13T11:15:30Z"

If your service needs timestamps in a specific timezone, you can configure it via environment variable or programmatic config:

Via environment variable (recommended for cloud-native deployments):

# Valid IANA timezone names: UTC, America/New_York, Europe/London, Asia/Tokyo, etc.
export GITLAB_LOG_TIMEZONE=America/New_York

Via programmatic configuration:

ny, _ := time.LoadLocation("America/New_York")
logger := log.NewWithConfig(&log.Config{
    Location: ny,
})
logger.Info("message")  // timestamp will be in New York timezone

If you need to use a custom clock (for testing or special use cases), you can provide one via Config.Clock:

// Custom clock for testing
logger := log.NewWithConfig(&log.Config{
    Clock: func() time.Time {
        return time.Date(2026, 4, 13, 12, 0, 0, 0, time.UTC)
    },
})

Note: When using a custom clock with a specific timezone:

ny, _ := time.LoadLocation("America/New_York")
logger := log.NewWithConfig(&log.Config{
    Clock: func() time.Time {
        return time.Now()  // Clock can return any time
    },
    Location: ny,  // Timestamp will be converted to this timezone
})

Writing to Files

If you need to write to specific files, you can achieve this like so:

logger, logCloser, err := log.NewWithFile(filePath, &log.Config{...})
if err != nil {
    // When an error occurs while opening a filePath NewWithFile() will return
    // an stderr logger alongside an error.
    // You can choose how to proceed, here we return the error.
    return err
}

// You are responsible for closing the log file.
defer logCloser.Close()

logger.Info("hello!")

This would create or append logs to a file at filePath file.

Testing Your Observability

The logs that our systems output represent an often-neglected part of our API. Additional reporting systems and alerts are typically built on top of log lines and a lack of testing makes these setups rather fragile in nature.

It's strongly encouraged that all engineers bake in some form of assertions on the logs that they rely on for additional observability configuration within their tests.

// NewWithRecorder returns a logRecorder struct that
// captures all log lines emitted by the `logger`
logger, recorder := logtest.NewWithRecorder(nil)

// We can then perform assertions on things like how many log lines
// have been emitted
assert.Len(t, recorder.Records, 1)

// As well as the shape of individual log lines
assert.Equal(t, "test message", recorder.Records[0].Message)
assert.Equal(t, tt.expectedLevel, recorder.Records[0].Level.String())
assert.Contains(t, recorder.Records[0].Attrs, slog.Attr{Key: "key", Value: slog.AnyValue("value")})

These log lines are captured in a Records field which is a slice of type testRecord:

// testRecord - a representation of a single log message
type testRecord struct {
    Level   slog.Level
    Message string
    Attrs   []slog.Attr
}

Structured Logging with Fields

To emit important information with every log message, use the logger's .With() method to attach attributes. This is the canonical pattern for request-scoped enrichment throughout your request lifecycle.

logger := log.New()

// Create a logger enriched with request-level fields
enriched := logger.With(
    slog.String(fields.GitLabUserName, "1234abc"),
    slog.String(fields.CorrelationID, "req-xyz"),
)

// All logs from this logger will include these fields
enriched.Info("processing request")
enriched.Info("completed request")
// Both lines will include gl_user_name and correlation_id

Request Lifecycle Pattern

For services that handle requests, create an enriched logger at the request boundary and pass it through your execution flow:

// HTTP handler - request boundary
func handleRequest(w http.ResponseWriter, r *http.Request) {
    baseLogger := log.New()

    // Create a logger enriched with request-level fields
    requestLogger := baseLogger.With(
        slog.String("request_id", r.Header.Get("X-Request-ID")),
        slog.String("user_id", extractUserID(r)),
    )

    // Pass the enriched logger through your call stack
    if err := handleBusiness(requestLogger); err != nil {
        requestLogger.Error("request failed", slog.Any("error", err))
    }
    requestLogger.Info("request completed")
}

// Business logic - enrich further if needed
func handleBusiness(logger *slog.Logger) error {
    // Add business-specific fields by creating a new logger from the passed logger
    opLogger := logger.With(slog.String("operation", "process_order"))
    opLogger.Info("starting operation")

    return processOrder(opLogger)
}

// Data access layer - enrich further if needed
func processOrder(logger *slog.Logger) error {
    // Add data-specific fields
    dbLogger := logger.With(slog.String("database", "orders_db"))

    // All fields from the entire chain are included:
    // request_id, user_id, operation, and database
    dbLogger.Info("executing query")
    return nil
}

In this pattern:

• Request boundary: Create an enriched logger with request-level fields
• Business logic: Add operation-specific fields by calling .With() on the passed logger
• Data layer: Add implementation details by calling .With() again
• Final logs: All accumulated fields flow through the entire call stack

Storing Loggers in Context

When you need to store and retrieve a logger from context (e.g., in middleware), use WithLogger() and FromContext():

ctx := context.Background()

// Store the logger in context
ctx = log.WithLogger(ctx, logger)

// Retrieve it later (panics if not present)
logger := log.FromContext(ctx)

Performance

The v2 log package is a significant performance improvement over the v1 Logrus-based approach.

v2 vs v1 performance comparison

The migration from Logrus (v1) to slog (v2) provides substantial performance improvements across all logging scenarios:

Memory allocation is also significantly better:

Real-world impact

For a service making 10,000 logs/second with average 3 attributes per log:

• Logrus v1: 39.6 seconds of CPU per hour on logging
• LabKit v2: 23.4 seconds per hour (41% reduction)

On an 8-core server, this saves approximately 3 minutes per hour of CPU time.

See the description of !427 for real-world impact analysis with concrete GitLab service examples.

Best practices

To get the best performance from the v2 logger:

1. Create enriched loggers at request boundaries: When handling requests, create an enriched logger once with request-level fields and pass it through your call stack:

   // ✓ GOOD - created once per request
   requestLogger := logger.With(
       slog.String("request_id", reqID),
       slog.String("user_id", userID),
   )
   // Pass requestLogger through your handlers
   

2. Reuse loggers for service-level attributes: When you have attributes that don't change (service name, environment), create a logger once at startup:

   // ✓ GOOD - created once at startup, reused for all requests
   serviceLogger := logger.With(
       slog.String("service", "my-service"),
       slog.String("env", os.Getenv("ENVIRONMENT")),
   )
   // Enrich further per-request
   requestLogger := serviceLogger.With(slog.String("request_id", reqID))
   

3. Chain .With() calls for nested contexts: When adding fields at different layers, chain .With() calls on the logger passed to you:

   // Handler passes a logger
   // Business layer enriches it
   opLogger := logger.With(slog.String("operation", "checkout"))
   // Data layer enriches it further
   dbLogger := opLogger.With(slog.String("table", "orders"))
   // All fields accumulate in the final logger