log

Log Package

The log package acts as a helpful abstraction around the slog Logger that is built into the standard library in Go.

Engineers in GitLab must use this package in order to instantiate loggers for their application to ensure that there is consistency across all of our services in how we emit logs from our systems.

Usage

// returns a new JSON logger that outputs logs to the stdout
logger := log.New()

// A standard Info line
// Note: We strongly encourage that you use the *Context methods
// for observability purposes to ensure you'll be benefiting from
// field enrichment.
logger.InfoContext(ctx, "some info")

Environment Variables

The logger can be configured via environment variables, which is the recommended approach for cloud-native deployments:

Values are case-insensitive. If an unrecognised value is set, the logger falls back to INFO and prints a warning to stderr.

# Kubernetes example
env:
  - name: GITLAB_LOG_LEVEL
    value: "WARN"

# Local debugging
GITLAB_LOG_LEVEL=DEBUG ./my-service

Programmatic Configuration

Explicit configuration takes precedence over environment variables:

level := slog.LevelError
logger := log.NewWithConfig(&log.Config{
    // log.WithWriter - allows you to pass in a custom io.Writer
    // should you wish.
    Writer: os.Stderr,
    // allows you to define the minimum logging level for said logger.
    // When set, this overrides GITLAB_LOG_LEVEL.
    LogLevel: &level,
    // allows you to set the output to text format
    UseTextFormat: true,
})

Writing to Files

If you need to write to specific files, you can achieve this like so:

logger := log.NewWithFile(filePath, &log.Config{...})
logger.Info("hello!")

This would create or append logs to a file at filePath file.

Testing Your Observability

The logs that our systems output represent an often-neglected part of our API. Additional reporting systems and alerts are typically built on top of log lines and a lack of testing makes these setups rather fragile in nature.

It's strongly encouraged that all engineers bake in some form of assertions on the logs that they rely on for additional observability configuration within their tests.

// NewWithRecorder returns a logRecorder struct that
// captures all log lines emitted by the `logger`
logger, recorder := logtest.NewWithRecorder(nil)

// We can then perform assertions on things like how many log lines
// have been emitted
assert.Len(t, recorder.Records, 1)

// As well as the shape of individual log lines
assert.Equal(t, "test message", recorder.Records[0].Message)
assert.Equal(t, tt.expectedLevel, recorder.Records[0].Level.String())
assert.Contains(t, recorder.Records[0].Attrs, slog.Attr{Key: "key", Value: slog.AnyValue("value")})

These log lines are captured in a Records field which is a slice of type testRecord:

// testRecord - a representation of a single log message
type testRecord struct {
    Level   slog.Level
    Message string
    Attrs   []slog.Attr
}

Context Logger

There is some important information we need to emit for every log message whenever a request is processed. This could be things such as the pipeline_id or the GitLab project ID.

You can store this information within the context and it will automatically enrich subsequent log emissions with these fields. This is super handy if you need to correlate log lines together by various fields for investigatory purposes.

ctx := log.WithFields(
    slog.String(fields.GitLabUserName, "1234abc")
)

logger.InfoContext(ctx, "hello world")
// emitted log line will contain the `gl_user_name` field alongside
// "hello world" - this is imported from the `fields` package.

Canonical Logger

Canonical logging is a technique that can be used to help reduce the volume of logs being emitted from your systems. You are effectively aggregating all of the fields that you care about through the lifecycle of a request and then logging a single line at the end of the lifecycle.

// pass this logger along the request lifecycle
ctx := context.Background()
ctx = log.WithFields(ctx, slog.String("step", "1"))

// at the point at which you need to emit the
// final log line which will then include all of the
// fields that have been collected within the
// context up until this point.
logger.InfoContext(ctx, "canonical_log")