config

LabKit v2 Config

Protobuf-first configuration management for GitLab Go services.

Overview

The config package provides a standardized way to load, validate, and migrate configuration files using Protocol Buffers as the single source of truth for configuration schemas. Validation rules are defined using protovalidate constraints directly in .proto files.

Features

• Format autodetection: Automatically detects YAML, JSON, or TOML by file extension
• Strong validation: Uses protovalidate for rich constraint validation (ranges, patterns, cross-field rules)
• Rollback safety: Unknown fields silently ignored by default (forwards-compatible)
• Version migrations: Automatic migration from version N-1 to N with pre/post-validation
• Excellent error messages: File, line, column information for parse errors (YAML)
• Extensible: Add custom format parsers via simple interface

Installation

# Install buf (if not using mise/asdf)
# See: https://buf.build/docs/installation

# Or with mise/asdf (recommended)
mise install

Quick Start

1. Define your configuration proto

// proto/config/v1/config.proto
edition = "2023";
package myapp.config.v1;

option go_package = "myapp/gen/config/v1;configv1";
option features.field_presence = IMPLICIT;

import "buf/validate/validate.proto";

message Config {
  int32 version = 1;
  ServerConfig server = 2 [(buf.validate.field).required = true];
}

message ServerConfig {
  string host = 1 [(buf.validate.field).string.min_bytes = 1];
  uint32 port = 2 [(buf.validate.field).uint32 = {
    gte: 1
    lte: 65535
  }];
}

2. Generate Go code

# Using buf directly (from v2/config directory)
cd v2/config
buf generate

# Or use the provided script for config module + examples (from repo root)
./scripts/generate-config-protos.sh

3. Load and validate configuration

package main

import (
    "context"
    "os"

    "gitlab.com/gitlab-org/labkit/v2/config"
    "gitlab.com/gitlab-org/labkit/v2/log"
    configv1 "myapp/gen/config/v1"
)

func main() {
    // Initialize structured logger
    logger := log.New()
    ctx := context.Background()

    loader, err := config.New()
    if err != nil {
        logger.ErrorContext(ctx, "Failed to create config loader", "error", err)
        os.Exit(1)
    }

    var cfg configv1.Config
    if err := loader.Load("config.yaml", &cfg); err != nil {
        logger.ErrorContext(ctx, "Failed to load config", "error", err, "path", "config.yaml")
        os.Exit(1)
    }

    // cfg is now loaded and validated - emit structured log
    logger.InfoContext(ctx, "Config loaded successfully",
        "server_host", cfg.Server.Host,
        "server_port", cfg.Server.Port,
        "version", cfg.Version)
}

Format Support

Always Available

• YAML (.yaml, .yml) - Recommended for human-authored configs
• JSON (.json) - Useful for machine-generated configs

Opt-in

• TOML (.toml) - Add via WithParser(toml.NewTOMLParser())

import (
    "gitlab.com/gitlab-org/labkit/v2/config"
    "gitlab.com/gitlab-org/labkit/v2/config/toml"
)

loader, _ := config.New(
    config.WithParser(toml.NewTOMLParser()),
)

Validation

Validation uses protovalidate constraints defined inline in proto files.

Standard Constraints

message ServerConfig {
  string host = 1 [(buf.validate.field).string = {
    min_bytes: 1
    max_bytes: 255
  }];

  uint32 port = 2 [(buf.validate.field).uint32 = {
    gte: 1
    lte: 65535
  }];

  repeated string tags = 3 [(buf.validate.field).repeated.min_items = 1];
}

Cross-Field Validation with CEL

message TLSConfig {
  string cert_path = 1;
  string key_path = 2;

  option (buf.validate.message).cel = {
    id: "tls_pair"
    message: "cert_path and key_path must both be set or both be empty"
    expression: "(this.cert_path == '') == (this.key_path == '')"
  };
}

Migrations

Configure version migrations to automatically upgrade configs from version N-1 to N.

1. Define versioned protos

v1 config:

message Config {
  int32 version = 1;
  ServerConfig server = 2;
}

message ServerConfig {
  string host = 1;
  uint32 port = 2;
}

v2 config (renamed field):

message Config {
  int32 version = 1;
  ServerConfig server = 2;
}

message ServerConfig {
  string address = 1;  // renamed from "host"
  uint32 port = 2;
}

2. Write typed migration function

func migrateV1ToV2(source *configv1.Config) (*configv2.Config, error) {
    return &configv2.Config{
        Version: 2,
        Server: &configv2.ServerConfig{
            Address: source.Server.Host, // Renamed field
            Port:    source.Server.Port,
        },
    }, nil
}

3. Register migration

import (
    "context"
    "os"
    "gitlab.com/gitlab-org/labkit/v2/log"
)

logger := log.New()
ctx := context.Background()

loader, err := config.New(
    config.WithMigration(migrateV1ToV2),
)
if err != nil {
    logger.ErrorContext(ctx, "Failed to create loader with migration", "error", err)
    os.Exit(1)
}

var cfg configv2.Config
// If config file is v1, migration runs automatically
if err := loader.Load("config.yaml", &cfg); err != nil {
    logger.ErrorContext(ctx, "Failed to load config", "error", err)
    os.Exit(1)
}

logger.InfoContext(ctx, "Config loaded with migration",
    "version", cfg.Version,
    "migrated", cfg.Version > 1)

Migration Flow

1. Parse config file into target proto (v2) to detect version
2. Detect version mismatch (e.g., file has v1, proto expects v2)
3. Re-parse config file into source type (v1)
4. Pre-migration validation: Validate against v1 schema
5. Run typed migration function to transform v1 → v2
6. Post-migration validation: Validate against v2 schema

The migration function receives fully-typed, validated v1 config and returns v2 config. This provides type safety and makes migrations easier to write and test.

Strict Mode

By default, unknown fields are silently ignored (rollback-safe). Enable strict mode to catch typos:

loader, _ := config.New(config.WithStrictMode())

When to use strict mode

✓ Use in:

• CI pipelines validating config examples
• Pre-deployment validation
• Development environments

✗ Don't use in:

• Production deployments (breaks rollback safety)
• Canary deployments (version skew)

Why rollback safety matters

If you deploy v2 with a new config field, then roll back to v1, the v1 binary will encounter an unknown field. With strict mode enabled, it crashes. With strict mode disabled (default), it ignores the field and continues.

Custom Parsers

Add support for custom formats by implementing the Parser interface:

type Parser interface {
    Extensions() []string
    Unmarshal(data []byte, path string, msg proto.Message, strict bool) error
}

Example: XML parser

type xmlParser struct{}

func (p *xmlParser) Extensions() []string {
    return []string{".xml"}
}

func (p *xmlParser) Unmarshal(
    data []byte, path string, msg proto.Message, strict bool,
) error {
    // Parse XML, convert to proto message
    return nil
}

// Register the parser
loader, _ := config.New(
    config.WithParser(&xmlParser{}),
)

API Reference

Constructors

// Create loader with default settings (YAML + JSON parsers)
func New(opts ...Option) (*Loader, error)

Options

// Add custom format parser
func WithParser(p Parser) Option

// Enable strict mode (reject unknown fields)
func WithStrictMode() Option

// Register migration from v(N-1) to v(N)
func WithMigration(fn MigrationFunc, oldProto proto.Message) Option

Methods

// Load config from file (format auto-detected)
func (l *Loader) Load(path string, msg proto.Message) error

// Load config from bytes with explicit format
func (l *Loader) LoadBytes(data []byte, ext string, msg proto.Message) error

Helpers

// Extract version field from proto message
func GetVersion(msg proto.Message) (int, error)

Best Practices

1. Use YAML for human-authored configs - Better readability, comments supported
2. Use YAML or JSON for machine-generated configs - Simpler, no ambiguity
3. Keep migrations simple - Only support N-1 → N, not arbitrary jumps
4. Use protovalidate for all constraints - Single source of truth
5. Test config examples in CI with strict mode - Catch typos early
6. Never use strict mode in production - Breaks rollback safety

Example

See go-service-template for a complete working example demonstrating config management in a production service.

Related Documentation

• protovalidate documentation
• Protocol Buffers
• CEL (Common Expression Language)