ch-migrator-example

ch-migrator-example

A reference implementation of a ClickHouse schema migration tool built on goose, with OpenTelemetry collector schemas as the example migration set. Fork it, replace the bundled migrations with your own, and you have a production-ready migrator for your ClickHouse deployment.

What this does

This project provides a migration framework for ClickHouse databases that supports:

• SQL migrations for schema DDL
• Go migrations for dynamic, programmatic schema changes (e.g., altering TTL across tables matching a pattern)
• Revision targeting — migrate up to latest, or up/down to a specific revision
• Rollback support — each migration has an up and down path

The included migrations create the full OpenTelemetry schema for ClickHouse, as defined by the OTel Collector ClickHouse exporter:

• Logs — otel_logs (Map attributes) and otel_logs_json (JSON attributes)
• Traces — otel_traces, otel_traces_json, trace ID timestamp lookup table + materialized view
• Metrics — gauge, sum, histogram, exponential histogram, summary

Plus two example Go migrations demonstrating:

• Setting TTL on metrics tables by pattern
• Adding a column to a table

Quick start

Requires: Go 1.25+, Make, a running ClickHouse instance.

# Build
make build

# Run against a local ClickHouse (no TLS)
./bin/ch-migrator-example \
  --dbAddr localhost \
  --dbPort 9000 \
  --enableTLS=false \
  --dbUsername default \
  --dbPassword ''

Configuration

Configuration via CLI flags, environment variables, or YAML config file (--config path/to/config.yaml).

Using as a library

The migration package can be imported directly:

import migrations "github.com/ClickHouse/ch-migrator-example/pkg"

Note: the directory is named pkg but the Go package is migrations, so an import alias is needed. The key exports are:

• migrations.Config — migration configuration struct
• migrations.InitiateMigrations(db, config, fs) — run migrations
• migrations.EmbeddedMigrations — the bundled OTel migration files
• migrations.RevisionLatest — sentinel for "migrate to latest"

Adding your own migrations

See pkg/migrations/README.md for details on creating SQL and Go migrations.

Testing

# Unit tests
make test

# Integration tests (requires Docker — uses testcontainers-go)
make integration-test