Central-Config-Stream

central-config

A control plane for microservice configuration: feature flags, per-service appsettings, and localization bundles for a fleet of services.

A relational database is the source of truth. Every admin write is also written through to NATS JetStream KV, which consuming services watch and cache in memory. Consumers never call this service on the request path and never query the database — a config read in a consumer is a field access.

The repository is Central-Config-Stream; the project itself is central-config — the Go module path, the binary, the compose service and the default SERVICE_NAME. Only the repository name differs.

One concept wears several names. The per-service JSON tree is appsettings in prose, /configs on the API, MICROCONFIG in KV, microconfig in the Go packages, MicroSettings() on the client, and SETTINGS_JSON in the database. They are all the same thing.

The problem

Fleet-wide configuration has to reach running processes quickly, without every service polling a database. Polling gives you a choice between stale config and a database that carries the read load of the whole fleet. Pushing from the admin API to each service directly means the admin API has to know who is running.

The split here is:

• Control plane — this service. An HTTP admin API backed by PostgreSQL. Admins write here. Consumers never call it.
• Data plane — NATS JetStream KV. Consumers watch it and serve every read from an in-memory cache.

If the control plane or the database is down, consumers keep running on cached values. If NATS is down after a consumer has started, values freeze at last known good rather than erroring.

Requirements

SQLite is the local test stack only. DB_DRIVER=sqlite swaps in a parallel set of repositories so the JetStream path can be exercised on a laptop without a PostgreSQL instance. The schema is the same shape and the SQL differs only in bind-parameter syntax, but running that stack does not exercise the PostgreSQL repositories at all — those are covered by their own suite, internal/pgintegration, which CI runs against a real server. See Known limitations.

Quickstart

The local stack runs the whole path on one machine: an admin write hits the HTTP API, lands in the database, is written through to KV, and is pushed to a consumer that serves it from memory.

docker compose -f deploy/compose/docker-compose.yml up --build
# console: http://localhost:8090   admin API: :8080   NATS: nats://localhost:4222

Compose builds the React console inside the image, so nothing else is needed. The stack seeds everything into environment 1 (dev) — environments 2 (staging) and 3 (prod) exist as rows but hold almost nothing. Every example below is environment 1.

Read the seeded flag values. Reads need a bearer token now; the stack's is local-dev-token:

curl -s -H "Authorization: Bearer local-dev-token" \
  "http://localhost:8080/flags/values?environmentId=1"

[{"id":100,"environmentId":1,"flagId":7,"flagKey":"search_v2","value":"on","enabled":1,"updatedAt":"..."},
 {"id":101,"environmentId":1,"flagId":8,"flagKey":"dark_mode","value":"off","enabled":0,"updatedAt":"..."},
 {"id":102,"environmentId":1,"flagId":9,"flagKey":"new_pricing","value":"0.25","enabled":1,"updatedAt":"..."}]

Now turn dark_mode on. id is the flag-value row id from the listing above:

curl -s -X PUT http://localhost:8080/flags/values \
  -H "Authorization: Bearer local-dev-token" \
  -H "Content-Type: application/json" \
  -d '{"id":101,"enabled":1,"value":"on"}'

{"id":101,"environmentId":1,"flagId":8,"value":"on","enabled":1,"updatedAt":"2026-08-05T23:29:01Z"}

That request wrote the database (SQLite here), published KV key FLAGS/1.dark_mode, and pushed it to every watching consumer. The console's own consumer has it already:

curl -s http://localhost:8090/api/state | jq '.flags.dark_mode'

{"enabled": true, "value": "on", "updatedAt": "2026-08-05T23:29:01Z"}

Note "enabled": 1 going in and "enabled": true coming out — see the int/bool trap.

deploy/compose/README.md is the full walkthrough: the seeded data table, four things worth trying, and a two-terminal setup that needs no Docker at all.

Install

Nothing is tagged yet. There are no releases and no published image, so the source paths below are the only ones that work today — a ghcr.io pull or a release-page download 404s, and will keep doing so until the first v* tag is cut:

# from source — the module path carries the repository name, the binary does not
go install github.com/ErasedKyte/Central-Config-Stream/cmd/central-config@latest

# or from a checkout
go build ./cmd/central-config && ./central-config --version

Either way the binary honestly says dev (commit none, built unknown): neither go install nor a plain go build passes -ldflags, so nothing is stamped in. Use it for a quick look, not for telling two deployments apart — make build stamps the real values.

Once the first tag exists, .github/workflows/release.yml publishes a distroless, non-root image for linux/amd64 and linux/arm64 to ghcr.io/erasedkyte/central-config and cross-compiled .tar.gz binaries with a checksums.txt to the releases page, both stamped with a real version, because the pipeline passes the -ldflags a laptop build does not. docs/RELEASING.md is the other side of this: how a tag is cut and how to check one landed.

That image will be the service alone; it serves no browser UI. The console is a separate binary, and its React bundle is not in the archives — web/ is gitignored and building it needs Node — so an unpacked testconsole answers /api/state and /api/events but tells the browser the UI has not been built. Getting the console with its UI means the compose stack above, which builds the bundle in a Node stage, or cd webui && npm ci && npm run build beside the binary.

The whole repository, console included:

go build ./... && go vet ./... && go test ./...
cd webui && npm ci && npm run build   # React console, writes ../web

The console

webui/ + cmd/testconsole is a React console that plays a consuming microservice. It holds a live configclient cache warmed from KV and serves a browser UI showing that cache change as you write. It is the only showable artefact in an otherwise headless project, and it demonstrates things the API alone cannot:

• Seven views. Overview (health, drift, recent pushes), Audit log, System (health, metrics, inventory), Flags (definitions and an environment × flag matrix), Appsettings, Localization, and Environments & services.
• An environment switcher in the header that narrows every view to one environment, or shows all of them.
• Live SSE push. GET /api/events streams every KV update the consumer receives; the right-hand panel fills in as writes land, with no polling.
• A drift panel. The one place anything compares the database against what a consumer is actually serving. It reads the admin API and the consumer's cache and reports each disagreement — a row the consumer never received, a cached key with no row behind it, a value that differs. Rows outside the watched environment are counted as out of scope, never as drift.
• A measured propagation latency. A write is marked before the request goes out, and the KV push that follows is timestamped on arrival; the header shows last push +NNms and each event carries its own figure. It is the number that makes "write-through to KV" concrete.

The console proxies its admin calls so the browser never holds the token. It binds to 127.0.0.1 by default, and it warns loudly when it is not on loopback, because it attaches a full-scope admin token to everything it forwards. PORT is read against BIND_ADDR: both 8090 and :8090 bind loopback with BIND_ADDR unset, and only a PORT that names a host — 0.0.0.0:8090, which is what the compose stack sets so its published port is reachable — widens the bind on its own. The embedded NATS server follows the same variable.

It then checks the Host header against an allowlist before it looks at Origin, because Origin-versus-Host alone is defeated by DNS rebinding: a page on evil.example whose DNS answer is 127.0.0.1 sets both to itself and they agree. Only Host gives it away, since a browser fills that in from the address it was pointed at. Loopback is always accepted under all of its names; so is the address the console is bound to, unless that is a wildcard, which names no host. ALLOWED_HOSTS is where any other name is said out loud — so reaching the console from another machine needs both a wider bind and that name listed, or it answers 403. /api/state and /api/events are behind the same guard, because they hand over the consumer's whole cache.

Beyond that it forwards only application/json to a named list of API paths, and — when web/ has not been built — serves a page telling you to build it instead of a bare 404.

How it works

flowchart LR
    admin[Admin / deploy script] -->|HTTP PUT| cc

    subgraph proc["central-config (one process)"]
        cc[admin API]
        rec[reconciler]
    end

    cc -->|1. write, source of truth| pg[(PostgreSQL)]
    cc -->|2. write-through| kv[(JetStream KV<br/>FLAGS · MICROCONFIG · LOCALIZATION)]
    rec -.->|periodic resync, heals drift| kv
    pg -.-> rec
    kv -->|watch| s1[Service A<br/>in-memory cache]
    kv -->|watch| s2[Service B<br/>in-memory cache]
    kv -->|watch| s3[Service C<br/>in-memory cache]

The reconciler is not a separate deployable — it runs in-process, started by the same binary when PUBLISH_ENABLED=true.

The dual write is not transactional. PostgreSQL is written first and the KV write follows; a KV failure is logged and does not fail the admin request. The reconciler sweeps the database against KV on RECONCILE_INTERVAL and republishes anything missing or stale, so drift is bounded by that interval rather than permanent. See docs/PRODUCTION_READINESS.md §3.2 for what that costs and what a transactional outbox would buy.

Behaviour when NATS is down

The service starts and serves reads with NATS unreachable, retrying bucket provisioning in the background on every reconcile cycle. Startup takes about fifteen seconds longer while the first provisioning attempt times out. GET /health reports "nats":"disconnected" and answers 503; GET /livez stays 200, so an orchestrator takes the pod out of the load balancer instead of killing it. Writes still return 200 — the database is the source of truth and accepts them — but the KV publish fails, is logged, and is healed by the reconciler once NATS returns.

KV key layout

Three buckets. Keys are environment-scoped so a consumer can watch one prefix and receive only what belongs to its environment.

Values:

• FLAGS — {"enabled": true, "value": "0.25", "updatedAt": "..."}. value is a free-form string, used for things like rollout percentages.
• MICROCONFIG — the entire appsettings tree as one JSON document, so one push replaces it atomically and a consumer can never observe half an update.
• LOCALIZATION — one JSON object per locale: {"catalog.title": "..."}.

Buckets keep five historical values per key, which is the rollback depth, and a single value may not exceed 512 KiB. The admin API enforces the same ceiling before the database write and answers 400 (settings json exceeds the maximum value size), so a payload can never be accepted with a 201 and then refused by JetStream forever afterwards.

No secrets in KV. Anyone with NATS credentials can read every key, and every consumer holds it in plaintext memory. The convention is that secret-shaped fields carry a marker instead of a value — "accessKeyId": "env:STORAGE_ACCESS_KEY_ID" — which the consumer resolves at bind time from its own secret store. The seeded example document in internal/database/sqlite.go demonstrates the shape.

Admin API

Every route except GET /health, GET /livez and GET /metrics requires a bearer token. Reads used to be anonymous; they are not any more, because a single unauthenticated GET was enough to walk off with every appsettings tree and bundle in the estate.

Writes are additionally rate-limited (WRITE_RATE_LIMIT_PER_MINUTE, default 120) and recorded in an audit log with the actor, the target and a redacted request body. The limiter has three key namespaces: ip:<ip> at the edge for a write that arrives with a configured credential, anon:<ip> at the edge for one that does not — charged at four times the rate, so a quarter of the budget, and kept separate so a flood of credential-less writes cannot spend what an authenticated caller behind the same proxy IP is about to need — and t:<name> per credential after authentication. GET /inventory is metered too, under inv:t:<name>; it is the only rate-limited read, because it is the only one that is not bounded by a page in SQL.

Token scope narrows reads as well as writes

ADMIN_TOKENS=ci-dev:1|2:secret gives a token the environments 1 and 2. That scope applies in both directions:

• Writes to any other environment answer 403.
• Listings (/flags/values, /configs/values, /localization, /environments) return only rows in environments 1 and 2.
• GET /inventory and GET /audit are narrowed the same way; audit rows that carry no environment at all are invisible to a scoped token.
• A single row fetched by id that lives outside the scope answers 404, not 403 — any other answer would confirm it exists.

ADMIN_TOKEN (singular) is the shared full-scope form, recorded as the actor shared. With neither variable set, auth is disabled and the service warns at startup — dev only.

Routes

GET  /health      readiness: pings the database AND reports NATS; 503 if either is down
GET  /livez       liveness: static, touches no dependency
GET  /metrics     Prometheus: publish success/failure, reconcile drift, HTTP

GET  /inventory   every editable row with its id — paged, ?limit (default 100, max 500) &offset; rate-limited
GET  /audit       the write audit trail — ?actor=&from=&to=&limit=&offset=

GET    /environments                 ?limit=&offset=
POST   /environments
DELETE /environments/{id}

GET    /microservices                ?limit=&offset=
POST   /microservices
DELETE /microservices/{id}

GET    /configs/{id}                 the microservice definition (belongs to no environment)
GET    /configs/values               ?microserviceId=&environmentId=&limit=&offset=
GET    /configs/values/{id}
POST   /configs/values
PUT    /configs/values               the id goes in the body, not the path
DELETE /configs/values/{id}

GET    /flags                        ?flagKey=&limit=&offset=
GET    /flags/{id}
POST   /flags
DELETE /flags/{id}

GET    /flags/values                 ?environmentId=&flagKey=&limit=&offset=
GET    /flags/values/{id}
POST   /flags/values
PUT    /flags/values                 the id goes in the body, not the path
DELETE /flags/values/{id}

GET    /localization                 ?microserviceId=&environmentId=&locale=&limit=&offset=
GET    /localization/{id}
GET    /localization/lookup/{msId}/{envId}/{locale}
POST   /localization
PUT    /localization/values          the id goes in the body, not the path
DELETE /localization/{id}

Two shapes to note. DELETE is registered on /{id} only — there is no delete on a collection path. And localization is the odd one out: it creates at /localization and deletes at /localization/{id}, but updates at /localization/values.

Every response carries X-Content-Type-Options: nosniff, and Strict-Transport-Security when the connection is really TLS. When TLS_CERT_FILE/TLS_KEY_FILE are set the server floor is TLS 1.3 — nothing in the stack (browsers, the console proxy, the Go client, Prometheus) needs 1.2 kept open.

Writing

Creating an appsettings row and a bundle, against the local stack:

curl -s -X POST http://localhost:8080/configs/values \
  -H "Authorization: Bearer local-dev-token" \
  -H "Content-Type: application/json" \
  -d '{"microserviceId":2,"environmentId":2,"settingsJson":{"http":{"timeoutMs":1500}}}'
# 201 {"id":203,"microserviceId":2,"environmentId":2,"settingsJson":{...},"updatedAt":"..."}

curl -s -X POST http://localhost:8080/localization \
  -H "Authorization: Bearer local-dev-token" \
  -H "Content-Type: application/json" \
  -d '{"microserviceId":1,"environmentId":2,"locale":"en-US","bundleJson":{"catalog.title":"Catalog"}}'
# 201 {"id":302,"microserviceId":1,"environmentId":2,"locale":"en-US","bundleJson":{...},"updatedAt":"..."}

settingsJson and bundleJson must be JSON objects — a top-level array or scalar is well-formed JSON that would break every consumer's deserialization at once, so it is refused with 400.

A flag value has its own three rules, applied identically on POST and PUT: value must be non-empty and at most 4000 characters — runes, because the VARCHAR(4000) column counts characters and not bytes — and enabled (like isActive on a flag definition) must be exactly 0 or 1. The column would take any SMALLINT, but only 0 and 1 mean anything, since the KV payload collapses everything non-zero to true. Each of these used to be a 500 from the driver, or — for an empty value — a silently published empty string that arrives at every consumer in the environment looking like a parse bug.

Optimistic concurrency

Every PUT accepts an optional expectedUpdatedAt. The write applies only while the stored row still carries that timestamp; otherwise it answers 409 and changes nothing. Absent, the original last-write-wins behaviour stands. It is implemented and tested in all three domains — flag values, appsettings and bundles — against both the PostgreSQL and SQLite repositories.

# read the row, keep its updatedAt
curl -s -H "Authorization: Bearer local-dev-token" \
  http://localhost:8080/flags/values/101
# {"id":101,...,"updatedAt":"2026-08-05T23:27:56Z"}

# a stale timestamp loses the race
curl -s -o /dev/null -w '%{http_code}\n' -X PUT http://localhost:8080/flags/values \
  -H "Authorization: Bearer local-dev-token" \
  -H "Content-Type: application/json" \
  -d '{"id":101,"enabled":0,"value":"off","expectedUpdatedAt":"2020-01-01T00:00:00Z"}'
# 409

The int/bool trap

enabled is an integer on the admin API and a boolean in KV. Same concept, two types, because the API mirrors the SMALLINT column and KV mirrors what a consumer wants to branch on:

The conversion is Enabled != 0, done once on the publish path. A tool that reads a flag value from the API and writes it back unchanged is fine; a tool that reads from KV and posts what it found is not.

Integrating a consumer

Go — import pkg/configclient:

c, err := configclient.New(ctx, configclient.Options{
    NATSURL:        "nats://nats.example.internal:4222",
    EnvironmentID:  1, // the environment the local stack seeds
    MicroserviceID: 1, // scopes the watch to this service's own keys
})
if err != nil {
    return fmt.Errorf("configclient: %w", err)
}
defer c.Close()

if c.FlagEnabled("search_v2") { ... }              // true in the seeded env 1
raw, ok := c.MicroSettings(1)                       // catalog-api's appsettings tree
text, ok := c.Translate(1, "pt-BR", "catalog.title") // "Catálogo"

Check the error before the defer: on failure c is nil and c.Close() panics.

New blocks until the initial values for the client's scope are loaded, so a service does not start serving on an empty cache — that is how a "feature disabled everywhere" incident happens. Reads after that are memory accesses. Setting MicroserviceID matters: without it the client caches every service's configuration in the process, and Status().FleetWide reports that it did.

Options.HTTPFallback is an optional cold-start path: if JetStream is unreachable at boot it hydrates what it can over the admin API instead of booting with no config at all. It runs only from New, never from a read. Because every admin route now needs a credential, it requires HTTPFallback.Token — scoped to the same environment as the client, since a read outside a token's scope answers 404 like a row that does not exist. AllowUnauthenticated exists for a deployment running with auth switched off, which is dev only.

Any other language — there is no shipped client, but the consumer contract is small and fully specified in docs/CONSUMER_CONTRACT.md: what to watch, what the values look like, how to gate startup, why the watch handler must be idempotent, how to resolve the secret markers, and how to test the whole loop against the local stack.

Configuration

Entirely environment-driven. .env.example documents every variable both binaries read, with its default in brackets — read it rather than guessing. The service auto-loads a .env file at startup via godotenv.Load(); the test console does not, so its variables have to be exported or set on the command line.

Beyond the obvious DB_DRIVER / CONN_STRING / NATS_URL / PUBLISH_ENABLED:

Console-only: BIND_ADDR [127.0.0.1], ALLOWED_ORIGINS, ALLOWED_HOSTS, ENVIRONMENT_ID [1], CENTRAL_CONFIG_URL, NATS_EMBEDDED [false], WEB_DIR [web].

ADMIN_TOKENS is NAME:ENV_SCOPE:SECRET, comma separated. A name is limited to [A-Za-z0-9._-] and 100 characters, and a secret may not contain a comma — entries are split on commas before they are split on colons, so a comma inside a secret silently starts an entry, and possibly a full-scope token, of its own. The name check is what catches that at startup rather than in production.

A malformed value is a fatal startup error, and the line is drawn after trimming. ADMIN_TOKENS= — which is what an env file renders for an absent value — is indistinguishable from the variable being unset, so it takes the documented, loudly-warned development path. A value that holds separators but yields no entry (" , , ", what a Helm value referencing an empty secret renders to) is not blank: something was meant to be there, so it stops the process rather than quietly disabling authentication. A whitespace-only ADMIN_TOKEN is fatal for the opposite reason — it is a live full-scope credential whose secret is a space, with no warning printed at all.

Documentation

Read in this order: the compose walkthrough to see it work, the consumer contract to integrate against it, then security and production readiness before deploying it.

PostgreSQL DDL for every table is in migrations/, numbered in the order it applies.

Known limitations

• The dual write is not transactional. A crash between the database commit and the KV write leaves KV stale until the next reconcile. Bounded by RECONCILE_INTERVAL, not eliminated. A transactional outbox would close it and is deliberately not built — see docs/PRODUCTION_READINESS.md §3.2.
• PostgreSQL is newly adopted and has never run a deployment. The SQL is exercised by internal/pgintegration against a real server on every CI run, which is a great deal more than compilation, but no production instance has yet carried this schema. Treat the first deployment accordingly.
• There is no migration runner. migrations/ is DDL you apply yourself; nothing records what has been applied and there is no down path. The file numbers are the apply order, so psql -f over the directory in lexical order works — see docs/PRODUCTION_READINESS.md §3.4.
• Almost everything above the repositories is covered against SQLite only. internal/pgintegration now drives the whole service over a real PostgreSQL in one file — app_scope_test.go builds the app through app.NewApp and exercises token scoping, the row-environment lookup and the audit insert through HTTP, because the security middleware has a Postgres branch nothing else executed. That is the exception, not the rule: every other end-to-end and handler test still runs on SQLite, so the rest of the wiring above the repositories is verified against the mirror schema rather than the real one.
• KV has no per-key ACLs out of the box. With a single shared credential, any consumer holding it can read every environment's keys. The environment prefix is a scoping convention for watches, not an access control boundary. This is why secrets must not be in KV; docs/SECURITY.md has the NATS permission model that narrows it.
• TLS is opt-in. With TLS_CERT_FILE/TLS_KEY_FILE unset the admin API serves plain HTTP and warns at startup, on the assumption that TLS terminates at an ingress.
• The HTTP cold-start fallback fetches flags by row id. GET /flags/values?environmentId= does exist — it is the first thing the quickstart above calls — but configclient's fallback does not use it. It fetches GET /flags/values/{id} one row at a time, so HTTPFallback.FlagValueIDs has to name the row ids by hand, and what is left unconfigured is simply not fetched. Closing this is a change to pkg/configclient/httpfallback.go, not a new server route.

Licence

MIT — see LICENSE.