Documentation
¶
Index ¶
- Constants
- func CheckDataDir(dir string) error
- func LogStorageInitError(kind, path string, err error)
- func WarnIfFreshDataDir(kind, dir string)
- type Auth
- type Cache
- type CacheBackend
- type CacheRedisConfig
- type CacheRedisTLS
- type ClickHouse
- type Config
- type Coord
- type CoordBackend
- type CoordNATSConfig
- type Dedupe
- type DedupeBackend
- type DedupeDynamoDBConfig
- type MQ
- type MQBackend
- type MQNATSConfig
- type MQNATSTLS
- type OTel
- type OTelLogs
- type OTelMetrics
- type OTelTraces
- type Prometheus
- type Role
- type Server
- type Settings
Constants ¶
const ( RedisStandalone = "standalone" RedisCluster = "cluster" RedisSentinel = "sentinel" )
Redis deployment modes for cache.redis.mode. RedisSentinel is refused until the backend supports it (#656).
const ( EnvConfig = "WH_CONFIG" EnvLogLevel = "WH_LOG_LEVEL" )
EnvConfig names the boot-config file (default: ./config.yaml) and EnvLogLevel the minimum log level. Neither is a Config field — the first locates the file the struct is read from, the second is read by main directly — so they are declared here as the process-level names unboundEnv must not flag.
const EnvSettingsDir = "WH_SETTINGS_DIR"
EnvSettingsDir is the environment variable naming the settings directory — the same variable the Settings.Dir struct tag binds. Subcommands (`wavehouse validate`) read it directly without loading config, so the name is exported here as the single authority; a struct tag must be a literal, so a config test pins the tag to this constant to prevent drift.
const MaxNATSShards = 256
MaxNATSShards is the largest mq.nats.shards boot accepts: internal/mq's own bound, which config cannot import (internal/app pins the two together).
Variables ¶
This section is empty.
Functions ¶
func CheckDataDir ¶ added in v0.2.0
CheckDataDir reports whether boot can use dir as data_dir: a directory that exists must be writable, and one that doesn't (boot creates it — a first run, or a missing mount that WarnIfFreshDataDir calls out) must have a writable nearest existing ancestor so that creation can succeed. Run before anything dials out, so a data_dir the process cannot write to refuses boot before ClickHouse discovery rather than after it; a permission denial — on the write probe, or on reaching dir at all through a parent without search permission — carries the UID-65532 hint, since a bind mount owned by root is the typical cause. Writability is probed by creating and removing one temp file: the only portable test that exercises the mount's ownership and mode. A blank dir — reachable through `WH_DATA_DIR=` or `data_dir: ""` — is refused outright: the ancestor walk would otherwise probe the working directory and pass, and NATS and Pebble state would land under it.
func LogStorageInitError ¶
LogStorageInitError emits an error log for a storage-init failure, with a UID-65532 hint when the failure looks like a permission denial — the dominant signature of a Docker bind mount whose host directory is owned by root rather than the distroless `nonroot` user. Keeps the actionable remediation right next to the error line so operators don't have to dig.
`kind` is a short label (e.g. "mq", "dedupe"). The caller still decides whether to exit; this only logs.
func WarnIfFreshDataDir ¶
func WarnIfFreshDataDir(kind, dir string)
WarnIfFreshDataDir logs a startup `WARN` if dir doesn't already exist or is empty, signalling that we're about to start with no prior state.
On a first-ever run this is expected; on every subsequent run it should be silent. So when the warning *does* fire on a redeploy, it's the most direct possible signal that the persistent volume isn't actually persisting: either it wasn't mounted, the wrong path was mounted, or the volume was recreated.
`kind` is a short label for the log message (e.g. "nats", "pebble").
Types ¶
type Auth ¶
type Auth struct {
OperatorKey string `yaml:"operator_key" env:"WH_AUTH_OPERATOR_KEY"`
}
Auth holds the platform's own credential. The verifier wiring — `jwks_url`, `jwt_secret`, `role_claim` — is the settings directory's `auth` block (hot-reloadable: a change rebuilds the tenant's verifier; the HMAC secret is a tenant's own, #529). There is no on/off switch: the middleware always runs. A request with no token, or an invalid/expired one, falls back to the policy default_role; elevated access requires a valid token whose role claim matches a granted role (or the policy admin_role). With neither jwt_secret nor jwks_url set, no token can validate, so every request is the default role — a pure public deployment.
OperatorKey is an optional non-JWT credential for the operator running the deployment: a request presenting it via an "Authorization: Operator <key>" header (or the X-Operator-Key alias) is authorized as a full-access platform operator, independent of the JWT verifier, and keeps working even if the policy is missing/deleted (break-glass recovery, e.g. fixing policies.json). Empty (the default) disables it. Treat it as an admin secret.
type Cache ¶
type Cache struct {
Backend CacheBackend `yaml:"backend" env:"WH_CACHE_BACKEND"`
L1MaxCost int64 `yaml:"l1_max_cost" env:"WH_CACHE_L1_MAX_COST"`
Redis CacheRedisConfig `yaml:"redis"`
}
Cache selects and sizes the query-result cache. The time-range bucket structured queries normalize to is a settings-directory key (query.timestamp_bucket_seconds) — query shaping, not process memory.
type CacheBackend ¶ added in v0.2.0
type CacheBackend string
CacheBackend names the query-result cache implementation.
const ( // CacheLocal is the in-process Ristretto cache, sized by // cache.l1_max_cost. CacheLocal CacheBackend = "local" // CacheRedis is one Redis-compatible server shared by every process, // configured by cache.redis. CacheRedis CacheBackend = "redis" )
type CacheRedisConfig ¶ added in v0.2.0
type CacheRedisConfig struct {
// Addrs are host:port pairs: the server, or seeds for a cluster.
Addrs []string `yaml:"addrs" env:"WH_CACHE_REDIS_ADDRS"`
Mode string `yaml:"mode" env:"WH_CACHE_REDIS_MODE"`
Username string `yaml:"username" env:"WH_CACHE_REDIS_USERNAME"`
Password string `yaml:"password" env:"WH_CACHE_REDIS_PASSWORD"`
DB int `yaml:"db" env:"WH_CACHE_REDIS_DB"`
TLS CacheRedisTLS `yaml:"tls"`
// KeyPrefix leads every key, so deployments can share one server.
KeyPrefix string `yaml:"key_prefix" env:"WH_CACHE_REDIS_KEY_PREFIX"`
Timeout time.Duration `yaml:"timeout" env:"WH_CACHE_REDIS_TIMEOUT"`
DialTimeout time.Duration `yaml:"dial_timeout" env:"WH_CACHE_REDIS_DIAL_TIMEOUT"`
// MaxValueBytes is the largest value stored, after compression.
MaxValueBytes int `yaml:"max_value_bytes" env:"WH_CACHE_REDIS_MAX_VALUE_BYTES"`
// CompressMinBytes is the smallest value zstd-compressed; 0 never
// compresses.
CompressMinBytes int `yaml:"compress_min_bytes" env:"WH_CACHE_REDIS_COMPRESS_MIN_BYTES"`
// VersionTTL is how long a version token outlives its last bump.
VersionTTL time.Duration `yaml:"version_ttl" env:"WH_CACHE_REDIS_VERSION_TTL"`
}
CacheRedisConfig configures cache.backend=redis: one Redis-compatible server (Redis, Valkey, Dragonfly, ElastiCache, MemoryDB) shared by every process. Read only when that backend is selected.
type CacheRedisTLS ¶ added in v0.2.0
type CacheRedisTLS struct {
Enabled bool `yaml:"enabled" env:"WH_CACHE_REDIS_TLS_ENABLED"`
CAFile string `yaml:"ca_file" env:"WH_CACHE_REDIS_TLS_CA_FILE"`
CertFile string `yaml:"cert_file" env:"WH_CACHE_REDIS_TLS_CERT_FILE"`
KeyFile string `yaml:"key_file" env:"WH_CACHE_REDIS_TLS_KEY_FILE"`
ServerName string `yaml:"server_name" env:"WH_CACHE_REDIS_TLS_SERVER_NAME"`
InsecureSkipVerify bool `yaml:"insecure_skip_verify" env:"WH_CACHE_REDIS_TLS_INSECURE_SKIP_VERIFY"`
}
CacheRedisTLS is cache.redis.tls. The files are paths, read at boot.
type ClickHouse ¶
type ClickHouse struct {
// MaxTotalConns caps the native connections the process may hold open
// across its pools: the settings directory's clickhouse.max_open_conns
// must not exceed it. 0, the default, is no ceiling.
MaxTotalConns int `yaml:"max_total_conns" env:"WH_CH_MAX_TOTAL_CONNS"`
}
ClickHouse holds the connection ceiling. The wiring — address, HTTP port and scheme, database, username, password, query timeout, TLS, headers, pool size — is the settings directory's `clickhouse` block (hot-reloadable: a change swaps the connection; the password is a tenant's own, #529). The ceiling stays here because it is capacity, sized once per process, not wiring.
type Config ¶
type Config struct {
// DataDir is the root for embedded state. NATS JetStream lives at
// `<DataDir>/nats`; Pebble, holding every tenant's dedupe store while any
// tenant has dedupe enabled, at `<DataDir>/pebble`.
// Subdirectory names are conventions, not config — one knob, one mount.
// In a container this MUST resolve to a host-backed volume; the relative
// `./data` default is fine for local binary use only.
DataDir string `yaml:"data_dir" env:"WH_DATA_DIR"`
// Roles are the components this process runs (every role by default);
// a Deployment per role differs only in this. See Role.
Roles []Role `yaml:"roles" env:"WH_ROLES"`
// InstanceID names this process: logged at boot, and the holder a
// coord.backend=nats lease names. Empty resolves to <hostname>-<8 hex> at
// Load.
InstanceID string `yaml:"instance_id" env:"WH_INSTANCE_ID"`
Server Server `yaml:"server"`
ClickHouse ClickHouse `yaml:"clickhouse"`
MQ MQ `yaml:"mq"`
Cache Cache `yaml:"cache"`
Dedupe Dedupe `yaml:"dedupe"`
Coord Coord `yaml:"coord"`
Auth Auth `yaml:"auth"`
OTel OTel `yaml:"otel"`
Prometheus Prometheus `yaml:"prometheus"`
Settings Settings `yaml:"settings"`
}
Config is the top-level application configuration.
func Load ¶
Load reads config from a YAML file (if it exists) with env var overrides. Both sources are strict: a YAML key the Config struct doesn't declare, or a WH_* environment variable no field binds, is an error naming every offender, so a tunable that moved to the settings directory (or a typo) can't be silently ignored. The environment always carries unrelated names, so only the WH_ prefix is checked there. Boot is the validator for this half of configuration — there is no dry run; a refused boot is the loud signal.
func (*Config) Distributed ¶ added in v0.2.0
Distributed reports whether the message queue is shared with other processes. The embedded one listens on no port, so while it is selected every process is an island: nothing else can reach its queue.
func (*Config) NeedsDataDir ¶ added in v0.2.0
NeedsDataDir reports whether a backend this process opens keeps state under data_dir, and so whether boot must probe it (CheckDataDir). Only the api role opens the dedupe stores.
func (*Config) Warnings ¶ added in v0.2.0
Warnings returns what a valid configuration is still likely to get wrong, one line each, for boot to log at WARN. None is an error: an ignored block is harmless, and the shared-queue ones are correct for a single replica, which one process cannot tell from many.
type Coord ¶ added in v0.2.0
type Coord struct {
Backend CoordBackend `yaml:"backend" env:"WH_COORD_BACKEND"`
// NATS is read only when Backend is nats.
NATS CoordNATSConfig `yaml:"nats"`
}
Coord selects the coordination layer.
type CoordBackend ¶ added in v0.2.0
type CoordBackend string
CoordBackend names where leases are held: the sweeper's on the embedded MQ, and the ingest processes' on a shared one.
const CoordLocal CoordBackend = "local"
CoordLocal holds leases in this process, which is enough while no other process shares its queue.
const CoordNATS CoordBackend = "nats"
CoordNATS holds leases in a KV bucket on mq.nats's connection, so every process on the shared queue contends for the same ones. It needs mq.backend=nats; its settings are the coord.nats block.
type CoordNATSConfig ¶ added in v0.2.0
type CoordNATSConfig struct {
// Bucket is the KV bucket the leases live in; empty is
// <mq.nats.subject_prefix>_coord, the name the generated manifests give it.
Bucket string `yaml:"bucket" env:"WH_COORD_NATS_BUCKET"`
}
CoordNATSConfig names the operator's KV bucket. There is no connection block: coord.backend=nats rides mq.nats's connection and credentials.
type Dedupe ¶
type Dedupe struct {
Backend DedupeBackend `yaml:"backend" env:"WH_DEDUPE_BACKEND"`
// Lease is how long a claimed id stays pending while its record is
// published; a claim its request never settles lapses after it.
Lease time.Duration `yaml:"lease" env:"WH_DEDUPE_LEASE"`
// ReserveConcurrency bounds the parallel calls one Reserve, Commit or
// Release makes to a remote backend, and sizes its idle connection pool
// to match. Pebble ignores it.
ReserveConcurrency int `yaml:"reserve_concurrency" env:"WH_DEDUPE_RESERVE_CONCURRENCY"`
DynamoDB DedupeDynamoDBConfig `yaml:"dynamodb"`
}
Dedupe selects the dedupe store. Whether a tenant dedupes, on which field, and for how long are settings-directory keys, not this block's.
type DedupeBackend ¶ added in v0.2.0
type DedupeBackend string
DedupeBackend names where ingest dedupe keeps the ids it has seen.
const ( // DedupePebble is the Pebble instance inside this process, under // <data_dir>/pebble, opened while any tenant has dedupe on. Seen ids are // per process. DedupePebble DedupeBackend = "pebble" // DedupeDynamoDB is one DynamoDB table every tenant and every process // shares, configured by dedupe.dynamodb. DedupeDynamoDB DedupeBackend = "dynamodb" )
type DedupeDynamoDBConfig ¶ added in v0.2.0
type DedupeDynamoDBConfig struct {
// Table is the shared table; WaveHouse never creates it outside
// dynamodb-local. Required.
Table string `yaml:"table" env:"WH_DEDUPE_DYNAMODB_TABLE"`
// Region overrides the SDK chain's (AWS_REGION).
Region string `yaml:"region" env:"WH_DEDUPE_DYNAMODB_REGION"`
// Endpoint points the client at dynamodb-local.
Endpoint string `yaml:"endpoint" env:"WH_DEDUPE_DYNAMODB_ENDPOINT"`
Timeout time.Duration `yaml:"timeout" env:"WH_DEDUPE_DYNAMODB_TIMEOUT"`
MaxAttempts int `yaml:"max_attempts" env:"WH_DEDUPE_DYNAMODB_MAX_ATTEMPTS"`
RetryMode string `yaml:"retry_mode" env:"WH_DEDUPE_DYNAMODB_RETRY_MODE"`
// CreateTable creates the table at boot if it is missing. Development
// only: refused unless Endpoint is set.
CreateTable bool `yaml:"create_table" env:"WH_DEDUPE_DYNAMODB_CREATE_TABLE"`
}
DedupeDynamoDBConfig is the dynamodb backend's block, read only when it is selected. Credentials are the AWS SDK's default chain (EKS Pod Identity, IRSA, AWS_* variables), never keys here.
type MQ ¶
type MQ struct {
Backend MQBackend `yaml:"backend" env:"WH_MQ_BACKEND"`
// NATS is read only when Backend is nats.
NATS MQNATSConfig `yaml:"nats"`
}
MQ selects the message queue. The per-tenant byte budget, mq.max_bytes_gb, is a settings-directory key, not this block's.
type MQBackend ¶ added in v0.2.0
type MQBackend string
MQBackend names the message queue implementation.
const MQEmbedded MQBackend = "embedded"
MQEmbedded is the NATS JetStream server inside this process, under <data_dir>/nats.
const MQNATS MQBackend = "nats"
MQNATS is a NATS JetStream cluster the operator runs, holding the streams and durables deployments/nats describes; every process naming it shares one queue. Its settings are the mq.nats block.
type MQNATSConfig ¶ added in v0.2.0
type MQNATSConfig struct {
URLs []string `yaml:"urls" env:"WH_MQ_NATS_URLS"`
// Name is the connection name the server reports; empty is
// wavehouse-<hostname>.
Name string `yaml:"name" env:"WH_MQ_NATS_NAME"`
// CredsFile, NKeySeedFile and User are exclusive: one way to
// authenticate, or none.
CredsFile string `yaml:"creds_file" env:"WH_MQ_NATS_CREDS_FILE"`
NKeySeedFile string `yaml:"nkey_seed_file" env:"WH_MQ_NATS_NKEY_SEED_FILE"`
User string `yaml:"user" env:"WH_MQ_NATS_USER"`
PasswordFile string `yaml:"password_file" env:"WH_MQ_NATS_PASSWORD_FILE"`
TLS MQNATSTLS `yaml:"tls"`
// JSDomain is the JetStream domain, for a leafnode or hub-and-spoke
// deployment.
JSDomain string `yaml:"js_domain" env:"WH_MQ_NATS_JS_DOMAIN"`
SubjectPrefix string `yaml:"subject_prefix" env:"WH_MQ_NATS_SUBJECT_PREFIX"`
Partitions int `yaml:"partitions" env:"WH_MQ_NATS_PARTITIONS"`
// Shards is how many shards each partition has, one durable each.
Shards int `yaml:"shards" env:"WH_MQ_NATS_SHARDS"`
// IngestConsumer names the shard durables: <ingest_consumer>-<shard>.
IngestConsumer string `yaml:"ingest_consumer" env:"WH_MQ_NATS_INGEST_CONSUMER"`
// HistoryStream is looked up by name; empty is <SUBJECT_PREFIX>_HISTORY,
// the name the generated manifests give it.
HistoryStream string `yaml:"history_stream" env:"WH_MQ_NATS_HISTORY_STREAM"`
ConnectTimeout time.Duration `yaml:"connect_timeout" env:"WH_MQ_NATS_CONNECT_TIMEOUT"`
PublishTimeout time.Duration `yaml:"publish_timeout" env:"WH_MQ_NATS_PUBLISH_TIMEOUT"`
TopologyWait time.Duration `yaml:"topology_wait" env:"WH_MQ_NATS_TOPOLOGY_WAIT"`
}
MQNATSConfig is how to reach the operator's NATS and what topology to expect there (mq.NATSConfig, which internal/app builds from it). Secrets are file paths only: nothing inline.
type MQNATSTLS ¶ added in v0.2.0
type MQNATSTLS struct {
CAFile string `yaml:"ca_file" env:"WH_MQ_NATS_TLS_CA_FILE"`
CertFile string `yaml:"cert_file" env:"WH_MQ_NATS_TLS_CERT_FILE"`
KeyFile string `yaml:"key_file" env:"WH_MQ_NATS_TLS_KEY_FILE"`
ServerName string `yaml:"server_name" env:"WH_MQ_NATS_TLS_SERVER_NAME"`
HandshakeFirst bool `yaml:"handshake_first" env:"WH_MQ_NATS_TLS_HANDSHAKE_FIRST"`
}
MQNATSTLS is the client side of TLS to the NATS servers.
type OTel ¶
type OTel struct {
Enabled bool `yaml:"enabled" env:"WH_OTEL_ENABLED"`
Traces OTelTraces `yaml:"traces"`
Metrics OTelMetrics `yaml:"metrics"`
Logs OTelLogs `yaml:"logs"`
}
OTel configures the OpenTelemetry pipeline. `enabled` is the master switch; when false, no signals are initialized regardless of the per-signal toggles. The OTLP destination — endpoint, TLS, custom CA, mutual TLS, and auth headers — is configured through the standard OTEL_EXPORTER_OTLP_* environment variables read by the OpenTelemetry SDK, not WaveHouse config. See docs/src/content/docs/configuration.mdx.
type OTelLogs ¶
type OTelLogs struct {
Enabled bool `yaml:"enabled" env:"WH_OTEL_LOGS_ENABLED"`
SampleRate float64 `yaml:"sample_rate" env:"WH_OTEL_LOGS_SAMPLE_RATE"`
}
OTelLogs sample rate applies to OTLP export of DEBUG/INFO only. WARN and ERROR always export at 100% — dropping them silently during incidents is too dangerous to expose as a knob. Stdout receives 100% of records regardless of this rate (sampling for scraped-log pipelines like Loki/Promtail belongs at the scraper, not the application).
type OTelMetrics ¶
type OTelMetrics struct {
Enabled bool `yaml:"enabled" env:"WH_OTEL_METRICS_ENABLED"`
}
type OTelTraces ¶
type Prometheus ¶
type Prometheus struct {
Enabled bool `yaml:"enabled" env:"WH_PROMETHEUS_ENABLED"`
Path string `yaml:"path" env:"WH_PROMETHEUS_PATH"`
Port int `yaml:"port" env:"WH_PROMETHEUS_PORT"`
}
Prometheus controls a Prometheus exposition endpoint served alongside (or independently of) the OTLP push exporter. It is its own top-level block so operators using Prometheus scraping (Grafana Alloy / Mimir / etc.) don't have to hunt through `[otel]` for an output they don't otherwise care about. Internally the same OTel MeterProvider drives both — the Prometheus exporter is an additional Reader — but the user-facing config keeps the two outputs distinct.
Works in any of three combinations: OTel only, Prometheus only, or both. If only Prometheus is enabled, the MeterProvider is still created so runtime + custom metrics flow to `/metrics`; no OTLP push is attempted.
Port `0` mounts the endpoint on the existing API server router. A non-zero port spins up a dedicated HTTP listener — useful for firewalling metrics off the public API surface in production.
type Role ¶ added in v0.2.0
type Role string
Role is one part of the work a process can run.
const ( // RoleAPI serves the HTTP API and everything that answers it: schema // discovery, the auth verifiers, the dedupe stores, and the SSE hub with // its bridge off the queue and its keepalive wheels. Per process: every // API process runs its own. RoleAPI Role = "api" // RoleIngest runs the ingest worker, queue to ClickHouse. Under nats each // shard of the queue is delivered to one ingest process at a time. RoleIngest Role = "ingest" // RoleSweeper runs the sweeper, one per queue, under the sweeper lease, on // the embedded MQ only: under nats the streams' own retention replaces it. RoleSweeper Role = "sweeper" )
type Server ¶
type Server struct {
Port int `yaml:"port" env:"WH_SERVER_PORT"`
ShutdownTimeout int `yaml:"shutdown_timeout" env:"WH_SERVER_SHUTDOWN_TIMEOUT"`
}
Server holds listener wiring. The CORS allowlist is a tenant tunable and lives in the settings directory's config.json (internal/settings), as do the SSE keepalive and gap-window knobs (stream.*).
type Settings ¶ added in v0.2.0
type Settings struct {
Dir string `yaml:"dir" env:"WH_SETTINGS_DIR"`
}
Settings locates the hot-reloadable settings directory — the four JSON documents (roles.json, policies.json, pipes.json, config.json) validated by internal/settings. Boot-tier by necessity: it's the pointer the reload machinery follows, so it can't live behind itself. REQUIRED, with no default: a baked-in path would turn a missing mount into silent misconfiguration instead of an explicit operator choice, and the binary has no compiled tunable defaults to fall back on — `wavehouse bootstrap [dir]` writes the starter directory (the container images preset WH_SETTINGS_DIR=/app/settings and expect a mount there).