config

package
v0.18.55 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 7, 2026 License: AGPL-3.0 Imports: 14 Imported by: 0

Documentation

Overview

Package config provides YAML configuration loading for Wadjet.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ChangedKeys added in v0.18.18

func ChangedKeys(old, new *Config) []string

ChangedKeys returns the registry keys whose values differ between two configurations, in registry order.

func DeferredReason added in v0.18.18

func DeferredReason(name string) string

DeferredReason returns the recorded mechanism for a deferred key.

func EnvNames added in v0.18.18

func EnvNames() []string

EnvNames returns every environment variable the registry reads, in registry order.

Types

type ABACCondition

type ABACCondition struct {
	Attribute string `yaml:"attribute"` // e.g. "subject.role", "resource.name", "env.source_ip"
	Operator  string `yaml:"operator"`  // eq, neq, in, not_in, gt, lt, gte, lte, contains, regex, exists, not_exists
	Value     string `yaml:"value"`     // single value or comma-separated for "in"/"not_in"
}

ABACCondition defines a condition that must match for the rule to apply.

type ABACObligation

type ABACObligation struct {
	Type   string `yaml:"type"`   // row_filter, mask_column, deny_column, query_limit
	Target string `yaml:"target"` // column name or table name
	Value  string `yaml:"value"`  // filter expression, mask function, limit value
}

ABACObligation defines a side-effect obligation on an allow rule.

type ABACPolicy

type ABACPolicy struct {
	Name        string     `yaml:"name"`
	Description string     `yaml:"description"`
	Priority    int        `yaml:"priority"` // lower = evaluated first
	Enabled     *bool      `yaml:"enabled"`  // nil = true
	Rules       []ABACRule `yaml:"rules"`
}

ABACPolicy defines an attribute-based access control policy in config.

type ABACRule

type ABACRule struct {
	Effect      string           `yaml:"effect"` // "allow" or "deny"
	Conditions  []ABACCondition  `yaml:"conditions"`
	Obligations []ABACObligation `yaml:"obligations"` // only for "allow" rules
}

ABACRule defines a single rule within an ABAC policy.

type Alerts added in v0.18.18

type Alerts struct {
	Enabled bool `yaml:"enabled"` // enable CREATE ALERT DDL and the scheduler
}

Alerts configures the alert DDL and scheduler.

type Auth

type Auth struct {
	Enabled      bool         `yaml:"enabled"`
	APIKeys      []AuthAPIKey `yaml:"api_keys"`
	JWT          AuthJWT      `yaml:"jwt"`
	MTLS         AuthMTLS     `yaml:"mtls"`
	Roles        []AuthRole   `yaml:"roles"`
	Policies     []AuthPolicy `yaml:"policies"`      // cell-level access policies (legacy)
	ABACPolicies []ABACPolicy `yaml:"abac_policies"` // ABAC access control policies
}

Auth configures authentication and authorization.

type AuthAPIKey

type AuthAPIKey struct {
	Key  string `yaml:"key"`
	Name string `yaml:"name"`
	Role string `yaml:"role"`
}

AuthAPIKey defines an API key credential.

type AuthJWT

type AuthJWT struct {
	Enabled       bool   `yaml:"enabled"`
	Secret        string `yaml:"secret"`
	PublicKeyFile string `yaml:"public_key_file"`
	RoleClaim     string `yaml:"role_claim"`
	Issuer        string `yaml:"issuer"`
}

AuthJWT configures JWT authentication.

type AuthMTLS

type AuthMTLS struct {
	Enabled     bool              `yaml:"enabled"`
	CAFile      string            `yaml:"ca_file"`
	CertFile    string            `yaml:"cert_file"` // server TLS cert
	KeyFile     string            `yaml:"key_file"`  // server TLS key
	RoleMap     map[string]string `yaml:"role_map"`  // CN/SAN -> role
	DefaultRole string            `yaml:"default_role"`
}

AuthMTLS configures mutual TLS authentication.

type AuthPolicy

type AuthPolicy struct {
	Table     string            `yaml:"table"`
	Role      string            `yaml:"role"`
	Columns   map[string]string `yaml:"columns"`    // column -> "allow", "mask", "deny"
	RowFilter string            `yaml:"row_filter"` // SQL WHERE predicate
}

AuthPolicy defines a cell-level access policy for a table+role.

type AuthRole

type AuthRole struct {
	Name        string       `yaml:"name"`
	Tables      []string     `yaml:"tables"`       // table names or "*" for all
	Allow       []string     `yaml:"allow"`        // "read", "write", "admin"
	QueryLimits *QueryLimits `yaml:"query_limits"` // per-role overrides (nil = use global)
}

AuthRole defines a role with table access and permissions.

type ChangeEvent

type ChangeEvent struct {
	Old *Config
	New *Config
}

ChangeEvent describes what changed in a configuration update.

type Config

type Config struct {
	Mode        string      `yaml:"mode"` // standalone, coordinator, worker
	Storage     Storage     `yaml:"storage"`
	NATS        NATS        `yaml:"nats"`
	HTTP        HTTP        `yaml:"http"`
	GRPC        GRPC        `yaml:"grpc"`
	Worker      Worker      `yaml:"worker"`
	Parquet     Parquet     `yaml:"parquet"`
	Auth        Auth        `yaml:"auth"`
	GeoIP       GeoIP       `yaml:"geoip"`
	Alerts      Alerts      `yaml:"alerts"`       // CREATE ALERT DDL + scheduler
	QueryLimits QueryLimits `yaml:"query_limits"` // global query cost limits
	Query       Query       `yaml:"query"`        // coordinator query lifecycle
	Telemetry   Telemetry   `yaml:"telemetry"`    // OpenTelemetry tracing export
}

Config is the top-level configuration for Wadjet.

func DefaultConfig

func DefaultConfig() Config

DefaultConfig returns a configuration with sensible defaults.

func Load

func Load(path string) (*Config, error)

Load reads a YAML config file and merges with defaults.

The decode is STRICT: a key the schema does not define is an error naming it. Before the precedence loader a mistyped `storage:` key was inert anyway, because the whole section was; now the difference between `bucket:` and `buckett:` is the difference between reading the right bucket and the wrong one, with nothing said at startup. PostgreSQL refuses an unrecognised parameter in postgresql.conf for the same reason, and ADR-0029 already takes its precedence order.

The cost is forward compatibility: an older binary refuses a file written for a newer one. That is the trade this repo wants — a silently ignored key is a silently wrong deployment, which is the whole of #808.

func LoadOrDefault

func LoadOrDefault(path string) *Config

LoadOrDefault loads a config file if it exists, otherwise returns defaults. Environment variables with WADJET_ prefix always override file/default values.

func LoadWithKeys added in v0.18.18

func LoadWithKeys(path string) (*Config, map[string]bool, error)

LoadWithKeys reads a YAML config file and returns the config merged over the defaults (exactly Load's result) together with the set of registry keys the file actually set.

Presence is decided by unmarshalling a SECOND time into a zero Config and asking whether the key came out non-zero. A key written into the file with its type's zero value is therefore indistinguishable from an absent one — the same convention the environment tier has always used for an empty string, and the reason Resolve treats both alike.

func (*Config) EffectiveQueryLimits added in v0.18.15

func (c *Config) EffectiveQueryLimits() (*QueryLimits, map[string]*QueryLimits)

EffectiveQueryLimits extracts the cost guard from a loaded config: the global limits, and the per-role map a planner resolves against for the identity answering a query.

EVERY configured role gets an entry, nil meaning unlimited — a role that declares no `query_limits` OVERRIDES the global limits rather than inheriting them, which is what docs/security.md's "Per-Role Limits" section says and how its `admin` example is meant to read. A role nobody configured (and an unauthenticated request) falls back to the global limits.

Returns (nil, nil) for a config with neither, so an unconfigured deployment stays unlimited and pays nothing.

type FlagValues added in v0.18.18

type FlagValues interface {
	// Changed reports whether the flag was set on the command line.
	Changed(flag string) bool
	// Value returns the flag's current value (its default when not
	// Changed), or ok=false when the resolver's flag name is unknown.
	Value(flag string) (any, bool)
	// Default returns the flag's registered default, snapshotted before the
	// command line was parsed. It is the DEFAULT TIER for every key that
	// has a flag: the binary runs on the flag's default today, and
	// DefaultConfig()'s value is not always the same one (DefaultConfig
	// sets storage.access_key to "minioadmin" where --access-key defaults
	// to "" and means "auto-detect from env/IAM").
	Default(flag string) (any, bool)
}

FlagValues is the resolver's view of a parsed command line.

Value returns the variable the flag is bound to, which holds the flag's DEFAULT until the operator sets it — so a single accessor serves both the default tier and the explicit-flag tier, and Changed is what separates them. That is the whole of ADR-0029's "a flag counts only if Changed".

type GRPC

type GRPC struct {
	Addr string `yaml:"addr"`
}

GRPC configures the gRPC API server.

type GeoIP

type GeoIP struct {
	CityDB string `yaml:"city_db"` // path to GeoLite2-City.mmdb
	ASNDB  string `yaml:"asn_db"`  // path to GeoLite2-ASN.mmdb
}

GeoIP configures MaxMind GeoIP database paths.

type HTTP

type HTTP struct {
	Addr string `yaml:"addr"`
}

HTTP configures the HTTP API server.

type Inputs added in v0.18.18

type Inputs struct {
	// File is the config file merged over DefaultConfig() — i.e. Load()'s
	// result — or nil when no --config was given. Sections outside the
	// registry (auth, abac_policies, per-role query limits) are carried
	// through from here unchanged.
	File *Config
	// FileKeys names the registry keys the file actually set. Use
	// LoadWithKeys to build it; a key absent here never wins its tier even
	// when File carries a merged default for it.
	FileKeys map[string]bool
	// Lookup reads an environment variable. nil means os.LookupEnv.
	Lookup func(string) (string, bool)
	// Flags is the parsed command line. nil means no flag tier, in which
	// case DefaultConfig() supplies the default tier.
	Flags FlagValues
}

Inputs are the tiers a resolution draws from.

type Key added in v0.18.18

type Key struct {
	// Name is the dotted config-file path, e.g. "storage.bucket".
	Name string
	// Env is the environment variable that sets this key ("" = none).
	Env string
	// Flag is the root command's persistent flag name ("" = none).
	Flag string
	// Kind is the value type.
	Kind Kind
	// Secret marks a value that must never be echoed back (admin GET
	// reports its SOURCE but redacts the value).
	Secret bool
	// Deferred marks a key that is parsed, resolved and reported but does
	// NOT reach a runtime consumer, with the structural change that would
	// be required named in DeferredWhy. Rule 11 of
	// docs/design/correctness-fix-protocol.md: such a key is deferred
	// explicitly, never left half-live — every write path refuses it.
	Deferred    bool
	DeferredWhy string
	// contains filtered or unexported fields
}

Key describes one configuration setting: where it can come from, what it carries, and how to read or write it on a Config.

The registry is the single source of truth for the precedence machinery. The resolver (resolve.go), applyEnvOverrides, the admin endpoint's effective-value report and the docs' name-set gate all read this table, so a setting cannot exist on one of those paths and be missing from another.

func KeyByName added in v0.18.18

func KeyByName(name string) (Key, bool)

KeyByName looks a key up by its dotted name.

func Keys added in v0.18.18

func Keys() []Key

Keys returns the configuration registry.

func (Key) Get added in v0.18.18

func (k Key) Get(c *Config) any

Get reads the key's value from a config.

func (Key) IsZero added in v0.18.18

func (k Key) IsZero(v any) bool

IsZero reports whether v is the zero value for the key's Kind.

func (Key) ParseEnv added in v0.18.18

func (k Key) ParseEnv(s string) (any, bool)

ParseEnv converts an environment variable's text to the key's Kind. An unparseable value is reported as not-ok and leaves the lower tier in place, which is what applyEnvOverrides has always done.

func (Key) Set added in v0.18.18

func (k Key) Set(c *Config, v any)

Set writes the key's value into a config. v must carry the key's Kind.

type Kind added in v0.18.18

type Kind int

Kind is the Go type a configuration key carries. It decides how an environment variable's string is parsed and how a flag value is asserted.

const (
	KindString Kind = iota
	KindInt
	KindInt64
	KindBool
	KindFloat64
	KindDuration
	KindStringSlice
)

func (Kind) String added in v0.18.18

func (k Kind) String() string

type Manager

type Manager struct {
	// contains filtered or unexported fields
}

Manager provides atomic access to configuration with change notification. Reads are lock-free via atomic.Pointer. Writes are serialized and notify subscribers.

func NewManager

func NewManager(initial *Config, logger *slog.Logger) *Manager

NewManager creates a ConfigManager with the given initial config.

func NewManagerFromResolution added in v0.18.18

func NewManagerFromResolution(res *Resolution, logger *slog.Logger) *Manager

NewManagerFromResolution creates a Manager over a resolved configuration, keeping the per-key source so GET /v1/admin/config can report where each effective value came from (#828).

func (*Manager) Apply

func (m *Manager) Apply(newCfg *Config) error

Apply atomically replaces the configuration and notifies subscribers.

Every registry key that no subscriber consumes is PRESERVED from the current config. The manager is what GET /v1/admin/config reports, so a value it holds must be a value the process is actually running on: a file edit or an admin write that moves a key nothing re-reads would otherwise make the endpoint report a configuration that does not exist until the next restart (#828). The freeze used to be a hardcoded list — Mode, HTTP.Addr and the NATS fields — which left worker.* and parquet.* free to drift away from the running process.

Sections outside the registry (auth and its policies) are not touched: that is where the hot-reload path actually lives.

func (*Manager) Current

func (m *Manager) Current() *Config

Current returns the current configuration. Lock-free.

func (*Manager) HotReloadable added in v0.18.18

func (m *Manager) HotReloadable(key string) bool

HotReloadable reports whether a runtime change to key would reach a consumer — that is, whether any subscriber registered a prefix covering it. A deferred key (registry.Key.Deferred) is never hot-reloadable.

func (*Manager) Reload

func (m *Manager) Reload(path string) error

Reload reads the config file and applies changes.

func (*Manager) ReloadWithReport added in v0.18.18

func (m *Manager) ReloadWithReport(path string) ([]string, error)

ReloadWithReport reads the config file, applies it, and returns the registry keys THE FILE SETS that no subscriber consumes — the ones Apply preserved.

Apply preserving them is right: the running process is not going to re-read a startup-only key, and the manager must report the running configuration. But saying nothing about the part that was ignored is how an operator edits `worker.max_concurrent`, sees "reloaded", and believes it took effect. The PUT path answers 409 naming such a key; a file reload cannot refuse (the file legitimately carries startup-only keys for the NEXT start), so it reports instead.

The report is FileKeys ∩ !HotReloadable, and it has to be. Diffing the running config against Load(path) instead names keys the file never mentions: the running config's default tier is the FLAG's default, while Load merges over DefaultConfig(), and decision 2 of ADR-0029 exists precisely because those two differ — DefaultConfig() sets storage.access_key to "minioadmin" where --access-key defaults to "", and worker.cache_bytes to 256 MiB where --cache-bytes defaults to 0. That diff reported three keys on every reload of any file in any deployment before this, plus one per key taken from a flag or the environment, so the config-file WATCHER emitted the warning on every legitimate auth edit and the one true positive arrived buried in sixteen false ones.

func (*Manager) Resolution added in v0.18.18

func (m *Manager) Resolution() *Resolution

Resolution returns the current configuration with its per-key sources.

func (*Manager) Subscribe

func (m *Manager) Subscribe(fn Subscriber)

Subscribe registers a callback for configuration changes. Subscribers are called synchronously in order during Apply.

A bare Subscribe claims NO config keys, so it does not make anything hot-reloadable. Use SubscribeKeys to declare which keys the callback actually applies.

func (*Manager) SubscribeKeys added in v0.18.18

func (m *Manager) SubscribeKeys(prefixes []string, fn Subscriber)

SubscribeKeys registers a callback and declares the config-key prefixes it applies at runtime. Those prefixes are exactly the keys the admin API will accept a write for; everything else is refused as not hot-reloadable.

type NATS

type NATS struct {
	Port        int      `yaml:"port"`
	URL         string   `yaml:"url"`          // for worker mode: coordinator's NATS URL
	StoreDir    string   `yaml:"store_dir"`    // JetStream storage directory
	ClusterID   string   `yaml:"cluster_id"`   // unique cluster identifier (e.g., "central", "afb-east")
	LeafRemotes []string `yaml:"leaf_remotes"` // remote NATS URLs for leaf node connections
	TLSCert     string   `yaml:"tls_cert"`     // TLS certificate file (server or client)
	TLSKey      string   `yaml:"tls_key"`      // TLS private key file
	TLSCA       string   `yaml:"tls_ca"`       // CA certificate for verifying peers (enables mTLS)
}

NATS configures the embedded NATS server or client connection.

type Parquet

type Parquet struct {
	Compression    string `yaml:"compression"`      // snappy, zstd, gzip, lz4, none
	RowGroupSize   int    `yaml:"row_group_size"`   // rows per row group
	PageBufferSize int    `yaml:"page_buffer_size"` // page size in bytes
}

Parquet configures Parquet file writing.

type Query added in v0.18.18

type Query struct {
	// IntermediateTTL is the age after which the coordinator's periodic
	// sweep reclaims a queries/<id>/* prefix the per-query cleanup did not.
	IntermediateTTL time.Duration `yaml:"intermediate_ttl"`
	// IntermediateSweep is how often that sweep runs.
	IntermediateSweep time.Duration `yaml:"intermediate_sweep"`
}

Query configures the coordinator's query lifecycle. Zero values mean "use the built-in default", matching the flags of the same name.

type QueryLimits

type QueryLimits struct {
	MaxScanBytes            int64 `yaml:"max_scan_bytes"`             // max estimated bytes across all scans
	MaxScanRows             int64 `yaml:"max_scan_rows"`              // max estimated rows across all scans
	MaxScanFiles            int   `yaml:"max_scan_files"`             // max files across all scans
	RequireFilterAboveBytes int64 `yaml:"require_filter_above_bytes"` // require WHERE on tables exceeding this size
	RequireLimitAboveRows   int64 `yaml:"require_limit_above_rows"`   // require LIMIT on scans exceeding this row count
}

QueryLimits configures cost-based query guards. Zero values mean unlimited. Per-role limits in Auth.Roles override these global defaults.

type Resolution added in v0.18.18

type Resolution struct {
	// contains filtered or unexported fields
}

Resolution is a fully resolved configuration plus the tier each key came from.

func Resolve added in v0.18.18

func Resolve(in Inputs) *Resolution

Resolve applies the precedence order settled in ADR-0029:

explicit flag > environment variable > config file > default

A flag counts only when FlagValues reports it Changed; a flag's DEFAULT never beats an environment variable or a config-file value. An empty string never overrides a lower tier, on any tier — that is how the environment layer has always behaved, and applying it to flags too means `--nats-tls-cert=""` reads as "unset" rather than as "explicitly blank", which is what every string flag here means by an empty value.

func (*Resolution) Config added in v0.18.18

func (r *Resolution) Config() *Config

Config returns the resolved configuration.

func (*Resolution) DeferredKeysSet added in v0.18.18

func (r *Resolution) DeferredKeysSet() []string

DeferredKeysSet returns every DEFERRED key this resolution took from a tier other than the default — that is, every key an operator configured that no runtime consumer reads.

Rule 11's "never left half-live" is what this serves. Parsing such a section silently and refusing it only on the admin WRITE path leaves the operator's startup configuration accepted and inert, which is the exact defect #808 was filed for; the caller turns this into a refusal that names the keys.

func (*Resolution) Source added in v0.18.18

func (r *Resolution) Source(key string) Source

Source returns the tier key came from, or SourceDefault for an unknown key.

func (*Resolution) Sources added in v0.18.18

func (r *Resolution) Sources() map[string]Source

Sources returns a copy of the per-key source map.

type Source added in v0.18.18

type Source string

Source names the tier a resolved value came from.

const (
	// SourceDefault is the built-in default — the flag's registered default
	// where the key has a flag, DefaultConfig()'s value otherwise.
	SourceDefault Source = "default"
	// SourceFile is the YAML configuration file.
	SourceFile Source = "file"
	// SourceEnv is a WADJET_* environment variable.
	SourceEnv Source = "env"
	// SourceFlag is a command-line flag the operator actually typed.
	SourceFlag Source = "flag"
	// SourceAdmin is a runtime change through the admin API.
	SourceAdmin Source = "admin"
)

type Storage

type Storage struct {
	Type      string `yaml:"type"`     // "s3" (default) or "file"
	DataDir   string `yaml:"data_dir"` // local directory for type=file
	Endpoint  string `yaml:"endpoint"`
	AccessKey string `yaml:"access_key"`
	SecretKey string `yaml:"secret_key"`
	Bucket    string `yaml:"bucket"`
	UseSSL    bool   `yaml:"use_ssl"`
	Region    string `yaml:"region"`
	// Circuit configures the per-operation-class object-store circuit
	// breaker (ADR-0028).
	Circuit StorageCircuit `yaml:"circuit"`
}

Storage configures the object store connection.

type StorageCircuit added in v0.18.18

type StorageCircuit struct {
	FailureThreshold int           `yaml:"failure_threshold"` // consecutive failures in one class before the class opens
	ResetTimeout     time.Duration `yaml:"reset_timeout"`     // how long an open breaker stays open
	RequestTimeout   time.Duration `yaml:"request_timeout"`   // per-request timeout for non-streaming operations
}

StorageCircuit configures the object-store circuit breaker. Zero values mean "use the built-in default", matching the flags of the same name.

type Subscriber

type Subscriber func(event ChangeEvent)

Subscriber is called when configuration changes.

type Telemetry

type Telemetry struct {
	Endpoint   string  `yaml:"endpoint"`    // OTLP gRPC endpoint (e.g., "localhost:4317")
	Insecure   bool    `yaml:"insecure"`    // use plaintext gRPC (no TLS)
	SampleRate float64 `yaml:"sample_rate"` // 0.0-1.0 (default: 1.0 = always)
}

Telemetry configures OpenTelemetry tracing export.

type Watcher

type Watcher struct {
	// contains filtered or unexported fields
}

Watcher polls a config file for changes and triggers reload. Uses file modtime + size instead of fsnotify to avoid external dependencies.

func NewWatcher

func NewWatcher(cfg WatcherConfig, manager *Manager, logger *slog.Logger) *Watcher

NewWatcher creates a file watcher for configuration hot-reload.

func (*Watcher) Watch

func (w *Watcher) Watch(ctx context.Context)

Watch starts polling in a goroutine and stops when ctx is cancelled.

type WatcherConfig

type WatcherConfig struct {
	Path     string        // config file path
	Interval time.Duration // poll interval (default 2s)
	Debounce time.Duration // debounce window after change detected (default 500ms)
}

WatcherConfig configures the file watcher.

type Worker

type Worker struct {
	MaxConcurrent    int    `yaml:"max_concurrent"`
	CacheBytes       int64  `yaml:"cache_bytes"`
	MemoryBudget     int64  `yaml:"memory_budget"`      // per-task memory budget in bytes (0 = unlimited, no spill)
	SpillDir         string `yaml:"spill_dir"`          // directory for spill files (default: os temp dir)
	ResultStoreBytes int64  `yaml:"result_store_bytes"` // in-memory result store capacity (0 = disabled)
}

Worker configures the worker.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL