Documentation
¶
Overview ¶
Package config provides YAML configuration loading for Wadjet.
Index ¶
- func ChangedKeys(old, new *Config) []string
- func DeferredReason(name string) string
- func EnvNames() []string
- type ABACCondition
- type ABACObligation
- type ABACPolicy
- type ABACRule
- type Alerts
- type Auth
- type AuthAPIKey
- type AuthJWT
- type AuthMTLS
- type AuthPolicy
- type AuthRole
- type ChangeEvent
- type Config
- type FlagValues
- type GRPC
- type GeoIP
- type HTTP
- type Inputs
- type Key
- type Kind
- type Manager
- func (m *Manager) Apply(newCfg *Config) error
- func (m *Manager) Current() *Config
- func (m *Manager) HotReloadable(key string) bool
- func (m *Manager) Reload(path string) error
- func (m *Manager) ReloadWithReport(path string) ([]string, error)
- func (m *Manager) Resolution() *Resolution
- func (m *Manager) Subscribe(fn Subscriber)
- func (m *Manager) SubscribeKeys(prefixes []string, fn Subscriber)
- type NATS
- type Parquet
- type Query
- type QueryLimits
- type Resolution
- type Source
- type Storage
- type StorageCircuit
- type Subscriber
- type Telemetry
- type Watcher
- type WatcherConfig
- type Worker
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func ChangedKeys ¶ added in v0.18.18
ChangedKeys returns the registry keys whose values differ between two configurations, in registry order.
func DeferredReason ¶ added in v0.18.18
DeferredReason returns the recorded mechanism for a deferred key.
Types ¶
type ABACCondition ¶
type ABACCondition struct {
Attribute string `yaml:"attribute"` // e.g. "subject.role", "resource.name", "env.source_ip"
Operator string `yaml:"operator"` // eq, neq, in, not_in, gt, lt, gte, lte, contains, regex, exists, not_exists
Value string `yaml:"value"` // single value or comma-separated for "in"/"not_in"
}
ABACCondition defines a condition that must match for the rule to apply.
type ABACObligation ¶
type ABACObligation struct {
Type string `yaml:"type"` // row_filter, mask_column, deny_column, query_limit
Target string `yaml:"target"` // column name or table name
Value string `yaml:"value"` // filter expression, mask function, limit value
}
ABACObligation defines a side-effect obligation on an allow rule.
type ABACPolicy ¶
type ABACPolicy struct {
Name string `yaml:"name"`
Description string `yaml:"description"`
Priority int `yaml:"priority"` // lower = evaluated first
Enabled *bool `yaml:"enabled"` // nil = true
Rules []ABACRule `yaml:"rules"`
}
ABACPolicy defines an attribute-based access control policy in config.
type ABACRule ¶
type ABACRule struct {
Effect string `yaml:"effect"` // "allow" or "deny"
Conditions []ABACCondition `yaml:"conditions"`
Obligations []ABACObligation `yaml:"obligations"` // only for "allow" rules
}
ABACRule defines a single rule within an ABAC policy.
type Alerts ¶ added in v0.18.18
type Alerts struct {
Enabled bool `yaml:"enabled"` // enable CREATE ALERT DDL and the scheduler
}
Alerts configures the alert DDL and scheduler.
type Auth ¶
type Auth struct {
Enabled bool `yaml:"enabled"`
APIKeys []AuthAPIKey `yaml:"api_keys"`
JWT AuthJWT `yaml:"jwt"`
MTLS AuthMTLS `yaml:"mtls"`
Roles []AuthRole `yaml:"roles"`
Policies []AuthPolicy `yaml:"policies"` // cell-level access policies (legacy)
ABACPolicies []ABACPolicy `yaml:"abac_policies"` // ABAC access control policies
}
Auth configures authentication and authorization.
type AuthAPIKey ¶
type AuthAPIKey struct {
Key string `yaml:"key"`
Name string `yaml:"name"`
Role string `yaml:"role"`
}
AuthAPIKey defines an API key credential.
type AuthJWT ¶
type AuthJWT struct {
Enabled bool `yaml:"enabled"`
Secret string `yaml:"secret"`
PublicKeyFile string `yaml:"public_key_file"`
RoleClaim string `yaml:"role_claim"`
Issuer string `yaml:"issuer"`
}
AuthJWT configures JWT authentication.
type AuthMTLS ¶
type AuthMTLS struct {
Enabled bool `yaml:"enabled"`
CAFile string `yaml:"ca_file"`
CertFile string `yaml:"cert_file"` // server TLS cert
KeyFile string `yaml:"key_file"` // server TLS key
RoleMap map[string]string `yaml:"role_map"` // CN/SAN -> role
DefaultRole string `yaml:"default_role"`
}
AuthMTLS configures mutual TLS authentication.
type AuthPolicy ¶
type AuthPolicy struct {
Table string `yaml:"table"`
Role string `yaml:"role"`
Columns map[string]string `yaml:"columns"` // column -> "allow", "mask", "deny"
RowFilter string `yaml:"row_filter"` // SQL WHERE predicate
}
AuthPolicy defines a cell-level access policy for a table+role.
type AuthRole ¶
type AuthRole struct {
Name string `yaml:"name"`
Tables []string `yaml:"tables"` // table names or "*" for all
Allow []string `yaml:"allow"` // "read", "write", "admin"
QueryLimits *QueryLimits `yaml:"query_limits"` // per-role overrides (nil = use global)
}
AuthRole defines a role with table access and permissions.
type ChangeEvent ¶
ChangeEvent describes what changed in a configuration update.
type Config ¶
type Config struct {
Mode string `yaml:"mode"` // standalone, coordinator, worker
Storage Storage `yaml:"storage"`
NATS NATS `yaml:"nats"`
HTTP HTTP `yaml:"http"`
GRPC GRPC `yaml:"grpc"`
Worker Worker `yaml:"worker"`
Parquet Parquet `yaml:"parquet"`
Auth Auth `yaml:"auth"`
GeoIP GeoIP `yaml:"geoip"`
Alerts Alerts `yaml:"alerts"` // CREATE ALERT DDL + scheduler
QueryLimits QueryLimits `yaml:"query_limits"` // global query cost limits
Query Query `yaml:"query"` // coordinator query lifecycle
Telemetry Telemetry `yaml:"telemetry"` // OpenTelemetry tracing export
}
Config is the top-level configuration for Wadjet.
func DefaultConfig ¶
func DefaultConfig() Config
DefaultConfig returns a configuration with sensible defaults.
func Load ¶
Load reads a YAML config file and merges with defaults.
The decode is STRICT: a key the schema does not define is an error naming it. Before the precedence loader a mistyped `storage:` key was inert anyway, because the whole section was; now the difference between `bucket:` and `buckett:` is the difference between reading the right bucket and the wrong one, with nothing said at startup. PostgreSQL refuses an unrecognised parameter in postgresql.conf for the same reason, and ADR-0029 already takes its precedence order.
The cost is forward compatibility: an older binary refuses a file written for a newer one. That is the trade this repo wants — a silently ignored key is a silently wrong deployment, which is the whole of #808.
func LoadOrDefault ¶
LoadOrDefault loads a config file if it exists, otherwise returns defaults. Environment variables with WADJET_ prefix always override file/default values.
func LoadWithKeys ¶ added in v0.18.18
LoadWithKeys reads a YAML config file and returns the config merged over the defaults (exactly Load's result) together with the set of registry keys the file actually set.
Presence is decided by unmarshalling a SECOND time into a zero Config and asking whether the key came out non-zero. A key written into the file with its type's zero value is therefore indistinguishable from an absent one — the same convention the environment tier has always used for an empty string, and the reason Resolve treats both alike.
func (*Config) EffectiveQueryLimits ¶ added in v0.18.15
func (c *Config) EffectiveQueryLimits() (*QueryLimits, map[string]*QueryLimits)
EffectiveQueryLimits extracts the cost guard from a loaded config: the global limits, and the per-role map a planner resolves against for the identity answering a query.
EVERY configured role gets an entry, nil meaning unlimited — a role that declares no `query_limits` OVERRIDES the global limits rather than inheriting them, which is what docs/security.md's "Per-Role Limits" section says and how its `admin` example is meant to read. A role nobody configured (and an unauthenticated request) falls back to the global limits.
Returns (nil, nil) for a config with neither, so an unconfigured deployment stays unlimited and pays nothing.
type FlagValues ¶ added in v0.18.18
type FlagValues interface {
// Changed reports whether the flag was set on the command line.
Changed(flag string) bool
// Value returns the flag's current value (its default when not
// Changed), or ok=false when the resolver's flag name is unknown.
Value(flag string) (any, bool)
// Default returns the flag's registered default, snapshotted before the
// command line was parsed. It is the DEFAULT TIER for every key that
// has a flag: the binary runs on the flag's default today, and
// DefaultConfig()'s value is not always the same one (DefaultConfig
// sets storage.access_key to "minioadmin" where --access-key defaults
// to "" and means "auto-detect from env/IAM").
Default(flag string) (any, bool)
}
FlagValues is the resolver's view of a parsed command line.
Value returns the variable the flag is bound to, which holds the flag's DEFAULT until the operator sets it — so a single accessor serves both the default tier and the explicit-flag tier, and Changed is what separates them. That is the whole of ADR-0029's "a flag counts only if Changed".
type GeoIP ¶
type GeoIP struct {
CityDB string `yaml:"city_db"` // path to GeoLite2-City.mmdb
ASNDB string `yaml:"asn_db"` // path to GeoLite2-ASN.mmdb
}
GeoIP configures MaxMind GeoIP database paths.
type Inputs ¶ added in v0.18.18
type Inputs struct {
// File is the config file merged over DefaultConfig() — i.e. Load()'s
// result — or nil when no --config was given. Sections outside the
// registry (auth, abac_policies, per-role query limits) are carried
// through from here unchanged.
File *Config
// FileKeys names the registry keys the file actually set. Use
// LoadWithKeys to build it; a key absent here never wins its tier even
// when File carries a merged default for it.
FileKeys map[string]bool
// Lookup reads an environment variable. nil means os.LookupEnv.
Lookup func(string) (string, bool)
// Flags is the parsed command line. nil means no flag tier, in which
// case DefaultConfig() supplies the default tier.
Flags FlagValues
}
Inputs are the tiers a resolution draws from.
type Key ¶ added in v0.18.18
type Key struct {
// Name is the dotted config-file path, e.g. "storage.bucket".
Name string
// Env is the environment variable that sets this key ("" = none).
Env string
// Flag is the root command's persistent flag name ("" = none).
Flag string
// Kind is the value type.
Kind Kind
// Secret marks a value that must never be echoed back (admin GET
// reports its SOURCE but redacts the value).
Secret bool
// Deferred marks a key that is parsed, resolved and reported but does
// NOT reach a runtime consumer, with the structural change that would
// be required named in DeferredWhy. Rule 11 of
// docs/design/correctness-fix-protocol.md: such a key is deferred
// explicitly, never left half-live — every write path refuses it.
Deferred bool
DeferredWhy string
// contains filtered or unexported fields
}
Key describes one configuration setting: where it can come from, what it carries, and how to read or write it on a Config.
The registry is the single source of truth for the precedence machinery. The resolver (resolve.go), applyEnvOverrides, the admin endpoint's effective-value report and the docs' name-set gate all read this table, so a setting cannot exist on one of those paths and be missing from another.
func (Key) IsZero ¶ added in v0.18.18
IsZero reports whether v is the zero value for the key's Kind.
type Kind ¶ added in v0.18.18
type Kind int
Kind is the Go type a configuration key carries. It decides how an environment variable's string is parsed and how a flag value is asserted.
type Manager ¶
type Manager struct {
// contains filtered or unexported fields
}
Manager provides atomic access to configuration with change notification. Reads are lock-free via atomic.Pointer. Writes are serialized and notify subscribers.
func NewManager ¶
NewManager creates a ConfigManager with the given initial config.
func NewManagerFromResolution ¶ added in v0.18.18
func NewManagerFromResolution(res *Resolution, logger *slog.Logger) *Manager
NewManagerFromResolution creates a Manager over a resolved configuration, keeping the per-key source so GET /v1/admin/config can report where each effective value came from (#828).
func (*Manager) Apply ¶
Apply atomically replaces the configuration and notifies subscribers.
Every registry key that no subscriber consumes is PRESERVED from the current config. The manager is what GET /v1/admin/config reports, so a value it holds must be a value the process is actually running on: a file edit or an admin write that moves a key nothing re-reads would otherwise make the endpoint report a configuration that does not exist until the next restart (#828). The freeze used to be a hardcoded list — Mode, HTTP.Addr and the NATS fields — which left worker.* and parquet.* free to drift away from the running process.
Sections outside the registry (auth and its policies) are not touched: that is where the hot-reload path actually lives.
func (*Manager) HotReloadable ¶ added in v0.18.18
HotReloadable reports whether a runtime change to key would reach a consumer — that is, whether any subscriber registered a prefix covering it. A deferred key (registry.Key.Deferred) is never hot-reloadable.
func (*Manager) ReloadWithReport ¶ added in v0.18.18
ReloadWithReport reads the config file, applies it, and returns the registry keys THE FILE SETS that no subscriber consumes — the ones Apply preserved.
Apply preserving them is right: the running process is not going to re-read a startup-only key, and the manager must report the running configuration. But saying nothing about the part that was ignored is how an operator edits `worker.max_concurrent`, sees "reloaded", and believes it took effect. The PUT path answers 409 naming such a key; a file reload cannot refuse (the file legitimately carries startup-only keys for the NEXT start), so it reports instead.
The report is FileKeys ∩ !HotReloadable, and it has to be. Diffing the running config against Load(path) instead names keys the file never mentions: the running config's default tier is the FLAG's default, while Load merges over DefaultConfig(), and decision 2 of ADR-0029 exists precisely because those two differ — DefaultConfig() sets storage.access_key to "minioadmin" where --access-key defaults to "", and worker.cache_bytes to 256 MiB where --cache-bytes defaults to 0. That diff reported three keys on every reload of any file in any deployment before this, plus one per key taken from a flag or the environment, so the config-file WATCHER emitted the warning on every legitimate auth edit and the one true positive arrived buried in sixteen false ones.
func (*Manager) Resolution ¶ added in v0.18.18
func (m *Manager) Resolution() *Resolution
Resolution returns the current configuration with its per-key sources.
func (*Manager) Subscribe ¶
func (m *Manager) Subscribe(fn Subscriber)
Subscribe registers a callback for configuration changes. Subscribers are called synchronously in order during Apply.
A bare Subscribe claims NO config keys, so it does not make anything hot-reloadable. Use SubscribeKeys to declare which keys the callback actually applies.
func (*Manager) SubscribeKeys ¶ added in v0.18.18
func (m *Manager) SubscribeKeys(prefixes []string, fn Subscriber)
SubscribeKeys registers a callback and declares the config-key prefixes it applies at runtime. Those prefixes are exactly the keys the admin API will accept a write for; everything else is refused as not hot-reloadable.
type NATS ¶
type NATS struct {
Port int `yaml:"port"`
URL string `yaml:"url"` // for worker mode: coordinator's NATS URL
StoreDir string `yaml:"store_dir"` // JetStream storage directory
ClusterID string `yaml:"cluster_id"` // unique cluster identifier (e.g., "central", "afb-east")
LeafRemotes []string `yaml:"leaf_remotes"` // remote NATS URLs for leaf node connections
TLSCert string `yaml:"tls_cert"` // TLS certificate file (server or client)
TLSKey string `yaml:"tls_key"` // TLS private key file
TLSCA string `yaml:"tls_ca"` // CA certificate for verifying peers (enables mTLS)
}
NATS configures the embedded NATS server or client connection.
type Parquet ¶
type Parquet struct {
Compression string `yaml:"compression"` // snappy, zstd, gzip, lz4, none
RowGroupSize int `yaml:"row_group_size"` // rows per row group
PageBufferSize int `yaml:"page_buffer_size"` // page size in bytes
}
Parquet configures Parquet file writing.
type Query ¶ added in v0.18.18
type Query struct {
// IntermediateTTL is the age after which the coordinator's periodic
// sweep reclaims a queries/<id>/* prefix the per-query cleanup did not.
IntermediateTTL time.Duration `yaml:"intermediate_ttl"`
// IntermediateSweep is how often that sweep runs.
IntermediateSweep time.Duration `yaml:"intermediate_sweep"`
}
Query configures the coordinator's query lifecycle. Zero values mean "use the built-in default", matching the flags of the same name.
type QueryLimits ¶
type QueryLimits struct {
MaxScanBytes int64 `yaml:"max_scan_bytes"` // max estimated bytes across all scans
MaxScanRows int64 `yaml:"max_scan_rows"` // max estimated rows across all scans
MaxScanFiles int `yaml:"max_scan_files"` // max files across all scans
RequireFilterAboveBytes int64 `yaml:"require_filter_above_bytes"` // require WHERE on tables exceeding this size
RequireLimitAboveRows int64 `yaml:"require_limit_above_rows"` // require LIMIT on scans exceeding this row count
}
QueryLimits configures cost-based query guards. Zero values mean unlimited. Per-role limits in Auth.Roles override these global defaults.
type Resolution ¶ added in v0.18.18
type Resolution struct {
// contains filtered or unexported fields
}
Resolution is a fully resolved configuration plus the tier each key came from.
func Resolve ¶ added in v0.18.18
func Resolve(in Inputs) *Resolution
Resolve applies the precedence order settled in ADR-0029:
explicit flag > environment variable > config file > default
A flag counts only when FlagValues reports it Changed; a flag's DEFAULT never beats an environment variable or a config-file value. An empty string never overrides a lower tier, on any tier — that is how the environment layer has always behaved, and applying it to flags too means `--nats-tls-cert=""` reads as "unset" rather than as "explicitly blank", which is what every string flag here means by an empty value.
func (*Resolution) Config ¶ added in v0.18.18
func (r *Resolution) Config() *Config
Config returns the resolved configuration.
func (*Resolution) DeferredKeysSet ¶ added in v0.18.18
func (r *Resolution) DeferredKeysSet() []string
DeferredKeysSet returns every DEFERRED key this resolution took from a tier other than the default — that is, every key an operator configured that no runtime consumer reads.
Rule 11's "never left half-live" is what this serves. Parsing such a section silently and refusing it only on the admin WRITE path leaves the operator's startup configuration accepted and inert, which is the exact defect #808 was filed for; the caller turns this into a refusal that names the keys.
func (*Resolution) Source ¶ added in v0.18.18
func (r *Resolution) Source(key string) Source
Source returns the tier key came from, or SourceDefault for an unknown key.
func (*Resolution) Sources ¶ added in v0.18.18
func (r *Resolution) Sources() map[string]Source
Sources returns a copy of the per-key source map.
type Source ¶ added in v0.18.18
type Source string
Source names the tier a resolved value came from.
const ( // SourceDefault is the built-in default — the flag's registered default // where the key has a flag, DefaultConfig()'s value otherwise. SourceDefault Source = "default" // SourceFile is the YAML configuration file. SourceFile Source = "file" // SourceEnv is a WADJET_* environment variable. SourceEnv Source = "env" // SourceFlag is a command-line flag the operator actually typed. SourceFlag Source = "flag" // SourceAdmin is a runtime change through the admin API. SourceAdmin Source = "admin" )
type Storage ¶
type Storage struct {
Type string `yaml:"type"` // "s3" (default) or "file"
DataDir string `yaml:"data_dir"` // local directory for type=file
Endpoint string `yaml:"endpoint"`
AccessKey string `yaml:"access_key"`
SecretKey string `yaml:"secret_key"`
Bucket string `yaml:"bucket"`
UseSSL bool `yaml:"use_ssl"`
Region string `yaml:"region"`
// Circuit configures the per-operation-class object-store circuit
// breaker (ADR-0028).
Circuit StorageCircuit `yaml:"circuit"`
}
Storage configures the object store connection.
type StorageCircuit ¶ added in v0.18.18
type StorageCircuit struct {
FailureThreshold int `yaml:"failure_threshold"` // consecutive failures in one class before the class opens
ResetTimeout time.Duration `yaml:"reset_timeout"` // how long an open breaker stays open
RequestTimeout time.Duration `yaml:"request_timeout"` // per-request timeout for non-streaming operations
}
StorageCircuit configures the object-store circuit breaker. Zero values mean "use the built-in default", matching the flags of the same name.
type Subscriber ¶
type Subscriber func(event ChangeEvent)
Subscriber is called when configuration changes.
type Telemetry ¶
type Telemetry struct {
Endpoint string `yaml:"endpoint"` // OTLP gRPC endpoint (e.g., "localhost:4317")
Insecure bool `yaml:"insecure"` // use plaintext gRPC (no TLS)
SampleRate float64 `yaml:"sample_rate"` // 0.0-1.0 (default: 1.0 = always)
}
Telemetry configures OpenTelemetry tracing export.
type Watcher ¶
type Watcher struct {
// contains filtered or unexported fields
}
Watcher polls a config file for changes and triggers reload. Uses file modtime + size instead of fsnotify to avoid external dependencies.
func NewWatcher ¶
func NewWatcher(cfg WatcherConfig, manager *Manager, logger *slog.Logger) *Watcher
NewWatcher creates a file watcher for configuration hot-reload.
type WatcherConfig ¶
type WatcherConfig struct {
Path string // config file path
Interval time.Duration // poll interval (default 2s)
Debounce time.Duration // debounce window after change detected (default 500ms)
}
WatcherConfig configures the file watcher.
type Worker ¶
type Worker struct {
MaxConcurrent int `yaml:"max_concurrent"`
CacheBytes int64 `yaml:"cache_bytes"`
MemoryBudget int64 `yaml:"memory_budget"` // per-task memory budget in bytes (0 = unlimited, no spill)
SpillDir string `yaml:"spill_dir"` // directory for spill files (default: os temp dir)
ResultStoreBytes int64 `yaml:"result_store_bytes"` // in-memory result store capacity (0 = disabled)
}
Worker configures the worker.