Documentation
¶
Overview ¶
Package settings is the runtime half of the settings directory: the Registry of tenants and the Store each one's adopted snapshot lives in, the reload triggers, the directory watcher, and WriteSeed. The model it adopts — the document types, the validator, the embedded seed — is pkg/settings, importable from outside the module.
Index ¶
- Constants
- func WriteSeed(dir string) error
- type Auth
- type ClickHouse
- type Dedupe
- type Registry
- func (r *Registry) AfterAdopt(fn func(adopted []tenant.ID))
- func (r *Registry) All() iter.Seq2[tenant.ID, *Store]
- func (r *Registry) Dir() string
- func (r *Registry) For(id tenant.ID) (*Store, bool)
- func (r *Registry) Known() iter.Seq2[tenant.ID, *Store]
- func (r *Registry) Nested() bool
- func (r *Registry) Reconciled(component string, err error)
- func (r *Registry) Reload(trigger string) ([]settings.Finding, bool)
- func (r *Registry) ReloadTenant(id tenant.ID, trigger string) (findings []settings.Finding, adopted, known bool)
- func (r *Registry) Resolve(id tenant.ID) (store *Store, known bool)
- func (r *Registry) Stats() ReloadStats
- func (r *Registry) Watch(ctx context.Context) error
- type ReloadKey
- type ReloadStats
- type Store
- func (s *Store) Auth() Auth
- func (s *Store) CORSOrigins() []string
- func (s *Store) ClickHouse() ClickHouse
- func (s *Store) DLQFor(table string) bool
- func (s *Store) DedupeEnabled() bool
- func (s *Store) DedupeFor(table string) Dedupe
- func (s *Store) DedupeRetentions() map[string]time.Duration
- func (s *Store) DefaultMaxRows() int
- func (s *Store) GapWindow() time.Duration
- func (s *Store) Keepalive() (period time.Duration, buckets int)
- func (s *Store) MQMaxBytes() int64
- func (s *Store) Pipe(name string) *pipes.NamedQuery
- func (s *Store) Pipes() []*pipes.NamedQuery
- func (s *Store) Policy() *policy.Policy
- func (s *Store) SchemaRefreshInterval() time.Duration
- func (s *Store) Tenant() tenant.ID
- func (s *Store) TimestampBucketSeconds() int
- type TLS
Constants ¶
const ( // ResultAdopted is a reload that adopted every tenant's folder, and that // every consumer reporting through Reconciled reconciled with. ResultAdopted = "adopted" // ResultPartial is a reload that applied, but with a tenant's folder // rejected or a consumer that failed to reconcile with it. ResultPartial = "partial" // ResultRejected is a reload refused whole: a flat directory with an // error, a finding about the directory itself, or the one tenant // ReloadTenant read rejected. A nested reload rejecting every folder // is partial. ResultRejected = "rejected" )
Reload outcomes, the Result of a ReloadKey.
Variables ¶
This section is empty.
Functions ¶
Types ¶
type Auth ¶
Auth is the adopted verifier wiring, HMAC secret included (the operator key is boot config).
type ClickHouse ¶
type ClickHouse struct {
Addr string
HTTPPort int
HTTPScheme string
Database string
Username string
Password string
QueryTimeout time.Duration
TLS TLS
// Headers is this reader's own copy of the HTTP-interface headers.
Headers map[string]string
MaxOpenConns int
MaxIdleConns int
}
ClickHouse is the adopted connection wiring, resolved as one value from one snapshot so a reconnect never mixes the address of one document with the database of another.
type Dedupe ¶
type Dedupe struct {
Enabled bool
IDField string
RequireID bool
// Retention is how long a committed id stays a duplicate; 0 is forever.
Retention time.Duration
}
Dedupe is a table's effective dedupe settings.
type Registry ¶
type Registry struct {
// contains filtered or unexported fields
}
Registry maps a tenant id to the Store holding that tenant's adopted settings, and owns everything that changes one. Open validates the directory and adopts it up front. Reload is the single code path every later trigger — SIGHUP, the directory watcher, and POST /v1/ops/settings/reload — funnels through. The stores are passive: a document pointer their getters read.
What a rejected directory costs depends on its shape (#583), which is fixed at Open — switching shapes is stop, restructure, start:
- Flat, the four files in the directory itself: tenant.Default alone. Open refuses an invalid directory, so a flat *Registry never exists without a good document behind it, and a reload swaps the snapshot only when no finding is an error, so a bad edit (or a deleted file, or a vanished directory) can never evict the last good document.
- Nested, one folder per tenant: fail closed per tenant, at Open and on reload alike. A folder with an error finding stops being served — the registry still knows the tenant, so its requests are refused rather than unknown — and every other tenant carries on; there is no previous-snapshot fallback. A reload mirrors the folders: a new one is served and a removed one is forgotten, the last one too. A finding about the directory itself (a loose file, an unreadable directory, a changed shape) rejects the reload whole and leaves every tenant as it was; at Open it refuses boot. So does a nested directory that would serve no tenant at Open — every folder rejected, or none named by a tenant id, as a fresh volume holding only lost+found is — since a server serving no tenant is what a wrong mount looks like, not what anyone meant; on reload the same tree drops every tenant and keeps running (#599).
func NewRegistry ¶
NewRegistry returns a flat Registry serving store as tenant.Default, with no directory behind it: what a test that fixes its settings holds. The store is stamped with that id, as one a Registry creates is.
func Open ¶
Open validates dir and returns a Registry serving it. A rejected directory — flat and invalid, of either shape with a finding about the directory itself, or nested with no tenant to serve — returns a nil Registry with the findings: the caller (boot) refuses to start. A nested directory with a rejected tenant folder beside a valid one still opens; the rejected tenant alone is not served.
func (*Registry) AfterAdopt ¶
AfterAdopt registers fn to run after each subsequent reload the registry applied, serialized with the reload itself, with the tenants it adopted: none for a reload that only rejected or removed a tenant, which a consumer holding a resource per tenant needs to hear of as much as an adoption. A whole-directory reload rejected whole runs nothing, since nothing changed. Open's boot adoption has already happened by the time a caller can register, so the caller applies the boot state itself. A hook whose work can fail reports how it went through Reconciled.
func (*Registry) All ¶
All iterates over the tenants being served, in id order — for a consumer that owns one resource every tenant shares and has to weigh their settings against each other.
func (*Registry) For ¶
For returns the store of tenant id, or false when the registry is not serving that tenant — it has no such tenant, or the tenant's folder was rejected.
func (*Registry) Known ¶
Known iterates over every tenant the registry holds, served or rejected, in id order, each with the store holding its last adopted settings — nil for a tenant whose folder has not validated since boot — for a consumer that must keep a rejected tenant's resources current too: the tenant comes back into service with them, and a rejection is the common reload failure (a typo, fixed and reloaded minutes later). A rejected tenant's store serves no request; it is handed out for what those resources read of it.
func (*Registry) Nested ¶
Nested reports the directory's shape: one folder per tenant (true) or the four files of tenant.Default (false).
func (*Registry) Reconciled ¶ added in v0.2.1
Reconciled reports how a consumer that follows the adopted settings (component names it: "clickhouse", "dedupe", "mq") fared at reconciling with them, from its AfterAdopt hook or from the boot apply that follows Open. A nil err records nothing. A failure is counted, and makes the reload it follows partial: the one whose hooks are running, or, from the boot apply, Open's own, which then no longer counts as adopted — so read Stats for a counter only once boot is done, or it would go down.
func (*Registry) Reload ¶
Reload re-validates the directory and adopts what it finds (see Registry for what a rejection costs in each shape). Warnings don't block adoption, matching `wavehouse validate`. The returned bool reports whether everything was adopted: no finding is an error. In a nested directory false can therefore mean adopted in part — the tenants whose folders validated were adopted, warnings and all, and the ones with an error finding were not. trigger names the path that fired ("boot", "sighup", "watch", "api") and tags every log line so operators can tell them apart.
func (*Registry) ReloadTenant ¶
func (r *Registry) ReloadTenant(id tenant.ID, trigger string) (findings []settings.Finding, adopted, known bool)
ReloadTenant is Reload for one tenant's folder of a nested directory: the rest of the directory is not read, so it can neither adopt nor drop another tenant — what the writer of one folder calls when that folder is complete. The folder is adopted, or the tenant stops being served. known is false for a tenant the registry does not hold, and nothing is read: a whole-tree Reload is what picks up a new folder. A flat directory is its default tenant's folder, so there this is Reload.
func (*Registry) Resolve ¶
Resolve is For for a caller that answers the two misses differently: a nil store with known=true is a tenant whose folder was rejected, and with known=false a tenant the registry has never heard of.
func (*Registry) Stats ¶ added in v0.2.1
func (r *Registry) Stats() ReloadStats
Stats returns how this process's reloads have gone so far.
func (*Registry) Watch ¶
Watch blocks until ctx is done, watching the settings directory and running the shared reload path when its contents change. A change that fails validation is logged and skipped — the previous good snapshot stays — so an operator mid-edit degrades to a log line, never to a broken server.
The directory itself is watched (not the four files): atomic-writer flows replace files wholesale, and Kubernetes ConfigMap mounts swap a symlink, both of which per-file watches lose track of. The parent directory is watched too, because fsnotify silently drops a watch whose directory is removed or renamed: with only the directory watch, a delete-and-recreate (or a rename-over, the atomic way to replace a whole directory) would leave every later edit unwatched. Parent events are filtered to the settings directory's own name, and the directory watch is re-added on every reload so it is restored once the directory exists again.
Once the watch is registered the directory is reloaded immediately, so an edit that landed between the boot read and the watch is not lost.
The setup error is returned (directory missing, fd limits); runtime watcher errors are logged and the loop continues — SIGHUP and the ops reload endpoint remain as triggers even if the watcher degrades.
Flat directories only, and a nested registry is refused rather than trusted to its caller: "the previous good snapshot stays" does not hold there. A watcher would validate a tenant's folder halfway through being written, and with no previous-snapshot fallback that drops the tenant.
type ReloadKey ¶ added in v0.2.1
type ReloadKey struct{ Trigger, Result string }
ReloadKey counts reloads by the trigger that fired them and their result.
type ReloadStats ¶ added in v0.2.1
type ReloadStats struct {
// Served and Rejected count the tenants the registry holds now.
Served, Rejected int
// Removed counts the tenants reloads have dropped since boot.
Removed int64
// Reloads counts every reload since boot, Open's included.
Reloads map[ReloadKey]int64
// ReconcileFailures counts the failures reported through Reconciled,
// by component.
ReconcileFailures map[string]int64
// LastSuccessful is whether the latest reload's result was
// ResultAdopted.
LastSuccessful bool
// LastSuccess is when a reload's result last was ResultAdopted; zero
// if none has been.
LastSuccess time.Time
}
ReloadStats is how this process's settings reloads have gone (#776), for the metrics internal/app exports: reloads are per process, so a tenant can be rejected or half-reconciled on one process and fine on the next.
type Store ¶
type Store struct {
// contains filtered or unexported fields
}
Store holds the settings snapshot one tenant has adopted, and nothing else: the Registry validates, swaps the document in, and owns every reload trigger. Readers go through one lock-free atomic load per lookup; the typed accessors below each resolve from a single snapshot load, so a reload lands between lookups, never inside one.
There are no compiled defaults here on purpose, but one: every key but dedupe.retention (missing means "0", forever) is required by Validate, so the snapshot is exactly what the files said when they were adopted. Defaults live in the seed directory (Seed / WriteSeed).
func (*Store) CORSOrigins ¶
CORSOrigins returns the allowed CORS origins; an empty list denies every browser origin and ["*"] is the only allow-all spelling.
func (*Store) ClickHouse ¶
func (s *Store) ClickHouse() ClickHouse
ClickHouse returns the adopted ClickHouse wiring.
func (*Store) DLQFor ¶
DLQFor reports whether a poison row for table is parked on the DLQ (true) or left unacked for redelivery (false): the table override when present, the global switch otherwise.
func (*Store) DedupeEnabled ¶
DedupeEnabled reports the adopted dedupe.enabled switch.
func (*Store) DedupeFor ¶
DedupeFor resolves the effective dedupe settings for a table: the switch, then the table override for each field it names, the global value otherwise. Every field resolves from one snapshot load, so a reload can never hand a record the id_field of one document and the require_id, retention or switch of another.
func (*Store) DedupeRetentions ¶
DedupeRetentions is the retention of the default (key "") and of every table override that names one, from one snapshot; 0 is forever.
func (*Store) DefaultMaxRows ¶
DefaultMaxRows returns the fallback result LIMIT for structured queries.
func (*Store) GapWindow ¶
GapWindow returns how much ACKed history the sweeper keeps for SSE gap-fill.
func (*Store) Keepalive ¶
Keepalive returns the SSE keepalive period and bucket count together, from one snapshot, so the wheel is never rebuilt from a mixed pair.
func (*Store) MQMaxBytes ¶
MQMaxBytes returns the disk budget of the tenant's ingest queue in bytes.
func (*Store) Pipe ¶
func (s *Store) Pipe(name string) *pipes.NamedQuery
Pipe returns the adopted named query, or nil when pipes.json defines none by that name. Satisfies pipes.Source with Pipes.
func (*Store) Pipes ¶
func (s *Store) Pipes() []*pipes.NamedQuery
Pipes returns every adopted named query.
func (*Store) Policy ¶
Policy returns the adopted access-control policy (policies.json). nil is the deliberate lockout an empty document spells: every token-based request is denied until a policy is adopted. Satisfies policy.Source.
func (*Store) SchemaRefreshInterval ¶
SchemaRefreshInterval returns the schema-discovery auto-refresh period.
func (*Store) Tenant ¶
Tenant returns the id of the tenant this store holds the settings of: how a handler holding the request's store names its tenant to a per-tenant resource (#583) without a second read of the context.
func (*Store) TimestampBucketSeconds ¶
TimestampBucketSeconds returns the time-range bucket structured queries truncate to (0 = no bucketing).