Documentation
¶
Overview ¶
Package config reads a SemEngine process's configuration and declares the JetStream streams it needs, and its Manager keeps that configuration in a NATS key-value bucket across restarts.
Loading ¶
A Loader reads one JSON document, lays it over built-in defaults, applies the environment overrides, and checks that platform.org and platform.id leave room for the entity IDs the deployment will create (ADR-104). It does not call Validate; the caller does:
cfg, err := config.NewLoader().LoadFile("config.json")
if err != nil {
return err
}
if err := cfg.Validate(); err != nil {
return err
}
LoadFromBytes does the same for a document already in memory. A key the document sets replaces the default, including false, 0 or an empty list; a key it leaves out keeps the default; objects merge key by key.
Write every key exactly as the README shows it. A key that differs from a declared key only in letter case fails the load, naming the right spelling. Names a deployment chooses (component, service, stream, archival and model registry names) are not checked, and a component's or service's config payload belongs to its owner, which decodes it.
Loading fails, returning no Config, when the document:
- is not exactly one JSON object (null is refused too), is larger than 10 MB, or nests deeper than 100 levels;
- has a value that does not fit its field, such as a string where a list is expected, or an unknown key inside a services entry (the error carries the decoder's message);
- sets platform.instance_id, which was removed (ADR-102);
- sets a NATS setting nothing in SemEngine applies yet: nats.username, nats.password, nats.token, nats.tls, nats.max_reconnects, nats.reconnect_wait, nats.jetstream.domain, nats.jetstream.retention_policy or nats.jetstream.replication_factor. These are refused when present, whatever their value and in any letter case, so a setting an operator wrote is never silently ignored;
- has a key that differs from a declared key only in letter case, such as "NATS" or nats "URLs": the error names the key as written and as declared, for example "config key nats.URLs must be written nats.urls". When the document also sets one of the keys above, that refusal wins.
Other unknown keys are accepted and ignored.
Environment overrides ¶
When set and not empty, SEMENGINE_PLATFORM_ID, SEMENGINE_PLATFORM_TYPE, SEMENGINE_PLATFORM_REGION and SEMENGINE_NATS_URLS replace the document's platform.id, platform.type, platform.region and nats.urls. The URL list is split at commas and spaces are not trimmed, so write a,b, not a, b. Loading fails when SEMENGINE_NATS_USERNAME, SEMENGINE_NATS_PASSWORD or SEMENGINE_NATS_TOKEN is set and not empty, naming the variable: those credentials are not applied yet.
Which files LoadFile reads ¶
The path must end in .json or .json5; the content is read as plain JSON either way. A relative path must stay inside the current working directory once . and .. are resolved. An absolute path is not confined to any directory, but is refused if a name in it contains "..". Symbolic links are followed, so a link to a regular file is read wherever that file is. A directory or device is refused.
Sharing a configuration ¶
SafeConfig wraps a Config for concurrent use. Get returns a deep copy. Mutate holds the write lock from the read to the swap, so use it, not Get followed by Update, to change part of a configuration another goroutine may also change.
The configuration manager ¶
Manager keeps a deployment's configuration in a NATS key-value bucket, the configuration bucket, so it outlives the process and another process can change what the next boot runs. BucketName names the bucket from the document's platform.org and declared platform.id: semengine_config_<org>_<stem>.
Start(ctx) does its startup work under ctx: it acquires the bucket, establishes the deployment's platform identity from the bucket's identity record (ADR-104), decides whether the file or the bucket wins, and opens the watches. It refuses a nil or ended ctx, and refuses every Start after the first with errs.ErrAlreadyStarted. Ending ctx after Start returned stops nothing (ADR-113): the watches run until Stop.
Stop(ctx) closes every channel the Manager handed out, stops every watch and waits for the Manager's goroutines. It refuses a nil ctx; any non-nil ctx (live or already ended) initiates teardown at once and bounds only the join wait, returning nil once all goroutines exit or ctx's error if ctx ended first so a later Stop tries again. Pass a fresh bounded context on shutdown so watches have a graceful drain budget:
stopCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), 5*time.Second) defer cancel() err := mgr.Stop(stopCtx)
Which copy wins at Start: on a first boot, and when the file's version is newer than the bucket's version key (CompareVersions), Start writes the file to the bucket. Otherwise (equal versions, an older file, or either version not a semantic version, a file with no version among them) the bucket wins: its services and components replace the file's, so one deleted at run time stays deleted. A missing version key, and a stored version that fails to decode into a string, such as a number (logged as a warning), compare as 0.0.0, so a file at 1.0.0 wins; a stored string that is not a semantic version selects the bucket, and so does a stored null, which decodes as an empty string. A stored model_registry replaces the file's; a stored nats is decoded over the file's nats section, so a key it leaves out keeps the file's value. The stored platform key is a copy for readers and is never applied. Start writes the version key last, so a write cut short leaves the old version, or none; the next boot writes again only when the file's version is newer than that. A file with no version, or 0.0.0, is not, and selects the partly written bucket.
Each service and component name must be one NATS KV key token once its spaces are written as '_': no '.', '*' or '>'. A name with a space is kept under its key's name from Start on, so GetConfig reports a file component "a b" as "a_b", and two names that map to one key are refused.
Start fails closed. It returns an error and leaves every writer refusing when a name or value cannot be written, the bucket cannot be listed or read, a stored services, components, nats or model_registry entry does not decode, a stored nats entry sets a key loading refuses, the write to the bucket fails, or a watch does not open. The error names the key. A refused name or value is found before the bucket is touched, so nothing is written, not even the identity record. A failed Start rolls back what it acquired before returning; if that rollback fails, its error is joined into Start's and the rest of the cleanup is kept for Stop, so a caller installs its bounded Stop before calling Start.
PutComponentToKV and DeleteComponentFromKV change the components the next boot runs. They update GetConfig before they return; a put whose key the watcher has already applied at a newer revision keeps that newer value, and no watcher event is applied between a delete reaching the bucket and its removal from GetConfig, so a delete never erases a later put. They never start, stop or restart a running component. WatchModelRegistry delivers the model registry each time its key changes.
A key family (NewKeyFamily, WithKeyFamily) hands another owner the keys "<prefix>.<name>" in the same bucket: its handler receives the entries present when Start opened its watch, then each put and delete, and its Get, Put, Create, Delete and Names reach only those keys. The Manager never decodes a family's values. A prefix the Manager uses itself (services, components, platform, nats, model_registry, platform_identity) is refused.
Streams ¶
StreamsManager.EnsureStreams creates or repairs the JetStream streams a Config needs: the framework's own, those in its streams section, and those its components' JetStream output ports imply. Every stream must declare a finite max_age, a max_bytes above zero and a discard policy, unless it is declared archival. Config.Validate checks those declarations without a server (ValidateStreamDeclarations).
The package README gives an example document and the rules in full.
Index ¶
- Constants
- Variables
- func BucketName(org, stem string) (string, error)
- func CompareVersions(v1, v2 string) (int, error)
- func DeriveStreamName(subject string) string
- type ArchivalStream
- type ArchivalStreamStatus
- type ArchivalStreams
- type ComponentConfigs
- type Config
- type JetStreamConfig
- type KeyFamily
- func (f *KeyFamily) Create(ctx context.Context, name string, value []byte) error
- func (f *KeyFamily) Delete(ctx context.Context, name string) error
- func (f *KeyFamily) Get(ctx context.Context, name string) ([]byte, error)
- func (f *KeyFamily) Names(ctx context.Context) ([]string, error)
- func (f *KeyFamily) Put(ctx context.Context, name string, value []byte) error
- type KeyFamilyEntry
- type KeyFamilyOperation
- type Loader
- type Manager
- func (cm *Manager) DeleteComponentFromKV(ctx context.Context, name string) error
- func (cm *Manager) GetConfig() *SafeConfig
- func (cm *Manager) PutComponentToKV(ctx context.Context, name string, compConfig types.ComponentConfig) error
- func (cm *Manager) Start(ctx context.Context) error
- func (cm *Manager) Stop(ctx context.Context) error
- func (cm *Manager) WatchModelRegistry() <-chan *model.Registry
- type ManagerOption
- type NATSConfig
- type PlatformConfig
- type SafeConfig
- type StreamConfig
- type StreamConfigs
- type StreamsManager
Constants ¶
const ( // StreamDiscardOld evicts the oldest messages once the stream reaches a // limit. The write always succeeds; the data loss is silent and at the tail. StreamDiscardOld = "old" // StreamDiscardNew refuses the write once the stream reaches a limit. Nothing // is evicted; producers see the rejection instead. See discardPolicyGuidance. StreamDiscardNew = "new" )
Discard-policy declaration values for StreamConfig.Discard. These are the operator-facing spellings of jetstream.DiscardOld / jetstream.DiscardNew; the mapping happens once, in buildStreamConfig.
Variables ¶
var ( // ErrStreamBoundsUndeclared is returned when an ordinary stream carries no // explicitly declared finite MaxAge, finite MaxBytes, or discard policy and // is not admitted by an archival declaration. // // It is natsclient's value rather than a second one of the same name. The same // requirement is enforced at two seams — this declarative path and // natsclient.Client.EnsureStream — and a caller testing for "bounds are not // declared" should not have to know which door refused it. ErrStreamBoundsUndeclared = natsclient.ErrStreamBoundsUndeclared // ErrArchivalStreamInvalid is returned when an archival declaration omits its // owner or its reason, so archival cannot become a silent way to opt out of // bounds. ErrArchivalStreamInvalid = errors.New("archival stream declaration is incomplete") )
Bounds-contract sentinels. Classifiable so a boot path, an operator tool, or a test can tell an undeclared bound from a malformed archival declaration, which fail validation for different reasons and have different fixes.
var ErrBackingStreamNotProvisionable = natsclient.ErrBackingStreamNotProvisionable
ErrBackingStreamNotProvisionable is the refusal sentinel, re-exported from natsclient so config callers and tests need not reach across for it. It is the SAME error value, so errors.Is matches a refusal raised at any provisioning seam — this provisioner, Client.EnsureStream, or Client.CreateStream.
var ErrStreamNonEditableDrift = errors.New("existing stream diverges in a field that cannot be reconciled in place")
ErrStreamNonEditableDrift is returned when an existing ordinary stream's live configuration diverges from its declaration in a field the provisioner will not change on a live stream. Readiness fails rather than accepting the divergence in silence, because a stream that reports one storage tier or retention policy while its declaration states another is a configuration two people can read opposite answers from.
Functions ¶
func BucketName ¶
BucketName is the ONE derivation of a deployment's configuration bucket name: "semengine_config_<org>_<stem>", from the configuration document's platform.org and its DECLARED platform.id — never the minted identifier, which lives inside the bucket and so cannot name it. The bucket is looked up by the same name it was created under.
It refuses an empty part, a byte outside the NATS bucket grammar [A-Za-z0-9_-] (nats.go jetstream/kv.go validBucketRe), and a result longer than NATS accepts. Every pair Config.Validate admits passes: a segment is already a subset of that grammar, and a declared pair is at most 163 bytes.
The separator is not injective: `_` is legal inside both parts, so org "a_b" with stem "c" and org "a" with stem "b_c" name the same bucket. That alias is refused at Start by the identity record's org/stem compare, not prevented here.
func CompareVersions ¶
CompareVersions compares two semver version strings Returns:
-1 if v1 < v2 0 if v1 == v2 1 if v1 > v2 error if either version is invalid
func DeriveStreamName ¶
DeriveStreamName extracts stream name from subject convention. Convention: subject "component.action.type" → stream "COMPONENT" Examples:
"objectstore.stored.entity" → "OBJECTSTORE" "sensor.processed.entity" → "SENSOR" "rule.triggered.alert" → "RULE"
Types ¶
type ArchivalStream ¶
type ArchivalStream struct {
// Owner is the team or component accountable for this stream's growth.
// Required — an archival stream's only remaining lever is capacity, and
// capacity questions need an addressee.
Owner string `json:"owner"`
// Reason states why permanence is the contract. Required — without it,
// "archival" is just an unbounded stream with better vocabulary.
Reason string `json:"reason"`
}
ArchivalStream declares ONE ordinary stream whose contract is permanence: nothing may ever be evicted from it. It is exempt from the finite-bounds requirement by declaration, and validation reports it as a named PERMANENT exception.
Both fields are required, so archival cannot become an unowned, unexplained way to opt out of bounds.
type ArchivalStreamStatus ¶
type ArchivalStreamStatus struct {
Stream string // the permanently unbounded ordinary stream
Owner string // who is accountable for its growth
Reason string // why permanence is the contract
}
ArchivalStreamStatus is one permanent exception as validation reports it. It has no expiry: archival is permanent by contract.
func ValidateStreamDeclarations ¶
func ValidateStreamDeclarations(cfg *Config) ([]ArchivalStreamStatus, error)
ValidateStreamDeclarations resolves every ordinary stream this configuration would provision and enforces the bounds contract on it, returning one entry per archival_streams entry, sorted by stream name, nil when there is none.
It is pure and I/O-free, which is what lets Config.Validate run it: without that, a validation run would report a configuration valid that hard-fails at boot.
type ArchivalStreams ¶
type ArchivalStreams map[string]ArchivalStream
ArchivalStreams maps ordinary stream name to its archival declaration.
type ComponentConfigs ¶
type ComponentConfigs map[string]types.ComponentConfig
ComponentConfigs holds component instance configurations. The map key is the instance name (e.g., "udp-sensor-main"). Components are only created if both: 1. Their factory has been registered via init() 2. They have an entry in this config map with enabled=true
type Config ¶
type Config struct {
Version string `json:"version"` // Semantic version (e.g., "1.0.0") for KV sync control
Platform PlatformConfig `json:"platform"`
Security security.Config `json:"security,omitempty"` // Platform-wide security configuration
NATS NATSConfig `json:"nats"`
Services types.ServiceConfigs `json:"services"` // Map of service configs
Components ComponentConfigs `json:"components"` // Map of component instance configs
Streams StreamConfigs `json:"streams,omitempty"` // Explicit JetStream stream definitions
ModelRegistry *model.Registry `json:"model_registry,omitempty"` // Unified model endpoint registry
// ArchivalStreams declares ordinary streams whose contract is permanence:
// the one exception to the finite-bounds requirement.
ArchivalStreams ArchivalStreams `json:"archival_streams,omitempty"`
}
Config represents the complete application configuration Simplified to 6 fields: Version (semver), Platform (identity), Security (TLS), NATS (connection), Services, Components
func (*Config) GetPlatform ¶
GetPlatform returns the minting deployment authority: platform.id, the one identity field (ADR-102, ruled O-2; platform.instance_id was removed).
func (*Config) Validate ¶
Validate checks the configuration for use on this host: ValidateDocument, then the host checks: each configured server certificate, server key and client CA file exists, and a warning on standard error when client certificate verification is switched off.
func (*Config) ValidateDocument ¶
ValidateDocument checks the configuration document alone: it reads no file and writes nothing, so offline review (composition.Validate) judges a deployment's document the same way on any machine. It reads nothing from outside the document. Component entries are visited in instance name order, so the first error for a document is the same on every call. Like Validate, it lowercases platform.org in place.
type JetStreamConfig ¶
type JetStreamConfig struct {
Enabled bool `json:"enabled"`
// MaxMemory is the operator's expected server-side
// max_memory_store limit. VerifyJetStreamLimits Warns at boot if
// this exceeds the server's actual AccountInfo.Limits.MaxMemory.
// 0 disables the check (the default for operators who manage
// JetStream sizing entirely via nats.conf without surfacing intent
// in the framework config).
MaxMemory int64 `json:"max_memory,omitempty"`
// MaxFileStore is the operator's expected server-side
// max_file_store limit. Same Warn-on-gap semantics as MaxMemory.
MaxFileStore int64 `json:"max_file_store,omitempty"`
}
JetStreamConfig for JetStream settings. MaxMemory and MaxFileStore are verification hints; JetStream account limits are exclusively server-side configuration via nats.conf — the nats.go SDK exposes AccountInfo as read-only and offers no client-side mutation surface, so the framework cannot push operator intent. What it CAN do is compare configured-vs-server at boot and Warn on gap (see StreamsManager.VerifyJetStreamLimits) so an operator who set max_file_store: 10GB in the framework config but didn't update nats.conf isn't left wondering why stream-create eventually fails with "insufficient storage resources".
type KeyFamily ¶
type KeyFamily struct {
// contains filtered or unexported fields
}
KeyFamily is the family of keys "<prefix>.<name>" in the configuration bucket, together with the handler that receives the family's entries.
Its reads and writes are scoped to the family's "<prefix>." keys, and NewConfigManager refuses a family whose prefix is one of the Manager's own key prefixes, so a holder cannot reach the identity record or any other key the Manager owns, and never learns the bucket's name. They return the not-acquired error until a Manager that registered the family has started successfully.
func NewKeyFamily ¶
func NewKeyFamily(prefix string, handle func(KeyFamilyEntry)) (*KeyFamily, error)
NewKeyFamily builds a family for the keys "<prefix>.<name>".
prefix and every member name are each one NATS KV literal token (natsclient.ValidateKVLiteralToken): the family's watch is "<prefix>.*", which matches exactly one token after the prefix, so a member name holding a "." would be stored but never delivered.
handle runs on the Manager's watch goroutine, in delivery order, and MUST NOT block: the Manager's Stop waits for that goroutine. A handler that has real work to do should signal a goroutine it owns.
func (*KeyFamily) Create ¶
Create writes one member only if it is absent, or returns natsclient.ErrKVKeyExists.
type KeyFamilyEntry ¶
type KeyFamilyEntry struct {
// Name is the member name: the key with "<prefix>." removed.
Name string
// Value is the entry's value. It is empty for a delete.
Value []byte
// Operation says whether the entry was put or deleted.
Operation KeyFamilyOperation
// Initial is true for an entry that was present when the watch opened.
Initial bool
}
KeyFamilyEntry is one entry of a key family, delivered to its handler.
type KeyFamilyOperation ¶
type KeyFamilyOperation uint8
KeyFamilyOperation says what happened to one family entry.
const ( // KeyFamilyPut is a created or updated entry. KeyFamilyPut KeyFamilyOperation = iota + 1 // KeyFamilyDelete is a deleted or purged entry. KeyFamilyDelete )
type Loader ¶
type Loader struct {
// contains filtered or unexported fields
}
Loader loads one configuration document over the defaults.
func (*Loader) LoadFile ¶
LoadFile loads configuration from a single file. It applies the defaults, the document, the environment overrides and the authority-pair reserve; it does not call Validate, which the caller does.
func (*Loader) LoadFromBytes ¶
LoadFromBytes loads configuration from JSON bytes. This is useful when you need to pre-process the configuration (e.g., environment variable expansion) before loading.
The data is merged with the defaults exactly as LoadFile merges a file.
type Manager ¶
type Manager struct {
// contains filtered or unexported fields
}
Manager provides centralized configuration management with channel-based updates
func NewConfigManager ¶
func NewConfigManager(cfg *Config, natsClient *natsclient.Client, logger *slog.Logger, opts ...ManagerOption) (*Manager, error)
NewConfigManager creates a new configuration manager
func (*Manager) DeleteComponentFromKV ¶
DeleteComponentFromKV removes a component from durable next-boot desired state. A push only puts keys that exist in memory; it does not delete keys that are absent.
The removal is also applied synchronously to the Manager's current desired-state view. It does not tear down or otherwise change the running ComponentManager; composition changes only on process restart.
The NATS API does not return the Delete's revision, so the bucket delete and the in-memory removal run under revMu, the lock every revisioned apply takes: no watcher event applies between them. A later put of the key therefore applies after the removal, never before it, and the delete's own event is either an idempotent reapply or, after that later put, skipped as older.
func (*Manager) GetConfig ¶
func (cm *Manager) GetConfig() *SafeConfig
GetConfig returns the current configuration
func (*Manager) PutComponentToKV ¶
func (cm *Manager) PutComponentToKV(ctx context.Context, name string, compConfig types.ComponentConfig) error
PutComponentToKV writes a single component's configuration to NATS KV. This is more efficient than a full push when only one component has changed, and avoids race conditions with KV watchers when multiple operations are in flight.
The method performs write KV → apply desired state in memory and record the key's revision. It records a component candidate for the next process start; it does not add or restart a component in the running ComponentManager. The revision returned by KV.Put lets handleUpdate skip the redundant reapply while still notifying desired-state observers. KV-write is first so a failed Put leaves memory untouched.
func (*Manager) Start ¶
Start acquires the configuration bucket, establishes the platform identity, arbitrates between the file and the bucket, and opens the watches, all under ctx. It refuses a nil or ended ctx, and any Start after the first with errs.ErrAlreadyStarted, before contacting NATS. A Start that fails arms no writer and rolls back what it acquired; a rollback that fails is joined into Start's error and leaves its cleanup for Stop.
Ending ctx after Start returned stops nothing: the watches run under a run context Start creates and Stop ends (ADR-113). Stop is the only stop.
func (*Manager) Stop ¶
Stop stops the watches, closes every channel onChange and WatchModelRegistry returned, and waits for every goroutine the Manager started. It refuses a nil ctx; any non-nil ctx (live or already ended) initiates teardown at once and bounds only the join wait, returning nil once all goroutines exit or ctx's error if ctx ended first so a later Stop tries again.
Pass a fresh bounded context on shutdown, for example context.WithTimeout(context.WithoutCancel(ctx), 5*time.Second), so watches have time to drain cleanly rather than returning ctx.Err() immediately.
func (*Manager) WatchModelRegistry ¶
WatchModelRegistry returns a channel that emits the latest *model.Registry whenever the model_registry KV key changes. The channel is buffered (cap 1); slow consumers see the most recent registry on their next read — intermediate updates coalesce.
Use this for external library consumers that deliberately maintain a live model-registry view. SemEngine components receive the registry selected at boot; later writes are durable desired state for the next process start and do not restart or rewire the running ComponentManager.
See model.Watch for a one-line consumer pattern.
The channel closes when Stop runs, and a reader still holding it does not hold Stop. A channel asked for after Stop is returned closed.
type ManagerOption ¶
type ManagerOption func(*Manager)
ManagerOption configures a Manager at construction.
func WithKeyFamily ¶
func WithKeyFamily(family *KeyFamily) ManagerOption
WithKeyFamily registers a key family whose owner is not the Manager.
The Manager is the configuration bucket's only acquirer, and the bucket's name is derived from the deployment's declared authority pair, which only the Manager holds. A second writer therefore cannot find the bucket on its own. It registers its key family here and reaches the bucket through the family instead.
type NATSConfig ¶
type NATSConfig struct {
URLs []string `json:"urls,omitempty"`
JetStream JetStreamConfig `json:"jetstream,omitempty"`
}
NATSConfig defines NATS connection settings. It carries only the settings something reads: boot dials URLs, and VerifyJetStreamLimits reads JetStream. Credentials, TLS and reconnect settings are refused at load until boot applies them (unappliedNATSKeys).
type PlatformConfig ¶
PlatformConfig is an alias for platform.Config.
PlatformConfig was promoted to its own leaf package so message/ (and any future consumer of platform identity) can reference it without importing all of config. Keeping the alias preserves config.PlatformConfig usage at every call site that already had it. New code should prefer platform.Config directly. Removing the alias would be a mechanical rename across ~10 files; defer until there's a reason.
type SafeConfig ¶
type SafeConfig struct {
// contains filtered or unexported fields
}
SafeConfig provides thread-safe access to configuration
func NewSafeConfig ¶
func NewSafeConfig(cfg *Config) *SafeConfig
NewSafeConfig creates a new thread-safe config wrapper
func (*SafeConfig) Get ¶
func (sc *SafeConfig) Get() *Config
Get returns a deep copy of the current configuration
func (*SafeConfig) Mutate ¶
func (sc *SafeConfig) Mutate(fn func(*Config) error) error
Mutate atomically applies a read-modify-write to the configuration. It holds the write lock across the WHOLE clone → mutate → validate → swap sequence, so two concurrent mutations cannot lose one another's change. Without this, a caller doing Get() (a clone) → mutate → Update() races another doing the same: each starts from the same base and the second swap clobbers the first. -race does not flag it — each Get/Update is individually locked; the atomicity violation is at the compound level.
fn receives a private deep copy it may mutate freely; returning a non-nil error aborts the mutation with no change. The result is validated before the swap.
Re-entrancy contract: fn MUST NOT call any SafeConfig method (Get/Update/Mutate) — it operates only on the draft it is handed, or it self-deadlocks on sc.mu. Do post-mutation side effects (KV writes, subscriber notification) AFTER Mutate returns; capture any value they need into a variable inside fn.
Unlike Update, validation runs INSIDE the write lock (it must, to keep the whole RMW atomic). Validate does filesystem I/O (os.Stat) when TLS is configured, so a mutation briefly blocks concurrent Get() readers — acceptable because config mutations are rare and off the hot path.
func (*SafeConfig) Update ¶
func (sc *SafeConfig) Update(cfg *Config) error
Update atomically updates the configuration after validation
type StreamConfig ¶
type StreamConfig struct {
Subjects []string `json:"subjects"` // Subjects captured by this stream
Storage string `json:"storage,omitempty"` // "file" or "memory" (default: file)
MaxAge string `json:"max_age,omitempty"` // Required: message TTL (e.g., "168h", "7d")
MaxBytes int64 `json:"max_bytes,omitempty"` // Required: max storage in bytes; must be > 0
Retention string `json:"retention,omitempty"` // "limits", "interest", "workqueue" (default: limits)
Replicas int `json:"replicas,omitempty"` // Replication factor (default: 1)
// Discard is the operator's choice of what happens when the stream reaches
// MaxBytes or MaxAge: StreamDiscardOld evicts the oldest messages,
// StreamDiscardNew refuses the write (producers see NATS 503
// err_code=10077). Required for an ordinary stream — it was hardcoded to
// DiscardOld before, so the policy was never the operator's choice.
Discard string `json:"discard,omitempty"`
// Duplicates is the server-side duplicate-detection window for the
// Nats-Msg-Id header (e.g. "2m", "30m", "1h"). Producers using
// Client.PublishToStreamWithMsgID rely on this window to collapse
// redeliveries of the same logical event (ADR-055 §5, "T1"). Empty
// leaves the window unset, so the NATS server applies its default of
// 2 minutes — adequate for steady-state redelivery but shorter than a
// restart/recovery replay horizon. Streams whose producers need
// dedup across that horizon set this explicitly. Must be <= MaxAge:
// the NATS server REJECTS (does not clamp) an explicit window larger
// than MaxAge, so createStream clamps it down to MaxAge with a warning
// rather than letting EnsureStreams abort boot.
Duplicates string `json:"duplicates,omitempty"`
}
StreamConfig defines configuration for a JetStream stream.
MaxAge, MaxBytes, and Discard are REQUIRED for an ordinary stream and are never supplied by a framework default. A bound the operator never chose is indistinguishable in the operator surface from one they did, which is the condition the bounds contract exists to end. The one way out is an explicit declaration of its own: `archival_streams` for a stream whose contract is permanence. See ValidateStreamDeclarations.
type StreamConfigs ¶
type StreamConfigs map[string]StreamConfig
StreamConfigs is a map of stream name to configuration.
type StreamsManager ¶
type StreamsManager struct {
// contains filtered or unexported fields
}
StreamsManager handles JetStream stream creation and management.
func NewStreamsManager ¶
func NewStreamsManager(natsClient *natsclient.Client, logger *slog.Logger) *StreamsManager
NewStreamsManager creates a new StreamsManager.
func (*StreamsManager) EnsureStreams ¶
func (sm *StreamsManager) EnsureStreams(ctx context.Context, cfg *Config) error
EnsureStreams creates all required JetStream streams based on: 1. System streams (LOGS for out-of-band logging) 2. Explicit streams defined in config.Streams (highest priority) 3. Streams derived from component JetStream output ports
Every resolved ordinary stream must carry an explicitly declared finite MaxAge, a finite MaxBytes, and a discard policy, unless an archival declaration admits it. That check runs BEFORE any stream is created, so a configuration missing a bound fails closed with a complete diagnostic rather than provisioning half its streams and then stopping — a partially-provisioned account is harder to reason about than one that never started.
func (*StreamsManager) VerifyJetStreamLimits ¶
func (sm *StreamsManager) VerifyJetStreamLimits(ctx context.Context, cfg *Config) error
VerifyJetStreamLimits reads the operator's MaxMemory / MaxFileStore hints from cfg.NATS.JetStream and logs a Warn for each value that exceeds the server's actual account limit. JetStream account limits are server-side configuration (nats.conf or jetstream-domain configuration); the nats.go SDK exposes AccountInfo as read-only, so the framework cannot push the operator's intent — but it CAN surface the gap loudly so an operator who set max_file_store: 10GB in the framework config but didn't update nats.conf isn't left wondering why stream-create fails with "insufficient storage resources" at runtime. Zero or unset values skip the check.
Best-effort: a failure to fetch AccountInfo (server down, JetStream disabled, AccountInfo unsupported) logs at Debug and returns nil — the check is diagnostic, not gating. EnsureStreams runs anyway and any actual capacity miss surfaces as a CreateStream error there.