Documentation
¶
Overview ¶
Package storagereport produces and reads the account storage report.
A StorageInventoryCollector lists every JetStream-backed resource the account holds (ordinary streams, KV buckets and ObjectStores, whoever created them) through AccountStreamLister. A Publisher derives each resource's growth, projection and pressure from successive inventories and writes one KV key per resource, plus the account row, to the storage report bucket. A Consumer watches that bucket and serves the current report as a Snapshot, telling an optional Observer about each change.
The bucket itself (its name, owner and retention) is declared in graph/kvcatalog; this package holds no bucket name and imports no graph package. It reads the account through natsclient.
Index ¶
- Constants
- func ResourceKey(resource string) (string, error)
- type AccountReport
- type AttributionState
- type Capacity
- type CapacityState
- type Consumer
- type ConsumerConfig
- type Growth
- type GrowthState
- type InventoryPublisher
- type Observer
- type OvercommitmentState
- type OwnerResolver
- type Pressure
- type PressureBasis
- type PressureInput
- type PressureState
- type Projection
- type PublishResult
- type Publisher
- type PublisherConfig
- type ReportStore
- type ReportWatchStore
- type ResolvedPressureThresholds
- type ResourceReport
- type Snapshot
- type StorageInventory
- type StorageInventoryCollector
- type StorageInventoryConfig
- type StoragePressureThresholds
- type StorageResource
- type StorageTier
- type StreamLister
- type StreamListerSource
- type ThresholdSource
- type TierComparison
Constants ¶
const ( // limit for the tier. The resources are still named and their declared sum // still reported; only the verdict is withheld. OvercommitmentUnavailableUnboundedLimit = "account-limit-unbounded" // determined — AccountInfo was unreadable, or the value the server returned // is ambiguous. OvercommitmentUnavailableUnknownLimit = "account-limit-unknown" )
Reasons an over-commitment comparison is not applicable. They are distinct because they resolve differently: the first is a deliberate operator choice and needs no action, the second is a gap in what this process can read.
const ( // this resource exist yet — a new resource, or a fresh report bucket. GrowthUnavailableNoPriorObservation = "no-prior-observation" // so there is nothing to difference. GrowthUnavailableUnknownUsage = "unknown-usage" // is separated from the current one by at least 5 s. GrowthUnavailableObservationsTooClose = "observations-too-close" )
Reasons a growth rate is unavailable. They are distinct because they resolve differently: the first resolves on the next collection, the second needs the resource to become describable, and the third needs the collection interval (or the publication cadence) to be wider than 5 s, the shortest interval a rate is measured over.
const ( // DefaultStorageInventoryInterval is how often the account is enumerated // when the caller does not choose. DefaultStorageInventoryInterval = time.Minute // DefaultStorageInventoryTimeout bounds ONE collection, both listings // included. It is the collector's own bound, not the caller's, so an // unbounded caller context cannot make a collection unbounded. DefaultStorageInventoryTimeout = 15 * time.Second )
Collection defaults. Both are configuration: every process polling account-wide multiplies cost by deployment size, so an operator running many instances turns the interval down rather than losing the view entirely.
const ( // be read. Projecting from it would fabricate a number. ProjectionUnavailableUnknownCapacity = "unknown-capacity" // There is nothing to have headroom against and nothing to run out of. ProjectionUnavailableUnbounded = "unbounded" // yet, so no time can be projected. Headroom is still reported. ProjectionUnavailableUnknownGrowth = "unknown-growth-rate" // or negative: the threshold is never reached. Distinct from // unknown-growth-rate, because "measured, and not growing" is a finding an // operator can rely on. ProjectionUnavailableNotGrowing = "not-growing" )
Reasons a projection is unavailable. Each names a different missing input, so an operator reading a suppressed projection can tell whether to fix the server, the declaration, or simply wait for the next collection.
const ( // Reporting normal here would manufacture confidence about a resource // nobody can measure. PressureUnavailableUnknownCapacity = "unknown-capacity" // against declares no bound, so neither band has an input. // // For a RESOURCE it is not the final word: the publisher re-evaluates it // against the only ceiling it has, its storage tier's account limit, and the // more specific unbounded-* reasons below are what a resource ends up // carrying. This value // survives on a TIER row — pressure assessed over an unbounded account limit — // and as the fallback when a bounded tier reports no reason of its own. PressureUnavailableUnbounded = "unbounded" // bound AND its storage tier's account limit is itself unbounded. Nothing // anywhere constrains it, so there is genuinely nothing to project — as // distinct from a ceiling that exists and could not be read. PressureUnavailableUnboundedNoTierCeiling = "unbounded-no-account-tier-ceiling" // bound and its tier's account limit could not be read. The ceiling may well // exist; this process cannot see it, which is a gap rather than a licence to // report the resource as fine. PressureUnavailableUnboundedTierUnknown = "unbounded-account-tier-unknown" // bound and could not be filed under any tier, so no account ceiling applies // to it that this process can name. // // NOT REACHABLE from the production collector: the account report has a row // for every tier any resource reports, creating the unknown-tier row on demand, // so a collected resource always finds one. It exists as the fail-closed arm // for a hand-built inventory, and reporting no state is the right answer there // rather than borrowing an arbitrary tier's. PressureUnavailableUnboundedTierUnfiled = "unbounded-account-tier-unfiled" )
Reasons no pressure state exists for a resource.
const SkipReasonStaleInventory = "inventory is stale; last-good is not a new observation"
SkipReasonStaleInventory is the SkipReason for an inventory whose most recent collection did not succeed.
const StorageAccountReportKey = "_account.tiers"
StorageAccountReportKey is the reserved key carrying the per-tier account report (AccountReport) rather than a resource row.
It is RESERVED by construction, not by convention. Every resource key is exactly ONE key token: a JetStream stream name may not contain a dot, and ResourceKey's fallback opaque token is a single token too. A key containing a dot is therefore unreachable from any resource name, however hostile — which is what lets one bucket carry two row kinds with no possibility of one addressing the other. A consumer discriminates on the key.
const StorageReportGrowthObservations uint8 = 2
StorageReportGrowthObservations is how many observations of a resource the rate derivation needs: two, the current one and one retained prior one. It is the FLOOR on the report bucket's declared History depth, not the depth that bucket should carry.
The distinction is worth stating because the obvious argument for depth is wrong. A publisher seeds from the history BEFORE it writes, so the row it is about to compact is the row it just read: a single restart recovers its baseline at any depth, including 1. Depth becomes load-bearing in the case the design actually cares about — a crash loop, a deploy loop, or several processes publishing account-wide, where the newest retained rows are all too close together to measure against (less than 5 s apart) and the walk has to reach back PAST them. At depth 1 there is nothing to reach back to and the rate stays unknown for exactly as long as the loop lasts.
It is exported as a CROSS-PIN so a catalog editor who reconciles the STORAGE_REPORT row's History below the floor fails a test in graph rather than silently blanking every rate in the account.
Variables ¶
This section is empty.
Functions ¶
func ResourceKey ¶
ResourceKey is the report bucket key for one resource name.
An ordinary name IS its own key, so `nats kv get` reads naturally and an operator can address a resource by the name they already know. JetStream accepts stream names that are not legal KV keys, though — `$` and `+` among others are fine in a stream name and illegal in a key — and the resources most likely to be a growth problem are exactly the ones nobody in this process created. Dropping them would rebuild the silent omission the inventory exists to end, so an illegal name is addressed through the repository's opaque key codec instead. The row still carries the real name.
A name that is ITSELF a canonical opaque token is also encoded, so no two resources can ever collide on one key.
Types ¶
type AccountReport ¶
type AccountReport struct {
// CollectedAt is when the account was read. Stamped by the publisher from
// the inventory this report was derived with, so it always matches the
// resource rows it was computed alongside.
CollectedAt time.Time `json:"collected_at"`
// ProducedBy names the process that produced this report.
ProducedBy string `json:"produced_by"`
// Tiers holds file and memory always, and the unknown tier only when some
// resource could not be filed under either. An always-present unknown row
// would publish a permanently empty comparison for a limit that does not
// exist.
Tiers []TierComparison `json:"tiers"`
// when they were.
LimitsUnavailable string `json:"limits_unavailable,omitempty"`
}
AccountReport is the account-level half of the storage report: what each storage tier's ceiling is, and whether the declarations filed against it fit.
func (AccountReport) TierFor ¶
func (r AccountReport) TierFor(tier StorageTier) (TierComparison, bool)
TierFor returns one tier's comparison. ok is false when the report carries no row for that tier, which is a real answer rather than a zero comparison.
type AttributionState ¶
type AttributionState string
AttributionState discriminates why a resource does or does not name an owner.
Unattributed and not-applicable are DIFFERENT facts and only one of them is a finding. Logical ownership is defined for KV buckets, through the descriptor catalog; there is no owner registry for ordinary streams or ObjectStores, and the inventory enumerates the ACCOUNT, so a resource another process declared has no declaration this process could read. Reporting both as an empty owner would say "the framework has no owner concept here" and "this bucket escaped the catalog" in the same breath.
const ( // AttributionAttributed means the descriptor catalog declares an owner for // this resource's bucket, and Owner carries it. AttributionAttributed AttributionState = "attributed" // AttributionUnattributed means the resource IS a KV bucket, so an owner is // meaningful, but the catalog declares none. This is the finding state: a // bucket outside framework ownership, reported rather than omitted or // force-fit. AttributionUnattributed AttributionState = "unattributed" // AttributionNotApplicable means ownership is not defined for this kind of // resource at all. Not a finding. AttributionNotApplicable AttributionState = "not-applicable" )
Attribution states.
type Capacity ¶
type Capacity struct {
State CapacityState `json:"state"`
// ConfiguredLimit is the declared finite bound. Non-nil ONLY when State is
// CapacityBounded.
ConfiguredLimit *int64 `json:"configured_limit,omitempty"`
// Used is observed usage. Nil when State is CapacityUnknown, because an
// unreadable usage must not read back as zero.
Used *int64 `json:"used,omitempty"`
}
Capacity is one bounded dimension of a resource — a configured limit paired with observed usage.
The numeric fields are pointers on purpose. An absent value must be ABSENT, never a zero that a downstream reader can mistake for a real measurement: "limit 0" is the exact shape that turns an unreadable resource into a reported-healthy one. Read them through Limit and Usage, which return an ok flag, and build them through NewCapacity, which is the only classifier.
func NewCapacity ¶
NewCapacity classifies one capacity axis. It is the ONE place a limit/usage pair becomes a state, so no caller can invent a fourth interpretation.
known reports whether the resource's configuration AND state were readable. They arrive together in a single listing entry today; a future collector that reads them from separate calls MUST pass false when either fails, because the requirement is "limit OR usage cannot be determined".
A non-positive limit is JetStream's unlimited encoding (both 0 and the -1 sentinel appear in practice) and yields CapacityUnbounded — never a bounded zero.
type CapacityState ¶
type CapacityState string
CapacityState discriminates the three — not two — states a capacity axis can be in. Collapsing any pair produces the phantom-signal class this capability exists to remove: an unreadable limit is not an absent limit, and an absent limit is not a safe one.
const ( // CapacityBounded means a finite configured limit was read. Only this state // carries a limit value, and only this state supports headroom or // time-to-threshold projection. CapacityBounded CapacityState = "bounded" // CapacityUnbounded means the resource is deliberately unlimited. Usage is // still observable; headroom is not, because there is no bound to have // headroom against. CapacityUnbounded CapacityState = "unbounded" // CapacityUnknown means the limit or the usage could not be determined. // It is NOT unlimited, NOT zero, and NOT healthy, and it suppresses // projection rather than emitting a fabricated one. CapacityUnknown CapacityState = "unknown" )
Capacity states.
type Consumer ¶
type Consumer struct {
// contains filtered or unexported fields
}
Consumer maintains the in-process view of the published report.
func NewConsumer ¶
func NewConsumer( store ReportWatchStore, cfg ConsumerConfig, ) (*Consumer, error)
NewConsumer builds a consumer over the report bucket.
func (*Consumer) Run ¶
Run watches the report bucket until ctx ends, re-establishing the watch when it cannot be created or when it ends.
The retry is not optional resilience. A watch that dies leaves every operator surface frozen on its last values, and a frozen report is indistinguishable from a calm account — which is the exact failure this capability exists to end. Call Run in its own goroutine.
Run returns after nats.go has closed the delivery of the watch it stopped, for at most 5 s after the watcher's own Stop returns, whether or not ctx has ended: at nats.go v1.55.0 a delivery blocked on a full updates buffer outlives the watch's Stop until the buffer is read (TestIntegration_ReportConsumer_RunLeavesNoWatchDelivery). A failure to stop the watch is logged at Warn.
func (*Consumer) Snapshot ¶
Snapshot returns the current view without doing any I/O. Safe to call from a health check, an HTTP handler, or a metrics scrape. The result shares no memory with the consumer, so a caller may edit it.
While a replacement watch delivers its initial values, Snapshot returns the previous view unchanged; at that watch's sync marker the view becomes exactly the bucket's current values in one step, so a row or account row the bucket no longer holds is gone even when no delete marker for it was replayed.
type ConsumerConfig ¶
type ConsumerConfig struct {
// Observer receives each applied change. Optional: a consumer without one
// still maintains its snapshot.
Observer Observer
// RetryBackoff is the wait before re-establishing a watch. Defaults to
// 5 s.
RetryBackoff time.Duration
// Logger receives watch failures. Defaults to slog.Default().
Logger *slog.Logger
}
ConsumerConfig configures the report consumer.
type Growth ¶
type Growth struct {
State GrowthState `json:"state"`
// BytesPerSecond is the measured rate. Non-nil ONLY when State is
// GrowthKnown. Negative means the resource is shrinking.
BytesPerSecond *float64 `json:"bytes_per_second,omitempty"`
// ObservedOver is the interval between the two observations the rate was
// measured across. Published so a consumer can weigh a rate measured over
// one minute against one measured over a day; a rate with no interval is
// not interpretable.
ObservedOver time.Duration `json:"observed_over_ns,omitempty"`
// ObservedFrom is when the baseline observation was taken. A POINTER, like
// the numeric absences: `omitempty` does nothing for a time.Time, so a
// value field would publish "observed_from":"0001-01-01T00:00:00Z" on every
// unknown rate, and a consumer could read that zero as a real baseline.
ObservedFrom *time.Time `json:"observed_from,omitempty"`
// GrowthKnown.
Unavailable string `json:"unavailable,omitempty"`
}
Growth is a resource's observed rate of change.
BytesPerSecond is a pointer for the same reason the capacity numbers are: an absent rate must be ABSENT, never a zero a downstream reader can mistake for "measured, and not growing".
type GrowthState ¶
type GrowthState string
GrowthState discriminates a known rate from an absent one. There is no third state: a rate is either measured across two observations or it is unknown.
const ( // GrowthKnown means a rate was measured across two observations. Zero and // negative rates are KNOWN — a stable resource and a shrinking one are // facts, not absences. GrowthKnown GrowthState = "known" // GrowthUnknown means no usable pair of observations exists. It suppresses // time-to-threshold rather than being treated as zero growth, because "not // growing" and "not measured" license different operator decisions. GrowthUnknown GrowthState = "unknown" )
Growth states.
type InventoryPublisher ¶
type InventoryPublisher interface {
Publish(ctx context.Context, inv StorageInventory) (PublishResult, error)
}
InventoryPublisher publishes one collection's inventory. *Publisher satisfies it; the interface keeps the collector from depending on the report's whole surface.
type Observer ¶
type Observer interface {
ObserveResource(row ResourceReport)
ForgetResource(row ResourceReport)
ObserveAccount(row AccountReport)
ForgetAccount(row AccountReport)
}
Observer receives each change the consumer applies.
It exists so a metrics surface can update ONE series per event instead of rebuilding every series on every tick. ForgetResource carries the whole last known row rather than a key, because a resource whose name is not a legal KV key is addressed by an opaque token — a bare key cannot say which label set a gauge was registered under. ForgetAccount likewise carries the last account row the consumer held, so the surface can retract the per-tier series it registered for it.
Implementations are called from the consumer's watch goroutine and must not block on it.
type OvercommitmentState ¶
type OvercommitmentState string
OvercommitmentState is the verdict on one tier's declared-versus-limit comparison.
const ( // OvercommitmentWithin means the declared bounds in this tier fit inside // the account limit for the tier. It says nothing about the tier's // unbounded resources, whose declarations are absent from the sum by // construction — read UnboundedResources alongside it. OvercommitmentWithin OvercommitmentState = "within-limit" // OvercommitmentOver means the declared bounds in this tier exceed the // account limit for the tier. Every resource can still be within its own // bound: this is the failure mode a per-resource view cannot see. OvercommitmentOver OvercommitmentState = "over-committed" // OvercommitmentNotApplicable means there is no comparison to make, // because the account limit for this tier is unbounded or unreadable. It // is NOT a passing verdict, and Unavailable says which case it is. OvercommitmentNotApplicable OvercommitmentState = "not-applicable" )
Over-commitment states.
type OwnerResolver ¶
OwnerResolver resolves a KV bucket name to its declared logical owner, returning "" for a bucket it does not declare.
This is a FUNCTION, not a map, and that is the point: attribution must be a read of the one descriptor catalog at collection time. kvcatalog.OwnerOf satisfies it directly. A retained copy could disagree with the acquisition seam about who owns a bucket, and would keep reporting a former owner after the catalog dropped the row.
type Pressure ¶
type Pressure struct {
// Evaluated reports whether a state could be derived at all.
Evaluated bool `json:"evaluated"`
// State is the worse of the two bands. Empty when Evaluated is false.
State PressureState `json:"state,omitempty"`
// RaisedBy names the input that produced State.
RaisedBy PressureInput `json:"raised_by,omitempty"`
// FromHeadroom and FromTimeToThreshold are the individual bands, published
// so a consumer can see the input that did NOT win.
// FromTimeToThreshold is empty when no projection was available.
//
// They describe whichever ceiling EvaluatedAgainst names. On a row whose basis
// is the account tier these are the TIER's bands, not this resource's — the
// resource has no bound of its own to produce any — so they will not agree with
// the row's own Projection, which reports headroom unavailable. Read the basis
// first; the numbers behind these bands are on the account row.
FromHeadroom PressureState `json:"from_headroom,omitempty"`
FromTimeToThreshold PressureState `json:"from_time_to_threshold,omitempty"`
// EvaluatedAgainst names the ceiling State was derived from. Set whenever
// Evaluated, so no consumer has to infer the basis from the resource's
// capacity state.
EvaluatedAgainst PressureBasis `json:"evaluated_against,omitempty"`
Unavailable string `json:"unavailable,omitempty"`
}
Pressure is one resource's derived pressure state.
Evaluated is explicit rather than inferred from an empty State, so a consumer cannot read "no state" as normal. A resource whose capacity is unknown or unbounded has NO pressure state at all — reporting normal for it would be the phantom-signal class this capability exists to remove.
type PressureBasis ¶
type PressureBasis string
PressureBasis names WHICH CEILING a pressure state was evaluated against.
It exists because this capability now evaluates two different ceilings and an operator acts on them differently. A resource at high pressure against its own declared bound is fixed by raising that bound or letting retention do its job. A resource at high pressure against its ACCOUNT TIER has no bound of its own to raise: the tier is filling, the finding is about every resource sharing it, and the levers are account capacity or deleting data. Publishing the state without the basis would leave those indistinguishable in the one field every alert rule reads.
const ( // PressureBasisOwnBound means the state came from the resource's own // declared limit. PressureBasisOwnBound PressureBasis = "own-bound" // PressureBasisAccountTier means the state came from the account limit of // the storage tier the resource lives in. For an unbounded resource that is // the only ceiling there is; for a tier row it is the tier's own limit. PressureBasisAccountTier PressureBasis = "account-tier" )
Pressure bases.
type PressureInput ¶
type PressureInput string
PressureInput names which input raised a pressure state, so an operator can tell a capacity problem from a rate problem.
const ( // PressureInputNone means nothing raised the state above normal. PressureInputNone PressureInput = "none" // PressureInputHeadroom means proportional headroom raised it. PressureInputHeadroom PressureInput = "headroom" // PressureInputTimeToThreshold means the projection raised it — the case // proportional headroom alone would have missed. PressureInputTimeToThreshold PressureInput = "time-to-threshold" // PressureInputBoth means both inputs landed on the same band. Reported // rather than tie-broken: an arbitrary winner would misreport the cause. PressureInputBoth PressureInput = "both" )
Pressure inputs.
type PressureState ¶
type PressureState string
PressureState is how worried an operator should be about one resource.
const ( // PressureNormal means neither input is inside a configured band. PressureNormal PressureState = "normal" // PressureWarning means an input crossed the widest band. PressureWarning PressureState = "warning" // PressureHigh means an input crossed the middle band. PressureHigh PressureState = "high" // PressureCritical means an input crossed the tightest band. It still // rejects nothing: pressure is report-only in this capability. PressureCritical PressureState = "critical" )
Pressure states, worst last.
type Projection ¶
type Projection struct {
// HeadroomBytes is the distance from current usage to the configured
// bound. It can be NEGATIVE for a resource already over its bound, which is
// reported rather than clamped.
HeadroomBytes *int64 `json:"headroom_bytes,omitempty"`
// HeadroomFraction is HeadroomBytes as a fraction of the bound.
HeadroomFraction *float64 `json:"headroom_fraction,omitempty"`
HeadroomUnavailable string `json:"headroom_unavailable,omitempty"`
// ThresholdBytes is the usage level the time projection targets: the
// critical-headroom level, not the bound itself. Capacity has to be
// corrected BEFORE the last byte fits.
ThresholdBytes *int64 `json:"threshold_bytes,omitempty"`
// TimeToThreshold is the projected time until usage reaches
// ThresholdBytes at the observed rate. Zero means the resource is already
// there; it is never negative.
TimeToThreshold *time.Duration `json:"time_to_threshold_ns,omitempty"`
TimeToThresholdUnavailable string `json:"time_to_threshold_unavailable,omitempty"`
}
Projection is what the report can say about a resource's future.
Headroom and time-to-threshold are suppressed INDEPENDENTLY, each with its own reason: headroom needs a readable bound, while a time projection additionally needs a measured, positive rate. Suppressing them together would discard the half that is knowable.
The numeric fields are pointers so an absent projection is ABSENT rather than a zero that reads as "no headroom left" or "exhausted now".
type PublishResult ¶
type PublishResult struct {
Published int
Deleted int
// AccountPublished reports that the per-tier account row was written. It is
// a separate field rather than part of Published so that count keeps meaning
// "resources", which is what every caller logging it reads it as.
AccountPublished bool
// Skipped reports that nothing was written at all, with SkipReason saying
// why. It is not an error: declining to republish a stale inventory is the
// correct behavior, not a failure.
Skipped bool
SkipReason string
}
PublishResult is what one publication did, for the caller that logs it.
It deliberately does NOT carry the published rows. Every operator-facing surface must be a CONSUMER of the bucket rather than a second report-producing path, and handing an in-process caller the derived values directly is exactly the shortcut that would let one surface disagree with another.
type Publisher ¶
type Publisher struct {
// contains filtered or unexported fields
}
Publisher writes the account report to its KV bucket.
Safe for concurrent use, though one publication at a time is the intended shape: the baseline map is the growth series' in-process cache and two overlapping publications would race to advance it.
func NewPublisher ¶
func NewPublisher(store ReportStore, cfg PublisherConfig) (*Publisher, error)
NewPublisher builds a publisher. It fails closed on a missing store or threshold source rather than evaluating every resource in the account against numbers no operator chose.
func (*Publisher) Publish ¶
func (p *Publisher) Publish(ctx context.Context, inv StorageInventory) (PublishResult, error)
Publish writes one row per inventoried resource and deletes the key of any resource the inventory no longer names.
A STALE inventory publishes nothing. Last-good is what the collector serves when a collection fails, and republishing it as a fresh observation would inject a duplicate sample into the growth series and move every row's collection timestamp forward onto data that is not new.
A resource that fails to publish does not stop the rest of the report: the failures are joined and returned, naming the resources they lost, so a partial publication is REPORTED rather than silently shrinking the report.
type PublisherConfig ¶
type PublisherConfig struct {
// Thresholds resolves the pressure thresholds per evaluation. REQUIRED:
// see ThresholdSource.
Thresholds ThresholdSource
// Logger receives publication and threshold warnings. Defaults to
// slog.Default().
Logger *slog.Logger
}
PublisherConfig configures the report publisher.
type ReportStore ¶
type ReportStore interface {
Put(ctx context.Context, key string, value []byte) (uint64, error)
Delete(ctx context.Context, key string, opts ...jetstream.KVDeleteOpt) error
History(ctx context.Context, key string, opts ...jetstream.WatchOpt) ([]jetstream.KeyValueEntry, error)
WatchFiltered(ctx context.Context, keys []string, opts ...jetstream.WatchOpt) (jetstream.KeyWatcher, error)
}
ReportStore is the narrow KV capability the report publisher needs. jetstream.KeyValue satisfies it.
Reclamation lists the bucket's keys through natsclient.FilteredKeys, which reads WatchFiltered and leaves no watch delivery running when the listing ends early (TestPublishStoppedMidReclaimLeavesNoDelivery). Migration: a custom store implements WatchFiltered in place of the ListKeys it implemented before (#496).
History rather than Get: the seed read has to be able to walk PAST an observation too close to the current one to measure (several processes may publish account-wide, so the newest revisions can all land inside one collection interval), and the retained series is what makes that possible.
type ReportWatchStore ¶
type ReportWatchStore interface {
WatchAll(ctx context.Context, opts ...jetstream.WatchOpt) (jetstream.KeyWatcher, error)
}
ReportWatchStore is the narrow KV capability the report consumer needs. jetstream.KeyValue satisfies it.
WatchAll rather than a range read: on restart a consumer needs the current report immediately (KV re-delivers every current value, which is correct recovery), and several surfaces should each react rather than one dequeuing. Deletes are NOT filtered out — a reclaimed row must retract its metric series rather than leave a resource reporting forever after it is gone.
type ResolvedPressureThresholds ¶
type ResolvedPressureThresholds struct {
WarningHeadroom float64
HighHeadroom float64
CriticalHeadroom float64
WarningHorizon time.Duration
HighHorizon time.Duration
CriticalHorizon time.Duration
}
ResolvedPressureThresholds is the applied form: defaults filled, durations parsed, ordering validated. Only Resolve produces one, so no evaluation can run against a threshold set nobody checked.
type ResourceReport ¶
type ResourceReport struct {
// Resource is the inventory row verbatim — name, kind, attribution, owner,
// tier, and the capacity states. The report is a VIEW of the inventory, not
// a second opinion about any of it.
Resource StorageResource `json:"resource"`
// CollectedAt is when this resource was read from the account. Every key
// carries its own, because a consumer ranging the bucket sees a mix of
// revisions rather than an atomic snapshot and needs to see the spread.
CollectedAt time.Time `json:"collected_at"`
// ProducedBy names the process that collected and published this row. A
// fleet of processes each polling account-wide is unreconcilable without
// it.
ProducedBy string `json:"produced_by"`
// Growth is the observed rate of change across successive observations.
Growth Growth `json:"growth"`
// Projection is headroom and time-to-threshold, each suppressed on its own
// terms rather than fabricated.
Projection Projection `json:"projection"`
// Pressure is the derived state. Report-only: no write is rejected, no
// component is throttled, no readiness gate is failed, and no retention is
// applied because of what it says.
Pressure Pressure `json:"pressure"`
}
ResourceReport is one published row: what the collector saw about one resource, and what it derived from it.
The row is plain JSON with no BaseMessage wrapper, following the GRAPH_STATUS readiness envelope: the payload registry governs polymorphic publishes on subjects, where a receiver must discriminate a type it did not choose. This is a KV value on a key whose type is fixed by the contract, and wrapping it would break every consumer's plain decode for nothing.
type Snapshot ¶
type Snapshot struct {
// Resources is every published resource row, sorted by resource name.
Resources []ResourceReport
// Account is the per-tier comparison, valid only when AccountKnown.
Account AccountReport
AccountKnown bool
// Synced reports that a watch has delivered every current value at least
// once. Before it, the snapshot is empty and means "not read yet" rather
// than "the account holds nothing" — the same distinction the inventory's
// Stale flag keeps. It stays true while a replacement watch is syncing,
// because the view is then the previous watch's complete one.
Synced bool
// UpdatedAt is when the last change was applied.
UpdatedAt time.Time
// PressureCounts tallies the EVALUATED rows by state.
PressureCounts map[PressureState]int
// NotEvaluated counts rows carrying no pressure state at all — unbounded or
// unknown capacity. Counted rather than folded into normal: a surface that
// filtered on "state != normal" would make exactly the unbounded resources
// invisible.
NotEvaluated int
// WorstPressure is the worst evaluated state, or empty when nothing was
// evaluated. Empty is NOT normal: an account whose every row declined to
// evaluate has no pressure verdict, and reporting one would manufacture it.
WorstPressure PressureState
}
Snapshot is the whole published report as one in-process view.
The counts are a TALLY of what the rows say, never a re-evaluation: pressure is derived at publication, and a consumer that recomputed it would be the second producer this design exists to prevent.
type StorageInventory ¶
type StorageInventory struct {
// ProducedBy names the process that collected this inventory. Never empty,
// because a fleet of processes each polling account-wide produces reports
// that cannot be reconciled without it.
ProducedBy string `json:"produced_by"`
// CollectedAt is when Resources were read from the account. Zero when no
// collection has ever succeeded.
CollectedAt time.Time `json:"collected_at"`
// Resources is the complete account listing, deduplicated and sorted by
// physical name. It is never a partial listing: a walk that fails part way
// through leaves the previous result in place rather than reporting a
// subset of the account as if it were all of it.
Resources []StorageResource `json:"resources"`
// Account is the per-tier declared-versus-account-limit comparison over
// exactly these Resources. It travels WITH them rather than being computed
// by each consumer, so no operator surface can disagree with another about
// whether a tier is over-committed. An account whose limits could not be
// read still yields a report — with every tier's limit unknown, which is
// distinct from unbounded and from satisfied.
Account AccountReport `json:"account"`
// Stale reports that the most recent collection attempt did not succeed, so
// Resources and CollectedAt describe an earlier moment. It is also true
// before the first successful collection, when Resources is legitimately
// empty but the account is NOT known to be empty.
Stale bool `json:"stale"`
// StaleSince is when the most recent failed attempt happened. A POINTER for
// the same reason the capacity numbers are: `omitempty` is a no-op on a
// time.Time, so a value field would publish a zero timestamp on a healthy
// inventory and invite a consumer to read it as a real failure time.
StaleSince *time.Time `json:"stale_since,omitempty"`
// StaleReason explains the failure in operator terms.
StaleReason string `json:"stale_reason,omitempty"`
}
StorageInventory is one account-wide collection result.
CollectedAt is when the RESOURCES were read, not when the report was rendered: a degraded inventory keeps the timestamp its data actually came from, so an operator reading a stale report can tell how stale it is.
type StorageInventoryCollector ¶
type StorageInventoryCollector struct {
// contains filtered or unexported fields
}
StorageInventoryCollector enumerates account storage on an interval and publishes the last good result.
Collect performs all of its I/O outside the publication lock and takes the write lock only to swap the finished snapshot, so Latest never waits behind a collection. Nothing here belongs on a component's Start or health path.
func NewStorageInventoryCollector ¶
func NewStorageInventoryCollector( source StreamListerSource, cfg StorageInventoryConfig, ) (*StorageInventoryCollector, error)
NewStorageInventoryCollector builds a collector. It fails closed on a missing lister source or owner resolver rather than degrading to a silently unattributed inventory.
func (*StorageInventoryCollector) Collect ¶
func (c *StorageInventoryCollector) Collect(parent context.Context) (StorageInventory, error)
Collect enumerates the account once, bounded by the configured timeout, and publishes the result. Collections are serialized, so two overlapping callers cannot publish out of order and walk CollectedAt backwards.
On failure it returns the last good inventory marked stale ALONGSIDE the error, so a caller that drops the error still cannot mistake a failed collection for an empty account. A failure caused by the CALLER's context ending — a graceful shutdown — leaves the last good result unmarked, because "context canceled" is not a storage finding.
func (*StorageInventoryCollector) Latest ¶
func (c *StorageInventoryCollector) Latest() StorageInventory
Latest returns the most recent inventory without doing any I/O. It never blocks on a collection and never fails, so a health check, a readiness evaluation, or an operator report can call it freely.
The returned Resources slice is a COPY. The published slice is built with spare capacity, so handing out the same backing array would let two callers appending to their own results write the same slots.
func (*StorageInventoryCollector) Run ¶
func (c *StorageInventoryCollector) Run(ctx context.Context)
Run collects on the configured interval until ctx ends. Call it in its own goroutine — never from a component's Start or from health evaluation.
The first collection happens immediately rather than one interval later, so a freshly booted process has a report to serve; because Run is already asynchronous, that cannot delay anything.
type StorageInventoryConfig ¶
type StorageInventoryConfig struct {
// Interval is how often Run enumerates the account. Defaults to
// DefaultStorageInventoryInterval.
Interval time.Duration
// Timeout bounds one collection. Defaults to
// DefaultStorageInventoryTimeout.
Timeout time.Duration
// ProducedBy names this process in the report. Defaults to host/pid.
ProducedBy string
// OwnerResolver attributes KV resources. REQUIRED: with no resolver every
// KV resource would report unattributed, which reads as "nothing is
// framework-owned" rather than as "attribution was never wired", so the
// constructor fails closed instead of defaulting.
OwnerResolver OwnerResolver
// Publisher writes the report after each SUCCESSFUL collection Run makes.
// Optional: a collector without one keeps its in-process inventory and
// publishes nothing, which is what the attribution unit tests want.
//
// It hangs off the collector rather than running its own timer because the
// publication cadence IS the observation cadence — the growth series is
// Δbytes over Δt across published observations, and a second timer could
// drift from the interval the operator configured.
Publisher InventoryPublisher
// Logger receives collection-failure warnings. Defaults to slog.Default().
Logger *slog.Logger
}
StorageInventoryConfig configures account storage collection.
type StoragePressureThresholds ¶
type StoragePressureThresholds struct {
// WarningHeadroom is the free fraction (0..1) at or below which a resource
// reports warning.
WarningHeadroom float64 `` /* 164-byte string literal not displayed */
// HighHeadroom is the free fraction at or below which a resource reports
// high.
HighHeadroom float64 `` /* 158-byte string literal not displayed */
// CriticalHeadroom is the free fraction at or below which a resource
// reports critical. It is also the level the time projection targets, so
// "time to threshold" means "time until this fraction is all that is left".
CriticalHeadroom float64 `` /* 209-byte string literal not displayed */
// WarningHorizon is the projected time-to-threshold at or below which a
// resource reports warning.
WarningHorizon string `` /* 177-byte string literal not displayed */
// HighHorizon is the projected time-to-threshold at or below which a
// resource reports high.
HighHorizon string `` /* 171-byte string literal not displayed */
// CriticalHorizon is the projected time-to-threshold at or below which a
// resource reports critical.
CriticalHorizon string `` /* 178-byte string literal not displayed */
}
StoragePressureThresholds is the OPERATOR-FACING pressure configuration.
Durations are strings ("72h", "90m") following this repository's operator configuration convention, parsed by Resolve rather than at decode time, so a malformed edit is a reported configuration error rather than a decode failure that takes a component down.
Every field is optional and an omitted field takes its documented default. An out-of-range or incoherent field is an ERROR, never a silent fallback: a silently defaulted threshold applies a number the operator did not choose and is indistinguishable from a working edit.
func DefaultStoragePressureThresholds ¶
func DefaultStoragePressureThresholds() StoragePressureThresholds
DefaultStoragePressureThresholds returns the shipped defaults in their operator-authored form, so a caller that wants them deliberately can say so in one line rather than passing a zero value and hoping.
func (StoragePressureThresholds) Resolve ¶
func (t StoragePressureThresholds) Resolve() (ResolvedPressureThresholds, error)
Resolve applies defaults, parses the horizon durations, and validates the whole set. It fails on any incoherence rather than repairing it, because a repaired threshold is a number the operator did not choose.
type StorageResource ¶
type StorageResource struct {
// Name is the physical JetStream stream name. Always present: a listing
// entry the collector cannot name fails the collection rather than becoming
// an unlookupable row.
Name string `json:"name"`
// Kind is the logical resource the physical name backs. Derived from the
// name alone, so it is available even for a resource the server declines to
// describe.
Kind natsclient.ResourceKind `json:"kind"`
// Bucket is the logical bucket name behind a backing stream, with exactly
// one reserved prefix stripped. Empty for an ordinary stream.
Bucket string `json:"bucket,omitempty"`
// Attribution says whether an owner is defined, undeclared, or not a
// meaningful question for this kind of resource.
Attribution AttributionState `json:"attribution"`
// Owner is the logical owner as the descriptor catalog declares it.
// Non-empty only when Attribution is AttributionAttributed.
Owner string `json:"owner,omitempty"`
// Tier is the storage tier this resource's usage counts against.
Tier StorageTier `json:"tier"`
// Bytes and Messages are the two capacity axes JetStream bounds.
Bytes Capacity `json:"bytes"`
Messages Capacity `json:"messages"`
}
StorageResource is one account storage resource as the inventory sees it.
func (StorageResource) Attributed ¶
func (r StorageResource) Attributed() bool
Attributed reports whether the resource resolved to a declared owner. It reads the Attribution state rather than testing Owner for emptiness, so a not-applicable resource is never mistaken for an escaped bucket.
type StorageTier ¶
type StorageTier string
StorageTier is the JetStream storage tier backing a resource. JetStream keeps SEPARATE memory and file account limits and this repository's own streams span both, so the tier is recorded per resource and must never be guessed: summing across tiers produces a number that means nothing.
const ( // TierFile is file-backed storage. TierFile StorageTier = "file" // TierMemory is memory-backed storage. TierMemory StorageTier = "memory" // TierUnknown is a resource whose tier could not be read, because the // server declined to describe it. It is reported rather than defaulted: // defaulting would silently file the resource under one account limit's // comparison. TierUnknown StorageTier = "unknown" )
Storage tiers.
type StreamLister ¶
type StreamLister interface {
ListStreams(context.Context, ...jetstream.StreamListOpt) jetstream.StreamInfoLister
StreamNames(context.Context, ...jetstream.StreamListOpt) jetstream.StreamNameLister
}
StreamLister is the narrow JetStream capability the inventory needs: the two paged account listings. jetstream.JetStream satisfies it.
BOTH are required, and neither is a per-resource round-trip. ListStreams returns configuration and state together, which is what keeps the inventory off an N+1 describe path. StreamNames is the completeness check: the server excludes an offline stream from the info listing but not from the name listing, so without the second listing the inventory silently omits exactly the resources nobody can read.
The interface is this narrow so the collector's classification, ordering, reconciliation, degradation, and timeout behavior are all testable without a server, while the production path still drives both real listings through an integration test.
A lister that also has the method AccountInfo(ctx context.Context) (*jetstream.AccountInfo, error), as jetstream.JetStream does, feeds the account tier comparison too; one without it gets an inventory whose comparison is reported unknown.
type StreamListerSource ¶
type StreamListerSource func() (StreamLister, error)
StreamListerSource yields the account lister for ONE collection. It is resolved per collection rather than captured at construction, so a collector built before its client connects starts working when the client does, and one that outlives a reconnect never holds a stale JetStream context.
func AccountStreamLister ¶
func AccountStreamLister(client *natsclient.Client) StreamListerSource
AccountStreamLister returns the source that resolves client's live JetStream context as the narrow account-listing capability the storage inventory needs. The source calls client.JetStream on each call, so resolution happens per collection.
type ThresholdSource ¶
type ThresholdSource func() StoragePressureThresholds
ThresholdSource yields the pressure thresholds for ONE evaluation.
It is a FUNCTION, and that is the whole point. SemStreams is flow-based and its components are runtime-reconfigurable — watchConfigUpdates is launched after the boot barrier specifically so post-boot component edits reach running components. A threshold captured into a value at composition-root construction would apply the stale number successfully and silently after an operator edit: nothing errors, the report just answers the wrong question. Resolving at the seam, from live configuration, is the same discipline the bucket catalog applied at the acquisition seam.
type TierComparison ¶
type TierComparison struct {
// Tier is the storage tier this row compares within. Memory and file have
// separate account limits; the unknown tier has none at all.
Tier StorageTier `json:"tier"`
// Limit is the account ceiling for this tier paired with the account's
// usage against it — bounded, unbounded, or unknown, on the same three-state
// model every capacity in this capability uses.
Limit Capacity `json:"limit"`
// DeclaredBytes is the sum of the configured bounds of the BOUNDED
// resources in this tier. Unbounded and unknown-capacity resources
// contribute nothing, which makes this a FLOOR rather than a total whenever
// UnboundedResources or UnknownResources is non-zero.
DeclaredBytes int64 `json:"declared_bytes"`
// The resource counts behind DeclaredBytes, published so a consumer can see
// how much of the tier the sum actually covers.
BoundedResources int `json:"bounded_resources"`
UnboundedResources int `json:"unbounded_resources"`
UnknownResources int `json:"unknown_resources"`
// State is the verdict. Report-only: nothing rejects, throttles, or
// degrades because a tier is over-committed.
State OvercommitmentState `json:"state"`
Unavailable string `json:"unavailable,omitempty"`
// Growth, Projection and Pressure are the tier ceiling's own capacity
// picture, measured against the account's usage of this tier rather than
// against any resource's declaration.
//
// They are here because a tier limit is a CEILING like any other, and an
// unbounded resource has no other one. Over-commitment answers a different
// question — "do the declarations filed against this tier fit inside it" —
// and answers it about declared bounds, so a tier can be comfortably
// within-limit while the bytes actually stored are about to exhaust it. Both
// are published because they fail independently.
//
// The rate is measured across the account row's own retained history, which
// carries this tier's usage at every past collection (storage_report.go). It
// is the tier's rate, not the sum of any subset of resource rates: resources
// appear and disappear between collections, and a sum over the ones that
// happen to be present would move for reasons that are not growth.
Growth Growth `json:"growth"`
Projection Projection `json:"projection"`
Pressure Pressure `json:"pressure"`
}
TierComparison is one storage tier's account-level capacity picture.