Documentation
¶
Overview ¶
Package resourcemanager emits metering.v1 billing data for durable billable resources such as trigger registrations, workflow specs, and log filters.
It emits two complementary, load-bearing billing streams, each covering exactly one resource identified by its ResourceIdentity:
- MeterRecords: the durable, first-class billing event stream. Each record captures one signed delta — the level change of one request against a durable resource (EmitDelta -> METER_ACTION_UPDATE) or one instantaneous occurrence of consumption (EmitUsage -> METER_ACTION_USAGE). Records give the consumer precise request-time edges and the audit trail.
- MeterSnapshots: the periodic absolute level of each active resource, emitted on a timer, one per resource a registered Meterable reports as active. Snapshots carry the level and the liveness signal.
The emitter is stateless with respect to metering: a delta is derived only from the request being fielded plus the producer's own store. Producers emit only when fielding a request that changes desired state.
event_id is the consumer's cross-node dedup/aggregation key: deltas have counter semantics, so at-least-once delivery must dedup by event_id, and snapshot reconciliation then bounds any residual level drift. It has two derivations depending on the stream:
- MeterRecord: the producer supplies event_id (the eventID argument to EmitDelta/EmitUsage), derived deterministically from the triggering request or reconciliation event so that every DON node fielding the same logical delta emits the identical value. The ResourceManager never generates it; an empty eventID is an error — the emit is skipped and the failure counter is incremented (fail-open), never replaced by a random id (which would silently break cross-node dedup on a multi-node DON). See EventID for the canonical derivation helper.
- MeterSnapshot: the ResourceManager derives event_id deterministically from the snapshot bucket, resource, and node identity (see SnapshotEventID). Snapshots are reduced by median across nodes, so node_id is intentionally part of the key — the dedup scope is a single node's retries of one tick.
Bucket semantics: An org draws down credits in a time bucket if the bucket contains any positive-delta UPDATE record, any USAGE record, or any nonzero snapshot for a resource attributed to that org. A +N and -N cancelling within one bucket still triggers drawdown via the positive delta.
The ResourceManager is the single owner of the snapshot tick: each producer starts the manager as a sub-service and only Registers itself; producers never run their own snapshot loop.
Emission is fail-open by design: EmitDelta, EmitUsage, and the snapshot loop return no error, and a metering failure must never gate, delay, or retry the resource operation being metered. Failures surface via error-level logs and the resource_manager_*_failure_total counters.
Index ¶
- Constants
- func EventID(namespace string, parts ...string) string
- func NewUtilization(value string, fields UtilizationFields) *meteringpb.Utilization
- func NewUtilizationBig(value *big.Int, fields UtilizationFields) *meteringpb.Utilization
- func NewUtilizationFloat(value float64, fields UtilizationFields) *meteringpb.Utilization
- func NewUtilizationInt(value int64, fields UtilizationFields) *meteringpb.Utilization
- func SnapshotEventID(nodeID, service, resourcePool, resourceType, resourceID string, ...) string
- type Config
- type DeploymentIdentity
- type DonIdentity
- type Emitter
- type Meterable
- type ResourceIdentity
- type ResourceManager
- func (rm *ResourceManager) EmitDelta(ctx context.Context, identity ResourceIdentity, eventID string, delta int64, ...)
- func (rm *ResourceManager) EmitUsage(ctx context.Context, identity ResourceIdentity, eventID string, quantity int64, ...)
- func (rm *ResourceManager) Register(m Meterable) (unregister func())
- type ResourceManagerConfig
- type SnapshotEntry
- type UtilizationFields
Constants ¶
const DefaultSnapshotInterval = 60 * time.Second
DefaultSnapshotInterval is the recommended snapshot period. It is NOT applied implicitly: a zero ResourceManagerConfig.SnapshotInterval disables snapshots. Callers that want snapshots pass this (or their own value) explicitly.
const UnsetProduct = "unset"
UnsetProduct is a non-silent default product for misconfigured producers.
Variables ¶
This section is empty.
Functions ¶
func EventID ¶
EventID derives a deterministic, cross-node-identical MeterRecord event_id from an action namespace and ordered identifier parts.
It is the canonical helper every producer must use so that all DON nodes fielding the same logical delta compute the identical event_id — the consumer's cross-node dedup/aggregation key (see metering.v1.Utilization.event_id). Correctness therefore depends entirely on the CALLER: every part must be DON-consistent, i.e. identical on every node that fields the same request. Safe inputs are fields of a DON-aggregated request (the remote capability layer runs mode consensus over workflow-DON senders, so the request delivered to every capability node is byte-identical — see core/capabilities/remote/trigger_publisher.go) or a reconciliation signature. NEVER pass a locally-generated value (a per-node random id, os.Hostname, time.Now, ...): that reintroduces per-node event_ids and silently breaks cross-node dedup.
The namespace (e.g. "register", "unregister", "evm-activate") both scopes the action so paired +N/-N deltas do not collide and makes the id human-recognizable; the parts are hashed so the id has bounded length and no delimiter-injection ambiguity. The result is "<namespace>:<hex sha256>".
func NewUtilization ¶
func NewUtilization(value string, fields UtilizationFields) *meteringpb.Utilization
NewUtilization builds a Utilization from a pre-formatted numeric string value. event_id is left empty; the ResourceManager stamps it at emit time.
func NewUtilizationBig ¶
func NewUtilizationBig(value *big.Int, fields UtilizationFields) *meteringpb.Utilization
NewUtilizationBig builds a Utilization from an arbitrary-precision integer.
func NewUtilizationFloat ¶
func NewUtilizationFloat(value float64, fields UtilizationFields) *meteringpb.Utilization
NewUtilizationFloat builds a Utilization from a floating-point value.
func NewUtilizationInt ¶
func NewUtilizationInt(value int64, fields UtilizationFields) *meteringpb.Utilization
NewUtilizationInt builds a Utilization with an int64 quantity encoded as a decimal string value. The value may be negative (e.g. an unregister or resize-down delta of -N for METER_ACTION_UPDATE).
func SnapshotEventID ¶
func SnapshotEventID(nodeID, service, resourcePool, resourceType, resourceID string, bucketUnix int64) string
SnapshotEventID derives the deterministic event_id the ResourceManager stamps on a MeterSnapshot utilization. Unlike MeterRecord event_ids, snapshots are RM-derived (never producer-supplied): the key is
snapshot:{node_id}:{service}:{resource_pool}:{resource_type}:{resource_id}:{bucket_unix}
where bucket_unix is the snapshot tick time truncated to the snapshot interval, as unix seconds. It is stable across a node's retransmits of the same tick and distinct across buckets. node_id is intentionally part of the key: snapshots are reduced by median across nodes, so the dedup scope is a single node's own tick retries, not cross-node convergence.
Types ¶
type Config ¶
type Config struct {
ResourceManagerConfig
// DeploymentIdentity carries the static deployment + node identity
// dimensions resolved from the LOOP environment.
DeploymentIdentity DeploymentIdentity
}
Config is everything a LOOP producer needs to wire metering from its resolved process environment: the ResourceManagerConfig for NewResourceManager, plus the DeploymentIdentity used to build base ResourceIdentities via NewBaseIdentity. See loop.EnvConfig.MeteringConfig, which builds one of these from a resolved LOOP environment so the mapping lives in one place instead of being copy-pasted into every producer main.
type DeploymentIdentity ¶
type DeploymentIdentity struct {
// Product is the deployment product, e.g. "cre".
Product string
// Tenant is the human-readable deployment tenant name, e.g. "mainline" or
// "enterprise".
Tenant string
// NumericTenantID is the numbered tenant identifier as a string.
NumericTenantID string
// Environment is the deployment environment, e.g. "production", "staging".
Environment string
// Zone is the deployment zone, e.g. "wf-zone-a".
Zone string
// NodeID is the node's logical name, e.g. "clp-cre-wf-zone-a-1". It is NOT
// the CSA public key; it is a stable name the billing service can use to
// look up the node's CSA key in the workflow registry. The CSA key itself is
// carried separately as the node_csa_key event attribute.
NodeID string
}
DeploymentIdentity carries the static deployment + node identity dimensions that are fixed for a LOOP plugin process. They are resolved once from node config by the host and delivered to every LOOP plugin over the environment (loop.EnvConfig), not the standard-capabilities boundary, so any LOOP plugin can populate the coarse metering rollup dimensions. Any field may be empty if the host did not provide it.
type DonIdentity ¶
type DonIdentity struct {
// DonID is the DON ID the emitting service belongs to.
DonID string
// NodeID is the node's logical name within the scope of the DON, e.g.
// "clp-cre-wf-zone-a-1". It is a human-readable ID, NOT the CSA
// public key. The prefix can be redundant with other fully-qualified
// dimensions, but helps readability. The CSA key is emitted separately via
// the node_csa_key attribute.
NodeID string
}
DonIdentity captures DON-specific identity dimensions as one unit.
type Emitter ¶
Emitter delivers an encoded metering message with its routing attributes. beholder.Emitter satisfies it, so production wiring is beholder.GetEmitter(); tests substitute a fake.
type Meterable ¶
type Meterable interface {
// ResourceIdentity returns the producer's base identity: the coarse
// dimensions (product, tenant, numeric_tenant_id, environment, zone, don, service) plus
// the service-level resource_pool / resource_pool_id. Per-resource billing
// fields (resource_type/resource_id/org_id/value) are carried by
// Utilizations on MeterRecord and MeterSnapshot; event_id is stamped by the
// ResourceManager at emit time.
ResourceIdentity() ResourceIdentity
// GetUtilization returns the current level of the producer's currently
// active resources, one SnapshotEntry per resource. The manager emits one
// MeterSnapshot per entry.
//
// It is called on the snapshot tick and MUST be a cheap, non-blocking
// read-snapshot of in-memory state: no network, no disk, no lock held
// across I/O. It must tolerate ctx cancellation (returning promptly, and
// nil/empty is acceptable) and tolerate concurrent registration of new
// resources. An empty or nil return is valid and means nothing is currently
// active: no snapshots are emitted, and billing zeroes the resource out by
// its absence from subsequent snapshots.
GetUtilization(ctx context.Context) []SnapshotEntry
}
Meterable is implemented by producers that manage durable billable resources (trigger registrations, workflow specs, log filters). A producer registers itself with a ResourceManager (see ResourceManager.Register) so it is polled once per snapshot tick for the absolute state of its currently active resources, in addition to emitting request-time deltas inline via EmitDelta / EmitUsage.
type ResourceIdentity ¶
type ResourceIdentity struct {
// Product is the deployment product, e.g. "cre".
Product string
// Tenant is the human-readable deployment tenant name, e.g. "mainline" or
// "enterprise".
Tenant string
// NumericTenantID is the numbered tenant identifier as a string.
NumericTenantID string
// Environment is the deployment environment, e.g. "production",
// "staging".
Environment string
// Zone is the deployment zone, e.g. "wf-zone-a".
Zone string
// Don groups DON-specific identity dimensions so consumers can
// branch on one struct instead of handling don/node permutations.
Don *DonIdentity
// Service is the stable service constant identifying the emitting service,
// e.g. "cron-trigger", "http-trigger", "evm-log-trigger",
// "workflow-syncer-v2". It must not
// encode deployment environment or zone.
Service string
// ResourcePool is the service-level resource pool the record applies to,
// e.g. "trigger_registrations", "log_filters", "workflow_storage".
ResourcePool string
// ResourcePoolID optionally scopes identity further within the resource pool.
ResourcePoolID string
}
ResourceIdentity is the structured, first-class identity of a durable resource. Its fields map 1:1 to metering.v1.ResourceIdentity so every emitted record carries each dimension as a discrete column rather than a parsed dotted string or out-of-band telemetry attribute.
func NewBaseIdentity ¶
func NewBaseIdentity(dep DeploymentIdentity, service, resourcePool string) ResourceIdentity
NewBaseIdentity builds a producer's base ResourceIdentity from its static deployment identity plus the service and resource-pool constants. Product falls back to UnsetProduct when empty, and the Don sub-identity is set only when there is a node ID to carry. DON ID is stamped separately via WithDonID once the producer knows it (e.g. after Initialise for LOOP plugins).
func (ResourceIdentity) DonID ¶
func (r ResourceIdentity) DonID() string
DonID returns the DON ID when present.
func (ResourceIdentity) NodeID ¶
func (r ResourceIdentity) NodeID() string
NodeID returns the node ID when present.
func (ResourceIdentity) WithDonID ¶
func (r ResourceIdentity) WithDonID(donID string) ResourceIdentity
WithDonID returns a copy of r stamped with donID (empty donID is a no-op), necessary for any resource that is scoped to a DON.
type ResourceManager ¶
ResourceManager emits MeterRecords and periodic MeterSnapshots for durable resources. It is a services.Service: callers start it (typically as a sub-service of the producer) and Register Meterables to be snapshotted. It is safe for concurrent use.
func NewResourceManager ¶
func NewResourceManager(lggr logger.Logger, cfg ResourceManagerConfig) *ResourceManager
NewResourceManager returns a ResourceManager. A failure to create a metric instrument is logged and that instrument is skipped; it never prevents construction. The manager must be Started before its snapshot loop runs; EmitDelta and EmitUsage work regardless of Start.
func (*ResourceManager) EmitDelta ¶
func (rm *ResourceManager) EmitDelta(ctx context.Context, identity ResourceIdentity, eventID string, delta int64, fields UtilizationFields)
EmitDelta emits a metering.v1.MeterRecord with METER_ACTION_UPDATE: a signed delta to the durable resource's level (register = +N, unregister = -N, resize = ±delta). delta may be negative. The record is timestamped with the raw clock (records are request-time edges, not bucket-aligned).
eventID must be a deterministic, cross-node-identical identifier derived from the triggering request (see EventID): every DON node fielding the same logical delta must pass the identical value, because it is the consumer's cross-node dedup/aggregation key. An empty eventID is rejected (logged + counted, emit skipped); the ResourceManager never generates one.
EmitDelta is fail-open and returns no error: when the manager is disabled or has no emitter it does nothing, and marshal or emit failures are recorded only via error-level logs and the failure counter. Callers must never gate the resource operation being metered on emission.
func (*ResourceManager) EmitUsage ¶
func (rm *ResourceManager) EmitUsage(ctx context.Context, identity ResourceIdentity, eventID string, quantity int64, fields UtilizationFields)
EmitUsage emits a metering.v1.MeterRecord with METER_ACTION_USAGE: a one-off instantaneous consumption event billed per occurrence (no level, never snapshotted). eventID has the same cross-node-determinism contract as EmitDelta. Same fail-open semantics as EmitDelta.
func (*ResourceManager) Register ¶
func (rm *ResourceManager) Register(m Meterable) (unregister func())
Register adds m to the snapshot registry and returns an idempotent function that removes it. Calling the returned function more than once is a no-op. The returned closure is safe for concurrent use.
type ResourceManagerConfig ¶
type ResourceManagerConfig struct {
// MeterRecordsEnabled is the meter-record rollout gate. When false (the
// default), EmitDelta and EmitUsage are no-ops.
MeterRecordsEnabled bool
// MeterSnapshotsEnabled gates snapshot emission. Snapshots are only emitted
// when this is true AND MeterRecordsEnabled is true.
MeterSnapshotsEnabled bool
// Emitter delivers encoded records, typically beholder.GetEmitter(). A nil
// Emitter makes EmitDelta and EmitUsage no-ops even when enabled and keeps
// the snapshot loop from starting.
Emitter Emitter
// SnapshotInterval is the period between snapshots. Zero (the default)
// DISABLES the snapshot loop; the manager still starts as a service and
// EmitDelta / EmitUsage still work. Callers that want snapshots set a
// positive value, e.g. DefaultSnapshotInterval. The default is not
// substituted for zero — zero means off.
SnapshotInterval time.Duration
// Clock drives snapshot tick timing and record timestamps. Nil selects the
// real clock.
Clock clockwork.Clock
}
ResourceManagerConfig configures a ResourceManager.
type SnapshotEntry ¶
type SnapshotEntry struct {
Identity ResourceIdentity
Utilizations []*meteringpb.Utilization
}
SnapshotEntry is the current level of one active resource at a snapshot tick. Identity is the base resource identity, and Utilizations carries one or more billed dimensions for that resource (resource_type/resource_id/org_id/value); event_id is stamped by the ResourceManager at emit time.
type UtilizationFields ¶
UtilizationFields identifies one billed utilization dimension.
EventID is intentionally NOT a field here: event_id is the consumer's dedup key and is generated fresh per emission by the ResourceManager. Producers must never populate it, so it cannot be supplied through this struct; the manager stamps it on every emitted Utilization.