Documentation
¶
Overview ¶
Package runs executes one billing period: it takes the period lock, opens a run row, chains metering, counters, rating, attribution and statement rendering over one reporting snapshot, and writes what came out under that run. What a period is billed as is decided by the packages this one calls; what belongs here is the order they run in, the run's bookkeeping, and the stats an operator reads a finished run from. Adjustment resolution runs between attribution and the statement build, and the adjustment records its lines become are written in the run's one write transaction beside the rest of the output.
A run either writes its whole output or none of it. The records, the supersede of the run it replaces, and the completion are one transaction, so a reader of the period never sees two completed runs or a completed run with half its records. A failure after the run row exists ends that row as 'failed' with the reason in its stats, and the previously completed run keeps standing.
A correction (Correct) is a run of kind 'correction' over a finalized period, through the same lock, the same run row, the same snapshot and the same write transaction. What it adds is the diff: its rated amounts are held against those of the latest finalized run of the period, and the differences are stored as correction deltas and as one credit note per affected project. The report ahead of it is DetectLate, which lists the resources whose events arrived after that run read the reporting database.
The normative specification is roadmap/03-phase-3-metering-rating.md, WP 3.8, with one deviation from its literal SQL: the roadmap takes the period lock as pg_advisory_xact_lock, which lives for one transaction, while a run opens several and holds the lock across all of them. The lock is a session lock here, taken and released on the one connection Execute holds for its whole length (the author's decision of 2026-08-24). It has to be released on that same connection: pgxpool does not reset session state when a connection is returned, and tally-engine tick executes several periods in one process, so a lock left behind would block the next run of that period until the process ends. A release that fails takes its connection out of the pool with it, because a connection that may still hold the lock is worse than one less.
Index ¶
- Constants
- Variables
- func Finalize(ctx context.Context, engine *pgxpool.Pool, periodFrom time.Time, ...) (string, error)
- func Reclaim(ctx context.Context, engine *pgxpool.Pool, periodFrom time.Time) ([]uuid.UUID, error)
- type CorrectOptions
- type CorrectionResult
- type CorrectionStats
- type LateReport
- type Options
- type Result
- type Stats
- type Warning
Constants ¶
const ( KindRegular = "regular" KindCorrection = "correction" )
KindRegular and KindCorrection are the two values runs.kind carries: a regular run meters a period, a correction re-meters a finalized one and stores the differences. statusFinalized is the period status this package refuses to meter.
const WarningPeriodNotEnded = "period_not_ended"
WarningPeriodNotEnded marks a run over a period that has not ended yet. Every resource that is still alive is billed to the period's end, so the numbers such a run produces are a forecast of the month rather than its invoice.
Variables ¶
var ErrLockReleaseFailed = errors.New("the period lock could not be released")
ErrLockReleaseFailed marks the one failure of a run that leaves its output standing: the run itself came through and only the release of the period lock did not. The Result beside such an error is the committed run, so a caller reports the month as billed and this as a warning rather than losing the run id and metering the month again. It marks nothing else: a run that broke comes back as the error that broke it, whether its lock was released or not.
A connection pooler in transaction pooling mode is what produces it in practice: the session lock lands on one server connection and its release on another, so every run of the deployment ends here.
var ErrPeriodFinalized = errors.New("the billing period is finalized")
ErrPeriodFinalized is what Execute returns for a period that is closed. A regular run leaves it alone, because its records are immutable; what changes a finalized period is a correction run (Correct).
var ErrPeriodMismatch = errors.New("the run bills another billing period")
ErrPeriodMismatch is what Finalize returns for a run of another month than the one being closed. Both months are named: the pair reaches the engine from two flags of one command, and either of them can be the mistyped one.
var ErrPeriodNotFinalized = errors.New("the billing period is not finalized")
ErrPeriodNotFinalized is what Finalize returns for a correction run of a period that is not closed, and what Correct returns for a period it is asked to correct that is not closed. A correction exists only over a finalized month: what it books is what arrived after the run that billed that month, and an open month is billed by metering it again.
var ErrRunInProgress = errors.New("another run of this period is in progress")
ErrRunInProgress is what Execute returns for a period another process is metering. Nothing is written for such a call: the period lock is taken before the first row.
var ErrRunNotCompleted = errors.New("the run is not completed")
ErrRunNotCompleted is what Finalize returns for a run that did not get to the end of its pass. A period is billed from the records of a completed run, so a running, failed or superseded one closes nothing.
var ErrRunNotFound = errors.New("the run does not exist")
ErrRunNotFound is what Finalize returns for a run id no row carries. Nothing is written for such a call. export.Load wraps the same sentinel for a run id it finds no row to export for, so one check tells a missing run from every other refusal either of them makes.
Functions ¶
func Finalize ¶
func Finalize( ctx context.Context, engine *pgxpool.Pool, periodFrom time.Time, runID uuid.UUID, ) (string, error)
Finalize closes a completed run, and is the whole of what tally-engine finalize does. Once its transaction commits, the D8 triggers of migration 0001 hold that run's usage records, rated records, correction deltas and statements against every write.
A regular run closes the billing period it meters. The run's status and the period's move in one transaction, so no reader ever sees a period pointing at a run that is not finalized itself. A correction run, which exists only over a period that is already closed, moves to 'finalized' alone: billing_periods keeps naming the regular run that closed the month, because those three columns say which run closed the period, and the latest finalized truth of a month is read through LatestFinalizedRun rather than off the period row.
The period is locked the way Execute locks it, and for the same length: a finalize must not land between the record writes of a run and the completion that ends it. A period another process is metering yields an error wrapping ErrRunInProgress.
The five refusals are a run no row carries (ErrRunNotFound), a run of another month (ErrPeriodMismatch), a run that is not completed (ErrRunNotCompleted), a regular run over a period that is already closed (ErrPeriodFinalized), and a correction run over a period that is not closed (ErrPeriodNotFinalized). None of them writes a row.
What comes back is the kind of the run that was closed, which decides what the caller reports: a regular run closed its period, a correction closed only itself. It is the empty string on every refusal.
func Reclaim ¶
Reclaim fails the runs of one period whose process is gone and returns the ids it took. It is what a caller that does not meter the period itself reclaims through: the scheduler counts a month's failed runs before it decides whether to meter it again, and a killed run that nothing has failed yet counts as no failure at all.
The period is locked the way Execute locks it, and for the same reason. What stands for the missing process is the age of the run row, and that age bounds no run of tally-engine run --period: a month large enough to meter for longer than it would be failed underneath the process that is still metering it, and the write that process is heading for would then be discarded whole. Under the lock a run that still reads as 'running' has no process behind it.
A period another process is metering yields an error wrapping ErrRunInProgress and nothing is written, which is the answer the reclaim was asking for: that run is the process the age was standing in for.
Types ¶
type CorrectOptions ¶
type CorrectOptions struct {
// PeriodFrom and PeriodTo are the half-open interval of the billing month,
// as internal/engine/period derives it.
PeriodFrom, PeriodTo time.Time
// AttributingRelationTypes are the relation types attribution walks. An
// empty list is attribution turned off.
AttributingRelationTypes []string
// AdjustmentRelationTypes are the relation types adjustment resolution
// walks from a statement's project. An empty list is adjustments turned
// off.
AdjustmentRelationTypes []string
// AdjustmentDepth bounds that walk: how many relation levels it follows
// from the statement's project, at least 1. The configuration checks it,
// and so does adjustments.New.
AdjustmentDepth int
// Counters is the counter sources file of the deployment, empty where it
// measures no counter metric.
Counters counters.Config
// VM answers the metricsql counter sources. It is nil unless
// Counters.HasMetricsQL() reports that any source needs it.
VM counters.Querier
}
CorrectOptions is what one correction meters. There is no cloud list: a correction inherits the clouds of the run it corrects, because it re-meters what that run billed rather than a set of clouds a caller chooses.
type CorrectionResult ¶
type CorrectionResult struct {
// RunID is the correction run that was opened. It is the zero id when the
// call was refused before any row was written.
RunID uuid.UUID
// CorrectsRunID is the finalized run the correction diffed against, which
// its run row and every delta it wrote name.
CorrectsRunID uuid.UUID
// PricingVersion is the model version the period was rated with, which is
// the one the corrected run recorded.
PricingVersion string
// Stats is what the correction counted, the same object it stored in
// runs.stats.
Stats CorrectionStats
// Superseded are the completed correction runs of the period this one
// replaced.
Superseded []uuid.UUID
// Reclaimed are the runs of the period that were still 'running' with no
// process behind them and were failed before this run opened.
Reclaimed []uuid.UUID
}
CorrectionResult is what one call to Correct did.
func Correct ¶
func Correct( ctx context.Context, engine *pgxpool.Pool, reporting *source.DB, opts CorrectOptions, ) (CorrectionResult, error)
Correct meters a finalized period again with the pricing version the latest finalized run of it recorded, diffs what came out against that run's amounts per the key of decision D6, and writes every non-zero difference as a correction_deltas row and one credit note per affected project, under a run of kind 'correction'. It is the whole of what tally-engine correct does. The engine pool is where the correction is written, the reporting database is where it is read from, and neither is closed here.
The baseline is the latest finalized run of the period by started_at: the regular run that closed the month for the first correction, and the last finalized correction after that. That run is what corrects_run_id names, on the run row and on every delta. A correction that is completed but not finalized is not the period's truth yet, so the next call diffs against the same baseline, produces the same deltas again, and supersedes it the way a regular run supersedes the completed run it replaces. The baseline cannot move while the pass runs, because Finalize takes the period lock this call holds for its whole length.
The clouds are the ones the corrected run metered, and the counter metrics are measured again rather than read off that run: a correction re-meters the period whole (D6).
The period is locked for the length of the call. A period another process is metering yields an error wrapping ErrRunInProgress, a period that is not closed one wrapping ErrPeriodNotFinalized, and a pricing version the database no longer holds one wrapping pricing.ErrVersionNotFound; none of them writes a row. Any failure once the run row exists ends that row as 'failed' with the reason in its stats and comes back as the error that caused it. The one exception is the release of the period lock: it happens after the correction is committed, so a failure there comes back wrapping ErrLockReleaseFailed beside the CorrectionResult of a correction that is done.
type CorrectionStats ¶
type CorrectionStats struct {
Stats
Deltas int `json:"deltas"`
AdjustmentDeltas int `json:"adjustment_deltas"`
}
CorrectionStats is what a correction stores in runs.stats: everything a regular run counts, under the same keys, and the deltas it wrote beside them. Statements counts the credit notes, which are the documents a correction renders. AdjustmentDeltas counts the adjustments the correction applies differently than the run it corrects.
type LateReport ¶
type LateReport struct {
// RunID is the latest finalized run of the period, the one a correction
// would diff against.
RunID uuid.UUID
// Kind is that run's kind, KindRegular or KindCorrection: the regular run
// that closed the month, or the last correction finalized over it.
Kind string
// SnapshotAt is the instant that run read the reporting database at, in
// UTC. It is what the events listed here arrived after.
SnapshotAt time.Time
// Resources are the resources whose events arrived late, with how many did
// and when the last of them did. It is empty when nothing arrived late, and
// holds at most source.LateResourceLimit entries.
Resources []source.LateResource
// Truncated is how many further resources have late events beyond the ones
// Resources names. It is zero where Resources names every one of them.
Truncated int
}
LateReport is what DetectLate found: the run the late events are late against, and the resources they belong to.
func DetectLate ¶
func DetectLate( ctx context.Context, engine *pgxpool.Pool, reporting *source.DB, periodFrom, periodTo time.Time, ) (LateReport, error)
DetectLate lists the resources of a finalized period whose events reached the reporting database after the latest finalized run of that period read it, and is the whole of what tally-engine detect-late does. The engine pool is where the finalized run is read from, the reporting database is where the events are counted, and neither is closed here.
The run the events are held against is the latest finalized run of the period by started_at, which is the one a correction diffs against: the regular run that closed the month, and the last finalized correction after that. The instant is the snapshot_at that run recorded in its stats.
The report names at most source.LateResourceLimit resources and counts the rest in Truncated, because a period finalized before its ingest had settled has one of them per resource of the fleet.
No period lock is taken and nothing is written. A report read while a correction or a finalization of the period is in flight can therefore name a run that is superseded a moment later, and what gives the current answer is reading it again once that pass is done.
The comparison against the snapshot time is strict. An ingest transaction that began before the snapshot and committed after it carries a received_at below it, so its events are not listed here, although a correction of the period does re-meter them. The window is the length of an ingest transaction; see ListLateEvents in internal/engine/source/queries.sql.
A period without a finalized run yields an error wrapping ErrPeriodNotFinalized, and a finalized run whose stats carry no snapshot_at one that names that run. A period whose chunks the reporting database has compressed is read whole, which its statement_timeout is the only bound on; see lateEventsError for what running into it reports.
type Options ¶
type Options struct {
// PeriodFrom and PeriodTo are the half-open interval of the billing month,
// as internal/engine/period derives it.
PeriodFrom, PeriodTo time.Time
// Clouds are the clouds to meter. An empty list meters every cloud the
// projection knows, which is the only meaning the engine can give it: it
// keeps no cloud list of its own.
Clouds []string
// AttributingRelationTypes are the relation types attribution walks. An
// empty list is attribution turned off.
AttributingRelationTypes []string
// AdjustmentRelationTypes are the relation types adjustment resolution
// walks from a statement's project. An empty list is adjustments turned
// off.
AdjustmentRelationTypes []string
// AdjustmentDepth bounds that walk: how many relation levels it follows
// from the statement's project, at least 1. The configuration checks it,
// and so does adjustments.New.
AdjustmentDepth int
// Counters is the counter sources file of the deployment, empty where it
// measures no counter metric.
Counters counters.Config
// VM answers the metricsql counter sources. It is nil unless
// Counters.HasMetricsQL() reports that any source needs it.
VM counters.Querier
}
Options is what one run meters.
type Result ¶
type Result struct {
// RunID is the run that was opened. It is the zero id when the call was
// refused before any row was written.
RunID uuid.UUID
// PricingVersion is the model version the period was rated with, which the
// CLI prints beside the run.
PricingVersion string
// Stats is what the run counted, the same object it stored in runs.stats.
Stats Stats
// Superseded are the completed regular runs of the period this one replaced.
Superseded []uuid.UUID
// Reclaimed are the runs of the period that were still 'running' with no
// process behind them and were failed before this run opened.
Reclaimed []uuid.UUID
}
Result is what one call to Execute did.
func Execute ¶
func Execute(ctx context.Context, engine *pgxpool.Pool, reporting *source.DB, opts Options) (Result, error)
Execute meters, rates and records one billing period, and is the whole of what tally-engine run does. The engine pool is where the run is written, the reporting database is where it is read from, and neither is closed here.
The period is locked for the length of the call. A period another process is already metering yields an error wrapping ErrRunInProgress, and a finalized one an error wrapping ErrPeriodFinalized; neither writes a row. Any failure once the run row exists ends that row as 'failed' with the reason in its stats and comes back as the error that caused it, so the caller reports what went wrong rather than that something did. The one exception is the release of the period lock: it happens after the run is committed, so a failure there comes back wrapping ErrLockReleaseFailed beside the Result of a run that is done.
type Stats ¶
type Stats struct {
SnapshotAt *time.Time `json:"snapshot_at,omitempty"`
Candidates int `json:"candidates"`
UsageRecords int `json:"usage_records"`
RatedRecords int `json:"rated_records"`
Statements int `json:"statements"`
AdjustmentRecords int `json:"adjustment_records,omitempty"`
Warnings []Warning `json:"warnings,omitempty"`
MeteringWarnings []metering.Warning `json:"metering_warnings,omitempty"`
CounterWarnings []counters.Warning `json:"counter_warnings,omitempty"`
AttributionWarnings []attribution.Warning `json:"attribution_warnings,omitempty"`
AdjustmentWarnings []adjustments.Warning `json:"adjustment_warnings,omitempty"`
Unpriced []rating.UnpricedResourceType `json:"unpriced,omitempty"`
Unreadable []rating.UnreadableQuantity `json:"unreadable,omitempty"`
UnregisteredProjects []statements.UnregisteredProject `json:"unregistered_projects,omitempty"`
Violations []metering.ResourceViolations `json:"violations,omitempty"`
Error string `json:"error,omitempty"`
}
Stats is the object a run stores in runs.stats. Every list is left out when it is empty, so a clean run reads as the four counts and nothing else.