Documentation
¶
Overview ¶
Package source is the metering engine's read seam over the Reporting database: a connection pool and, per run, one read-only REPEATABLE READ transaction that every read of that run goes through (D1, D3). The engine reaches the reporting data over its own connection rather than over the API, through the tally_engine_reader role, and this package writes nothing, here or anywhere else.
Every reporting read of the engine is a method on Snapshot, the event count a counter measures included (CountEvents) and the late events a correction is due for (LateEvents). The transaction itself stays unexported, so a later package that needs another query adds a method here rather than taking a handle on the transaction and running its own statements outside the snapshot.
A snapshot holds one connection for its lifetime and pins the reporting database's vacuum horizon while it is open, which keeps dead tuples from being reclaimed. A caller closes it as soon as metering has read what it needs rather than keeping it open across the rating and writing that follow, and the timeouts New sets keep a read that hangs from holding the horizon past them.
The static queries in queries.sql are compiled by sqlc into the sqlcgen subpackage. `make generate` rewrites that package, and it is committed so that a plain `go build` needs no generator.
The normative specification is roadmap/03-phase-3-metering-rating.md, WP 3.2.
Index ¶
- Constants
- type DB
- type LateResource
- type Project
- type Relation
- type Resource
- type Snapshot
- func (s *Snapshot) Candidates(ctx context.Context, clouds []string, periodFrom, periodTo time.Time) ([]Resource, error)
- func (s *Snapshot) Close(ctx context.Context) error
- func (s *Snapshot) CountEvents(ctx context.Context, r Resource, eventType string, from, to time.Time) (int64, error)
- func (s *Snapshot) History(ctx context.Context, r Resource, periodTo time.Time) ([]event.Stored, error)
- func (s *Snapshot) LateEvents(ctx context.Context, periodFrom, periodTo, since time.Time) ([]LateResource, int, error)
- func (s *Snapshot) Projects(ctx context.Context) ([]Project, error)
- func (s *Snapshot) Relations(ctx context.Context, relationTypes []string, periodFrom, periodTo time.Time) ([]Relation, error)
Constants ¶
const LateResourceLimit = 100
LateResourceLimit is how many resources one LateEvents call lists. A period finalized before its ingest had settled — a collector backlog that flushed afterwards, a cloud re-synced from its history — has one late group per resource of the fleet, and what the caller does with a group is print a line. A hundred of them name what a period needs a correction for; a fleet's worth of them is a terminal dump nobody reads and a row per resource carried across the wire for it. What the limit leaves out is counted rather than dropped, so how much arrived late is exact either way.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type DB ¶
type DB struct {
// contains filtered or unexported fields
}
DB owns the connection pool the engine reads the reporting database through.
func New ¶
New parses dbURL and prepares the pool for it. Connections are established lazily, so a database that is down does not keep the process from starting.
func (*DB) Close ¶
func (db *DB) Close()
Close releases every connection the pool holds and waits for in-flight ones to be returned.
func (*DB) Snapshot ¶
Snapshot opens the transaction a run reads everything through. It is REPEATABLE READ, so every query in it sees the same version of the data however long the run takes, and read-only as defense in depth under the grants the reader role holds.
Reading the snapshot time is the transaction's first statement, which is where REPEATABLE READ takes the MVCC snapshot the later queries see. The value it returns is that snapshot's time, which the run records as runs.stats.snapshot_at.
type LateResource ¶
LateResource is one resource of a period whose events reached the reporting database after a run read it: how many arrived late, and when the last of them did, in UTC.
type Relation ¶
type Relation struct {
ID uuid.UUID
SourceID uuid.UUID
TargetID uuid.UUID
RelationType string
// Metadata is the stored document as JSONB renders it, {} for a relation
// created without one. It is read by adjustment resolution.
Metadata json.RawMessage
ValidFrom time.Time
ValidTo *time.Time
}
Relation is one edge of the project graph. ValidTo is nil while the relation is open.
type Snapshot ¶
type Snapshot struct {
// At is when the snapshot was taken, in UTC. It is what the run records as
// runs.stats.snapshot_at: every row this snapshot returns is the data as of
// this instant.
At time.Time
// contains filtered or unexported fields
}
Snapshot is one run's consistent view of the reporting database.
func (*Snapshot) Candidates ¶
func (s *Snapshot) Candidates(ctx context.Context, clouds []string, periodFrom, periodTo time.Time) ([]Resource, error)
Candidates lists the resources the period can bill, ordered by cloud, resource type, and resource id. The projection is the index: its rows are never deleted, so a resource that lived during the period is still there to be found.
An empty clouds list meters every cloud the projection knows. The engine keeps no cloud list of its own, so "no filter" is the only meaning the empty list can carry.
func (*Snapshot) Close ¶
Close ends the transaction and returns its connection to the pool. A snapshot that is already closed is not an error, so a deferred Close next to an explicit one both report success.
func (*Snapshot) CountEvents ¶
func (s *Snapshot) CountEvents(ctx context.Context, r Resource, eventType string, from, to time.Time) (int64, error)
CountEvents counts the events of one type of r whose timestamp lies in the half-open interval [from, to). It is the count an events-kind counter source measures per usage interval, and it runs inside the snapshot transaction, so the count and the history the interval was folded from see the same data. A resource with no matching events counts as 0 and no error.
func (*Snapshot) History ¶
func (s *Snapshot) History(ctx context.Context, r Resource, periodTo time.Time) ([]event.Stored, error)
History loads one resource's events up to the period end, ordered by timestamp, received_at, and event id. The history starts at the first event there is, because the state the resource holds at the period start is whatever the events before it left behind.
func (*Snapshot) LateEvents ¶
func (s *Snapshot) LateEvents(ctx context.Context, periodFrom, periodTo, since time.Time) ( []LateResource, int, error, )
LateEvents counts, per (cloud, platform, resource_type, resource_id), the events dated in [periodFrom, periodTo) whose received_at lies strictly after since, which a caller passes as the snapshot time of the finalized run of the period. The groups come back ordered by those four fields, and LastReceivedAt is in UTC. A period nothing reached late is an empty slice rather than a nil one, and no error.
At most LateResourceLimit groups come back. The second return is how many resources arrived late altogether, which is the length of the slice where the limit left nothing out and larger where it did.
It is what tally-engine detect-late reports: the resources a correction of the period would re-meter from events the finalized run never saw.
func (*Snapshot) Projects ¶
Projects reads the project registry whole, ordered by cloud and external id. It is one of the two project graph loaders of WP 3.2; their consumer is attribution, which walks the graph over these UUIDs while the resources it walks from carry the external id rather than the registry's.
func (*Snapshot) Relations ¶
func (s *Snapshot) Relations(ctx context.Context, relationTypes []string, periodFrom, periodTo time.Time) ([]Relation, error)
Relations lists the relations of the given types whose validity overlaps the period (D4), ordered by id. It is the second project graph loader of WP 3.2, read by attribution.
An empty type list is attribution turned off, which is what an explicitly empty TALLY_ENGINE_ATTRIBUTING_RELATION_TYPES configures. No query runs then, because every relation type is one attribution would not walk.
Adjustment resolution is the second reader of these relations, and an empty type list turns its walk off the same way.
Source Files
Directories