Documentation
¶
Overview ¶
Package counters measures the counter metrics of a deployment per usage draft, from the counter sources file the deployment configures. The pass sits after metering.Meter and before anything is persisted, while the reporting snapshot is still open: the events reads go through the same snapshot as the history the drafts were folded from, so a counter and the drafts it slices into see the same data.
A source measures its metric in one of two ways. An events source counts the events of one type inside the draft's interval. A metricsql source runs an instant query against VictoriaMetrics at the draft's end, over the draft's length.
A metricsql source is required unless the file says required: false. A failed query of a required source fails the pass, whatever failed it, because billing data must not silently omit a revenue-relevant counter: an identity no MetricsQL query may carry would otherwise leave that omission to whoever writes an event. A failed query of an optional source leaves the metric out of that draft and yields a warning. So does an identity no query may carry, under its own code, because unlike a failed query it does not clear on a rerun.
This package persists nothing.
The normative specification is roadmap/03-phase-3-metering-rating.md, WP 3.4 and decision D7.
Index ¶
Constants ¶
const WarningCounterIdentityNotQueryable = "counter_identity_not_queryable"
WarningCounterIdentityNotQueryable marks a resource whose identity a MetricsQL query may not carry, for a source declared required: false. It is a second code rather than the one above because it does not clear on a rerun: the metric stays missing from that resource's drafts until the identity changes, which is a different thing to alert on than a store that was down.
const WarningCounterSourceFailed = "counter_source_failed"
WarningCounterSourceFailed marks a metricsql source declared required: false whose query failed for one draft. The metric is left out of that draft.
Variables ¶
var ErrAnswerShape = errors.New("the answer cannot be billed")
ErrAnswerShape marks the failures of a query that are a property of the one answer it got rather than of the store: a vector that did not aggregate, a value MetricsQL printed as NaN or Inf, a result of the wrong type. A Querier wraps it so that a caller can tell them apart from an outage, because the next resource's answer says nothing about them and the retry ladder an outage costs was never paid for them.
Functions ¶
func RenderQuery ¶
RenderQuery substitutes a metricsql source's placeholders with the draft they are measured for: {cloud}, {resource_id} and {project_id} with the draft's identity, {window} with its length.
The three identity values reach the engine from ingested event data, while the query around them is text an operator wrote. A value that is not inert in MetricsQL is therefore refused rather than escaped: nothing here can tell which of MetricsQL's string literal forms a placeholder sits in, or whether it sits in one at all, so an escaper for any single form would let such a value compose a query of its own. Only the placeholders the query uses are checked. The window is rendered from a count of seconds and needs no check.
Types ¶
type Config ¶
type Config struct {
Sources []Source
}
Config is the counter sources of a deployment, in file order.
func Load ¶
Load reads the counter sources at path. An empty path yields the zero Config and no error, which is what a deployment that measures no counter metric configures.
A path that cannot be read is an error rather than a run without counters: a misconfigured path would otherwise drop a billed metric from every draft of the period without anything saying so.
func Parse ¶
Parse decodes the counter sources and checks every entry. An empty document, one that holds only comments, and one without sources each yield the zero Config and no error.
A key the file form does not know is an error naming it, so that a misspelled event_typ fails the run instead of leaving the source measuring nothing.
func (Config) HasMetricsQL ¶
HasMetricsQL reports whether any source is a metricsql source, which is how a caller knows whether it needs a VictoriaMetrics client at all.
type EventCounter ¶
type EventCounter interface {
CountEvents(ctx context.Context, r source.Resource, eventType string, from, to time.Time) (int64, error)
}
EventCounter counts the events of one type inside an interval. It is the seam an events source is measured through, implemented by *source.Snapshot; the unit tests here and the engine's golden suite substitute their own.
type Measurer ¶
type Measurer struct {
// contains filtered or unexported fields
}
Measurer measures the counter metrics of a configuration into usage drafts.
func New ¶
func New(cfg Config, events EventCounter, vm Querier) (*Measurer, error)
New checks cfg the way a loaded file is checked and indexes its sources by the resource type they measure, so a hand-built configuration is held to the same rules as one Load returned.
A kind configured without the seam it is measured through is an error rather than a run that leaves those metrics out of every draft. A configuration without sources needs neither seam.
func (*Measurer) Apply ¶
func (m *Measurer) Apply(ctx context.Context, resources []metering.ResourceUsage) ([]Warning, error)
Apply measures the counter metrics of every resource it holds sources for into that resource's drafts. It runs over the resources of a metering pass, after metering.Meter and before anything is persisted, while the reporting snapshot is still open.
The values are written into each draft's usage object in place. That map is the one metering built and every copy of the draft shares it, so the caller's metering.Result carries the counters once Apply returns.
A source that fails where its metric may not be missing ends the pass. The drafts measured before it keep the values they were given, which is why a caller discards the whole result on an error instead of persisting what came back. A failed optional metricsql source yields a warning instead, and its metric is left out of that one draft; once it has failed maxSourceFailures drafts in a row it is left out of the remaining ones without being queried again, each of them still named by its own warning, and every probeEvery-th of those drafts queries it once more so that a store which came back is found while the pass still runs.
A resource whose identity a MetricsQL query may not carry fails a required source like any other failure: the identity comes from ingested event data, so warning over it instead would hand whoever writes an event the decision whether a billed counter is measured. An optional source is warned under WarningCounterIdentityNotQueryable, which unlike a failed query does not clear on a rerun. Neither that refusal nor an answer this one resource shaped is counted against the source: only a store that is down says anything about the next draft, so the resources whose own answers are fine keep the metric.
type Querier ¶
type Querier interface {
Query(ctx context.Context, expr string, at time.Time) (decimal.Decimal, error)
}
Querier runs an instant query and returns its single value. It is the seam a metricsql source is measured through, implemented by *VMClient; the unit tests here and the engine's golden suite substitute their own.
type Source ¶
type Source struct {
Platform, ResourceType, Metric string
Kind Kind
EventType string
Query string
Required bool
}
Source is one resolved entry of the counter sources file: which metric of which platform and resource type it measures, and how. EventType is set for events sources and Query for metricsql sources. Required is true unless the file said required: false, and it applies to metricsql sources only.
type VMClient ¶
type VMClient struct {
// contains filtered or unexported fields
}
VMClient is the instant-query client over the VictoriaMetrics read endpoint /api/v1/query.
func NewVMClient ¶
NewVMClient checks baseURL and returns the client that queries it. The url is the read endpoint's base, with or without a trailing slash and with or without a path prefix, such as http://victoriametrics:8428.
A nil httpClient selects the package default, which retries connection errors, 429, and 5xx below http.Client.Do and bounds one attempt by queryTimeout. Tests pass their own client.
func (*VMClient) Query ¶
Query runs expr as an instant query at at, which is a draft's end, and returns the single value it selects. The value comes back unrounded, because the quantity a counter contributes is rounded where it is merged into the draft.
An empty result is zero: a resource that reported no sample over the query's window used none of the metric. More than one series is an error rather than a sum or a pick of the first, because both would bill a number nobody configured; such a query has to aggregate to a single series.
type Warning ¶
type Warning struct {
Cloud string `json:"cloud"`
ResourceType string `json:"resource_type"`
ResourceID string `json:"resource_id"`
Metric string `json:"metric"`
FromTS time.Time `json:"from_ts"`
ToTS time.Time `json:"to_ts"`
Code string `json:"code"`
Detail string `json:"detail"`
}
Warning names a metric that is missing from one draft and why. It is a second warning type beside metering.Warning because it names a metric and an interval, which a metering warning has no field for; the run writes both lists to its stats. Detail is the failed source's error text.
Source Files