Help us shape the future of Go! Take the 2026 Go Developer Survey

obsquery

package
v1.0.14 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 2, 2026 License: Apache-2.0 Imports: 19 Imported by: 0

Documentation

Overview

Package obsquery compiles typed observability requests (log search, traces, metrics, dashboard panels, workloads, profiles) into parameterized ClickHouse SQL over the OTel exporter's tables.

One compiler serves two backends. The control plane runs it against an org's per-org database (t_<org>, provisioned by its obstenant schema), and forge runs it against a project's local ClickStack (LocalDatabase). The backend chooses the database through Scope.Database and runs the Stmt itself: this package never opens a connection. Tables a backend does not have (the control plane's browser tables, locally) fail at execution, not here.

It builds statements and parses results' shape. The caller runs a Stmt on its own connection, so the database, not this package, is the org boundary. Where a request takes several statements (EvalMetricPanel), the Runner is the caller's: obsquery opens no connection and holds no credential or setting. What this package adds is that nothing a caller sends can become SQL text:

  • every caller-supplied VALUE is a server-side parameter ({pN:Type}), never concatenated;
  • every IDENTIFIER (database, table, column) is either a constant here or the database name, which must pass IsIdentifier;
  • the log query language is parsed into an AST and compiled; there is no pass-through of query text.

Raw SQL is the escape hatch beside the compiler, not through it: SQLPanel binds a caller's statement to a scope with the dashspec.SQLParam* parameters, behind the alerting/dashspec lexical guard.

Index

Constants

View Source
const (
	AnomalySignalErrorRate  = "error_rate"
	AnomalySignalP95        = "p95"
	AnomalySignalThroughput = "throughput"
)

Signals a ServiceAnomaly finding names (alerting.Why.Signal).

View Source
const (
	LogKeyCodeFunction     = "code.function.name"
	LogKeyCodeFile         = "code.file.path"
	LogKeyProcedure        = "procedure"
	LogKeyExceptionType    = "exception.type"
	LogKeyExceptionMessage = "exception.message"
	LogKeyErrorGroup       = "error.group"
	LogKeyIssueSource      = "issue.source"
	// LogKeyError is the attribute a logger puts a failure's cause in when it
	// writes it as a string (slog's conventional "error" key).
	LogKeyError = "error"
)

Log attribute keys sfp1 reads. They are the keys forge/pkg/observe writes on every ERROR record (error_inputs.go) and the procedure key its rpc interceptor logs; the test pins that they agree.

View Source
const (
	// IssueEventFilterSQL selects every record an issue is built from: server
	// errors, and server records of any level that carry a source. The
	// control plane's ingest view filters on it.
	IssueEventFilterSQL = `ResourceAttributes['rum.sessionId'] = '' AND (SeverityNumber >= 17 OR LogAttributes['issue.source'] != '')`
	// IssueSourceSQL is a record's issue source, "" for an error.
	IssueSourceSQL = `LogAttributes['issue.source']`
)

Issue sources. A record carrying issue.source (observe.IssueSource) is an issue event of that source at ANY level: an agent's bug report, say, which must be grouped and triaged like an error without counting as one. Errors carry no source (""). Every issue has exactly one source, and sfp1 fingerprints both kinds the same way (error.group is how a source pins its grouping).

View Source
const (
	MaxQueryLen = 1024

	DefaultLogLimit = 100
	MaxLogLimit     = 1000
)

Log query limits.

View Source
const (
	WarnMixedBounds      = "mixed_bounds"
	WarnMixedTemporality = "mixed_temporality"
	WarnStepRaised       = "step_raised"
	WarnStoredAsNKinds   = "metric_stored_as_n_kinds"
	// WarnMetricAbsent: a series' metric has no data in the scope and range,
	// so the series is empty. Absence is a fact about the window, not a
	// malformed panel: the panel's other series are still drawn.
	WarnMetricAbsent = "metric_absent"
	// WarnKindMismatch: the series declares a metric kind the metric is not
	// stored as; the stored kind is read.
	WarnKindMismatch = "metric_kind_mismatch"
)

The warning codes.

View Source
const (
	MaxMetricGroupBy   = 3
	MaxMetricSeries    = 50
	MaxPointsPerSeries = 1500
	TargetPoints       = 120
	// MaxMetricRows bounds the rows one metric statement may return; a
	// result that reaches it is reported as truncated.
	MaxMetricRows = MaxMetricSeries * MaxPointsPerSeries * 2
	// ListMetricsDefault and ListMetricsMax bound the metric picker.
	ListMetricsDefault = 200
	ListMetricsMax     = 500
)

Metric limits.

View Source
const (
	MaxRangeLogs       = 7 * 24 * time.Hour
	MaxRangeTraces     = 7 * 24 * time.Hour
	MaxRangeOverview   = 7 * 24 * time.Hour
	MaxRangeMetrics    = 30 * 24 * time.Hour
	MaxRangeProfiles   = 30 * 24 * time.Hour
	MaxRangeDashboards = 30 * 24 * time.Hour
)

Per-RPC maximum time ranges.

View Source
const (
	MaxScopeFilters = 20
	MaxScopeEnvs    = 500
)

Scope limits.

View Source
const (
	MaxPanelSeries     = 10
	MaxPanelPredicates = 20
	MaxPanelValues     = 50
	MaxPanelGroupBy    = 3
	MaxTableRows       = 200
	// DefaultBarRows is how many groups a bar panel with no limit draws.
	DefaultBarRows = dashspec.DefaultBarLimit
	// SparklinePoints is about how many points an ungrouped number panel's
	// sparkline has.
	SparklinePoints = 30
	// MaxHeatmapCells caps the cells one heatmap panel returns: TargetPoints
	// time buckets by about 25 octaves of value at four buckets an octave.
	MaxHeatmapCells = 12_000
)

Panel limits.

View Source
const (
	MaxRangeFunnel   = 90 * 24 * time.Hour // product_events retention
	MaxRangeIssues   = 30 * 24 * time.Hour // error_hourly retention
	MaxRangeSessions = 30 * 24 * time.Hour // rum_sessions retention (replay days, capped at 30)

	MaxPanelIssues   = 50
	MaxPanelSessions = 50

	MinFunnelSteps = 2
	MaxFunnelSteps = 10

	DefaultFunnelWindow = "1h"
	DefaultFunnelBy     = "session"
)

Browser panel limits.

View Source
const (
	// DefaultReplayPageBytes is a page's byte budget when the request names none.
	DefaultReplayPageBytes = 4 << 20
	// MaxReplayPageBytes is the largest budget a request may name.
	MaxReplayPageBytes = 16 << 20
	// MaxReplayEventBytes is the largest event a page will carry. The recorder
	// shrinks an event over 5 MB by replacing data: URIs, so a real event is
	// below it.
	MaxReplayEventBytes = 16 << 20
	// MaxReplayPageEvents bounds the events of one page: many tiny mutations
	// must not make a page of a million rows inside the byte budget.
	MaxReplayPageEvents = 10000
	// MaxRangeReplay is the longest window a replay read covers (rum_replay
	// retention is capped at 30 days).
	MaxRangeReplay = MaxRangeSessions

	// ReplayPendingGrace is how long after an incomplete event the session must
	// have kept recording before the event's missing chunk is considered lost.
	// The recorder batches for 5 s; this leaves room for retries and ingest lag.
	ReplayPendingGrace = 30 * time.Second
)

Replay limits.

View Source
const (
	DefaultSessionLimit = 25
	MaxSessionLimit     = 100

	DefaultTimelineLimit = 200
	MaxTimelineLimit     = 2000

	// MaxRangeTimeline is the longest window a timeline reads: its spans come
	// from otel_traces, whose retention is the trace one.
	MaxRangeTimeline = MaxRangeTraces
)

Session limits.

View Source
const (
	IssueSortLast   = "last"   // most recently seen first (default)
	IssueSortFirst  = "first"  // most recently first-seen first
	IssueSortEvents = "events" // most events first
)

Issue list sort orders.

View Source
const (
	DefaultIssueLimit = 25
	MaxIssueLimit     = 100
	MaxIssueEvents    = 100
)

Issue page sizes and the caps on an issue's side lists.

View Source
const (
	AffectedUsers     = "users"
	AffectedChats     = "chats"
	AffectedThreads   = "threads"
	AffectedWorkflows = "workflows"
	AffectedTraces    = "traces"
	AffectedSessions  = "sessions"
)

Who an issue touched is counted over the first of these that at least half of the issue's events name, widest-first: the people (user.id), else the chats, threads and workflows (a worker or reconciler logs on behalf of a run, not a request), else the traces. A unit most events do not name is not used: "1 user" over an issue of hundreds of failures that mostly name nobody would read as a handful of people. The unit that was counted travels with the count, so "3 affected" is never a bare number.

View Source
const (
	DefaultTraceLimit = 50
	MaxTraceLimit     = 500
	// MaxSpansPerTrace caps GetTrace; the statement fetches one more to detect it.
	MaxSpansPerTrace = 10000
)

Trace limits.

View Source
const (
	DefaultValuesLimit = 50
	MaxValuesLimit     = 100
)

Attribute-value lookup limits.

View Source
const (
	MetricCPUUsage       = "container.cpu.usage"                      // {cpu}
	MetricMemWorkingSet  = "container.memory.working_set"             // By
	MetricCPULimitUtil   = "k8s.container.cpu_limit_utilization"      // ratio
	MetricCPUReqUtil     = "k8s.container.cpu_request_utilization"    // ratio
	MetricMemLimitUtil   = "k8s.container.memory_limit_utilization"   // ratio
	MetricMemReqUtil     = "k8s.container.memory_request_utilization" // ratio
	MetricCPULimit       = "k8s.container.cpu_limit"                  // {cpu}
	MetricCPURequest     = "k8s.container.cpu_request"                // {cpu}
	MetricMemLimit       = "k8s.container.memory_limit"               // By
	MetricMemRequest     = "k8s.container.memory_request"             // By
	MetricRestarts       = "k8s.container.restarts"                   // {restart}
	MetricContainerReady = "k8s.container.ready"                      // 1 = passed its readiness probe, 0 = not

	MaxWorkloadRows      = 20000
	MaxWorkloadEvents    = 500
	MaxWorkloadChanges   = 200
	MaxWorkloadSeriesRow = 200 * MaxPointsPerSeries
)

The infra telemetry the node agent delivers (console v2 contract K2). kubeletstats supplies usage and the optional utilization ratios; k8s_cluster supplies restarts and the absolute requests and limits. All are gauges, with these resource attributes: reliant.env.id, k8s.namespace.name, k8s.pod.name, k8s.pod.uid, k8s.container.name, k8s.workload.name, container.image.name, container.image.tag; service.name = the workload name.

View Source
const (
	JobEventCreated   = "SuccessfulCreate"
	JobEventCompleted = "Completed"
)

The reasons a Job's own events carry (the Job controller's, upstream kubernetes): one run starts with SuccessfulCreate and ends with Completed or one of the failure reasons.

View Source
const CatalogWindow = 24 * time.Hour

CatalogWindow is the longest range the catalog and attribute scans read: a 30-day scan finds no new names and costs 30 times as much.

View Source
const DefaultIncidentWindow = 10 * time.Minute

DefaultIncidentWindow is how close in first-seen time issues of one cause must be to be one incident.

View Source
const EnvIDParam = "reliant_env_id"

EnvIDParam is the ClickHouse parameter an alert rule's expr names the id of the env it is declared for by: {reliant_env_id:String}, forge's EnvIDPlaceholder. An alert evaluator binds it to that env's id. A sql panel binds it to the ONE environment its scope is restricted to, so a rule's query runs as written in a dashboard, the metrics explorer and `forge obs query`. A scope over several environments, every environment, or none leaves it unbound and Compile refuses a statement that uses it: it names one environment, and guessing which would read the wrong data.

View Source
const HeatmapZeroBucket = math.MinInt32

Heatmap value buckets are quarter powers of two: bucket i holds values in [2^(i/4), 2^((i+1)/4)). HeatmapZeroBucket holds zero and negative values.

View Source
const LocalDatabase = "default"

LocalDatabase is the database a local ClickStack's OTel exporter writes to.

View Source
const LocalIssuePrefix = "sfp1-"

LocalIssuePrefix starts the id of a query-time issue: "sfp1-<fingerprint>".

View Source
const MaxAnomalyServices = MaxMetricSeries

MaxAnomalyServices bounds the services one evaluation judges: the series cap of a line panel.

View Source
const MaxDeclaredKeys = 200

MaxDeclaredKeys bounds the keys one metric may declare.

View Source
const MaxExemplars = 20

MaxExemplars bounds one ListMetricExemplars answer and one stored-trace probe.

View Source
const MaxIncidentCandidates = 2000

MaxIncidentCandidates bounds the issues one incident search reads. A range with more distinct failing issues than this groups the newest of them.

View Source
const MaxIncidentWindow = time.Hour

MaxIncidentWindow bounds a requested window.

View Source
const MaxJobs = 200

MaxJobs bounds the Jobs one request returns (one more is read, to detect the cut).

View Source
const MaxREDGroups = 50

MaxREDGroups bounds the groups CompileREDGroups returns.

View Source
const MaxSQLRows = 10_000

MaxSQLRows is the most rows a sql panel may return. Like an alert rule's cap it throws: a longer result is an error, never a silently cut one.

View Source
const MaxStacks = 5000

MaxStacks caps the distinct stacks one flamegraph returns (the heaviest are kept); the statement fetches one more to detect the cut.

View Source
const MinReadinessStepSec = 60

MinReadinessStepSec is the finest Readiness bucket: the cluster collector delivers k8s.container.ready once a minute, so a finer bucket would be mostly empty.

View Source
const ServerCulpritSQL = `if(LogAttributes['code.file.path'] != '', concat(LogAttributes['code.file.path'], ' in ', ` + sqlFunction + `), ` + sqlFunction + `)`

ServerCulpritSQL is where an issue was logged: "file in function", or the function alone (ScopeName for a record with no source location).

View Source
const ServerErrorFilterSQL = `SeverityNumber >= 17 AND ResourceAttributes['rum.sessionId'] = ''`

ServerErrorFilterSQL selects server error events from otel_logs: ERROR or above (SeverityNumber 17+), and not from a browser session.

View Source
const ServerFingerprintCompatibleFrom = 1

ServerFingerprintCompatibleFrom is the oldest fp_version whose fingerprints still name the same issues under ServerFingerprintVersion (see the versioning note above): an issue stamped anywhere in [ServerFingerprintCompatibleFrom, ServerFingerprintVersion] is found by a lookup of its fingerprint.

View Source
const ServerFingerprintVersion = 2

ServerFingerprintVersion is the fp_version of the algorithm in force (sfp2): stamped on every newly fingerprinted row, and the version an issue is keyed by.

View Source
const StreamTypeAttr = "forge.rpc.stream_type"

StreamTypeAttr is the span attribute serverkit stamps on the server span of a streaming RPC: "server", "client" or "bidi".

Variables

View Source
var (

	// ServerFingerprintSQL is the server fingerprint (sfp1, refined to sfp2 for
	// the records RefinedByErrorClass selects) as a ClickHouse String expression
	// over an otel_logs row (ServiceName, ScopeName, Body, LogAttributes).
	// Evaluate it only on rows ServerErrorFilterSQL selects.
	ServerFingerprintSQL = `lower(hex(substring(SHA256(concat(` +
		`if(` + sqlRefined + `, 'sfp2', 'sfp1'), '\x1f', ServiceName, '\x1f', ` +
		`if(LogAttributes['error.group'] != '', LogAttributes['error.group'], ` + sqlGroupKey + `), ` +
		`if(` + sqlRefined + `, concat('\x1f', ` + sqlErrClass + `), '')` +
		`)), 1, 8)))`

	// ServerTitleSQL is an issue's title: "exception.type: norm(Body)", or
	// norm(Body) alone when the record carries no error. A record sfp2 refines
	// has no exception.type, and its error class stands in for it, so the issues
	// one message template splits into read apart.
	ServerTitleSQL = `if(LogAttributes['exception.type'] != '', concat(LogAttributes['exception.type'], ': ', ` + sqlBodyNorm + `), ` +
		`if(` + sqlRefined + `, concat(` + sqlErrClass + `, ': ', ` + sqlBodyNorm + `), ` + sqlBodyNorm + `))`

	// ServerCauseSQL is a record's incident cause: the ClickHouse spelling of
	// ServerCause, over the `error` attribute, else the SDK's exception.message.
	ServerCauseSQL = errCauseSQL(`if(LogAttributes['error'] != '', LogAttributes['error'], LogAttributes['exception.message'])`)

	// ServerEventCauseSQL is ServerCauseSQL over a server_error_events row, which
	// keeps the two attributes in its Attributes map (the control plane's ingest
	// view, tenant schema v9). A row stored before that keeps neither and has no
	// cause: an issue with no cause belongs to no incident.
	ServerEventCauseSQL = errCauseSQL(`if(Attributes['error'] != '', Attributes['error'], Attributes['exception.message'])`)
)
View Source
var AffectedKeys = []struct{ Unit, Attribute string }{
	{AffectedUsers, "user.id"},
	{AffectedChats, "reliant.chat.id"},
	{AffectedThreads, "thread.id"},
	{AffectedWorkflows, "workflow.id"},
}

AffectedKeys are the units an event can name, in preference order, and the log attribute each is read from.

View Source
var ErrInvalid = errors.New("invalid request")

ErrInvalid marks a request the caller must fix (InvalidArgument).

View Source
var ErrMetricNotFound = errors.New("metric not found")

ErrMetricNotFound is ParseLookup's refusal when no table holds the metric in the lookup window. It is not ErrInvalid: the request was well-formed.

View Source
var ErrSessionNotFound = errors.New("session not found")

ErrSessionNotFound is GetSession's answer for a session the scope cannot see (no such session, or one in another environment or service).

View Source
var JobFailureReasons = []string{"BackoffLimitExceeded", "DeadlineExceeded", "PodFailurePolicy"}

JobFailureReasons are the Job events that end a run in failure.

View Source
var ReplayColumns = []string{"Timestamp", "EventIndex", "Chunk", "TotalChunks", "Body", "more"}

ReplayColumns names the select list of Compile's statement, in order, for the code that scans it: one row per stored chunk of a selected event.

SeriesMetrics are the gauges GetWorkload charts (restarts are separate).

View Source
var ServerIssueEventColumns = []string{"timestamp", "trace_id", "span_id", "service", "version", "body", "attributes"}

ServerIssueEventColumns names the select list of ServerIssueEvents, in order.

View Source
var ServerIssueIncidentColumns = []string{"fingerprint", "service", "cause", "first_seen"}

ServerIssueIncidentColumns names the select list of ServerIssueIncidents, in order.

View Source
var ServerIssuesColumns = []string{
	"fingerprint", "service", "title", "culprit", "exception_type", "events", "affected",
	"first_seen", "last_seen", "latest_trace_id", "last_version", "first_version", "source", "affected_unit",
}

ServerIssuesColumns names the select list, in order, for the code that scans it.

UsageMetrics are the gauges ListWorkloads summarizes per container.

Functions

func AffectedSQL

func AffectedSQL() string

AffectedSQL selects the affected count of a group of error records of otel_logs, and AffectedUnitSQL the unit it counted.

func AffectedUnitSQL

func AffectedUnitSQL() string

AffectedUnitSQL is AffectedSQL's unit column.

func Allowed

func Allowed(key string) bool

Allowed reports whether key is on the allowlist.

func AllowedKeys

func AllowedKeys() []string

AllowedKeys lists the allowlisted keys, sorted.

func CheckRange

func CheckRange(start, end time.Time, max time.Duration) error

CheckRange validates the half-open range against a per-RPC maximum.

func CheckSessionID

func CheckSessionID(id string) error

CheckSessionID refuses anything that is not the shape of an rum.sessionId (a 32-digit hex id from the SDK; the pattern is wider so a custom id survives).

func ErrInvalidf

func ErrInvalidf(format string, a ...any) error

ErrInvalidf builds an ErrInvalid error for callers outside this package.

func ErrorClass

func ErrorClass(err string) string

ErrorClass is the Go reference for ErrorClassSQL: what kind of failure the string in a record's `error` attribute names, stable across the ids, values and timings its message carries. A SQLSTATE code (with the Go type that prefixes it, when there is one) beats a bare type name, which beats the message run through NormalizeMessage.

func FunnelWindow

func FunnelWindow(w string) (time.Duration, error)

FunnelWindow resolves a funnel window to a duration.

func HeatmapBounds

func HeatmapBounds(i int32) (low, high float64)

HeatmapBounds is the value range of heatmap bucket i.

func HistogramQuantile

func HistogramQuantile(bounds []float64, counts []float64, q float64) (float64, bool)

HistogramQuantile estimates the q-th quantile from explicit bucket bounds and per-bucket counts (len(counts) == len(bounds)+1), interpolating linearly inside the bucket the rank falls in. The first bucket's lower edge is 0 when its upper bound is positive; the open top bucket reports the last bound. ok is false when there are no observations.

func IncidentKey

func IncidentKey(cause string, anchor time.Time) string

IncidentKey is the key issues of one incident share: stable for a given cause and anchor, and short enough to read aloud.

func IncidentOf

func IncidentOf(groups []IncidentGroup) map[string]IncidentGroup

IncidentOf indexes groups by member id.

func IsBrowserKind

func IsBrowserKind(kind string) bool

IsBrowserKind reports whether kind is read from the browser tables.

func IsBuiltinMetric

func IsBuiltinMetric(name string) bool

IsBuiltinMetric reports whether name is a runtime or forge built-in metric (the rpc., go. and forge. prefixes) as opposed to an application's own. The console mirrors this rule in TypeScript; this is the one Go source to copy.

func IsIdentifier

func IsIdentifier(s string) bool

IsIdentifier reports whether s may be used unquoted as a ClickHouse database name.

func IsNotOnLogs

func IsNotOnLogs(err error) bool

IsNotOnLogs reports whether err is the not-applicable-to-logs marker.

func LocalIssueID

func LocalIssueID(fingerprint string) string

LocalIssueID is the id a query-time issue carries.

func MergeHistograms

func MergeHistograms(bounds, counts [][]float64) (outBounds, outCounts []float64, mixed bool)

MergeHistograms folds several (bounds, counts) of one bucket into one. When every source has the same bounds the counts are summed. Otherwise the result is on the union of bounds, each source bucket's count spread over the union intervals it covers in proportion to their width (the assumption HistogramQuantile makes inside a bucket); an open top bucket lands in the union's open top bucket. mixed reports that the bounds differed.

func MessageNormSQL

func MessageNormSQL(expr string) string

MessageNormSQL is the ClickHouse expression that normalizes the String expression expr the way NormalizeMessage does. expr is spliced verbatim: pass a column or a constant expression, never caller input.

func NeedsDeclared

func NeedsDeclared(keys ...string) bool

NeedsDeclared reports whether any key is outside the allowlist, so a metric query must look up the metric's declared attribute keys before compiling.

func NormalizeMessage

func NormalizeMessage(s string) string

NormalizeMessage is the Go reference for MessageNormSQL: the first line of s with URLs, UUIDs, quoted strings, long hex runs and integers masked, cut to 200 bytes.

func NormalizeTraceID

func NormalizeTraceID(id string) (string, error)

NormalizeTraceID lowercases a hex trace id, or returns ErrInvalid.

func OperatorSource

func OperatorSource(source string) bool

OperatorSource reports whether a reliant.source value marks Reliant operator data (the platform's own telemetry, present only in the operator's own org).

func PanelMaxRange

func PanelMaxRange(signal string) (time.Duration, error)

PanelMaxRange is the maximum range for a panel of the given signal, or ErrInvalid for an unknown signal.

func ParseLocalIssueID

func ParseLocalIssueID(id string) (string, error)

ParseLocalIssueID extracts the fingerprint of a query-time issue id.

func ParseStoredTraceIDs

func ParseStoredTraceIDs(rows Rows) (map[string]bool, error)

ParseStoredTraceIDs reads ExemplarsStored's rows into a set of lowercase ids.

func ServerCause

func ServerCause(r ServerErrorRecord) string

ServerCause is the Go reference for ServerCauseSQL: what an incident shares across the callers it breaks, read from the `error` attribute, else the SDK's exception.message. The SQLSTATE the text carries; else the target a dial was refused at; else its error class. "" when the record carries no error text, which no incident can be built from.

func ServerCulprit

func ServerCulprit(r ServerErrorRecord) string

ServerCulprit is the Go reference for ServerCulpritSQL.

func ServerFingerprint

func ServerFingerprint(r ServerErrorRecord) string

ServerFingerprint is the Go reference for ServerFingerprintSQL.

func ServerTitle

func ServerTitle(r ServerErrorRecord) string

ServerTitle is the Go reference for ServerTitleSQL.

func SortByTotal

func SortByTotal(keys []string, total map[string]float64)

SortByTotal orders series keys by descending total, ties by key, for the series cap.

func SparklineStep

func SparklineStep(rng time.Duration) int64

SparklineStep is the bucket width of an ungrouped number panel's sparkline over a range.

func StepFor

func StepFor(d time.Duration, wantPoints int, minSec int64) int64

StepFor returns the bucket width for a range: about 120 points, snapped to a round number of seconds, never below minSec.

func StepForMetric

func StepForMetric(rng time.Duration, requested int32, intervalSec int64) (step int64, raised bool)

StepForMetric picks the bucket width. The floor is the larger of the width that keeps a series within MaxPointsPerSeries and the export interval, so a bucket never holds less than one sample. The caller's step is honoured at or above the floor; below it, the step is raised and raised is true (the response says so). Unset, it is about TargetPoints per series.

func SuggestedBreakdowns

func SuggestedBreakdowns(stats []AttributeStat) []string

SuggestedBreakdowns lists the keys of ParseAttributes' result worth grouping by, best first: enumerated keys, then keys with 2 to 50 distinct values, leaving out any that could exceed the series cap.

func ValidFingerprint

func ValidFingerprint(s string) bool

ValidFingerprint reports whether s is the shape of an sfp1 fingerprint: 16 lowercase hex digits.

func ValidIssueSource

func ValidIssueSource(s string) bool

ValidIssueSource reports whether s can name an issue source: 1-64 of lowercase letters, digits, '_', '.' and '-', starting with a letter or digit.

func ValueUnit

func ValueUnit(info MetricInfo, agg Agg) string

ValueUnit is the unit of an aggregation's values: a rate is per second, a count is a count, everything else keeps the metric's unit.

func WarningCode

func WarningCode(w string) string

WarningCode is the machine-readable part of a warning.

Types

type Agg

type Agg int

Agg is a metric aggregation.

const (
	AggDefault Agg = iota
	AggAvg
	AggSum
	AggMin
	AggMax
	AggCount
	AggIncrease
	AggRate
	AggP50
	AggP95
	AggP99
	AggQuantile
	AggLast
)

The aggregations. AggDefault resolves per kind (see ResolveAgg). AggQuantile carries MetricQuery.Quantile (any level in (0, 1]); AggP50, AggP95 and AggP99 are aliases of it for those levels.

func PanelSeriesAgg

func PanelSeriesAgg(ps PanelSeries) (Agg, float64, error)

PanelSeriesAgg maps a metrics series' dashspec aggregation onto an Agg and its quantile level.

func ResolveAgg

func ResolveAgg(info MetricInfo, a Agg) (Agg, error)

ResolveAgg applies the per-kind default and refuses the combinations that have no meaning (the table in DESIGN §3.2):

monotonic sum   default rate; sum IS increase; count and quantile refused
up-down sum     default sum (of last values); rate, increase, count and quantile refused
gauge           default avg; rate, increase and quantile refused
histogram       default p95; every aggregation but last
exp histogram   as a histogram
summary         default avg; avg, sum, count, increase and rate over its Count and
                Sum; quantile only for a level every series stores (checked against
                the data); min, max and last refused

A monotonic sum resolves AggSum to AggIncrease, so one word never means two things.

type Anomaly

type Anomaly struct {
	Service string
	// Signal is an AnomalySignal* value.
	Signal string
	// Observed, Baseline (the median) and Spread (the MAD) are in Unit:
	// "fraction" for the error rate, "s" for p95, "requests" for throughput.
	Observed, Baseline, Spread float64
	Unit                       string
	// Limit is the nearest value of the signal that would not have fired,
	// on the side the observation is on.
	Limit float64
	// Up is whether the signal rose; throughput can also fall.
	Up bool
	// Requests is the window's request count.
	Requests float64
	// BaselineSamples is how many slots the baseline had.
	BaselineSamples int
	HistoryDays     int
}

Anomaly is one signal of one service that left its seasonal baseline.

func JudgeService

func JudgeService(in ServiceREDInput, p AnomalyParams) []Anomaly

JudgeService returns the signals of one service that left their baseline, in the order error_rate, p95, throughput; nil when the service is young, has no baseline, or everything is inside it.

func (Anomaly) Why

func (a Anomaly) Why(env string, q AnomalyQuery) alerting.Why

Why is the shared why-fired payload of the finding, filled with what the detector knows: observed, baseline, spread, the method, the window, the limits and the forge obs commands that lead to the evidence. The releases, activity and issues in the window are the caller's (they live in its registries): it fills Context before posting. env names the environment in the next commands.

type AnomalyParams

type AnomalyParams struct {
	// Window is the span the observed value covers, and the width of one
	// baseline slot, so a slot is directly comparable to the window.
	Window time.Duration
	// Delay is how far the window's end lags now: the metric exporters'
	// interval, so a sample that has not landed does not read as a drop.
	Delay time.Duration
	// Weeks is how many previous weeks make the baseline (1 to 4: metrics are
	// kept 30 days).
	Weeks int
	// K is the multiple of the baseline's MAD the observed value must leave.
	K float64

	// MinRequests: a window with fewer requests never fires an error-rate or
	// latency finding, and a baseline whose median slot has fewer never fires
	// a throughput one.
	MinRequests float64
	// MinErrors is the fewest server errors in the window that fire an
	// error-rate finding, and MinErrorDelta the least rise in the error ratio
	// (0..1) that does.
	MinErrors     float64
	MinErrorDelta float64
	// P95Ratio and MinLatencyDelta (seconds): the p95 must be at least this
	// multiple of the baseline's AND this much above it.
	P95Ratio        float64
	MinLatencyDelta float64
	// ThroughputRatio: requests must be at least this multiple of the
	// baseline's, or at most its reciprocal.
	ThroughputRatio float64

	// MinHistory is how much history a service must have to be judged.
	MinHistory time.Duration
	// MinBaselineSamples is the fewest baseline slots a signal needs.
	MinBaselineSamples int
}

AnomalyParams are the detector's limits. The zero value is not usable: start from DefaultAnomalyParams.

func DefaultAnomalyParams

func DefaultAnomalyParams() AnomalyParams

DefaultAnomalyParams are the platform defaults: a 15 minute window against the same hour of the previous 4 weeks, 4 MADs, at least 50 requests, a 1 point (and 5 errors) error-rate rise or a 1.5x (and 50 ms) p95 rise, after 7 days of history.

func (AnomalyParams) Validate

func (p AnomalyParams) Validate() error

Validate refuses limits the detector cannot work with.

type AnomalyQuery

type AnomalyQuery struct {
	Scope  Scope
	Params AnomalyParams
	// Now is the evaluation time; zero is time.Now().
	Now time.Time
}

AnomalyQuery is one evaluation: which services (the scope's), when, and with which limits. Scope.Start and End are ignored: the windows come from Now and the params.

func (AnomalyQuery) FirstSeenStmt

func (q AnomalyQuery) FirstSeenStmt() (Stmt, error)

FirstSeenStmt reads each service's first rpc.server.calls sample within the weeks the baseline reaches: columns service (String) and first (DateTime). A service whose first sample is the lookback's start has at least that much history.

type AnomalyResult

type AnomalyResult struct {
	// Inputs are the services judged, with the samples they were judged on.
	Inputs []ServiceREDInput
	// Anomalies are the findings, by service then signal.
	Anomalies []Anomaly
	// Window is the observed window's start and end.
	WindowStart, WindowEnd time.Time
	// Truncated is set when more services existed than MaxAnomalyServices:
	// the busiest were judged.
	Truncated bool
	Warnings  []string
}

AnomalyResult is what EvalAnomalies returns.

func EvalAnomalies

func EvalAnomalies(ctx context.Context, run Runner, q AnomalyQuery) (AnomalyResult, error)

EvalAnomalies runs the observed window, one baseline window per past week and the first-seen lookup through run, and judges every service the lookup found. A service with no first sample in the lookback has no metrics and is not judged.

type AttrFilter

type AttrFilter struct {
	Key    string
	Op     string // "" | "eq" | "neq" | "in" | "contains" | "exists"
	Value  string
	Values []string
}

AttrFilter is a structured attribute predicate (K1 filters[]).

type AttrValues

type AttrValues struct {
	Scope  Scope
	Key    string
	Prefix string
	// Signal is where to read; zero picks the key's natural home: span keys
	// read traces, event keys logs, a metric's attribute metrics, and resource
	// keys traces AND metrics (logs only when asked, they are the largest).
	Signal Signal
	// Metric names the metric whose declared attribute Key is; required for a
	// key that is not on the allowlist.
	Metric string
	Limit  int
}

AttrValues is a ListAttributeValues request: the values one allowlisted key takes in the scope, most frequent first, for filter autocomplete.

func (AttrValues) Compile

func (q AttrValues) Compile() (Stmt, error)

Compile builds the value statement. Columns: v (String), c (UInt64); it fetches Limit+1 rows so the caller can tell the list was cut.

Scope filters on the SAME key are dropped so the list is not narrowed to the current selection of that very control.

type AttributeStat

type AttributeStat struct {
	Key      string
	Resource bool
	// DistinctEstimate is the estimated number of values over the range.
	DistinctEstimate int64
	// Enumerated is the declared closed value set; empty when open.
	Enumerated []string
	// HighCardinality: grouping by the key (with the rest of the panel's
	// group-by) could exceed MaxMetricSeries.
	HighCardinality bool
}

AttributeStat is one attribute key of a metric.

func ParseAttributes

func ParseAttributes(rows Rows, declared *metrics.Spec, otherGroupCard int64) ([]AttributeStat, error)

ParseAttributes reads MetricAttributes' rows into at most MaxDeclaredKeys stats, the most useful breakdown first: enumerated keys, then keys with 2 to 50 distinct values, then keys with one value, then the rest. declared (may be nil) supplies the closed value sets. otherGroupCard is the product of the distinct counts of the panel's other group-by keys (0 or 1 for none).

type CatalogEntry

type CatalogEntry struct {
	Name        string
	Kind        CatalogKind
	Unit        string
	Description string
	// Monotonic: a sum that is a counter, not an up-down counter.
	Monotonic   bool
	Temporality Temporality
	// Services are up to 10 service.name values that emit it; ServiceCount is
	// how many do.
	Services     []string
	ServiceCount int
	// LastSeen is the newest data point in the catalog range; zero for a
	// declared metric never seen.
	LastSeen       time.Time
	SeriesEstimate int64
	// Queryable is false for a kind the evaluator cannot run.
	Queryable bool
	Builtin   bool
	// Declared: the release's metrics schema lists it. Seen: it has data in
	// the range.
	Declared bool
	Seen     bool
	// Stability is the declaration's "stable", "experimental" or
	// "deprecated"; empty when not declared.
	Stability          string
	DeclaredAttributes []metrics.Attribute
}

CatalogEntry is one metric as the explorer lists it.

func MergeDeclared

func MergeDeclared(observed []CatalogEntry, declared []metrics.Schema) []CatalogEntry

MergeDeclared joins the catalog with the release's declared metrics schemas. It is pure and never modifies its inputs.

  • declared and seen: the declaration's description and stability, its attributes and Declared = true; the observed unit, kind and numbers stay, because they describe the data actually stored;
  • declared, not seen: listed with Seen = false, so a typo'd or never-called metric is visible;
  • seen, not declared: unchanged (infrastructure and third-party metrics);
  • deprecated declarations are kept and flagged through Stability.

A metric declared by several schemas takes the first. The result is ordered by name, then kind.

func ParseCatalog

func ParseCatalog(rows Rows, limit int) (entries []CatalogEntry, truncated bool, err error)

ParseCatalog reads MetricCatalog's rows. limit is the request's Limit: a row beyond it reports truncated instead of being returned. The slice is never nil. The caller closes rows.

type CatalogKind

type CatalogKind int

CatalogKind is the table a metric is stored in. 1-3 equal MetricType's, and all five equal the control plane's MetricType wire enum.

const (
	CatalogGauge        CatalogKind = 1
	CatalogSum          CatalogKind = 2
	CatalogHistogram    CatalogKind = 3
	CatalogExpHistogram CatalogKind = 4
	CatalogSummary      CatalogKind = 5
)

The kinds.

func (CatalogKind) Queryable

func (k CatalogKind) Queryable() bool

Queryable reports whether the evaluator can run the kind: all five. A summary supports a narrower set of aggregations (see ResolveAgg); asking for another is refused with the reason.

func (CatalogKind) String

func (k CatalogKind) String() string

type DeclaredKeys

type DeclaredKeys struct {
	Scope Scope
	Name  string
	Type  MetricType
}

DeclaredKeys lists the attribute keys a metric's own data points carry, the extension to the key allowlist for metric filters and group-bys. It is scoped by environment and service only (the filters are what it enables). Column: key.

func (DeclaredKeys) Compile

func (q DeclaredKeys) Compile() (Stmt, error)

Compile builds the declared-keys statement.

type Env

type Env struct{ ID, Name string }

Env is a Reliant environment the caller may see.

type EvalOptions

type EvalOptions struct {
	// StepSeconds is the requested bucket width; 0 picks one. It is raised to
	// the metrics' export interval, with a step_raised warning.
	StepSeconds int32
	// Explain compiles every statement and runs nothing (the Runner is not
	// called). The kind of each metric is taken from the series' metricType.
	Explain bool
	// Now decides which buckets are partial; zero is time.Now().
	Now time.Time
	// MaxSeries lowers the series cap (default MaxMetricSeries).
	MaxSeries int
	// Filters are ANDed onto every series' predicates.
	Filters []PanelPredicate
}

EvalOptions are the request-level knobs of EvalMetricPanel.

type Exemplar

type Exemplar struct {
	Time       time.Time
	Value      float64
	TraceID    string
	SpanID     string
	Attributes map[string]string
	// Stored: the trace is in the trace store. false means it was sampled out
	// and the trace view would be empty.
	Stored bool
}

Exemplar is one sampled observation of a metric.

func MarkStored

func MarkStored(in []Exemplar, stored map[string]bool) []Exemplar

MarkStored sets Stored from the ids ParseStoredTraceIDs found and returns a copy with the stored exemplars first, each group keeping its order (newest first).

func ParseExemplars

func ParseExemplars(rows Rows) ([]Exemplar, error)

ParseExemplars reads MetricExemplars' rows. The caller closes rows.

type ExemplarsStored

type ExemplarsStored struct {
	Scope    Scope
	TraceIDs []string
	// From and To are the exemplar window; both zero means the scope's range.
	// The probe reads one hour either side.
	From, To time.Time
	// UseLookupTable reads the otel_traces_trace_id_ts lookup table (a hosted
	// backend has it) instead of otel_traces itself (the local ClickStack).
	UseLookupTable bool
}

ExemplarsStored is the probe for which of a set of trace ids are in the trace store. It is not narrowed to the scope's environments: the ids come from exemplars the caller was already allowed to read, and it reveals presence only.

func (ExemplarsStored) Compile

func (q ExemplarsStored) Compile() (Stmt, error)

Compile builds the probe. Column: trace_id (String), each stored id once.

type Flamegraph

type Flamegraph struct {
	Scope      Scope
	Service    string
	SampleType string
	TraceID    string
}

Flamegraph is a GetFlamegraph request.

func (Flamegraph) Compile

func (q Flamegraph) Compile() (Stmt, error)

Compile builds the stack statement from forge's profilequery (the SQL forge itself reads profiles back with), pointed at the scope's database and bounded to MaxStacks. Columns: frames (root to leaf), unit, value.

profilequery's statements name the tables `default.*` (LocalDatabase) and end in FORMAT JSONEachRow, which is a client-side directive; both are rewritten here, and a statement that still names `default.` afterwards is refused rather than run — unless the scope is Local, where `default` is the database.

func (Flamegraph) ProfileRange

func (q Flamegraph) ProfileRange() (time.Time, time.Time)

ProfileRange is the half-open range a profile covers, for the Profile value.

func (Flamegraph) ProfileTypes

func (q Flamegraph) ProfileTypes() (Stmt, error)

ProfileTypes lists the sample types a service has in the range. Columns: SampleType.

type IncidentCandidate

type IncidentCandidate struct {
	// ID is the caller's handle for the issue; it comes back in IncidentGroup.
	ID        string
	Cause     string
	FirstSeen time.Time
}

IncidentCandidate is one issue's incident inputs: the issue's identity, the cause of its newest event and its first event in the range.

type IncidentGroup

type IncidentGroup struct {
	Key   string
	Cause string
	// First is the first-seen time of the incident's earliest issue.
	First time.Time
	// IDs are the members, earliest first seen first.
	IDs []string
}

IncidentGroup is one incident: two or more issues of one cause first seen within the window.

func GroupIncidents

func GroupIncidents(cands []IncidentCandidate, window time.Duration) []IncidentGroup

GroupIncidents clusters candidates into incidents. An issue with no cause, or alone in its window, belongs to no incident and is not returned. The result is ordered by the incident's first-seen time, then its key.

func (IncidentGroup) RelatedCount

func (g IncidentGroup) RelatedCount() int

RelatedCount is how many other issues share g with one of its members.

type LogCursor

type LogCursor struct {
	TsNano int64  `json:"t"`
	Hash   uint64 `json:"h,string"`
	Finger string `json:"f"`
}

LogCursor is the keyset position of the last row of a page.

func DecodeLogCursor

func DecodeLogCursor(tok string) (*LogCursor, error)

DecodeLogCursor parses a page token.

type LogQuery

type LogQuery struct {
	Scope      Scope
	Query      string
	Severities []string
	Oldest     bool
	Limit      int // rows to return; the statement fetches one more
	Cursor     *LogCursor
}

LogQuery is a SearchLogs request.

func (LogQuery) Compile

func (q LogQuery) Compile() (Stmt, error)

Compile builds the SearchLogs statement. Columns, in order: Timestamp, SeverityText, SeverityNumber, ServiceName, Body, TraceId, SpanId, LogAttributes, ResourceAttributes, row hash.

func (LogQuery) Fingerprint

func (q LogQuery) Fingerprint() string

Fingerprint identifies the filters a cursor belongs to. The time range is deliberately not part of it: the keyset position anchors the page, so a client that slides a relative range between pages still pages correctly.

func (LogQuery) NextCursor

func (q LogQuery) NextCursor(ts time.Time, hash uint64) string

NextCursor builds the page token from the last returned row.

type MetricAttributes

type MetricAttributes struct {
	Scope Scope
	Name  string
	Kind  CatalogKind
}

MetricAttributes is a GetMetric request's attribute scan: every attribute key a metric's data points and its resources carry, with an estimate of how many values each takes.

func (MetricAttributes) Compile

func (q MetricAttributes) Compile() (Stmt, error)

Compile builds the attribute statement over the last CatalogWindow of the scope's range. Columns: key (String), resource (UInt8: 1 for a resource attribute), distinct (UInt64, uniqCombined). Resource keys come back only when they vary. At most MaxDeclaredKeys rows: data-point keys first, then the most varied.

type MetricCatalog

type MetricCatalog struct {
	Scope        Scope
	NameContains string
	Limit        int
}

MetricCatalog is a ListMetrics request: every metric with data in the last 24 hours of the scope's range, one entry per (name, kind).

func (MetricCatalog) Compile

func (q MetricCatalog) Compile() (Stmt, error)

Compile builds the catalog statement over all five metric tables. Only RESOURCE-level scope filters narrow it: any other key is valid for one metric only, and is checked when that metric is queried. Columns: name, kind (UInt8), unit, description, mono (UInt8), temporality (Int32), services (Array(String), at most 10), service_count (UInt64), last_seen (DateTime), series (UInt64). It fetches Limit+1 rows so ParseCatalog can tell the list was cut.

type MetricExemplars

type MetricExemplars struct {
	Scope Scope
	Name  string
	Kind  CatalogKind
	// Where narrows to series matching the predicates (a panel's vocabulary).
	Where []PanelPredicate
	// Group narrows to one series group: group-by key to value.
	Group map[string]string
	// From and To bound the exemplars' own timestamps (the bucket the user
	// clicked), inside the scope's range; both zero means the scope's range.
	From, To time.Time
	// MinValue keeps only exemplars at or above it (the tail of a latency
	// histogram).
	MinValue *float64
	// Limit is at most MaxExemplars; zero means MaxExemplars.
	Limit int
}

MetricExemplars is a ListMetricExemplars request: the sampled observations recorded with a metric's data points, newest first.

func (MetricExemplars) Compile

func (q MetricExemplars) Compile() (Stmt, error)

Compile builds the exemplar statement. Columns: time (DateTime), value (Float64), trace_id (String, lowercase), span_id (String), attributes (Map(String, String)). Newest first, one row per (trace, span), only exemplars that carry a trace id.

type MetricInfo

type MetricInfo struct {
	Type MetricType
	Unit string
	// Monotonic is a Sum's IsMonotonic. An up-down counter is not.
	Monotonic        bool
	Temporality      Temporality
	MixedTemporality bool
	// Series is the number of distinct series in the lookup window; it
	// decides whether the series cap is applied in SQL.
	Series int64
	// IntervalSeconds estimates the export interval; it is the floor of the
	// bucket width.
	IntervalSeconds int64
	// StoredKinds is how many metric kinds hold this name (normally 1).
	StoredKinds int
}

MetricInfo is what one lookup statement learns about a metric: which table holds it and the facts the semantics depend on (see MetricLookup).

func ParseLookup

func ParseLookup(rows Rows, want MetricType) (MetricInfo, error)

ParseLookup reads MetricLookup's rows into the info of the requested kind, or of the first kind stored when want is MetricUnknown or is not stored. It closes rows. No rows is ErrMetricNotFound.

type MetricLookup

type MetricLookup struct {
	Scope Scope
	Name  string
}

MetricLookup finds which tables hold a metric and what is known about it. Columns: kind, unit, mono, temporality, mixed, series, points, first, last, interval. ParseLookup turns the rows into a MetricInfo.

func (MetricLookup) Compile

func (q MetricLookup) Compile() (Stmt, error)

Compile builds the lookup statement. interval is the median over series of the mean gap between a series' points: unlike (last-first)/points it does not grow with series churn over a long range.

type MetricPanelResult

type MetricPanelResult struct {
	Meta   []SeriesMeta
	Series []MetricSeriesResult
	// SeriesTotal is how many groups existed before the series cap.
	SeriesTotal int
	Truncated   bool
	StepSeconds int64
	// Warnings are "<code>" or "<code>: <detail>"; see WarningCode.
	Warnings []string
	// Absent are the metrics of panel series that have no data in the scope
	// and range (each with a metric_absent warning). Their series are empty.
	Absent     []string
	Statements []NamedStmt
}

MetricPanelResult is what EvalMetricPanel returns.

func EvalMetricPanel

func EvalMetricPanel(ctx context.Context, run Runner, scope Scope, p Panel, o EvalOptions) (MetricPanelResult, error)

EvalMetricPanel runs a metrics panel end to end: look every series' metric up, resolve its aggregation, compile and run its statements through run, fold the rows into series, and combine them (formula or ratio). It is the one implementation of metric semantics: the control plane's QueryPanel and forge's local backend both call it. The panel's kind decides the shape: a line is a time series; a number is the range reduction plus a sparkline; a table or bar is the top groups by their range reduction.

A series whose metric has no data in the scope and range is empty, with a metric_absent warning and its name in Absent, never an error: a counter that only exists once something fails, or a metric outside the selected environments, is "no data in this window", and the panel's other series still draw. Its declared aggregation is still validated against its declared kind.

type MetricPoint

type MetricPoint struct {
	Time          time.Time
	Value         float64
	Null, Partial bool
}

MetricPoint is one bucket of a series. Null is a gap (a formula whose operand is absent); Partial says the bucket was still filling when it was read.

type MetricQuery

type MetricQuery struct {
	Scope Scope
	Name  string
	// Info is what MetricLookup found (see ParseLookup); the kind selects the
	// table and the statement shape, and Info.Series decides the SQL series cap.
	Info MetricInfo
	// Agg is resolved against Info (ResolveAgg), so AggDefault is allowed.
	Agg Agg
	// Quantile is the level for AggQuantile, in (0, 1].
	Quantile float64
	// GroupBy are allowlisted keys, the metric's declared attributes
	// (Scope.Declared), or panel-vocabulary keys (metric.attr.<k>,
	// resource.attr.<k>).
	GroupBy []string
	// Where are dashboard-panel predicates (metric.attr.*, service.name,
	// resource.attr.*), ANDed with Scope.Filters.
	Where []PanelPredicate
	// StepSeconds is the resolved bucket width (see StepForMetric). Unused
	// with Reduce.
	StepSeconds int64
	// MaxSeries caps a grouped query in SQL at the top MaxSeries groups; 0 is
	// MaxMetricSeries, and the most a table or bar panel may ask is MaxTableRows.
	MaxSeries int
	// Reduce compiles one bucket spanning the whole range: a rate becomes the
	// range-average rate, a quantile is over the merged range, a level
	// aggregation is the last value.
	Reduce bool
}

MetricQuery is one metric statement: a series' time-series (or, with Reduce, its one-bucket reduction) over Scope's range.

func (MetricQuery) Capped

func (q MetricQuery) Capped() bool

Capped reports whether the statement keeps only the top seriesCap groups in SQL: it is grouped and the metric has more series than the cap.

func (MetricQuery) Compile

func (q MetricQuery) Compile() (Stmt, error)

Compile builds the statement. Columns for every aggregation but a histogram quantile: bucket (DateTime), labels (Array(String), one per GroupBy key), value (Float64), series_total (UInt64; the group count before the cap when Capped, else 0). For a histogram quantile: bucket, labels, bounds (Array(Float64)), counts (Array(Float64)), series_total; one row per distinct bound set in a (bucket, labels), which MergeHistograms folds.

Counters are rates per series first, Σinc/Σdt over the series' samples in the bucket, and then summed across series. A cumulative sample's increment is its value less its predecessor's, or its whole value when it follows a reset (the StartTimeUnix moved, or the counter decreased); a delta sample's is its value.

func (MetricQuery) QuantileLevel

func (q MetricQuery) QuantileLevel() float64

QuantileLevel is the level a resolved histogram quantile aggregation reads.

type MetricSeriesResult

type MetricSeriesResult struct {
	Label   string
	Labels  map[string]string
	Points  []MetricPoint
	Reduced *float64
}

MetricSeriesResult is one group of one panel series (Label), or of the panel's "formula" or "ratio". A number, table or bar panel fills Reduced (nil: the reduction is null) and a number panel's Points are its sparkline.

type MetricType

type MetricType int

MetricType is the OTel metric kind a metric is stored as.

const (
	MetricUnknown MetricType = iota
	MetricGauge
	MetricSum
	MetricHistogram
	// MetricExpHistogram and MetricSummary equal their CatalogKind.
	MetricExpHistogram
	MetricSummary
)

The supported metric kinds.

func (MetricType) String

func (t MetricType) String() string

type NamedStmt

type NamedStmt struct {
	Purpose string
	Stmt    Stmt
}

NamedStmt is a compiled statement and what it is for (an explain's rows).

type Node

type Node interface{}

Node is a parsed log query expression.

func ParseLogQuery

func ParseLogQuery(q string) (Node, error)

ParseLogQuery parses the log query language into an AST. An empty query matches every record (nil Node).

type Overview

type Overview struct {
	Scope Scope
	// GroupBy splits each service by these allowlisted keys.
	GroupBy []string
}

Overview is a GetServiceOverview request.

func (Overview) Logs

func (q Overview) Logs() (Stmt, error)

Logs builds the per-service log counts. Columns: ServiceName, total, error_logs, source, then the group columns. Only resource-level filters and group keys apply: a span-level filter does not describe a log record, and would silently zero the count. IsNotOnLogs(err) means a group key is span-level, so there is no log figure to give.

func (Overview) RED

func (q Overview) RED() (Stmt, error)

RED builds the per-service request statement over server spans. Columns: ServiceName, requests, errors, q (p50,p95,p99 in ms), source, then one String column per group-by key. Requests and errors count every server span; the latency quantiles are over unary ones only, since a streaming RPC's span lasts as long as the stream (spanStreamExpr).

func (Overview) SeenServices

func (q Overview) SeenServices() (Stmt, error)

SeenServices lists service names with spans or logs in the scope, for the synthesized dashboards. Columns: ServiceName, has_spans.

func (Overview) Trend

func (q Overview) Trend(stepSec int64) (Stmt, error)

Trend builds the per-service per-bucket request and error counts. Columns: ServiceName, bucket (DateTime), requests, errors, then the group columns.

type Panel

type Panel struct {
	ID      string
	Kind    string // number | line | table | bar | heatmap | sql | funnel | issues | sessions  (search is not executable here)
	Signal  string // traces | logs | metrics; browser kinds: traces or empty; sql: optional
	Series  []PanelSeries
	Ratio   bool
	Where   []PanelPredicate
	GroupBy []string
	// Formula plots one expression over the series (forge dashspec.ParseFormula)
	// instead of the series, on line and number panels.
	Formula string
	// Limit keeps the top Limit groups of a table or bar panel; zero is the
	// kind's default (RowCap).
	Limit int
	// SQL is a sql panel's statement (see sql_panel.go).
	SQL string
	// Funnel is the definition of a funnel panel; HasError narrows a sessions
	// panel to sessions that hit an error. See panel_browser.go.
	Funnel   *PanelFunnel
	HasError bool
}

Panel is one dashboard tile (forge obsdash.Panel), decoupled from the wire type.

func (Panel) MaxRange

func (p Panel) MaxRange() (time.Duration, error)

MaxRange is the maximum range of a panel: by kind for the browser kinds, by signal for the rest.

func (Panel) ParsedFormula

func (p Panel) ParsedFormula() (*dashspec.Formula, error)

ParsedFormula is the panel's Formula, parsed against its series; nil when the panel has none.

func (Panel) RowCap

func (p Panel) RowCap() int

RowCap is how many groups a table or bar panel keeps: its Limit, else the kind's default.

func (Panel) Validate

func (p Panel) Validate() error

Validate checks everything about a panel that does not need the scope.

type PanelFunnel

type PanelFunnel struct {
	Steps  []string
	Window string // "<n><s|m|h|d>", at most MaxRangeFunnel; empty = DefaultFunnelWindow
	By     string // "session" | "user"; empty = DefaultFunnelBy
}

PanelFunnel is the definition of a funnel panel (forge.Funnel).

type PanelPredicate

type PanelPredicate struct {
	Field  string
	Op     string // eq neq in not_in prefix match exists gte
	Value  string
	Values []string
}

PanelPredicate is one condition of a dashboard panel (forge obsdash.Predicate).

type PanelQuery

type PanelQuery struct {
	Scope   Scope
	Panel   Panel
	Filters []PanelPredicate
	// StepSeconds is the resolved bucket width for line panels.
	StepSeconds int64
}

PanelQuery is a QueryPanel request for a traces or logs panel.

func (PanelQuery) Compile

func (q PanelQuery) Compile() (Stmt, PanelShape, error)

Compile builds the statement for a traces or logs panel.

line:    bucket, labels (Array(String)), v0..vN
number:  labels, v0..vN; ungrouped with StepSeconds set (a sparkline):
         bucket, total (UInt8), labels, v0..vN, the total row last
table:   labels, v0..vN   (heaviest first, at most RowCap+1 rows)
bar:     as table
heatmap: bucket, vb (Int32 value bucket), n (Float64)

type PanelSeries

type PanelSeries struct {
	Label      string
	Agg        string // count count_distinct sum avg max quantile last
	Quantile   float64
	Field      string // a semantic field; the metric NAME for metrics panels
	MetricType string // "sum" | "gauge" | "histogram" (metrics panels)
	Where      []PanelPredicate
}

PanelSeries is one aggregate of a panel (forge obsdash.Series).

type PanelShape

type PanelShape struct {
	Bucketed bool // first column is the bucket (line panels, sparklines)
	// Total: after the bucket, a UInt8 that is 1 on the row aggregating the
	// whole range (an ungrouped number panel's value; the others are its
	// sparkline).
	Total  bool
	Groups int // length of the labels array
	Series int // trailing Float64 columns
	// Rows is how many rows a table or bar panel keeps; the statement returns
	// one more when there were more.
	Rows int
	// Heatmap: the columns are bucket, value bucket (Int32), count.
	Heatmap bool
}

Shape describes the columns of the compiled statement.

type REDSample

type REDSample struct {
	Requests, Errors float64
	// P95 is in seconds; valid when HasP95.
	P95    float64
	HasP95 bool
}

REDSample is one window of one service: requests, server errors and p95.

func (REDSample) ErrorRate

func (s REDSample) ErrorRate() float64

ErrorRate is Errors / Requests, 0 when there were no requests.

type ReplayCursor

type ReplayCursor struct {
	TsNano int64  `json:"t"`
	Index  uint32 `json:"e"`
	Finger string `json:"f"`
}

ReplayCursor is the position after the last event of a page.

func DecodeReplayCursor

func DecodeReplayCursor(tok string) (*ReplayCursor, error)

DecodeReplayCursor parses a page token; "" is no cursor.

type ReplayEvent

type ReplayEvent struct {
	// Time is the event's timestamp (rrweb's own, to the millisecond).
	Time time.Time
	// Index is the recorder's rr-web.event counter. It restarts on every page
	// load, so it orders events only within one timestamp.
	Index uint32
	// Chunks is how many stored records the event was reassembled from.
	Chunks int
	// Data is the event's JSON text exactly as recorded: opaque, masked at
	// record time, never altered.
	Data []byte
}

ReplayEvent is one whole rrweb event.

type ReplayPage

type ReplayPage struct {
	Events []ReplayEvent
	// More says more complete events follow now. NextToken is set exactly then.
	More      bool
	NextToken string
	// Resume continues after the last returned event and is set whenever the
	// page holds one, even on the last page: passing it later returns the events
	// recorded since (a session still being recorded). Empty for an empty page.
	Resume string
	// Bytes is the total size of Events' Data.
	Bytes int
	// Withheld counts events the statement picked that Parse refused because
	// they were not whole JSON objects. It should be zero; a nonzero value is a
	// storage fault worth logging.
	Withheld int
}

ReplayPage is one page of ReadReplay.

func ReadReplay

func ReadReplay(ctx context.Context, run Runner, q ReplayQuery) (ReplayPage, error)

ReadReplay compiles q, runs it on run and reassembles the page.

type ReplayQuery

type ReplayQuery struct {
	// Scope bounds the read: the database, the environments, the services and
	// the time range (which should cover the session; rum_replay is partitioned
	// by day). Scope.Filters do not apply to a read of one named session.
	Scope     Scope
	SessionID string
	// MaxBytes is the page's byte budget; 0 = DefaultReplayPageBytes.
	MaxBytes int
	Cursor   *ReplayCursor
}

ReplayQuery is a ReadReplay request: one page of one session's rrweb events.

func (ReplayQuery) Compile

func (q ReplayQuery) Compile() (Stmt, error)

Compile builds the page statement. Columns: ReplayColumns, rows ordered by (Timestamp, EventIndex, Chunk).

The statement first reads only the SIZE of each chunk (length(Body) is read from the column's size stream, not its bytes) to plan the page: per event its chunk count, completeness and bytes, then a running total picks the events that fit. Only the chunks of the picked events are read for their bodies. Every caller-supplied value is a bound parameter.

func (ReplayQuery) Fingerprint

func (q ReplayQuery) Fingerprint() string

Fingerprint identifies the session a cursor belongs to. The time range and the byte budget are deliberately not part of it.

func (ReplayQuery) NextToken

func (q ReplayQuery) NextToken(ts time.Time, index uint32) string

NextToken builds the page token that continues after the event at ts with the given rr-web.event index.

func (ReplayQuery) Parse

func (q ReplayQuery) Parse(rows Rows) (ReplayPage, error)

Parse reassembles the rows of Compile's statement into whole events. The caller closes rows.

type Rows

type Rows interface {
	Next() bool
	Scan(dest ...any) error
	Err() error
	Close() error
}

Rows is the subset of a ClickHouse result obsquery reads (clickhouse-go's driver.Rows satisfies it; a backend with decoded rows adapts them). The caller closes a cursor it passes to a Parse* function.

type Runner

type Runner func(ctx context.Context, st Stmt) (Rows, error)

Runner executes a compiled statement on the caller's connection. obsquery opens nothing: the backend's connection, settings and access gate apply.

type SQLPanel

type SQLPanel struct {
	Scope       Scope
	SQL         string
	StepSeconds int64
}

SQLPanel is a sql panel's statement bound to a VERIFIED scope (the backend's own resolved scope, e.g. the control plane's org gate; never a request field): the dashboard's range, step, environments and services become the forge dashspec.SQLParam* ClickHouse parameters. Nothing is spliced into the statement, which runs as the caller wrote it; the parameters are values the server parses by the type the statement declares.

func (SQLPanel) Compile

func (q SQLPanel) Compile() (Stmt, error)

Compile checks the statement with the alert-rule guard and binds the scope.

from, to   the range as Unix seconds with nine decimals, which a
           DateTime64(9) parameter parses the same in every session
           timezone (a "YYYY-MM-DD hh:mm:ss" text would be read in the
           session's)
step       seconds
env_ids    the environments in scope; [] when unrestricted, ["(none)"]
           when restricted to none
envs       their names, in env_ids order
services   the services in scope; [] = all
reliant_env_id
           the one environment in scope (EnvIDParam); absent otherwise,
           and a statement that uses it is refused

type Scope

type Scope struct {
	// Database qualifies every table: the org's database (t_<org>) in
	// the control plane, LocalDatabase against a local ClickStack.
	Database string
	// Local marks a scope over a project's own local ClickStack. Only a local
	// scope may name LocalDatabase: in the control plane `default` is the
	// server's own database, and an org scope that names it is refused.
	Local bool
	// EnvIDs narrows to these environments (reliant.env.id) when EnvRestricted
	// is set; an empty restricted list matches nothing. Unrestricted = every
	// environment of the org, and rows that carry no environment at all.
	EnvIDs        []string
	EnvRestricted bool
	// Envs are the environments in scope, for resolving
	// deployment.environment.name to the Reliant environment's name.
	Envs []Env
	// Services narrows to these service.name values; empty = all.
	Services []string
	// Filters are generic attribute predicates, ANDed.
	Filters []AttrFilter
	// Declared are the attribute keys the queried metric's own data points
	// carry; the one extension to the key allowlist, for metrics.
	Declared map[string]bool
	// Start inclusive, End exclusive.
	Start, End time.Time
}

Scope is what every query is bounded by (console v2, contract K1).

func CatalogRange

func CatalogRange(s Scope) Scope

CatalogRange clamps the scope's range to the last CatalogWindow of it.

func (Scope) Fingerprint

func (s Scope) Fingerprint() []string

Fingerprint identifies the parts of a scope a page cursor belongs to. The time range is deliberately not part of it.

type SeriesMeta

type SeriesMeta struct {
	Label, Metric string
	Info          MetricInfo
	Agg           Agg
	Quantile      float64
	ValueUnit     string
}

SeriesMeta describes one panel series after its metric was looked up and its aggregation resolved.

type ServerErrorRecord

type ServerErrorRecord struct {
	ServiceName string
	ScopeName   string
	Body        string
	// Attributes are the row's LogAttributes.
	Attributes map[string]string
}

ServerErrorRecord is the part of an otel_logs row sfp1 reads.

func (ServerErrorRecord) RefinedByErrorClass

func (r ServerErrorRecord) RefinedByErrorClass() bool

RefinedByErrorClass reports whether sfp2 folds the error class into the fingerprint of r: it carries a cause in the `error` attribute and neither an error.group nor an exception.type. A record the SDK typed keeps its sfp1 fingerprint exactly: the type is the class.

type ServerIssueCoOccurring

type ServerIssueCoOccurring struct {
	Scope       Scope
	Fingerprint string
}

ServerIssueCoOccurring finds the other issues that fired in the same traces as one issue, most often first.

func (ServerIssueCoOccurring) Compile

func (q ServerIssueCoOccurring) Compile() (Stmt, error)

Compile builds the co-occurrence statement. Columns: fingerprint, shared (the number of traces both issues appear in).

type ServerIssueEvents

type ServerIssueEvents struct {
	Scope       Scope
	Fingerprint string
	Limit       int
}

ServerIssueEvents reads the newest events of one issue.

func (ServerIssueEvents) Compile

func (q ServerIssueEvents) Compile() (Stmt, error)

Compile builds the newest-first events statement. Columns: ServerIssueEventColumns.

type ServerIssueIncidents

type ServerIssueIncidents struct {
	Scope Scope
	// Service narrows to one service, as the list it marks does.
	Service string
}

ServerIssueIncidents reads, for every server error issue in the range, the cause of its newest event and when it first fired in the range: the inputs of GroupIncidents. Source issues are not errors and never form an incident.

func (ServerIssueIncidents) Compile

func (q ServerIssueIncidents) Compile() (Stmt, error)

Compile builds the incident-candidates statement, newest first, capped at MaxIncidentCandidates issues (a range with more failing issues than that groups the newest of them). Columns: ServerIssueIncidentColumns. An issue with no cause is left out.

type ServerIssueVersions

type ServerIssueVersions struct {
	Scope       Scope
	Fingerprint string
}

ServerIssueVersions counts one issue's events per service.version.

func (ServerIssueVersions) Compile

func (q ServerIssueVersions) Compile() (Stmt, error)

Compile builds the per-version statement, newest version first. Columns: version, events, affected, first_seen, last_seen, affected_unit.

type ServerIssues

type ServerIssues struct {
	Scope Scope
	// Service narrows to one service.name, beside Scope.Services.
	Service string
	// Fingerprint narrows to one issue; empty lists them all.
	Fingerprint string
	// Source lists the issues of this issue source (LogKeyIssueSource);
	// empty lists errors. Ignored with a Fingerprint, which names its issue
	// whatever its source.
	Source string
	Limit  int // issues to return; the statement fetches one more
	Sort   string
}

ServerIssues lists the server issues of a scope.

func (ServerIssues) Compile

func (q ServerIssues) Compile() (Stmt, error)

Compile builds the grouping statement. Columns: ServerIssuesColumns.

type ServiceREDInput

type ServiceREDInput struct {
	Service string
	// Observed is the window's aggregate.
	Observed REDSample
	// Baseline is one sample per slot of the previous weeks, a slot with no
	// traffic a zero sample, for the weeks the service already existed in.
	Baseline []REDSample
	// FirstSeen is when the service's metrics first appeared in the lookback
	// (zero: never).
	FirstSeen time.Time
	Now       time.Time
}

ServiceREDInput is what the judge needs of one service.

func (ServiceREDInput) HistoryDays

func (in ServiceREDInput) HistoryDays() int

HistoryDays is how many whole days of history the service had, at most the lookback the first-seen was read over.

type SessionCursor

type SessionCursor struct {
	TsMs      int64  `json:"t"`
	Service   string `json:"s"`
	SessionID string `json:"i"`
	Finger    string `json:"f"`
}

SessionCursor is the keyset position of the last session of a page.

func DecodeSessionCursor

func DecodeSessionCursor(tok string) (*SessionCursor, error)

DecodeSessionCursor parses a page token; "" is no cursor.

type SessionLookup

type SessionLookup struct {
	// Scope bounds the read; its range should cover the session (the days it
	// touched are read, and the day before the range start).
	Scope     Scope
	SessionID string
}

SessionLookup is a GetSession request: one session's summary.

func (SessionLookup) Compile

func (q SessionLookup) Compile() (Stmt, error)

Compile builds the statement. Columns: sessionCols. At most one row.

type SessionPage

type SessionPage struct {
	Sessions []SessionSummary
	// NextCursor continues after the last returned session; "" on the last page.
	NextCursor string
}

SessionPage is one page of ListSessions.

func ListSessions

func ListSessions(ctx context.Context, run Runner, q SessionQuery) (SessionPage, error)

ListSessions compiles q, runs it on run and parses the page.

type SessionQuery

type SessionQuery struct {
	Scope Scope
	// HasError keeps sessions with at least one browser error; HasReplay those
	// with recorded replay.
	HasError, HasReplay bool
	// UserID keeps sessions of that user id; empty keeps all.
	UserID string
	// PathContains keeps sessions whose entry path contains it, case-insensitive.
	PathContains string
	Limit        int // 0 = DefaultSessionLimit; the statement fetches one more
	Cursor       *SessionCursor
}

SessionQuery is a ListSessions request: the sessions that STARTED in the scope's range, newest first.

func (SessionQuery) Compile

func (q SessionQuery) Compile() (Stmt, error)

Compile builds the list statement. Columns: sessionCols. At most limit+1 rows.

func (SessionQuery) Fingerprint

func (q SessionQuery) Fingerprint() string

Fingerprint identifies the filters a cursor belongs to. The time range is deliberately not part of it.

func (SessionQuery) NextCursor

func (q SessionQuery) NextCursor(started time.Time, service, sessionID string) string

NextCursor builds the page token that continues after the session that started at started.

func (SessionQuery) Parse

func (q SessionQuery) Parse(rows Rows) (SessionPage, error)

Parse reads the rows of Compile's statement. The caller closes rows.

type SessionSummary

type SessionSummary struct {
	SessionID    string
	Service      string
	Version      string
	EnvID        string
	EnvName      string
	UserID       string // the userId global attribute, empty when never identified
	EntryPath    string // path of the first page, no query or fragment
	Started      time.Time
	Ended        time.Time
	PageViews    uint64
	Interactions uint64
	Errors       uint64
	HasReplay    bool
}

SessionSummary is one browser session, merged from its daily partial rows.

func GetSession

func GetSession(ctx context.Context, run Runner, q SessionLookup) (SessionSummary, error)

GetSession compiles q, runs it on run and parses the summary.

func ParseSession

func ParseSession(rows Rows, envs []Env) (SessionSummary, error)

ParseSession reads the rows of SessionLookup.Compile. envs resolve the environment's name. A session the statement did not find is ErrSessionNotFound.

func (SessionSummary) Duration

func (s SessionSummary) Duration() time.Duration

Duration is how long the session was observed.

type SessionTimeline

type SessionTimeline struct {
	// Scope bounds the read; its range (at most MaxRangeTimeline) should cover
	// the session, e.g. its start minus a minute to its end plus a minute.
	Scope     Scope
	SessionID string
	Limit     int // 0 = DefaultTimelineLimit; the statement fetches one more
	Cursor    *TimelineCursor
}

SessionTimeline is a ReadTimeline request: one session's navigations, clicks, failed requests, errors and custom events in time order.

func (SessionTimeline) Compile

func (q SessionTimeline) Compile() (Stmt, error)

Compile builds the timeline statement: five branches over the session's rows, merged and ordered by (ts, kind, span_id). Columns: timelineCols. At most limit+1 rows.

navigation       otel_traces: documentLoad, routeChange
click            otel_traces: mousedown (the SDK emits click two or three times per click)
request_failed   otel_traces: fetch / xml-http-request with status >= 400 or an error status
event            otel_traces: scope custom-action
error            error_events, left-joined to error_stacks for the fingerprint

A console.error(string) has no exception and is no error row, so it is no timeline entry either; the error spans of otel_traces are not read, since error_events holds them.

func (SessionTimeline) Fingerprint

func (q SessionTimeline) Fingerprint() string

Fingerprint identifies the session a cursor belongs to. The time range is deliberately not part of it.

func (SessionTimeline) NextCursor

func (q SessionTimeline) NextCursor(ts time.Time, kind, spanID string) string

NextCursor builds the page token that continues after the entry at ts.

func (SessionTimeline) Parse

func (q SessionTimeline) Parse(rows Rows) (TimelinePage, error)

Parse reads the rows of Compile's statement. The caller closes rows.

type Signal

type Signal int

Signal selects the table family a key or filter is compiled against.

const (
	SigTraces Signal = iota + 1
	SigLogs
	SigMetrics
)

The signals.

func SignalOf

func SignalOf(name string) (Signal, bool)

SignalOf maps a dashboard panel's signal name to a Signal.

type Stmt

type Stmt struct {
	SQL    string
	Params map[string]string
}

Stmt is a statement and the server-side parameter values it binds.

type Temporality

type Temporality int32

Temporality is how a sum or histogram's SDK exports its value; the values are the OTel AggregationTemporality enum.

const (
	TemporalityUnknown    Temporality = 0
	TemporalityDelta      Temporality = 1
	TemporalityCumulative Temporality = 2
)

The temporalities.

func (Temporality) String

func (t Temporality) String() string

type TimelineCursor

type TimelineCursor struct {
	TsNano int64  `json:"t"`
	Kind   string `json:"k"`
	SpanID string `json:"i"`
	Finger string `json:"f"`
}

TimelineCursor is the keyset position of the last entry of a page.

func DecodeTimelineCursor

func DecodeTimelineCursor(tok string) (*TimelineCursor, error)

DecodeTimelineCursor parses a page token; "" is no cursor.

type TimelineEntry

type TimelineEntry struct {
	Time time.Time
	Kind TimelineKind
	// Name: the span name of a navigation (documentLoad, routeChange), "click",
	// the HTTP method of a failed request, the span name of an error
	// (onerror, unhandledrejection, ...), the event name of a custom event.
	Name string
	// Path is the page (navigation, click, error, event) or the request's path.
	Path     string
	Method   string
	Status   int    // HTTP status of a failed request; 0 for a network failure
	Selector string // click: the element's xpath
	Element  string // click: the element's tag name
	// ErrorType is the exception type of an error, or the SDK's error.type of a
	// failed request. Message is the masked error message, or a failed
	// request's status text.
	ErrorType string
	Message   string
	// Fingerprint is the issue fingerprint of an error (error_stacks): empty
	// until the enrichment worker has seen its stack. The control plane maps it
	// to an issue id.
	Fingerprint string
	TraceID     string
	SpanID      string
	DurationMs  float64
	// Props: a custom event's properties (at most 32, values cut at 256
	// characters); a navigation's "from" (the previous path).
	Props   map[string]string
	Service string
}

TimelineEntry is one line of a session's timeline.

Nothing in it is a raw URL or element text. Paths have their query and fragment cut (they carry tokens); a click carries the element's tag and its xpath (tags and ids), never its text, which the recorder masks as well; an error carries the message with its volatile parts masked (MessageNorm).

type TimelineKind

type TimelineKind string

TimelineKind is what a timeline entry is.

const (
	TimelineNavigation    TimelineKind = "navigation"     // a document load or route change
	TimelineClick         TimelineKind = "click"          // one physical click or tap
	TimelineRequestFailed TimelineKind = "request_failed" // a fetch or XHR with status >= 400 or an error status
	TimelineError         TimelineKind = "error"          // a browser error (an error_events row)
	TimelineEvent         TimelineKind = "event"          // a custom event (track / HyperDX.addAction)
)

type TimelinePage

type TimelinePage struct {
	Entries []TimelineEntry
	// NextCursor continues after the last returned entry; "" on the last page.
	NextCursor string
}

TimelinePage is one page of ReadTimeline.

func ReadTimeline

func ReadTimeline(ctx context.Context, run Runner, q SessionTimeline) (TimelinePage, error)

ReadTimeline compiles q, runs it on run and parses the page.

type TraceCursor

type TraceCursor struct {
	TsNano  int64  `json:"t"`
	TraceID string `json:"i"`
	// DurNano is the last trace's duration, set only for OrderDurationDesc.
	DurNano int64  `json:"d,omitempty"`
	Finger  string `json:"f"`
}

TraceCursor is the keyset position of the last trace of a page.

func DecodeTraceCursor

func DecodeTraceCursor(tok string) (*TraceCursor, error)

DecodeTraceCursor parses a page token.

type TraceOrder

type TraceOrder string

TraceOrder is the row order of a ListTraces page.

const (
	// OrderNewest is newest first, the default (the zero TraceOrder).
	OrderNewest TraceOrder = ""
	// OrderDurationDesc is slowest first: the traces to open when latency
	// regressed. Duration is the trace's wall-clock span (dur_ms), not a
	// weighted figure, so the sampling weight does not move a trace in the
	// order; it still weights the RED summaries over the same filters.
	OrderDurationDesc TraceOrder = "duration_desc"
)

type TraceQuery

type TraceQuery struct {
	Scope         Scope
	ErrorsOnly    bool
	MinDurationMs float64
	RootSpanName  string
	// GroupBy are the allowlisted keys the RED summary is split by, valued at
	// each trace's root span (CompileREDGroups).
	GroupBy []string
	// Order is the page's row order; the zero value is newest first. It is
	// part of the cursor's fingerprint: a token only continues the order that
	// produced it.
	Order  TraceOrder
	Limit  int
	Cursor *TraceCursor
}

TraceQuery is a ListTraces request.

func (TraceQuery) Compile

func (q TraceQuery) Compile() (Stmt, error)

Compile builds the page statement in q.Order (newest first by default), one row more than Limit.

func (TraceQuery) CompileRED

func (q TraceQuery) CompileRED() (Stmt, error)

CompileRED builds the RED summary over every trace the filters match. Columns: trace_count, error_trace_count, p50, p95, p99.

func (TraceQuery) CompileREDGroups

func (q TraceQuery) CompileREDGroups() (Stmt, error)

CompileREDGroups builds the RED summary split by q.GroupBy (values read at each trace's root span), busiest group first. Columns: labels (Array(String)), trace_count, error_trace_count, q.

func (TraceQuery) Fingerprint

func (q TraceQuery) Fingerprint() string

Fingerprint identifies the filters a cursor belongs to.

func (TraceQuery) NextCursor

func (q TraceQuery) NextCursor(start time.Time, traceID string) string

NextCursor builds the page token from the last returned trace of a newest-first page. A duration-ordered page uses NextDurationCursor.

func (TraceQuery) NextDurationCursor

func (q TraceQuery) NextDurationCursor(start time.Time, traceID string, durMs float64) string

NextDurationCursor builds the page token from the last returned trace of an OrderDurationDesc page; durMs is that row's dur_ms column.

type TraceSpans

type TraceSpans struct {
	Scope   Scope
	TraceID string
}

TraceSpans is a GetTrace request.

func (TraceSpans) Compile

func (q TraceSpans) Compile() (Stmt, error)

Compile builds the all-spans statement. Columns: Timestamp, SpanId, ParentSpanId, SpanName, SpanKind, ServiceName, Duration, StatusCode, StatusMessage, SpanAttributes, ResourceAttributes, Events.Timestamp, Events.Name, Events.Attributes, Links.TraceId, Links.SpanId.

type Workloads

type Workloads struct {
	Scope Scope
	// Workload narrows to one k8s.workload.name (GetWorkload); empty = all.
	Workload string
	// Container narrows to one container; empty = all.
	Container string
}

Workloads is the common shape of the workload statements.

func (Workloads) Events

func (q Workloads) Events() (Stmt, error)

Events returns the workload's events, newest first (one more than MaxWorkloadEvents, to detect the cut). Columns as eventSelect.

func (Workloads) ImageChanges

func (q Workloads) ImageChanges() (Stmt, error)

ImageChanges lists the distinct images each container ran. Columns: env, container, img, tag, first_seen (DateTime), last_seen (DateTime); oldest first, one more than MaxWorkloadChanges.

func (Workloads) Jobs

func (q Workloads) Jobs() (Stmt, error)

Jobs finds the Job runs in the range from the Jobs' own Kubernetes events (k8s.object.kind = Job). The cluster collector delivers them with the Job's name as the workload. Columns: env, job, created_n (UInt64), started (DateTime), done_n (UInt64), finished (DateTime), failed_n (UInt64), failed_at (DateTime), reason, last (DateTime). Newest activity first, one more than MaxJobs. A count of 0 means the event was not seen, and the matching time is then meaningless.

func (Workloads) LatestWarnings

func (q Workloads) LatestWarnings() (Stmt, error)

LatestWarnings returns each workload's most recent Warning event. Columns as eventSelect.

func (Workloads) PodRestarts

func (q Workloads) PodRestarts() (Stmt, error)

PodRestarts gives, per pod and container, the growth of k8s.container.restarts in the range and the last terminated reason. Columns: env, workload, pod_name, pod_uid, container, restarts, first (DateTime: the container's first sample in the range), reason, reason_at (DateTime: when the reason was first reported). A reason already present at the first sample has reason_at == first: the real exit time is then unknown.

func (Workloads) Pods

func (q Workloads) Pods() (Stmt, error)

Pods summarizes every pod's presence in the range: first and last sample and the node. Columns: env, workload, pod_name, pod_uid, first_seen (DateTime), last_seen (DateTime), node. Newest first, MaxWorkloadRows.

The node is the k8s.node.name the node agent delivers; it is empty for data from an agent that strips it.

func (Workloads) Readiness

func (q Workloads) Readiness(stepSec int64) (Stmt, error)

Readiness is the continuous health signal: per workload and time bucket, how many pods reported, how many of them were ready, and how many restarts happened, read from the k8s_cluster receiver's k8s.container.ready and k8s.container.restarts gauges. It is a pure function of stored telemetry, so a verdict derived from it needs no reader to have looked at the time.

A pod counts as ready in a bucket when EVERY sample of EVERY one of its containers in the bucket is 1, so a dip inside a coarse bucket is not lost. A bucket in which a workload delivered no readiness sample has no row: a gap is missing data, never a healthy bucket, and the reader must treat it so. Restarts are the growth of the counter between consecutive samples of a container (a decrease is a reset and counts as zero), attributed to the bucket of the later sample.

Columns: env, workload, bucket (DateTime), pods (UInt64: distinct pods that reported readiness), ready (UInt64: of those, the ones fully ready), restarts (Float64), exit (String: the last terminated reason reported in the bucket, "" when none). Ordered by env, workload, bucket; at most MaxWorkloadSeriesRow rows. A row with pods = 0 carries restarts only.

func (Workloads) Replicas

func (q Workloads) Replicas() (Stmt, error)

Replicas counts the pods a workload has running: the distinct pods that reported within replicaWindow of the workload's own latest sample (so a historical range still answers, rather than counting zero). Columns: env, workload, replicas, last_seen (DateTime: the workload's latest sample, so a reader can tell a live workload from one that stopped reporting).

func (Workloads) RestartSeries

func (q Workloads) RestartSeries(stepSec int64) (Stmt, error)

RestartSeries charts restarts per bucket: the sum over pods of the growth of k8s.container.restarts (a decrease is a reset and counts as zero). Columns: env, container, bucket (DateTime), value.

func (Workloads) Restarts

func (q Workloads) Restarts() (Stmt, error)

Restarts counts restarts per container: for each pod the growth of k8s.container.restarts within the range, summed. Columns: env, workload, container, restarts.

func (Workloads) Series

func (q Workloads) Series(stepSec int64) (Stmt, error)

Series charts the gauges over time: per pod the last value in each bucket, then the worst (highest) replica. Columns: env, container, metric, bucket (DateTime), value.

func (Workloads) Terminations

func (q Workloads) Terminations() (Stmt, error)

Terminations finds each container's last terminated reason, read from the k8s.container.last_terminated_reason attribute of its restart gauge or of an event log (the agent may deliver either, contract K2). Columns: env, workload, container, reason, at (DateTime), in_window (UInt8).

From the gauge, `at` is when the reason was first reported, and in_window is 0 when it was already there at the container's first sample in the range (so the real time is unknown). From an event log `at` is the event time and in_window is 1.

func (Workloads) Usage

func (q Workloads) Usage() (Stmt, error)

Usage summarizes every container's usage, utilization, requests and limits. Columns: env, workload, container, metric, latest (highest per-pod latest), p95 (over every sample of every pod), ns, img, tag (latest non-empty).

func (Workloads) Versions

func (q Workloads) Versions(services []string) (Stmt, error)

Versions lists the service.version values the services' OWN telemetry (spans and metrics; service.name = the workload name) reported. Services limits it to those names; GetWorkload passes the workload. Columns: env, svc, version, first_seen (DateTime), last_seen (DateTime), oldest first.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL