Documentation
¶
Overview ¶
Package observe is the native observability storage layer for Rune ("RuneSight"). It defines a single LogStore seam between the control-plane ingest/query paths and a pluggable backend (an embedded default store, or optional ClickHouse / Loki sinks).
Everything in the observability subsystem is built against this package. The ObserveService parses a LogQL subset into the Query AST defined here exactly once, then hands the AST to a LogStore — no store ever parses LogQL itself. Stores declare what they can do via Capabilities, so the dashboard can feature-flag advanced widgets per backend.
Ported from the runesight/ scaffold (internal/storage) into Rune core; see _docs/plugins/RUNESIGHT_IMPLEMENTATION_PLAN.md §4 (storage abstraction) and §6 (what ports from the scaffold).
Index ¶
- Variables
- func ExtractFields(line string, stages []ParserStage) map[string]string
- type AggOp
- type Aggregation
- type Capabilities
- type CompiledLabelFilters
- type Direction
- type LabelFilter
- type LabelFilterOp
- type LabelValue
- type LineFilter
- type LineFilterOp
- type LogRecord
- type LogRow
- type LogStore
- type MatchOp
- type Matcher
- type MetricSample
- type ParserStage
- type Query
- type ResultStream
- type Selector
- type StatsProvider
- type StoreStats
- type Tier
Constants ¶
This section is empty.
Variables ¶
var ErrCapabilityUnsupported = errors.New("observe: capability not supported by backend")
ErrCapabilityUnsupported is returned when a Query asks for something the store does not advertise in Capabilities (e.g. RawSQL against the embedded store or Loki).
var ErrNotImplemented = errors.New("observe: not implemented")
ErrNotImplemented is returned by store methods that are scaffolded but not yet wired to a backend (the Loki / ClickHouse skeletons). Callers (and the conformance suite) can use errors.Is to detect a stubbed path.
Functions ¶
func ExtractFields ¶
func ExtractFields(line string, stages []ParserStage) map[string]string
ExtractFields runs the parser stages over a log line and returns the extracted fields. Stages apply in order; later stages overwrite duplicate keys. Unparseable lines extract nothing (never an error — a mixed stream must not fail the query; rows simply gain no fields and label filters drop them naturally).
Types ¶
type AggOp ¶
type AggOp uint8
AggOp is a range-vector aggregation over the matched lines.
const ( // AggCountOverTime is LogQL `count_over_time` — the core histogram op. AggCountOverTime AggOp = iota // AggRateOverTime is `rate` (count per second). AggRateOverTime // AggBytesOverTime is `bytes_over_time`. AggBytesOverTime // AggQuantileOverTime is a percentile over an extracted numeric field. // Advanced tier (ClickHouse only) — requires Quantile and Field set. AggQuantileOverTime )
type Aggregation ¶
type Aggregation struct {
// Op is the range-vector function.
Op AggOp
// Step is the histogram bucket width (the `[5m]` range / resolution).
Step time.Duration
// GroupBy is the `by (...)` grouping dimensions. Grouping by a
// high-cardinality field is Advanced tier on Loki.
GroupBy []string
// Field is the extracted numeric field for AggQuantileOverTime
// (e.g. "dur"). Ignored by the count/rate/bytes ops.
Field string
// Quantile in [0,1] for AggQuantileOverTime (e.g. 0.99). Advanced tier.
Quantile float64
}
Aggregation describes a metric query built over the selected log lines.
type Capabilities ¶
type Capabilities struct {
// Backend is the store identity ("embedded", "loki", or "clickhouse"),
// surfaced in the dashboard and in operational logs.
Backend string
// MaxTier is the highest tier the store supports. The embedded store and
// Loki report TierCore; ClickHouse reports TierAdvanced (which implies
// Core).
MaxTier Tier
// RawSQL is true when the store accepts Query.RawSQL (ClickHouse only).
RawSQL bool
// Percentiles is true when AggQuantileOverTime is supported.
Percentiles bool
// HighCardinalityFilters is true when filtering/grouping on arbitrary
// high-cardinality label keys (e.g. order_id, trace_id) is cheap enough to
// expose in the dashboard.
HighCardinalityFilters bool
// Parsers is true when the store evaluates field-extraction pipeline
// stages (`| logfmt`, `| json`) and post-parse label filters. The
// embedded store evaluates them in-process and Loki natively; the
// ClickHouse adapter rejects them for now (RawSQL is its escape hatch).
Parsers bool
}
Capabilities is the handshake a store reports on connect. The ObserveService forwards it to the dashboard; the dashboard shows SQL mode and advanced widgets only when the backend reports TierAdvanced.
func (Capabilities) Supports ¶
func (c Capabilities) Supports(t Tier) bool
Supports reports whether the store can serve the given tier.
type CompiledLabelFilters ¶
type CompiledLabelFilters []compiledLabelFilter
CompiledLabelFilters pre-compiles label filters for row-rate evaluation.
func CompileLabelFilters ¶
func CompileLabelFilters(fs []LabelFilter) (CompiledLabelFilters, error)
CompileLabelFilters validates and compiles the filters (regex compilation, numeric threshold parsing).
type LabelFilter ¶
type LabelFilter struct {
Label string
Op LabelFilterOp
Value string
}
LabelFilter is one post-parse predicate over a label (intrinsic or extracted by a parser stage).
type LabelFilterOp ¶
type LabelFilterOp string
LabelFilterOp is the comparison of a post-parse label filter. String ops compare lexically (regex ops compile the value); numeric ops parse both sides as floats and drop rows whose label value is not numeric.
const ( LabelFilterEq LabelFilterOp = "=" LabelFilterNeq LabelFilterOp = "!=" LabelFilterRe LabelFilterOp = "=~" LabelFilterNotRe LabelFilterOp = "!~" LabelFilterGT LabelFilterOp = ">" LabelFilterGTE LabelFilterOp = ">=" LabelFilterLT LabelFilterOp = "<" LabelFilterLTE LabelFilterOp = "<=" LabelFilterNumEq LabelFilterOp = "==" )
type LabelValue ¶
LabelValue is one value of a label dimension, with an optional occurrence count for ranking in the dashboard.
type LineFilter ¶
type LineFilter struct {
Op LineFilterOp
Value string
}
LineFilter is a single post-selection grep stage over the log line content.
type LineFilterOp ¶
type LineFilterOp uint8
LineFilterOp is the operator of a line-grep stage.
const ( // LineContains is `|= "x"` — keep lines containing x. LineContains LineFilterOp = iota // LineNotContains is `!= "x"` — drop lines containing x. LineNotContains // LineRegex is `|~ "re"` — keep lines matching re. LineRegex // LineNotRegex is `!~ "re"` — drop lines matching re. LineNotRegex )
type LogRecord ¶
type LogRecord struct {
// Timestamp is the event time of the line (not ingest time).
Timestamp time.Time
// Line is the raw log content (the "message").
Line string
// Stream is the source stream: "stdout" or "stderr". For Outbox-sourced
// agent/subsystem events this is "event".
Stream string
// Level is the parsed severity if known ("info", "warn", "error", ...).
// Empty when the line was not classified.
Level string
// Namespace the producing instance lives in (default: "default").
Namespace string
// Service is the parent service name.
Service string
// Instance is the instance ID the line came from.
Instance string
// Node is the Rune node ID (agent identity) that captured the line.
Node string
// Labels are arbitrary key/value stream selectors carried from the
// instance (pkg/types). High-cardinality labels are only safely
// queryable on the ClickHouse backend (Advanced tier).
Labels map[string]string
}
LogRecord is a single enriched log line on the ingest path.
The Rune agent forwarder stamps every record with the metadata already on the wire (see pkg/types/instance.go): namespace, service, instance, node, and arbitrary labels. The shape mirrors rune.api.LogResponse (logs.proto) plus the Rune-native identity fields that LogResponse omits.
func (LogRecord) StreamLabels ¶
StreamLabels returns the set of label keys that form the record's stream identity for backends that key chunks by stream (e.g. Loki). It is the union of the fixed Rune dimensions and any custom Labels keys.
type LogRow ¶
type LogRow struct {
Timestamp time.Time
Line string
Stream string
Level string
Labels map[string]string
}
LogRow is one returned log line on the query path. It mirrors LogRecord but is the read-side shape (a backend may not round-trip every ingest field).
type LogStore ¶
type LogStore interface {
// Write persists a batch of enriched records (the ingest path). Stores
// translate to the backend's native write. Must be safe for concurrent
// callers.
Write(ctx context.Context, batch []LogRecord) error
// Execute runs a parsed Query and streams results. It receives an AST,
// never raw LogQL text. Stores reject queries that exceed their
// Capabilities with ErrCapabilityUnsupported.
Execute(ctx context.Context, q *Query) (ResultStream, error)
// Capabilities reports the store's capability handshake for dashboard
// feature-flagging. Cheap and side-effect free.
Capabilities() Capabilities
// Labels returns known values for the dimensions named by sel — used to
// populate label chips and autocomplete in the dashboard.
Labels(ctx context.Context, sel Selector) ([]LabelValue, error)
// Health is a liveness/readiness probe against the backend.
Health(ctx context.Context) error
}
LogStore is the single seam between the observability control plane and its backend. Implementations live under embedded/ (the default), loki/, and clickhouse/. See plan §4.
type Matcher ¶
Matcher selects streams by a label dimension. Label is one of the Rune-native dimensions (namespace, service, instance, node) or a custom label key. The regex variants on a high-cardinality label key are Advanced-tier on Loki.
type MetricSample ¶
type MetricSample struct {
// Timestamp is the left edge of the histogram bucket.
Timestamp time.Time
// Value is the aggregated value (count, rate, percentile, ...).
Value float64
// GroupLabels are the resolved by(...) dimensions for this series
// (e.g. {"level":"error"} for a level breakdown). nil for ungrouped.
GroupLabels map[string]string
}
MetricSample is one bucket of an aggregation result: a value at a timestamp, tagged by the GroupBy dimensions that produced it.
type ParserStage ¶
type ParserStage string
ParserStage is a field-extraction stage.
const ( // ParserLogfmt extracts key=value pairs from the line (`| logfmt`). ParserLogfmt ParserStage = "logfmt" // ParserJSON extracts top-level scalar fields from a JSON line (`| json`). // Nested objects/arrays are skipped in this subset. ParserJSON ParserStage = "json" )
type Query ¶
type Query struct {
// Selectors are the stream matchers (the `{service="api"}` part of LogQL).
// ANDed together. Required — a query must select at least one stream.
Selectors []Matcher
// LineFilters are post-selection grep stages (`|= "boom"`, `!= "noise"`).
// Applied in order; ANDed together.
LineFilters []LineFilter
// Parsers are field-extraction stages (`| logfmt`, `| json`) applied
// after the line filters. Extracted fields become labels on the row —
// visible in results, filterable via LabelFilters, and groupable in
// aggregations. Our subset fixes the pipeline order as
// selectors → line filters → parsers → label filters.
Parsers []ParserStage
// LabelFilters are post-parse predicates (`| status = "500"`,
// `| dur > 250`) over intrinsic and extracted labels. ANDed together.
LabelFilters []LabelFilter
// Aggregation, when non-nil, turns the query from a log query into a
// metric query (e.g. count_over_time → a histogram). When nil the query
// returns raw log lines.
Aggregation *Aggregation
// Start and End bound the query window (event time). End is exclusive.
Start time.Time
End time.Time
// Limit caps the number of returned log lines (ignored for aggregations).
// Zero means "store default".
Limit int
// Direction controls ordering of returned lines.
Direction Direction
// RawSQL, when non-empty, is an Advanced-tier escape hatch: the operator
// asked for raw SQL mode. Only the ClickHouse store honours it; the Loki
// and embedded stores must reject a Query with RawSQL set (CapabilityAdvanced
// absent). When set, the other fields are advisory (time bounds may still be
// injected as query parameters).
RawSQL string
}
Query is the parsed, backend-agnostic representation of a user query. The ObserveService parses a LogQL subset into this AST exactly once; the embedded store evaluates it in-process, the Loki store re-emits it to Loki's HTTP API, and the ClickHouse store lowers it to SQL. No store parses LogQL text itself.
The AST is intentionally small: it expresses the Core capability tier (label select, line grep, count_over_time histograms, level breakdown) plus an escape hatch (RawSQL) for the ClickHouse-only Advanced tier.
func ParseLogQL ¶
ParseLogQL parses the Core-tier LogQL subset into a Query AST. The ObserveService calls this exactly once per request; stores never see raw LogQL text (plan §4). start/end/limit/forward come from the request envelope, not the LogQL string.
Supported subset (Core tier):
{label="v", label2=~"re"} |= "needle" != "noise" |~ "re" !~ "re"
count_over_time({label="v"}[5m])
rate({label="v"}[5m])
bytes_over_time({label="v"}[5m])
sum by (level) (count_over_time({label="v"}[5m]))
Anything outside this subset (parsers, unwrap, label_format, joins, quantile_over_time, ...) is Advanced tier and must go through RawSQL on a ClickHouse backend; ParseLogQL returns an error for it.
func (*Query) IsMetricQuery ¶
IsMetricQuery reports whether the query produces aggregated samples rather than raw log lines.
type ResultStream ¶
type ResultStream interface {
// IsMetric reports whether Next yields MetricSample (true) or LogRow.
IsMetric() bool
// Next advances to the next result. It returns false at end of stream or
// on error; check Err after Next returns false.
Next(ctx context.Context) bool
// Row returns the current log line. Valid only when IsMetric is false and
// the last Next returned true.
Row() LogRow
// Sample returns the current metric sample. Valid only when IsMetric is
// true and the last Next returned true.
Sample() MetricSample
// Err returns the first error encountered, if any.
Err() error
// Close releases backend resources. Safe to call multiple times.
Close() error
}
ResultStream is the result of Execute. Callers must Close it. A single Execute yields either log rows (raw query) or metric samples (aggregation), never both — IsMetric reports which.
type Selector ¶
type Selector struct {
// Name is the dimension whose values to enumerate (e.g. "service"). Empty
// means "enumerate the available label names rather than values".
Name string
// Match optionally constrains which streams contribute values (e.g. only
// services within a namespace). Empty means all streams.
Match []Matcher
// Start and End bound the time window considered. Zero values mean the
// store's default lookback.
Start time.Time
End time.Time
// Limit caps the number of returned values. Zero means store default.
Limit int
}
Selector narrows a Labels lookup.
type StatsProvider ¶
type StatsProvider interface {
Stats() StoreStats
}
StatsProvider is an optional interface a LogStore may implement to report StoreStats. The ObserveService type-asserts for it; stores that don't implement it surface as "managed by the backend" in the dashboard.
type StoreStats ¶
type StoreStats struct {
// Records currently held.
Records int64
// DiskUsedBytes is the on-disk footprint (0 for a purely in-memory store).
DiskUsedBytes int64
// DiskCapBytes is the hard disk bound (drop-oldest), 0 if unbounded.
DiskCapBytes int64
// Retention is the store's age bound (0 = store default semantics,
// negative = age-based eviction disabled).
Retention time.Duration
// OldestRecord is the timestamp of the oldest held record (zero when
// empty) — i.e. how far back queries can actually see.
OldestRecord time.Time
}
StoreStats are store-level facts for the dashboard's Sources page: how much is stored, how much disk it uses, and the store's bounds. Only stores that own their storage (the embedded store) can report them; external backends (Loki, ClickHouse) manage their own storage and report Supported=false.
type Tier ¶
type Tier uint8
Tier is a capability tier a store reports. The dashboard feature-flags advanced widgets (SQL mode, percentiles, cross-stream joins) off the reported tier so the embedded store, Loki and ClickHouse are not forced to capability parity (plan §4).
const ( // TierCore is supported by every install: label select, line grep, // count_over_time histograms, and level breakdown. TierCore Tier = iota // TierAdvanced is ClickHouse-only: raw SQL mode, percentiles on arbitrary // fields, cross-stream joins, and high-cardinality field filters. TierAdvanced )
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package alerting is the RuneSight alerter: it evaluates stored alert rules against the cluster's LogStore on a rolling cadence, drives the ok → pending → firing → resolved state machine, and notifies on transitions (a Rune event always; the rule's channels on firing/resolved).
|
Package alerting is the RuneSight alerter: it evaluates stored alert rules against the cluster's LogStore on a rolling cadence, drives the ok → pending → firing → resolved state machine, and notifies on transitions (a Rune event always; the rule's channels on firing/resolved). |
|
Package backend selects and constructs the observe.LogStore for the runefile-configured backend value (plan §5).
|
Package backend selects and constructs the observe.LogStore for the runefile-configured backend value (plan §5). |
|
Package clickhouse implements observe.LogStore against a ClickHouse backend (the analytical optional sink).
|
Package clickhouse implements observe.LogStore against a ClickHouse backend (the analytical optional sink). |
|
Package embedded implements observe.LogStore as a lightweight in-process store that ships in runed by default.
|
Package embedded implements observe.LogStore as a lightweight in-process store that ships in runed by default. |
|
Package loki implements observe.LogStore against a Loki backend (the light optional sink).
|
Package loki implements observe.LogStore against a Loki backend (the light optional sink). |