Documentation
¶
Overview ¶
Package storagebackend adapts the embeddable github.com/oteldb/storage engine to oteldb's query and ingestion interfaces, so any signal can be served from the native Go storage engine instead of ClickHouse.
All four signals are wired over a single shared *storage.Storage instance. Backend implements the metrics seam directly (Prometheus storage.Queryable + ExemplarQueryable, the PromQL engine's MetricsScanners, metricstorage.MetadataQuerier) and the ingestion sinks for every signal (ConsumeMetrics/ConsumeTraces/ConsumeLogs/ConsumeProfiles). Because the logs and profiles read interfaces declare colliding method names, each non-metric signal's query interface is implemented by a small wrapper obtained via Backend.Logs, Backend.Traces, and Backend.Profiles (see signals.go).
Index ¶
- type Backend
- func (b *Backend) ConsumeLogs(ctx context.Context, ld plog.Logs) error
- func (b *Backend) ConsumeMetrics(ctx context.Context, md pmetric.Metrics) error
- func (b *Backend) ConsumeProfiles(ctx context.Context, pd pprofile.Profiles) error
- func (b *Backend) ConsumeTraces(ctx context.Context, td ptrace.Traces) error
- func (b *Backend) ExemplarQuerier(context.Context) (promstorage.ExemplarQuerier, error)
- func (b *Backend) Logs() *LogQuerier
- func (b *Backend) MetricMetadata(context.Context, metricstorage.MetadataParams) (metricstorage.Metadata, error)
- func (b *Backend) MetricsScanners() (enginestorage.Scanners, error)
- func (b *Backend) Profiles() *ProfileQuerier
- func (b *Backend) Querier(mint, maxt int64) (promstorage.Querier, error)
- func (b *Backend) Traces() *TraceQuerier
- type LogQLOptimizer
- type LogQuerier
- func (q *LogQuerier) Capabilities() (caps logqlengine.QuerierCapabilities)
- func (q *LogQuerier) DetectedFields(ctx context.Context, opts logstorage.LabelsOptions) ([]logstorage.DetectedField, error)
- func (q *LogQuerier) DetectedLabels(ctx context.Context, opts logstorage.LabelsOptions) ([]logstorage.DetectedLabel, error)
- func (q *LogQuerier) LabelNames(ctx context.Context, opts logstorage.LabelsOptions) ([]string, error)
- func (q *LogQuerier) LabelValues(ctx context.Context, labelName string, opts logstorage.LabelsOptions) (iterators.Iterator[logstorage.Label], error)
- func (q *LogQuerier) Query(_ context.Context, selector []logql.LabelMatcher) (logqlengine.PipelineNode, error)
- func (q *LogQuerier) Series(ctx context.Context, opts logstorage.SeriesOptions) (logstorage.Series, error)
- type Option
- type ProfileQuerier
- func (q *ProfileQuerier) LabelNames(ctx context.Context, opts profilestorage.LabelNamesOptions) ([]string, error)
- func (q *ProfileQuerier) LabelValues(ctx context.Context, label string, opts profilestorage.LabelValuesOptions) ([]string, error)
- func (q *ProfileQuerier) ProfileTypes(ctx context.Context, opts profilestorage.ProfileTypesOptions) ([]profileql.ProfileType, error)
- func (q *ProfileQuerier) SelectMergeProfile(ctx context.Context, params profilestorage.SelectProfileParams) (*profilestorage.FlameTree, error)
- type TraceQuerier
- func (q *TraceQuerier) SearchTags(ctx context.Context, tags map[string]string, ...) (iterators.Iterator[tracestorage.Span], error)
- func (q *TraceQuerier) SelectSpansets(ctx context.Context, params traceqlengine.SelectSpansetsParams) (iterators.Iterator[traceqlengine.Trace], error)
- func (q *TraceQuerier) TagNames(ctx context.Context, opts tracestorage.TagNamesOptions) ([]tracestorage.TagName, error)
- func (q *TraceQuerier) TagValues(ctx context.Context, attr traceql.Attribute, ...) (iterators.Iterator[tracestorage.Tag], error)
- func (q *TraceQuerier) TraceByID(ctx context.Context, id otelstorage.TraceID, _ tracestorage.TraceByIDOptions) (iterators.Iterator[tracestorage.Span], error)
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Backend ¶
type Backend struct {
// contains filtered or unexported fields
}
Backend adapts a *storage.Storage to oteldb's metric query and ingestion interfaces. The zero value is not usable; construct with New.
func New ¶
New returns a Backend over store. The ingest side has no tenant callback, so every batch routes to the "default" tenant; the empty tenant id here normalizes to "default" on the read side, keeping reads and writes on the same tenant (which also makes cluster reads owner-aware).
func (*Backend) ConsumeLogs ¶
ConsumeLogs ingests an OTLP logs batch into the storage engine. It is the logs ingestion sink used when the storage backend serves logs.
func (*Backend) ConsumeMetrics ¶
ConsumeMetrics ingests an OTLP metrics batch into the storage engine. It is the metrics ingestion sink used by the oteldb collector exporter when the storage backend is selected. Histogram, exponential-histogram, summary, and value-less points are not representable in the storage engine yet and are silently dropped by the conversion.
func (*Backend) ConsumeProfiles ¶
ConsumeProfiles ingests an OTLP profiles batch into the storage engine. It is the profiles ingestion sink used when the storage backend serves profiles.
func (*Backend) ConsumeTraces ¶
ConsumeTraces ingests an OTLP traces batch into the storage engine. It is the traces ingestion sink used when the storage backend serves traces.
func (*Backend) ExemplarQuerier ¶
func (b *Backend) ExemplarQuerier(context.Context) (promstorage.ExemplarQuerier, error)
ExemplarQuerier implements storage.ExemplarQueryable. The storage engine does not store exemplars yet, so this returns an empty querier.
func (*Backend) Logs ¶
func (b *Backend) Logs() *LogQuerier
Logs returns the logs querier over the backend's storage engine.
func (*Backend) MetricMetadata ¶
func (b *Backend) MetricMetadata(context.Context, metricstorage.MetadataParams) (metricstorage.Metadata, error)
MetricMetadata implements metricstorage.MetadataQuerier. The storage engine does not expose metric metadata yet, so this returns an empty set.
func (*Backend) MetricsScanners ¶
func (b *Backend) MetricsScanners() (enginestorage.Scanners, error)
MetricsScanners implements the oteldb PromQL engine's scanner seam.
func (*Backend) Profiles ¶
func (b *Backend) Profiles() *ProfileQuerier
Profiles returns the profiles querier over the backend's storage engine.
func (*Backend) Querier ¶
func (b *Backend) Querier(mint, maxt int64) (promstorage.Querier, error)
Querier implements storage.Queryable.
func (*Backend) Traces ¶
func (b *Backend) Traces() *TraceQuerier
Traces returns the traces querier over the backend's storage engine.
type LogQLOptimizer ¶
type LogQLOptimizer struct{}
LogQLOptimizer offloads LogQL line filters to the storage fetch layer for the embedded backend.
The leading positive substring filters (`|= "x"`) of a pipeline are pushed into the log fetch as body conditions, so the storage drops non-matching records before they are materialized into entries with label sets — the expensive part of the scan. The conditions carry an exact contains Match, so the result is identical to evaluating the filter in the engine (the engine still applies the full pipeline on the surviving entries, so the offload only ever skips work).
It deliberately does not set a bloom token hint: the body bloom is word-tokenized, so a token hint would wrongly prune a part that holds a sub-word substring match (e.g. `|= "GET"` matching "xGETy"), violating the fetch contract's no-false-negatives guarantee. The offload is therefore a precise per-record prefilter, not a part-pruning hint.
Add it to a LogQL engine's optimizers when the querier is a storage LogQuerier; it is a no-op for any other node type.
func (*LogQLOptimizer) Name ¶
func (*LogQLOptimizer) Name() string
Name implements logqlengine.Optimizer.
func (*LogQLOptimizer) Optimize ¶
func (o *LogQLOptimizer) Optimize(_ context.Context, q logqlengine.Query) (logqlengine.Query, error)
Optimize implements logqlengine.Optimizer.
type LogQuerier ¶
type LogQuerier struct {
// contains filtered or unexported fields
}
LogQuerier adapts the storage engine to oteldb's logs query interfaces (logstorage.Querier and logqlengine.Querier).
func (*LogQuerier) Capabilities ¶
func (q *LogQuerier) Capabilities() (caps logqlengine.QuerierCapabilities)
Capabilities implements logqlengine.Querier. The storage backend does not push any pipeline filtering down, so it advertises no supported ops; the LogQL engine applies the whole pipeline (line filters, parsers, label filters) on top of the raw entry stream this backend returns.
func (*LogQuerier) DetectedFields ¶
func (q *LogQuerier) DetectedFields(ctx context.Context, opts logstorage.LabelsOptions) ([]logstorage.DetectedField, error)
DetectedFields implements logstorage.Querier. The storage backend does not parse record fields, so it reports the stream labels as string fields with their value cardinality.
func (*LogQuerier) DetectedLabels ¶
func (q *LogQuerier) DetectedLabels(ctx context.Context, opts logstorage.LabelsOptions) ([]logstorage.DetectedLabel, error)
DetectedLabels implements logstorage.Querier. It returns the cardinality of each stream label.
func (*LogQuerier) LabelNames ¶
func (q *LogQuerier) LabelNames(ctx context.Context, opts logstorage.LabelsOptions) ([]string, error)
LabelNames implements logstorage.Querier. It returns the distinct label names of the streams matching the options' selector.
func (*LogQuerier) LabelValues ¶
func (q *LogQuerier) LabelValues(ctx context.Context, labelName string, opts logstorage.LabelsOptions) (iterators.Iterator[logstorage.Label], error)
LabelValues implements logstorage.Querier. It returns the distinct values of labelName across the streams matching the options' selector.
func (*LogQuerier) Query ¶
func (q *LogQuerier) Query(_ context.Context, selector []logql.LabelMatcher) (logqlengine.PipelineNode, error)
Query implements logqlengine.Querier. It returns a node that streams the entries of the streams matching selector; the engine wraps it to evaluate the rest of the pipeline.
func (*LogQuerier) Series ¶
func (q *LogQuerier) Series(ctx context.Context, opts logstorage.SeriesOptions) (logstorage.Series, error)
Series implements logstorage.Querier. It returns the label sets of the streams matching any of the option selectors.
type Option ¶
type Option func(*Backend)
Option configures a Backend.
func WithLogParallelism ¶
WithLogParallelism enables concurrent materialization of LogQL query results across up to n workers. The fetched record set is split into contiguous chunks built in parallel and merged in order, so the result is identical to the sequential path regardless of scheduling. Opt-in: n <= 1 (the default) keeps the sequential path. Effective only above an internal record-count threshold.
func WithOverTimePushdown ¶ added in v0.42.0
WithOverTimePushdown toggles the instant *_over_time aggregate pushdown. It is on by default (the sidecar path is faster and correct); passing false restores the raw matrix-selector path (useful as a fallback or for differential testing).
type ProfileQuerier ¶
type ProfileQuerier struct {
// contains filtered or unexported fields
}
ProfileQuerier adapts the storage engine to oteldb's profiles query interface (profilestorage.Querier).
func (*ProfileQuerier) LabelNames ¶
func (q *ProfileQuerier) LabelNames(ctx context.Context, opts profilestorage.LabelNamesOptions) ([]string, error)
LabelNames implements profilestorage.Querier. It returns the distinct user-label names of the streams matching the selector, excluding the reserved profile-type labels.
func (*ProfileQuerier) LabelValues ¶
func (q *ProfileQuerier) LabelValues(ctx context.Context, label string, opts profilestorage.LabelValuesOptions) ([]string, error)
LabelValues implements profilestorage.Querier. It returns the distinct values the given label takes across the streams matching the selector.
func (*ProfileQuerier) ProfileTypes ¶
func (q *ProfileQuerier) ProfileTypes(ctx context.Context, opts profilestorage.ProfileTypesOptions) ([]profileql.ProfileType, error)
ProfileTypes implements profilestorage.Querier. It enumerates the distinct profile types of the tenant's streams in the window, reading the type out of each series' reserved labels.
func (*ProfileQuerier) SelectMergeProfile ¶
func (q *ProfileQuerier) SelectMergeProfile(ctx context.Context, params profilestorage.SelectProfileParams) (*profilestorage.FlameTree, error)
SelectMergeProfile implements profilestorage.Querier. It fetches every matching sample, resolves each sample's content-addressed stack to function frames, and merges them into a single flamegraph tree.
type TraceQuerier ¶
type TraceQuerier struct {
// contains filtered or unexported fields
}
TraceQuerier adapts the storage engine to oteldb's traces query interfaces (tracestorage.Querier and traceqlengine.Querier).
func (*TraceQuerier) SearchTags ¶
func (q *TraceQuerier) SearchTags(ctx context.Context, tags map[string]string, opts tracestorage.SearchTagsOptions) (iterators.Iterator[tracestorage.Span], error)
SearchTags implements tracestorage.Querier. It returns the spans whose attributes match every requested tag and whose duration is within the optional bounds.
func (*TraceQuerier) SelectSpansets ¶
func (q *TraceQuerier) SelectSpansets(ctx context.Context, params traceqlengine.SelectSpansetsParams) (iterators.Iterator[traceqlengine.Trace], error)
SelectSpansets implements traceqlengine.Querier. It returns every trace whose spans fall in the window, grouped by trace id; the TraceQL engine evaluates the spanset matchers on the result (mirroring the in-memory reference querier).
func (*TraceQuerier) TagNames ¶
func (q *TraceQuerier) TagNames(ctx context.Context, opts tracestorage.TagNamesOptions) ([]tracestorage.TagName, error)
TagNames implements tracestorage.Querier. It enumerates the distinct attribute names seen on the spans in the window, restricted to the requested scope.
func (*TraceQuerier) TagValues ¶
func (q *TraceQuerier) TagValues(ctx context.Context, attr traceql.Attribute, opts tracestorage.TagValuesOptions) (iterators.Iterator[tracestorage.Tag], error)
TagValues implements tracestorage.Querier. It enumerates the distinct values the attribute takes across the spans in the window.
func (*TraceQuerier) TraceByID ¶
func (q *TraceQuerier) TraceByID(ctx context.Context, id otelstorage.TraceID, _ tracestorage.TraceByIDOptions) (iterators.Iterator[tracestorage.Span], error)
TraceByID implements tracestorage.Querier. It fetches every span of one trace by id.
Source Files