Documentation
¶
Overview ¶
Package logical provides a logical query plan representation for data processing operations.
The logical plan is represented using static single-assignment (SSA) form of intermediate representation (IR) for the operations performed on log data.
For an introduction to SSA form, see https://en.wikipedia.org/wiki/Static_single_assignment_form.
The primary interfaces of this package are:
- Value, an expression that yields a value. - Instruction, a statement that consumes values and performs computation. - Plan, a sequence of instructions that produces a result.
A computation that also yields a result implements both the Value and Instruction interfaces. See the documentation comments on each type for which of those interfaces it implements.
Values are representable as either:
- A column value (such as in ColumnRef), - a relation (such as in Select), or - a value literal (such as in Literal).
The SSA form forms a graph: each Value may appear as an operand of one or more [Instruction]s.
Index ¶
- func PrintTree(w io.StringWriter, value Value)
- func WriteMermaidFormat(w io.Writer, p *Plan)
- type BinOp
- type Builder
- func (b *Builder) Cast(identifier string, operation types.UnaryOp) *Builder
- func (b *Builder) Compat(logqlCompatibility bool) *Builder
- func (b *Builder) Limit(skip uint32, fetch uint32) *Builder
- func (b *Builder) Parse(kind ParserKind) *Builder
- func (b *Builder) Project(all, expand, drop bool, expr ...Value) *Builder
- func (b *Builder) ProjectAll(expand, drop bool, expr ...Value) *Builder
- func (b *Builder) ProjectDrop(expr ...Value) *Builder
- func (b *Builder) ProjectExpand(expr ...Value) *Builder
- func (b *Builder) RangeAggregation(partitionBy []ColumnRef, operation types.RangeAggregationType, ...) *Builder
- func (b *Builder) Select(predicate Value) *Builder
- func (b *Builder) Sort(column ColumnRef, ascending, nullsFirst bool) *Builder
- func (b *Builder) ToPlan() (*Plan, error)
- func (b *Builder) Value() Value
- func (b *Builder) VectorAggregation(groupBy []ColumnRef, operation types.VectorAggregationType) *Builder
- type ColumnRef
- type Instruction
- type Limit
- type Literal
- type LogQLCompat
- type MakeTable
- type Parse
- type ParserKind
- type Plan
- type Projection
- type RangeAggregation
- type Return
- type Select
- type ShardInfo
- type Sort
- type UnaryOp
- type Value
- type VectorAggregation
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func PrintTree ¶
func PrintTree(w io.StringWriter, value Value)
PrintTree prints the given value and its dependencies as a tree structure to w.
func WriteMermaidFormat ¶
Types ¶
type BinOp ¶
The BinOp instruction yields the result of binary operation Left Op Right. BinOp implements both Instruction and Value.
type Builder ¶
type Builder struct {
// contains filtered or unexported fields
}
Builder provides an ergonomic interface for constructing a Plan.
func NewBuilder ¶
NewBuilder creates a new Builder from a Value, where the starting Value is usually a MakeTable.
func (*Builder) Cast ¶
Cast applies an Projection operation, with an UnaryOp cast operation, to the Builder.
func (*Builder) Compat ¶
Compat applies a LogQLCompat operation to the Builder, which is a marker to ensure v1 engine compatible results.
func (*Builder) Parse ¶
func (b *Builder) Parse(kind ParserKind) *Builder
Parse applies a Parse operation to the Builder.
func (*Builder) ProjectAll ¶
func (*Builder) ProjectDrop ¶
func (*Builder) ProjectExpand ¶
func (*Builder) RangeAggregation ¶
func (b *Builder) RangeAggregation( partitionBy []ColumnRef, operation types.RangeAggregationType, startTS, endTS time.Time, step time.Duration, rangeInterval time.Duration, ) *Builder
RangeAggregation applies a RangeAggregation operation to the Builder.
func (*Builder) Value ¶
Value returns the underlying Value. This is useful when you need to access the value directly, such as when passing it to a function that operates on values rather than a Builder.
func (*Builder) VectorAggregation ¶
func (b *Builder) VectorAggregation( groupBy []ColumnRef, operation types.VectorAggregationType, ) *Builder
VectorAggregation applies a VectorAggregation operation to the Builder.
type ColumnRef ¶
A ColumnRef referenes a column within a table relation. ColumnRef only implements Value.
func NewColumnRef ¶
func NewColumnRef(name string, ty types.ColumnType) *ColumnRef
type Instruction ¶
type Instruction interface {
// String returns the disassembled SSA form of the Instruction. This does not
// include the name of the Value if the Instruction also implements [Value].
String() string
// contains filtered or unexported methods
}
An Instruction is an SSA instruction that computes a new Value or has some effect.
Instructions that define a value (e.g., BinOp) also implement the Value interface; an Instruction that only has an effect (e.g., Return) does not.
type Limit ¶
type Limit struct {
Table Value // Table relation to limit.
// Skip is the number of rows to skip before returning results. A value of 0
// means no rows are skipped.
Skip uint32
// Fetch is the maximum number of rows to return. A value of 0 means all rows
// are returned (after applying Skip).
Fetch uint32
// contains filtered or unexported fields
}
The Limit instruction limits the number of rows from a table relation. Limit implements Instruction and Value.
type Literal ¶
A Literal represents a literal value known at plan time. Literal only implements Value.
The zero value of a Literal is a NULL value.
func NewLiteral ¶
func NewLiteral(value types.LiteralType) *Literal
type LogQLCompat ¶
type LogQLCompat struct {
Value Value
// contains filtered or unexported fields
}
The LOGQL_COMPAT instruction is a marker to indicate v1 engine compatibility. LogQLCompat implements Instruction and Value.
func (*LogQLCompat) Name ¶
func (c *LogQLCompat) Name() string
func (*LogQLCompat) String ¶
func (c *LogQLCompat) String() string
String returns the disassembled SSA form of r.
type MakeTable ¶
type MakeTable struct {
// Selector is used to generate a table relation. All streams for which the
// selector passes are included in the resulting table.
//
// It is invalid for Selector to include a [ColumnRef] that is not
// [ColumnTypeBuiltin] or [ColumnTypeLabel].
Selector Value
// Predicates are used to further filter the Selector table relation by utilising additional indexes in the catalogue.
// Unlike the Selector, there are no restrictions on the type of [ColumnRef] in Predicates.
Predicates []Value
// Shard is used to indicate that the table relation does not contain all data
// of the relation but only a subset of it.
// The Shard value must be of type [ShardRef].
Shard Value
// contains filtered or unexported fields
}
The MakeTable instruction yields a table relation from an identifier. MakeTable implements both Instruction and Value.
type Parse ¶
type Parse struct {
Table Value // The table relation to parse from
Kind ParserKind
// contains filtered or unexported fields
}
Parse represents a parsing instruction that extracts fields from log lines. It takes a table relation as input and produces a new table relation with additional columns for the parsed fields.
type ParserKind ¶
type ParserKind int
ParserKind represents the type of parser to use
const ( ParserInvalid ParserKind = iota ParserLogfmt ParserJSON )
func (ParserKind) String ¶
func (p ParserKind) String() string
type Plan ¶
type Plan struct {
Instructions []Instruction // Instructions of the plan in order.
}
A Plan represents a sequence of [Instruction]s that ultimately produce a Value.
The first Return instruction in the plan denotes the final output.
func BuildPlan ¶
BuildPlan converts a LogQL query represented as logql.Params into a logical Plan. It may return an error as second argument in case the traversal of the AST of the query fails.
type Projection ¶
type Projection struct {
Relation Value // The input relation.
Expressions []Value // The expressions to apply for projecting columns.
All bool // Marker for projecting all columns of input relation (similar to SQL `SELECT *`)
Expand bool // Indicates that projected columns should be added to input relation
Drop bool // Indicates that projected columns should be dropped from input Relation
// contains filtered or unexported fields
}
The Projection instruction projects (keeps/drops) columns from a relation. Projection implements both Instruction and Value.
func (*Projection) Name ¶
func (p *Projection) Name() string
Name returns an identifier for the Projection operation.
func (*Projection) String ¶
func (p *Projection) String() string
String returns the disassembled SSA form of the Projection instruction.
type RangeAggregation ¶
type RangeAggregation struct {
Table Value // The table relation to aggregate.
PartitionBy []ColumnRef // The columns to partition by.
Operation types.RangeAggregationType // The type of aggregation operation to perform.
Start time.Time
End time.Time
Step time.Duration
RangeInterval time.Duration
// contains filtered or unexported fields
}
RangeAggregation represents a logical plan node that performs aggregations over a time window. It is similar to window functions in SQL with a few important distinctions: 1. It evaluates the aggregation at step intervals unlike traditional window functions which are evaluated for each row. 2. It uses a time window defined by query [$range]. 3. It partitions by query-time streams if no partition by is specified.
func (*RangeAggregation) Name ¶
func (r *RangeAggregation) Name() string
Name returns an identifier for the RangeAggregation operation.
func (*RangeAggregation) String ¶
func (r *RangeAggregation) String() string
String returns the disassembled SSA form of the RangeAggregation instruction.
type Return ¶
type Return struct {
Value Value // The value to return.
}
The Return instruction yields a value to return from a plan. Return implements Instruction.
type Select ¶
type Select struct {
Table Value // The table relation to filter.
// Predicate is used to filter rows from Table. Each row is checked against
// the given Predicate, and only rows for which the Predicate is true are
// returned.
Predicate Value
// contains filtered or unexported fields
}
The Select instruction filters rows from a table relation. Select implements both Instruction and Value.
type ShardInfo ¶
type ShardInfo struct {
Shard uint32
Of uint32 // MUST be a power of 2 to ensure sharding logic works correctly.
}
A ShardInfo defines a subset of a table relation. ShardInfo only implements Value. It is the equivalent to the [index.ShardAnnotation] in the old query engine.
type Sort ¶
type Sort struct {
Table Value // The table relation to sort.
Column ColumnRef // The column to sort by.
Ascending bool // Whether to sort in ascending order.
NullsFirst bool // Controls whether NULLs appear first (true) or last (false).
// contains filtered or unexported fields
}
The Sort instruction sorts rows from a table relation. Sort implements both Instruction and Value.
type UnaryOp ¶
The UnaryOp instruction yields the result of unary operation Op Value. UnaryOp implements both Instruction and Value.
type Value ¶
type Value interface {
// Name returns an identifier for this Value (such as "%1"), which is used
// when this Value appears as an operand of an Instruction.
//
// If the Value was not created by the logical planner, Name instead returns
// the pointer address of the Value.
Name() string
// String returns human-readable information about the Value. If Value also
// implements [Instruction], String returns the disassembled form of the
// Instruction as documented by [Instruction.String].
String() string
// contains filtered or unexported methods
}
A Value is an SSA value that can be referenced by an Instruction.
type VectorAggregation ¶
type VectorAggregation struct {
Table Value // The table relation to aggregate.
// The columns to group by. If empty, all rows are aggregated into a single result.
GroupBy []ColumnRef
// The type of aggregation operation to perform (e.g., sum, min, max)
Operation types.VectorAggregationType
// contains filtered or unexported fields
}
VectorAggregation represents a logical plan node that performs vector aggregations. It computes aggregations over time series data at each timestamp instant grouping results by specified dimensions.
func (*VectorAggregation) Name ¶
func (v *VectorAggregation) Name() string
Name returns an identifier for the VectorAggregation operation.
func (*VectorAggregation) String ¶
func (v *VectorAggregation) String() string
String returns the disassembled SSA form of the VectorAggregation instruction.
Source Files
¶
- builder.go
- builder_convert.go
- column_ref.go
- format_tree.go
- logical.go
- node_binop.go
- node_limit.go
- node_literal.go
- node_logql_compat.go
- node_maketable.go
- node_parse.go
- node_project.go
- node_range_aggregate.go
- node_return.go
- node_select.go
- node_sort.go
- node_unaryop.go
- node_vector_aggregate.go
- planner.go
- printer.go
- shard_ref.go