Documentation
¶
Overview ¶
Package logical provides a logical query plan representation for data processing operations.
The logical plan is represented using static single-assignment (SSA) form of intermediate representation (IR) for the operations performed on log data.
For an introduction to SSA form, see https://en.wikipedia.org/wiki/Static_single_assignment_form.
The primary interfaces of this package are:
- Value, an expression that yields a value. - Instruction, a statement that consumes values and performs computation. - Plan, a sequence of instructions that produces a result.
A computation that also yields a result implements both the Value and Instruction interfaces. See the documentation comments on each type for which of those interfaces it implements.
Values are representable as either:
- A column value (such as in ColumnRef), - a relation (such as in Select), or - a value literal (such as in Literal).
The SSA form forms a graph: each Value may appear as an operand of one or more [Instruction]s.
Index ¶
- Variables
- func Optimize(p *Plan) error
- func PrintTree(w io.StringWriter, value Value)
- func WriteMermaidFormat(w io.Writer, p *Plan)
- type BinOp
- type Builder
- func (b *Builder) BinOpLeft(op types.BinaryOp, left Value) *Builder
- func (b *Builder) BinOpRight(op types.BinaryOp, right Value) *Builder
- func (b *Builder) Cast(identifier string, op types.UnaryOp) *Builder
- func (b *Builder) Compat(logqlCompatibility bool) *Builder
- func (b *Builder) Limit(skip uint32, fetch uint32) *Builder
- func (b *Builder) Parse(op types.VariadicOp, strict bool, keepEmpty bool) *Builder
- func (b *Builder) ParseRegexp(pattern string) *Builder
- func (b *Builder) Project(all, expand, drop bool, expr ...Value) *Builder
- func (b *Builder) ProjectAll(expand, drop bool, expr ...Value) *Builder
- func (b *Builder) ProjectDrop(expr ...Value) *Builder
- func (b *Builder) ProjectExpand(expr ...Value) *Builder
- func (b *Builder) RangeAggregation(grouping Grouping, operation types.RangeAggregationType, ...) *Builder
- func (b *Builder) Select(predicate Value) *Builder
- func (b *Builder) Sort(column ColumnRef, ascending, nullsFirst bool) *Builder
- func (b *Builder) ToPlan() (*Plan, error)
- func (b *Builder) TopK(sortBy *ColumnRef, K int, ascending, nullsFirst bool) *Builder
- func (b *Builder) Value() Value
- func (b *Builder) VectorAggregation(grouping Grouping, operation types.VectorAggregationType) *Builder
- type ColumnRef
- type FunctionOp
- type Grouping
- type Instruction
- type Limit
- type Literal
- type LogQLCompat
- type MakeTable
- type Node
- type Plan
- type Projection
- type RangeAggregation
- type Return
- type Select
- type ShardInfo
- type Sort
- type TopK
- type UnaryOp
- type Value
- type VectorAggregation
Constants ¶
This section is empty.
Variables ¶
var (
NoGrouping = Grouping{Without: true}
)
Functions ¶
func PrintTree ¶
func PrintTree(w io.StringWriter, value Value)
PrintTree prints the given value and its dependencies as a tree structure to w.
func WriteMermaidFormat ¶
Types ¶
type BinOp ¶
The BinOp instruction yields the result of binary operation Left Op Right. BinOp implements both Instruction and Value.
func (*BinOp) Operands ¶ added in v3.7.0
Operands appends the operands of b to the provided slice. The pointers may be modified to change operands of b.
func (*BinOp) Referrers ¶ added in v3.7.0
func (b *BinOp) Referrers() *[]Instruction
Referrers returns a list of instructions that reference the BinOp.
The list of instructions can be modified to update the reference list, such as when modifying the plan.
type Builder ¶
type Builder struct {
// contains filtered or unexported fields
}
Builder provides an ergonomic interface for constructing a Plan.
func NewBuilder ¶
NewBuilder creates a new Builder from a Value, where the starting Value is usually a MakeTable.
func (*Builder) BinOpLeft ¶ added in v3.7.0
BinOpLeft adds a binary arithmetic operation with a given left value
func (*Builder) BinOpRight ¶ added in v3.7.0
BinOpRight adds a binary arithmetic operation with a given right value
func (*Builder) Cast ¶
Cast applies an Projection operation, with an UnaryOp cast operation, to the Builder.
func (*Builder) Compat ¶
Compat applies a LogQLCompat operation to the Builder, which is a marker to ensure v1 engine compatible results.
func (*Builder) ParseRegexp ¶ added in v3.7.0
ParseRegexp applies a regex [Parse] operation to the Builder. The pattern must contain at least one named capture group like (?P<name>...).
func (*Builder) ProjectAll ¶
func (*Builder) ProjectDrop ¶
func (*Builder) ProjectExpand ¶
func (*Builder) RangeAggregation ¶
func (b *Builder) RangeAggregation( grouping Grouping, operation types.RangeAggregationType, startTS, endTS time.Time, step time.Duration, rangeInterval time.Duration, ) *Builder
RangeAggregation applies a RangeAggregation operation to the Builder.
func (*Builder) Value ¶
Value returns the underlying Value. This is useful when you need to access the value directly, such as when passing it to a function that operates on values rather than a Builder.
func (*Builder) VectorAggregation ¶
func (b *Builder) VectorAggregation( grouping Grouping, operation types.VectorAggregationType, ) *Builder
VectorAggregation applies a VectorAggregation operation to the Builder.
type ColumnRef ¶
A ColumnRef referenes a column within a table relation. ColumnRef only implements Value.
func NewColumnRef ¶
func NewColumnRef(name string, ty types.ColumnType) *ColumnRef
func (*ColumnRef) Name ¶
Name returns the identifier of the ColumnRef, which combines the column type and column name being referenced.
func (*ColumnRef) Referrers ¶ added in v3.7.0
func (c *ColumnRef) Referrers() *[]Instruction
Referrers returns a list of instructions that reference c.
The list of instructions can be modified to update the reference list, such as when modifying the plan.
type FunctionOp ¶ added in v3.7.0
type FunctionOp struct {
Op types.VariadicOp
Values []Value
// contains filtered or unexported fields
}
The FunctionOp instruction yields the result of function operation Op Value. UnaryOp implements both Instruction and Value.
func (*FunctionOp) Name ¶ added in v3.7.0
func (u *FunctionOp) Name() string
Name returns an identifier for the UnaryOp operation.
func (*FunctionOp) Operands ¶ added in v3.7.0
func (u *FunctionOp) Operands(buf []*Value) []*Value
Operands appends the operands of u to the provided slice. The pointers may be modified to change operands of u.
func (*FunctionOp) Referrers ¶ added in v3.7.0
func (u *FunctionOp) Referrers() *[]Instruction
Referrers returns a list of instructions that reference the FunctionOp.
The list of instructions can be modified to update the reference list, such as when modifying the plan.
func (*FunctionOp) String ¶ added in v3.7.0
func (u *FunctionOp) String() string
String returns the disassembled SSA form of the FunctionOp instruction.
type Grouping ¶ added in v3.7.0
type Grouping struct {
Columns []ColumnRef // The columns for grouping
Without bool // The grouping mode
}
Grouping represents the grouping by/without label(s) for vector aggregators and range vector aggregators.
type Instruction ¶
type Instruction interface {
Node
// String returns the disassembled SSA form of the Instruction. This does not
// include the name of the Value if the Instruction also implements [Value].
String() string
// Operands appends Instruction's operands to buf and returns the resulting
// slice.
//
// Operands are represented as pointers to permit updating an Instruction to
// use different operands.
Operands(buf []*Value) []*Value
// contains filtered or unexported methods
}
An Instruction is an SSA instruction that computes a new Value or has some effect.
Instructions that define a value (e.g., BinOp) also implement the Value interface; an Instruction that only has an effect (e.g., Return) does not.
type Limit ¶
type Limit struct {
Table Value // Table relation to limit.
// Skip is the number of rows to skip before returning results. A value of 0
// means no rows are skipped.
Skip uint32
// Fetch is the maximum number of rows to return. A value of 0 means all rows
// are returned (after applying Skip).
Fetch uint32
// contains filtered or unexported fields
}
The Limit instruction limits the number of rows from a table relation. Limit implements Instruction and Value.
func (*Limit) Operands ¶ added in v3.7.0
Operands appends the operands of l to the provided slice. The pointers may be modified to change operands of l.
func (*Limit) Referrers ¶ added in v3.7.0
func (l *Limit) Referrers() *[]Instruction
Referrers returns a list of instructions that reference the Limit.
The list of instructions can be modified to update the reference list, such as when modifying the plan.
type Literal ¶
type Literal struct {
// contains filtered or unexported fields
}
A Literal represents a literal value known at plan time. Literal only implements Value.
The zero value of a Literal is a NULL value.
func NewLiteral ¶
func (*Literal) Referrers ¶ added in v3.7.0
func (l *Literal) Referrers() *[]Instruction
Referrers returns a list of instructions that reference the Literal.
The list of instructions can be modified to update the reference list, such as when modifying the plan.
type LogQLCompat ¶
type LogQLCompat struct {
Value Value
// contains filtered or unexported fields
}
The LOGQL_COMPAT instruction is a marker to indicate v1 engine compatibility. LogQLCompat implements Instruction and Value.
func (*LogQLCompat) Name ¶
func (c *LogQLCompat) Name() string
func (*LogQLCompat) Operands ¶ added in v3.7.0
func (c *LogQLCompat) Operands(buf []*Value) []*Value
Operands appends the operands of c to the provided slice. The pointers may be modified to change operands of c.
func (*LogQLCompat) Referrers ¶ added in v3.7.0
func (c *LogQLCompat) Referrers() *[]Instruction
Referrers returns a list of instructions that reference the LogQLCompat.
The list of instructions can be modified to update the reference list, such as when modifying the plan.
func (*LogQLCompat) String ¶
func (c *LogQLCompat) String() string
String returns the disassembled SSA form of r.
type MakeTable ¶
type MakeTable struct {
// Selector is used to generate a table relation. All streams for which the
// selector passes are included in the resulting table.
//
// It is invalid for Selector to include a [ColumnRef] that is not
// [ColumnTypeBuiltin] or [ColumnTypeLabel].
Selector Value
// Predicates are used to further filter the Selector table relation by utilising additional indexes in the catalogue.
// Unlike the Selector, there are no restrictions on the type of [ColumnRef] in Predicates.
Predicates []Value
// Shard is used to indicate that the table relation does not contain all data
// of the relation but only a subset of it.
// The Shard value must be of type [ShardRef].
Shard Value
// contains filtered or unexported fields
}
The MakeTable instruction yields a table relation from an identifier. MakeTable implements both Instruction and Value.
func (*MakeTable) Operands ¶ added in v3.7.0
Operands appends the operands of t to the provided slice. The pointers may be modified to change operands of t.
func (*MakeTable) Referrers ¶ added in v3.7.0
func (t *MakeTable) Referrers() *[]Instruction
Referrers returns a list of instructions that reference the MakeTable.
The list of instructions can be modified to update the reference list, such as when modifying the plan.
type Node ¶ added in v3.7.0
type Node interface {
String() string
// contains filtered or unexported methods
}
Node is a node in the logical plan. Every type that implements Node is either a Value, Instruction, or both.
Node contains the common methods to Value and Instruction.
type Plan ¶
type Plan struct {
Instructions []Instruction // Instructions of the plan in order.
}
A Plan represents a sequence of [Instruction]s that ultimately produce a Value.
The first Return instruction in the plan denotes the final output.
func BuildPlan ¶
BuildPlan converts a LogQL query represented as logql.Params into a logical Plan. It may return an error as second argument in case the traversal of the AST of the query fails.
func BuildPlanWithDeletes ¶ added in v3.7.0
type Projection ¶
type Projection struct {
Relation Value // The input relation.
Expressions []Value // The expressions to apply for projecting columns.
All bool // Marker for projecting all columns of input relation (similar to SQL `SELECT *`)
Expand bool // Indicates that projected columns should be added to input relation
Drop bool // Indicates that projected columns should be dropped from input Relation
// contains filtered or unexported fields
}
The Projection instruction projects (keeps/drops) columns from a relation. Projection implements both Instruction and Value.
func (*Projection) Name ¶
func (p *Projection) Name() string
Name returns an identifier for the Projection operation.
func (*Projection) Operands ¶ added in v3.7.0
func (p *Projection) Operands(buf []*Value) []*Value
Operands appends the operands of p to the provided slice. The pointers may be modified to change operands of p.
func (*Projection) Referrers ¶ added in v3.7.0
func (p *Projection) Referrers() *[]Instruction
Referrers returns a list of instructions that reference the Projection.
The list of instructions can be modified to update the reference list, such as when modifying the plan.
func (*Projection) String ¶
func (p *Projection) String() string
String returns the disassembled SSA form of the Projection instruction.
type RangeAggregation ¶
type RangeAggregation struct {
Table Value // The table relation to aggregate.
Grouping Grouping // The grouping
Operation types.RangeAggregationType // The type of aggregation operation to perform.
Start time.Time
End time.Time
Step time.Duration
RangeInterval time.Duration
// contains filtered or unexported fields
}
RangeAggregation represents a logical plan node that performs aggregations over a time window. It is similar to window functions in SQL with a few important distinctions: 1. It evaluates the aggregation at step intervals unlike traditional window functions which are evaluated for each row. 2. It uses a time window defined by query [$range].
func (*RangeAggregation) Name ¶
func (r *RangeAggregation) Name() string
Name returns an identifier for the RangeAggregation operation.
func (*RangeAggregation) Operands ¶ added in v3.7.0
func (r *RangeAggregation) Operands(buf []*Value) []*Value
Operands appends the operands of r to the provided slice. The pointers may be modified to change operands of r.
func (*RangeAggregation) Referrers ¶ added in v3.7.0
func (r *RangeAggregation) Referrers() *[]Instruction
Referrers returns a list of instructions that reference the RangeAggregation.
The list of instructions can be modified to update the reference list, such as when modifying the plan.
func (*RangeAggregation) String ¶
func (r *RangeAggregation) String() string
String returns the disassembled SSA form of the RangeAggregation instruction.
type Return ¶
type Return struct {
Value Value // The value to return.
// contains filtered or unexported fields
}
The Return instruction yields a value to return from a plan. Return implements Instruction.
type Select ¶
type Select struct {
Table Value // The table relation to filter.
// Predicate is used to filter rows from Table. Each row is checked against
// the given Predicate, and only rows for which the Predicate is true are
// returned.
Predicate Value
// contains filtered or unexported fields
}
The Select instruction filters rows from a table relation. Select implements both Instruction and Value.
func (*Select) Operands ¶ added in v3.7.0
Operands appends the operands of s to the provided slice. The pointers may be modified to change operands of s.
func (*Select) Referrers ¶ added in v3.7.0
func (s *Select) Referrers() *[]Instruction
Referrers returns a list of instructions that reference the Select.
The list of instructions can be modified to update the reference list, such as when modifying the plan.
type ShardInfo ¶
type ShardInfo struct {
Shard uint32
Of uint32 // MUST be a power of 2 to ensure sharding logic works correctly.
// contains filtered or unexported fields
}
A ShardInfo defines a subset of a table relation. ShardInfo only implements Value. It is the equivalent to the [index.ShardAnnotation] in the old query engine.
func (*ShardInfo) Referrers ¶ added in v3.7.0
func (s *ShardInfo) Referrers() *[]Instruction
Referrers returns a list of instructions that reference the ShardInfo.
The list of instructions can be modified to update the reference list, such as when modifying the plan.
type Sort ¶
type Sort struct {
Table Value // The table relation to sort.
Column ColumnRef // The column to sort by.
Ascending bool // Whether to sort in ascending order.
NullsFirst bool // Controls whether NULLs appear first (true) or last (false).
// contains filtered or unexported fields
}
The Sort instruction sorts rows from a table relation. Sort implements both Instruction and Value.
func (*Sort) Operands ¶ added in v3.7.0
Operands appends the operands of s to the provided slice. The pointers may be modified to change operands of s.
func (*Sort) Referrers ¶ added in v3.7.0
func (s *Sort) Referrers() *[]Instruction
Referrers returns a list of instructions that reference the Sort.
The list of instructions can be modified to update the reference list, such as when modifying the plan.
type TopK ¶ added in v3.7.0
type TopK struct {
Table Value // The table relation to sort.
SortBy *ColumnRef // The column to sort by.
Ascending bool // Whether to sort in ascending order.
NullsFirst bool // Controls whether NULLs appear first (true) or last (false).
K int // Number of top rows to return.
// contains filtered or unexported fields
}
The TopK instruction find the top K rows from a table relation. TopK implements both Instruction and Value.
func (*TopK) Operands ¶ added in v3.7.0
Operands appends the operands of s to the provided slice. The pointers may be modified to change operands of s.
func (*TopK) Referrers ¶ added in v3.7.0
func (s *TopK) Referrers() *[]Instruction
Referrers returns a list of instructions that reference the TopK.
The list of instructions can be modified to update the reference list, such as when modifying the plan.
type UnaryOp ¶
The UnaryOp instruction yields the result of unary operation Op Value. UnaryOp implements both Instruction and Value.
func (*UnaryOp) Operands ¶ added in v3.7.0
Operands appends the operands of u to the provided slice. The pointers may be modified to change operands of u.
func (*UnaryOp) Referrers ¶ added in v3.7.0
func (u *UnaryOp) Referrers() *[]Instruction
Referrers returns a list of instructions that reference the UnaryOp.
The list of instructions can be modified to update the reference list, such as when modifying the plan.
type Value ¶
type Value interface {
Node
// Name returns an identifier for this Value (such as "%1"), which is used
// when this Value appears as an operand of an Instruction.
//
// If the Value was not created by the logical planner, Name instead returns
// the pointer address of the Value.
Name() string
// String returns human-readable information about the Value. If Value also
// implements [Instruction], String returns the disassembled form of the
// Instruction as documented by [Instruction.String].
String() string
// Referrers returns a list of instructions that reference this Value.
//
// Referrers may be mutated to update the reference list. Care should be
// taken to only perform modifications that preserve the validity of the
// reference list.
Referrers() *[]Instruction
// contains filtered or unexported methods
}
A Value is an SSA value that can be referenced by an Instruction.
type VectorAggregation ¶
type VectorAggregation struct {
Table Value // The table relation to aggregate.
Grouping Grouping // The grouping
// The type of aggregation operation to perform (e.g., sum, min, max)
Operation types.VectorAggregationType
// contains filtered or unexported fields
}
VectorAggregation represents a logical plan node that performs vector aggregations. It computes aggregations over time series data at each timestamp instant grouping results by specified dimensions.
func (*VectorAggregation) Name ¶
func (v *VectorAggregation) Name() string
Name returns an identifier for the VectorAggregation operation.
func (*VectorAggregation) Operands ¶ added in v3.7.0
func (v *VectorAggregation) Operands(buf []*Value) []*Value
Operands appends the operands of v to the provided slice. The pointers may be modified to change operands of v.
func (*VectorAggregation) Referrers ¶ added in v3.7.0
func (v *VectorAggregation) Referrers() *[]Instruction
Referrers returns a list of instructions that reference the VectorAggregation.
The list of instructions can be modified to update the reference list, such as when modifying the plan.
func (*VectorAggregation) String ¶
func (v *VectorAggregation) String() string
String returns the disassembled SSA form of the VectorAggregation instruction.
Source Files
¶
- builder.go
- builder_convert.go
- builder_referrers.go
- column_ref.go
- format_tree.go
- function_op.go
- grouping.go
- logical.go
- logical_optimize.go
- node_base.go
- node_binop.go
- node_limit.go
- node_literal.go
- node_logql_compat.go
- node_maketable.go
- node_project.go
- node_range_aggregate.go
- node_return.go
- node_select.go
- node_sort.go
- node_topk.go
- node_unaryop.go
- node_vector_aggregate.go
- planner.go
- printer.go
- shard_ref.go
- walk.go