Documentation
¶
Index ¶
- Variables
- type Array
- type ArrayStruct
- type BinaryFunction
- type BinaryFunctionRegistry
- type BufferedPipeline
- type CoalesceVector
- type ColumnVector
- type Config
- type Context
- type GenericPipeline
- func NewFilterPipeline(filter *physical.Filter, input Pipeline, evaluator expressionEvaluator, ...) *GenericPipeline
- func NewLimitPipeline(input Pipeline, skip, fetch uint32) *GenericPipeline
- func NewParsePipeline(parse *physical.ParseNode, input Pipeline, allocator memory.Allocator) *GenericPipeline
- type Merge
- type Pipeline
- type Scalar
- type UnaryFunc
- type UnaryFunction
- type UnaryFunctionRegistry
Constants ¶
This section is empty.
Variables ¶
var (
EOF = errors.New("pipeline exhausted") //nolint:revive,staticcheck
)
Functions ¶
This section is empty.
Types ¶
type Array ¶
type Array struct {
// contains filtered or unexported fields
}
Array represents a column of data, stored as an arrow.Array.
func (*Array) ColumnType ¶
func (a *Array) ColumnType() types.ColumnType
ColumnType implements ColumnVector.
type ArrayStruct ¶
type ArrayStruct struct {
// contains filtered or unexported fields
}
ArrayStruct represents multiple columns of data, stored as an array.Struct. It implements the ColumnVector interface while preserving access to individual fields.
func NewArrayStruct ¶
func NewArrayStruct(arr *array.Struct, ct types.ColumnType) *ArrayStruct
NewArrayStruct creates a new ArrayStruct from an array.Struct
func (*ArrayStruct) ColumnType ¶
func (a *ArrayStruct) ColumnType() types.ColumnType
ColumnType implements ColumnVector.
func (*ArrayStruct) Release ¶
func (a *ArrayStruct) Release()
Release decreases the reference count by 1 on underlying Arrow array
func (*ArrayStruct) ToArray ¶
func (a *ArrayStruct) ToArray() arrow.Array
ToArray implements ColumnVector. Returns the underlying struct array.
func (*ArrayStruct) Type ¶
func (a *ArrayStruct) Type() types.DataType
Type implements ColumnVector.
func (*ArrayStruct) Value ¶
func (a *ArrayStruct) Value(i int) any
Value implements ColumnVector. Returns a map of field names to values at the specified index.
type BinaryFunction ¶
type BinaryFunction interface {
Evaluate(lhs, rhs ColumnVector) (ColumnVector, error)
}
TODO(chaudum): Make BinaryFunction typed:
type BinaryFunction[L, R arrow.DataType] interface {
Evaluate(lhs ColumnVector[L], rhs ColumnVector[R]) (ColumnVector[arrow.BOOL], error)
}
type BinaryFunctionRegistry ¶
type BufferedPipeline ¶
type BufferedPipeline struct {
// contains filtered or unexported fields
}
BufferedPipeline is a pipeline implementation that reads from a fixed set of Arrow records. It implements the Pipeline interface and serves as a simple source for testing and data injection.
func NewBufferedPipeline ¶
func NewBufferedPipeline(records ...arrow.Record) *BufferedPipeline
NewBufferedPipeline creates a new BufferedPipeline from a set of Arrow records. The pipeline will return these records in sequence.
func (*BufferedPipeline) Close ¶
func (p *BufferedPipeline) Close()
Close implements Pipeline. It releases all unreturned records.
type CoalesceVector ¶
type CoalesceVector struct {
// contains filtered or unexported fields
}
CoalesceVector represents multiple columns with the same name but different types.ColumnType Vectors are ordered by precedence (highest precedence first).
func (*CoalesceVector) ColumnType ¶
func (m *CoalesceVector) ColumnType() types.ColumnType
ColumnType implements ColumnVector.
func (*CoalesceVector) Release ¶
func (m *CoalesceVector) Release()
Release implements ColumnVector.
func (*CoalesceVector) ToArray ¶
func (m *CoalesceVector) ToArray() arrow.Array
ToArray implements ColumnVector.
func (*CoalesceVector) Type ¶
func (m *CoalesceVector) Type() types.DataType
Type implements ColumnVector.
func (*CoalesceVector) Value ¶
func (m *CoalesceVector) Value(i int) any
Value returns the value at the specified index position considering the precedence rules.
type ColumnVector ¶
type ColumnVector interface {
// ToArray returns the underlying Arrow array representation of the column vector.
ToArray() arrow.Array
// Value returns the value at the specified index position in the column vector.
Value(i int) any
// Type returns the Loki data type of the column vector.
Type() types.DataType
// ColumnType returns the type of column the vector originates from.
ColumnType() types.ColumnType
// Len returns the length of the vector
Len() int64
// Release decreases the reference count by 1 on underlying Arrow array
Release()
}
ColumnVector represents columnar values from evaluated expressions.
type Context ¶
type Context struct {
// contains filtered or unexported fields
}
Context is the execution context
type GenericPipeline ¶
type GenericPipeline struct {
// contains filtered or unexported fields
}
func NewFilterPipeline ¶
func NewLimitPipeline ¶
func NewLimitPipeline(input Pipeline, skip, fetch uint32) *GenericPipeline
func NewParsePipeline ¶
type Merge ¶
type Merge struct {
// contains filtered or unexported fields
}
Merge is a pipeline that takes N inputs and sequentially consumes each one of them. It completely exhausts an input before moving to the next one.
type Pipeline ¶
type Pipeline interface {
// Read collects the next value ([arrow.Record]) from the pipeline and returns it to the caller.
// It returns an error if reading fails or when the pipeline is exhausted. In this case, the function returns EOF.
Read(context.Context) (arrow.Record, error)
// Close closes the resources of the pipeline.
// The implementation must close all the of the pipeline's inputs.
Close()
}
Pipeline represents a data processing pipeline that can read Arrow records. It provides methods to read data, access the current record, and close resources.
func NewProjectPipeline ¶
func NewProjectPipeline(input Pipeline, proj *physical.Projection, evaluator *expressionEvaluator) (Pipeline, error)
type Scalar ¶
type Scalar struct {
// contains filtered or unexported fields
}
Scalar represents a single value repeated any number of times.
func (*Scalar) ColumnType ¶
func (v *Scalar) ColumnType() types.ColumnType
ColumnType implements ColumnVector.
type UnaryFunc ¶
type UnaryFunc func(ColumnVector, memory.Allocator) (ColumnVector, error)
func (UnaryFunc) Evaluate ¶
func (f UnaryFunc) Evaluate(lhs ColumnVector, allocator memory.Allocator) (ColumnVector, error)
type UnaryFunction ¶
type UnaryFunction interface {
Evaluate(lhs ColumnVector, allocator memory.Allocator) (ColumnVector, error)
}
type UnaryFunctionRegistry ¶
Source Files
¶
- aggregator.go
- arrow_scalar_compare.go
- cast.go
- compat.go
- dataobjscan.go
- dataobjscan_predicate.go
- executor.go
- expressions.go
- filter.go
- functions.go
- limit.go
- merge.go
- parse.go
- parse_json.go
- parse_logfmt.go
- pipeline.go
- pipeline_utils.go
- project.go
- range_aggregation.go
- schema.go
- stream_injector.go
- streams_view.go
- topk.go
- topk_batch.go
- util.go
- vector_aggregate.go