plan

package
v0.11.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 24, 2026 License: Apache-2.0 Imports: 34 Imported by: 0

Documentation

Overview

Package plan is the vectorized measure-query plan tree (G8).

This package is a peer of pkg/query/logical/measure (deprecated, row path); the two share no plan-node types, executor wiring, or iterator machinery. Top-level dispatch (banyand/query/processor.go, G8d) routes requests to one OR the other based on VectorizedConfig.Enabled.

A VecPlan node knows its output BatchSchema, its children, and how to append itself to a *vectorized.PipelineBuilder during Build. Build is bottom-up: a node calls Build on its child first, then attaches its own operator. The root's Build returns a fully-composed PipelineBuilder that the executor (G8c) finalizes via builder.Build() to produce a *vectorized.Pipeline.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func BuildMultiGroupBatchSchema

func BuildMultiGroupBatchSchema(measureSchemas []*databasev1.Measure, req *measurev1.QueryRequest) (*vectorized.BatchSchema, error)

BuildMultiGroupBatchSchema unions the per-group BatchSchemas produced by BuildBatchSchema and returns a single merged schema whose column order is:

metadata (timestamp, version, sid, shardID), then tags in
(TagFamilyIdx, TagIdx) walk order across all groups with duplicates
skipped, then fields ordered by request projection (or schema-walk
order when no projection is given).

Tag type-divergence (same name, different type across groups) falls back to ColumnTypeTagValue so the frame v3 proto-bytes passthrough carries each cell without forced type conversion. Field type-divergence falls back to ColumnTypeFieldValue, mirroring the row path's FIELD_TYPE_UNSPECIFIED fallback (schema.go:165-176).

When len(measureSchemas) == 1 the result is identical to BuildBatchSchema applied to that single schema (single-group hot path is unchanged).

func Dispatch

func Dispatch(
	ctx context.Context,
	req *measurev1.QueryRequest,
	metadata *commonv1.Metadata,
	measureSchema *databasev1.Measure,
	logicalSchema logical.Schema,
	ec executor.MeasureExecutionContext,
	cfg measure.VectorizedConfig,
	emitPartial bool,
	skipProjectionValidation bool,
) (iter executor.MIterator, planStr string, handled bool, err error)

Dispatch is the G8d top-level entry into the vec measure subsystem.

Called from banyand/query/processor.go before the row-path Analyze runs. When the request is eligible for the vec subsystem, Dispatch:

  1. Analyzes the request into a VecPlan via plan.Analyze (G8b)
  2. Resolves the index.Query + entity table the storage layer needs (using inverted.BuildQuery / BuildIndexModeQuery — the same helpers the deprecated row path uses; the logical.Schema parameter threads through unchanged)
  3. Calls ec.Query(ctx, opts) to obtain the MeasureQueryResult
  4. Wraps the result as a vec PullOperator (BatchSourceFromBatchResult fast path when available; BatchScan fallback otherwise) and installs it on the leaf Scan node
  5. Executes the plan via plan.Execute (G8c) to return an MIterator

Returns (iter, planStr, true, nil) when the request is handled; the caller MUST return that iterator and skip the row plan. Returns (nil, "", false, nil) when the request is NOT eligible — the caller should fall through to the row path. Returns (nil, "", true, err) when the request was eligible but execution failed; the caller must surface the error rather than fall through (the storage query may have already touched state).

Eligibility gate (v1):

  • cfg.Enabled must be true
  • request may carry GroupBy and/or Agg in any combination (group+agg, scalar reduce, raw GroupBy); plan.Analyze auto-extends the projection so the keys / agg field always resolve
  • request may carry Top: the analyzer emits Scan → Top → Limit (or Scan → GroupByAgg → Top → Limit) and BatchTop reproduces the row path's top-N (G9a)
  • a nil request.TimeRange is accepted: Dispatch normalises it to an epoch→epoch window for row-path parity (G9c), so callers do not need to pre-populate one
  • hidden criteria tags (criteria tags absent from the projection) are projected for storage-side filtering, then stripped at egress by hiddenTagsMIterator so the wire format is byte-identical
  • measureSchema and logicalSchema must be non-nil

func Execute

Execute composes a vec plan tree into a runnable *vectorized.Pipeline and returns it as an executor.MIterator. The Scan node(s) in the plan must have Source already set — the executor (G8c) does not resolve storage on its own; G8d's top-level dispatch is responsible for building the scan source from a MeasureExecutionContext and stitching it in.

Lifecycle: a per-pipeline MemoryTracker is constructed from cfg.QueryMemoryMiB and threaded through the BuildContext so every memory-bookkeeping operator (BatchAggregation, future BatchGroupBy) charges against a single budget (G7a). Pipeline.Init is invoked before the iterator is returned so breaker stages (which lazily allocate their state on Init) are ready for the first Next.

On any error during Build, builder.Build, or Pipeline.Init, the function returns the error and closes any partially-constructed pipeline so source resources (BatchPool, underlying MeasureBatchResult) are released.

func FellThroughCount

func FellThroughCount() int64

FellThroughCount returns the cumulative number of times Dispatch declined to handle a request (returned handled=false, err=nil) in this process.

func HandledCount

func HandledCount() int64

HandledCount returns the cumulative number of vec dispatch successes observed by this process.

func PrintTree

func PrintTree(root VecPlan) string

PrintTree renders a plan tree as multi-line text. Leaves first column, each parent indented two spaces deeper than its child. Useful for debugging analyzer output.

func SupportsDistributedRows

func SupportsDistributedRows(req *measurev1.QueryRequest) bool

SupportsDistributedRows reports whether a non-aggregation request can use the native vectorized distributed row merge. Phase 2 lifts the OrderBy.IndexRuleName != "" gate; Phase 4 lifts the Top-without-Agg gate; Phase 5 lifts the GroupBy-without-Agg gate (raw GroupBy: first-seen row per group) and the multi-group + Top carve-out (per-group Limit is now calibrated rather than MaxUint32, so amplification is bounded).

Rejected: Agg != nil — aggregation requests always go through executeAgg.

func ValidateMultiGroupProjection

func ValidateMultiGroupProjection(req *measurev1.QueryRequest, schemas []logical.Schema, measures []*databasev1.Measure) error

ValidateMultiGroupProjection is the multi-measure counterpart of the single-group validation Dispatch runs inline. A projected tag/field is accepted if it resolves in ANY group's schema/measure — mirroring the row path's measure_analyzer.Analyze, which calls mergeSchema(ss) to union the per-group schemas before validating the projection. Without this, a multi-measure query that projects a tag/field added to one group but not another (the classic schema-evolution case the `multi_group_new_tag_field` integration test pins) would be rejected per-group inside Dispatch even though the row path accepts it. Returns the same byte-identical error message ("<tag>: tag is not defined" / "field <name> not found in schema") as the single-group path so test fixtures and operator-facing errors match across the two routes.

Types

type BuildContext

type BuildContext struct {
	Builder *vectorized.PipelineBuilder
	Tracker *vectorized.MemoryTracker
	Config  measure.VectorizedConfig
}

BuildContext is the cross-cutting state threaded through every Build call. Builder accumulates operators; Tracker is the shared per-pipeline MemoryTracker (G7a) every memory-bookkeeping operator must use; Config supplies BatchSize and other runtime knobs.

type DistributedPlan

type DistributedPlan struct {
	// contains filtered or unexported fields
}

DistributedPlan is the vectorized liaison-side distributed measure plan. It consumes data-node raw frame bodies as []byte values from RawFrameCodec, decodes them into vectorized batches, and runs the liaison operators without routing through the row-compatible logical distributedPlan.

func AnalyzeDistributed

func AnalyzeDistributed(
	req *measurev1.QueryRequest,
	measureSchemas []*databasev1.Measure,
	indexRules [][]*databasev1.IndexRule,
	cfg vmeasure.VectorizedConfig,
) (*DistributedPlan, error)

AnalyzeDistributed builds the vectorized distributed liaison plan. measureSchemas is the per-group slice of Measure schemas (one entry per req.Groups element). indexRules is the corresponding per-group slice of index rule sets (ec.GetIndexRules() for each group). Both slices must be in request-group order. Single-group callers may pass a length-1 slice for each (the existing behavior is preserved byte-for-byte).

When indexRules is nil or empty and req.OrderBy.IndexRuleName is non-empty, the resolver surfaces an "index rule X not found" error byte-equivalent to the row path.

func (*DistributedPlan) Execute

Execute broadcasts the internal query and executes the liaison-side vectorized plan. For single-group requests, one broadcast is issued for all groups (existing behavior). For multi-group requests, one broadcast is issued per group, each carrying a single-element Groups slice, so data nodes can answer with a schema that matches only their local group's columns.

func (*DistributedPlan) String

func (p *DistributedPlan) String() string

String returns a concise plan rendering for tracing.

type GroupByAgg

type GroupByAgg struct {
	Child   VecPlan
	GroupBy *model.MeasureGroupBy
	Agg     *model.MeasureAgg

	Mode vmeasure.AggMode
	// contains filtered or unexported fields
}

GroupByAgg is the vec aggregation/grouping node. It fuses GroupBy and a single Aggregation into one BatchAggregation operator (see G7d planner), and also covers the two single-sided shapes:

  • GroupBy + Agg → BatchAggregation: key columns + one agg result column (schema-rewriting; timestamp dropped, per G7 decision D2).
  • Agg without GroupBy → scalar reduce: BatchAggregation with no key columns, a single output row (first-seen tags + agg result).
  • GroupBy without Agg → raw GroupBy: a first-seen-row-per-group BatchGroupBy, schema-preserving.

The concrete operator (and therefore the output schema) is chosen by vmeasure.BuildOperators from which of GroupBy/Agg is set. Mode selects AggModeAll (single-node final reduce) vs AggModeMap (G9f.2 distributed Map phase emitting typed-column partials). Mode is irrelevant for raw GroupBy (BatchGroupBy doesn't carry partial state).

func NewGroupByAgg

func NewGroupByAgg(child VecPlan, groupBy *model.MeasureGroupBy, agg *model.MeasureAgg, mode vmeasure.AggMode) (*GroupByAgg, error)

NewGroupByAgg constructs a GroupByAgg node wrapping child. At least one of groupBy/agg must be set (BuildOperators routes on which); child must not be nil. mode selects AggModeAll for the single-node path or AggModeMap for the distributed Map phase (G9f.2); AggModeReduce is rejected here (the reduce plan is built liaison-side in G9f.3).

func (*GroupByAgg) Build

func (g *GroupByAgg) Build(ctx context.Context, bc *BuildContext) error

Build recurses into child, then constructs the BatchAggregation via BuildOperators and attaches it as a breaker. The pipeline-shared MemoryTracker from bc threads through.

func (*GroupByAgg) Children

func (g *GroupByAgg) Children() []VecPlan

Children returns the single child.

func (*GroupByAgg) Schema

func (g *GroupByAgg) Schema() *vectorized.BatchSchema

Schema returns the aggregation output schema. The schema is computed lazily on first call by running BuildOperators against the child schema; subsequent calls return the cached value.

func (*GroupByAgg) String

func (g *GroupByAgg) String() string

String returns a single-line debug description.

type Limit

type Limit struct {
	Child  VecPlan
	Offset uint32
	N      uint32
}

Limit applies offset+limit windowing as a fusible operator on the pipeline. Schema-preserving: emits the same column layout as its child.

Limit <= 0 means "no limit"; the analyzer normalises QueryRequest.Limit of zero to the row-path default (100) before constructing this node.

func NewLimit

func NewLimit(child VecPlan, offset, n uint32) *Limit

NewLimit constructs a Limit node wrapping child.

func (*Limit) Build

func (l *Limit) Build(ctx context.Context, bc *BuildContext) error

Build recurses into the child first, then attaches a BatchLimit as a fusible operator. If N is zero, no operator is attached — the caller's downstream nodes still see the source's full output.

func (*Limit) Children

func (l *Limit) Children() []VecPlan

Children returns the single child.

func (*Limit) Schema

func (l *Limit) Schema() *vectorized.BatchSchema

Schema returns the child's schema (Limit is schema-preserving).

func (*Limit) String

func (l *Limit) String() string

String returns a single-line debug description.

type Scan

type Scan struct {
	BatchSchema *vectorized.BatchSchema
	Source      vectorized.PullOperator
	Params      ScanParams
}

Scan is the leaf node of every vec plan. It carries the schema for downstream nodes to consult and the parameters the executor needs to build a source. Source is set by the executor immediately before Build is called; tests can populate it directly to drive Build with a fake source.

func NewScan

func NewScan(schema *vectorized.BatchSchema, params ScanParams) *Scan

NewScan constructs a Scan node with an analyzed schema and params. Source is unset; the executor or test populates it before Build.

func (*Scan) Build

func (s *Scan) Build(_ context.Context, bc *BuildContext) error

Build attaches Source as the pipeline source. The executor must set Source before invoking Build; an unset Source is treated as a programming error (the executor missed a step).

func (*Scan) Children

func (s *Scan) Children() []VecPlan

Children returns no children — Scan is the leaf.

func (*Scan) Schema

func (s *Scan) Schema() *vectorized.BatchSchema

Schema returns the BatchSchema of rows this node emits.

func (*Scan) String

func (s *Scan) String() string

String returns a single-line debug description.

type ScanParams

type ScanParams struct {
	Measure         *databasev1.Measure
	TimeRange       *timestamp.TimeRange
	Query           index.Query
	Entities        [][]*modelv1.TagValue
	GroupBy         *model.MeasureGroupBy
	Agg             *model.MeasureAgg
	TagProjection   []model.TagProjection
	FieldProjection []string
}

ScanParams holds everything the executor needs to materialize a batch source from a Measure. The analyzer populates these from the proto QueryRequest at plan-build time; the executor (G8c) consults the MeasureExecutionContext and constructs the MeasureBatchResult right before calling Scan.Build.

type Top

type Top struct {
	Child     VecPlan
	FieldName string
	N         int
	Asc       bool
}

Top selects the top-N (or bottom-N when Asc) rows by FieldName. Wraps `measure.BatchTop`, a single global heap. This matches the row path's req.Top handler (pkg/query/logical/measure.topOp), which also inserts every data point into one TopQueue — req.Top is a whole-result top-N, not the per-timestamp TopNQuery RPC (out of scope, see .omc/g9-plan.md G9a).

Schema-preserving.

func NewTop

func NewTop(child VecPlan, fieldName string, n int, asc bool) *Top

NewTop constructs a Top node wrapping child.

func (*Top) Build

func (t *Top) Build(ctx context.Context, bc *BuildContext) error

Build recurses into the child, locates FieldName in the propagated schema, then attaches a BatchTop as a breaker. The FieldName must reference a field column in the current schema.

func (*Top) Children

func (t *Top) Children() []VecPlan

Children returns the single child.

func (*Top) Schema

func (t *Top) Schema() *vectorized.BatchSchema

Schema returns the child's schema (Top is schema-preserving).

func (*Top) String

func (t *Top) String() string

String returns a single-line debug description.

type VecPlan

type VecPlan interface {
	Schema() *vectorized.BatchSchema
	Children() []VecPlan
	Build(ctx context.Context, bc *BuildContext) error
	String() string
}

VecPlan is the vectorized measure-query plan node interface.

Schema returns the BatchSchema of rows this node emits — for fusible or schema-preserving breakers it matches the input schema; for schema-rewriting breakers (BatchAggregation) it is the operator's OutputSchema.

Children returns the immediate child plan nodes (zero for leaves).

Build appends this node's contribution to bc.Builder. The leaf (Scan) calls Builder.From; fusible nodes call Builder.Apply; breakers call Builder.Break. Build must call Build on its children before attaching itself so the source flows in tree order.

String returns a single-line debug description ("Scan(measure=foo)", "GroupByAgg(keys=svc, fn=sum, field=value)", etc.) so plan trees can be pretty-printed with PrintTree.

func Analyze

func Analyze(req *measurev1.QueryRequest, measureSchema *databasev1.Measure, mode measure.AggMode) (VecPlan, error)

Analyze translates a measurev1.QueryRequest + its Measure schema into a VecPlan tree. It is the vec counterpart of pkg/query/logical/measure (deprecated) but produces vec plan nodes — there is no leaf substitution into a row plan.

The returned tree is structural: it carries the static (proto-derived) query parameters in `Scan.Params` and the BatchSchema for downstream nodes to consult. Runtime fields that depend on the executor's MeasureExecutionContext — the resolved `index.Query` and the entity table — are NOT populated here; the executor (G8c) fills them in before invoking Build.

Errors are returned for:

  • nil schema
  • tag/field projection naming columns not in the schema
  • GroupBy referencing a tag absent from the schema
  • Agg referencing a field absent from the schema

GroupBy and Agg may travel together (group + aggregate), or either alone: Agg without GroupBy is a scalar reduce (single output row); GroupBy without Agg is a raw GroupBy (first-seen row per group). Both mirror the row path (measure_plan_aggregation.go / measure_plan_groupby.go).

mode selects the BatchAggregation strategy when an agg operator is built: AggModeAll for single-node final reduce, AggModeMap for the distributed Map phase (G9f.2) that emits typed-column partials. AggModeReduce is rejected (the reduce plan is built liaison-side in G9f.3).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL