Documentation
¶
Overview ¶
Package plan is the vectorized measure-query plan tree (G8).
This package is a peer of pkg/query/logical/measure (deprecated, row path); the two share no plan-node types, executor wiring, or iterator machinery. Top-level dispatch (banyand/query/processor.go, G8d) routes requests to one OR the other based on VectorizedConfig.Enabled.
A VecPlan node knows its output BatchSchema, its children, and how to append itself to a *vectorized.PipelineBuilder during Build. Build is bottom-up: a node calls Build on its child first, then attaches its own operator. The root's Build returns a fully-composed PipelineBuilder that the executor (G8c) finalizes via builder.Build() to produce a *vectorized.Pipeline.
Index ¶
- func BuildMultiGroupBatchSchema(measureSchemas []*databasev1.Measure, req *measurev1.QueryRequest) (*vectorized.BatchSchema, error)
- func Dispatch(ctx context.Context, req *measurev1.QueryRequest, metadata *commonv1.Metadata, ...) (iter executor.MIterator, planStr string, handled bool, err error)
- func Execute(ctx context.Context, plan VecPlan, cfg measure.VectorizedConfig) (executor.MIterator, error)
- func FellThroughCount() int64
- func HandledCount() int64
- func PrintTree(root VecPlan) string
- func SupportsDistributedRows(req *measurev1.QueryRequest) bool
- func ValidateMultiGroupProjection(req *measurev1.QueryRequest, schemas []logical.Schema, ...) error
- type BuildContext
- type DistributedPlan
- type GroupByAgg
- type Limit
- type Scan
- type ScanParams
- type Top
- type VecPlan
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func BuildMultiGroupBatchSchema ¶
func BuildMultiGroupBatchSchema(measureSchemas []*databasev1.Measure, req *measurev1.QueryRequest) (*vectorized.BatchSchema, error)
BuildMultiGroupBatchSchema unions the per-group BatchSchemas produced by BuildBatchSchema and returns a single merged schema whose column order is:
metadata (timestamp, version, sid, shardID), then tags in (TagFamilyIdx, TagIdx) walk order across all groups with duplicates skipped, then fields ordered by request projection (or schema-walk order when no projection is given).
Tag type-divergence (same name, different type across groups) falls back to ColumnTypeTagValue so the frame v3 proto-bytes passthrough carries each cell without forced type conversion. Field type-divergence falls back to ColumnTypeFieldValue, mirroring the row path's FIELD_TYPE_UNSPECIFIED fallback (schema.go:165-176).
When len(measureSchemas) == 1 the result is identical to BuildBatchSchema applied to that single schema (single-group hot path is unchanged).
func Dispatch ¶
func Dispatch( ctx context.Context, req *measurev1.QueryRequest, metadata *commonv1.Metadata, measureSchema *databasev1.Measure, logicalSchema logical.Schema, ec executor.MeasureExecutionContext, cfg measure.VectorizedConfig, emitPartial bool, skipProjectionValidation bool, ) (iter executor.MIterator, planStr string, handled bool, err error)
Dispatch is the G8d top-level entry into the vec measure subsystem.
Called from banyand/query/processor.go before the row-path Analyze runs. When the request is eligible for the vec subsystem, Dispatch:
- Analyzes the request into a VecPlan via plan.Analyze (G8b)
- Resolves the index.Query + entity table the storage layer needs (using inverted.BuildQuery / BuildIndexModeQuery — the same helpers the deprecated row path uses; the logical.Schema parameter threads through unchanged)
- Calls ec.Query(ctx, opts) to obtain the MeasureQueryResult
- Wraps the result as a vec PullOperator (BatchSourceFromBatchResult fast path when available; BatchScan fallback otherwise) and installs it on the leaf Scan node
- Executes the plan via plan.Execute (G8c) to return an MIterator
Returns (iter, planStr, true, nil) when the request is handled; the caller MUST return that iterator and skip the row plan. Returns (nil, "", false, nil) when the request is NOT eligible — the caller should fall through to the row path. Returns (nil, "", true, err) when the request was eligible but execution failed; the caller must surface the error rather than fall through (the storage query may have already touched state).
Eligibility gate (v1):
- cfg.Enabled must be true
- request may carry GroupBy and/or Agg in any combination (group+agg, scalar reduce, raw GroupBy); plan.Analyze auto-extends the projection so the keys / agg field always resolve
- request may carry Top: the analyzer emits Scan → Top → Limit (or Scan → GroupByAgg → Top → Limit) and BatchTop reproduces the row path's top-N (G9a)
- a nil request.TimeRange is accepted: Dispatch normalises it to an epoch→epoch window for row-path parity (G9c), so callers do not need to pre-populate one
- hidden criteria tags (criteria tags absent from the projection) are projected for storage-side filtering, then stripped at egress by hiddenTagsMIterator so the wire format is byte-identical
- measureSchema and logicalSchema must be non-nil
func Execute ¶
func Execute(ctx context.Context, plan VecPlan, cfg measure.VectorizedConfig) (executor.MIterator, error)
Execute composes a vec plan tree into a runnable *vectorized.Pipeline and returns it as an executor.MIterator. The Scan node(s) in the plan must have Source already set — the executor (G8c) does not resolve storage on its own; G8d's top-level dispatch is responsible for building the scan source from a MeasureExecutionContext and stitching it in.
Lifecycle: a per-pipeline MemoryTracker is constructed from cfg.QueryMemoryMiB and threaded through the BuildContext so every memory-bookkeeping operator (BatchAggregation, future BatchGroupBy) charges against a single budget (G7a). Pipeline.Init is invoked before the iterator is returned so breaker stages (which lazily allocate their state on Init) are ready for the first Next.
On any error during Build, builder.Build, or Pipeline.Init, the function returns the error and closes any partially-constructed pipeline so source resources (BatchPool, underlying MeasureBatchResult) are released.
func FellThroughCount ¶
func FellThroughCount() int64
FellThroughCount returns the cumulative number of times Dispatch declined to handle a request (returned handled=false, err=nil) in this process.
func HandledCount ¶
func HandledCount() int64
HandledCount returns the cumulative number of vec dispatch successes observed by this process.
func PrintTree ¶
PrintTree renders a plan tree as multi-line text. Leaves first column, each parent indented two spaces deeper than its child. Useful for debugging analyzer output.
func SupportsDistributedRows ¶
func SupportsDistributedRows(req *measurev1.QueryRequest) bool
SupportsDistributedRows reports whether a non-aggregation request can use the native vectorized distributed row merge. Phase 2 lifts the OrderBy.IndexRuleName != "" gate; Phase 4 lifts the Top-without-Agg gate; Phase 5 lifts the GroupBy-without-Agg gate (raw GroupBy: first-seen row per group) and the multi-group + Top carve-out (per-group Limit is now calibrated rather than MaxUint32, so amplification is bounded).
Rejected: Agg != nil — aggregation requests always go through executeAgg.
func ValidateMultiGroupProjection ¶
func ValidateMultiGroupProjection(req *measurev1.QueryRequest, schemas []logical.Schema, measures []*databasev1.Measure) error
ValidateMultiGroupProjection is the multi-measure counterpart of the single-group validation Dispatch runs inline. A projected tag/field is accepted if it resolves in ANY group's schema/measure — mirroring the row path's measure_analyzer.Analyze, which calls mergeSchema(ss) to union the per-group schemas before validating the projection. Without this, a multi-measure query that projects a tag/field added to one group but not another (the classic schema-evolution case the `multi_group_new_tag_field` integration test pins) would be rejected per-group inside Dispatch even though the row path accepts it. Returns the same byte-identical error message ("<tag>: tag is not defined" / "field <name> not found in schema") as the single-group path so test fixtures and operator-facing errors match across the two routes.
Types ¶
type BuildContext ¶
type BuildContext struct {
Builder *vectorized.PipelineBuilder
Tracker *vectorized.MemoryTracker
Config measure.VectorizedConfig
}
BuildContext is the cross-cutting state threaded through every Build call. Builder accumulates operators; Tracker is the shared per-pipeline MemoryTracker (G7a) every memory-bookkeeping operator must use; Config supplies BatchSize and other runtime knobs.
type DistributedPlan ¶
type DistributedPlan struct {
// contains filtered or unexported fields
}
DistributedPlan is the vectorized liaison-side distributed measure plan. It consumes data-node raw frame bodies as []byte values from RawFrameCodec, decodes them into vectorized batches, and runs the liaison operators without routing through the row-compatible logical distributedPlan.
func AnalyzeDistributed ¶
func AnalyzeDistributed( req *measurev1.QueryRequest, measureSchemas []*databasev1.Measure, indexRules [][]*databasev1.IndexRule, cfg vmeasure.VectorizedConfig, ) (*DistributedPlan, error)
AnalyzeDistributed builds the vectorized distributed liaison plan. measureSchemas is the per-group slice of Measure schemas (one entry per req.Groups element). indexRules is the corresponding per-group slice of index rule sets (ec.GetIndexRules() for each group). Both slices must be in request-group order. Single-group callers may pass a length-1 slice for each (the existing behavior is preserved byte-for-byte).
When indexRules is nil or empty and req.OrderBy.IndexRuleName is non-empty, the resolver surfaces an "index rule X not found" error byte-equivalent to the row path.
func (*DistributedPlan) Execute ¶
Execute broadcasts the internal query and executes the liaison-side vectorized plan. For single-group requests, one broadcast is issued for all groups (existing behavior). For multi-group requests, one broadcast is issued per group, each carrying a single-element Groups slice, so data nodes can answer with a schema that matches only their local group's columns.
func (*DistributedPlan) String ¶
func (p *DistributedPlan) String() string
String returns a concise plan rendering for tracing.
type GroupByAgg ¶
type GroupByAgg struct {
Child VecPlan
GroupBy *model.MeasureGroupBy
Agg *model.MeasureAgg
Mode vmeasure.AggMode
// contains filtered or unexported fields
}
GroupByAgg is the vec aggregation/grouping node. It fuses GroupBy and a single Aggregation into one BatchAggregation operator (see G7d planner), and also covers the two single-sided shapes:
- GroupBy + Agg → BatchAggregation: key columns + one agg result column (schema-rewriting; timestamp dropped, per G7 decision D2).
- Agg without GroupBy → scalar reduce: BatchAggregation with no key columns, a single output row (first-seen tags + agg result).
- GroupBy without Agg → raw GroupBy: a first-seen-row-per-group BatchGroupBy, schema-preserving.
The concrete operator (and therefore the output schema) is chosen by vmeasure.BuildOperators from which of GroupBy/Agg is set. Mode selects AggModeAll (single-node final reduce) vs AggModeMap (G9f.2 distributed Map phase emitting typed-column partials). Mode is irrelevant for raw GroupBy (BatchGroupBy doesn't carry partial state).
func NewGroupByAgg ¶
func NewGroupByAgg(child VecPlan, groupBy *model.MeasureGroupBy, agg *model.MeasureAgg, mode vmeasure.AggMode) (*GroupByAgg, error)
NewGroupByAgg constructs a GroupByAgg node wrapping child. At least one of groupBy/agg must be set (BuildOperators routes on which); child must not be nil. mode selects AggModeAll for the single-node path or AggModeMap for the distributed Map phase (G9f.2); AggModeReduce is rejected here (the reduce plan is built liaison-side in G9f.3).
func (*GroupByAgg) Build ¶
func (g *GroupByAgg) Build(ctx context.Context, bc *BuildContext) error
Build recurses into child, then constructs the BatchAggregation via BuildOperators and attaches it as a breaker. The pipeline-shared MemoryTracker from bc threads through.
func (*GroupByAgg) Children ¶
func (g *GroupByAgg) Children() []VecPlan
Children returns the single child.
func (*GroupByAgg) Schema ¶
func (g *GroupByAgg) Schema() *vectorized.BatchSchema
Schema returns the aggregation output schema. The schema is computed lazily on first call by running BuildOperators against the child schema; subsequent calls return the cached value.
func (*GroupByAgg) String ¶
func (g *GroupByAgg) String() string
String returns a single-line debug description.
type Limit ¶
Limit applies offset+limit windowing as a fusible operator on the pipeline. Schema-preserving: emits the same column layout as its child.
Limit <= 0 means "no limit"; the analyzer normalises QueryRequest.Limit of zero to the row-path default (100) before constructing this node.
func (*Limit) Build ¶
func (l *Limit) Build(ctx context.Context, bc *BuildContext) error
Build recurses into the child first, then attaches a BatchLimit as a fusible operator. If N is zero, no operator is attached — the caller's downstream nodes still see the source's full output.
func (*Limit) Schema ¶
func (l *Limit) Schema() *vectorized.BatchSchema
Schema returns the child's schema (Limit is schema-preserving).
type Scan ¶
type Scan struct {
BatchSchema *vectorized.BatchSchema
Source vectorized.PullOperator
Params ScanParams
}
Scan is the leaf node of every vec plan. It carries the schema for downstream nodes to consult and the parameters the executor needs to build a source. Source is set by the executor immediately before Build is called; tests can populate it directly to drive Build with a fake source.
func NewScan ¶
func NewScan(schema *vectorized.BatchSchema, params ScanParams) *Scan
NewScan constructs a Scan node with an analyzed schema and params. Source is unset; the executor or test populates it before Build.
func (*Scan) Build ¶
func (s *Scan) Build(_ context.Context, bc *BuildContext) error
Build attaches Source as the pipeline source. The executor must set Source before invoking Build; an unset Source is treated as a programming error (the executor missed a step).
func (*Scan) Schema ¶
func (s *Scan) Schema() *vectorized.BatchSchema
Schema returns the BatchSchema of rows this node emits.
type ScanParams ¶
type ScanParams struct {
Measure *databasev1.Measure
TimeRange *timestamp.TimeRange
Query index.Query
Entities [][]*modelv1.TagValue
GroupBy *model.MeasureGroupBy
Agg *model.MeasureAgg
TagProjection []model.TagProjection
FieldProjection []string
}
ScanParams holds everything the executor needs to materialize a batch source from a Measure. The analyzer populates these from the proto QueryRequest at plan-build time; the executor (G8c) consults the MeasureExecutionContext and constructs the MeasureBatchResult right before calling Scan.Build.
type Top ¶
Top selects the top-N (or bottom-N when Asc) rows by FieldName. Wraps `measure.BatchTop`, a single global heap. This matches the row path's req.Top handler (pkg/query/logical/measure.topOp), which also inserts every data point into one TopQueue — req.Top is a whole-result top-N, not the per-timestamp TopNQuery RPC (out of scope, see .omc/g9-plan.md G9a).
Schema-preserving.
func (*Top) Build ¶
func (t *Top) Build(ctx context.Context, bc *BuildContext) error
Build recurses into the child, locates FieldName in the propagated schema, then attaches a BatchTop as a breaker. The FieldName must reference a field column in the current schema.
func (*Top) Schema ¶
func (t *Top) Schema() *vectorized.BatchSchema
Schema returns the child's schema (Top is schema-preserving).
type VecPlan ¶
type VecPlan interface {
Schema() *vectorized.BatchSchema
Children() []VecPlan
Build(ctx context.Context, bc *BuildContext) error
String() string
}
VecPlan is the vectorized measure-query plan node interface.
Schema returns the BatchSchema of rows this node emits — for fusible or schema-preserving breakers it matches the input schema; for schema-rewriting breakers (BatchAggregation) it is the operator's OutputSchema.
Children returns the immediate child plan nodes (zero for leaves).
Build appends this node's contribution to bc.Builder. The leaf (Scan) calls Builder.From; fusible nodes call Builder.Apply; breakers call Builder.Break. Build must call Build on its children before attaching itself so the source flows in tree order.
String returns a single-line debug description ("Scan(measure=foo)", "GroupByAgg(keys=svc, fn=sum, field=value)", etc.) so plan trees can be pretty-printed with PrintTree.
func Analyze ¶
func Analyze(req *measurev1.QueryRequest, measureSchema *databasev1.Measure, mode measure.AggMode) (VecPlan, error)
Analyze translates a measurev1.QueryRequest + its Measure schema into a VecPlan tree. It is the vec counterpart of pkg/query/logical/measure (deprecated) but produces vec plan nodes — there is no leaf substitution into a row plan.
The returned tree is structural: it carries the static (proto-derived) query parameters in `Scan.Params` and the BatchSchema for downstream nodes to consult. Runtime fields that depend on the executor's MeasureExecutionContext — the resolved `index.Query` and the entity table — are NOT populated here; the executor (G8c) fills them in before invoking Build.
Errors are returned for:
- nil schema
- tag/field projection naming columns not in the schema
- GroupBy referencing a tag absent from the schema
- Agg referencing a field absent from the schema
GroupBy and Agg may travel together (group + aggregate), or either alone: Agg without GroupBy is a scalar reduce (single output row); GroupBy without Agg is a raw GroupBy (first-seen row per group). Both mirror the row path (measure_plan_aggregation.go / measure_plan_groupby.go).
mode selects the BatchAggregation strategy when an agg operator is built: AggModeAll for single-node final reduce, AggModeMap for the distributed Map phase (G9f.2) that emits typed-column partials. AggModeReduce is rejected (the reduce plan is built liaison-side in G9f.3).