Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type LinkSpec ¶
type LinkSpec struct {
ProcessorID int
}
LinkSpec is the specification of a link. Link connects pipelines between different nodes.
type LogicalPlan ¶
type LogicalPlan interface {
GetTaskExtraParams() proto.ExtraParams
ToTaskMeta() ([]byte, error)
FromTaskMeta([]byte) error
ToPhysicalPlan(PlanCtx) (*PhysicalPlan, error)
}
LogicalPlan represents a logical plan in distribute framework. A normal flow of distribute framework is: logical plan -> physical plan -> pipelines. To integrate with current distribute framework, the flow becomes: logical plan -> task meta -> physical plan -> subtaskmetas -> pipelines.
type OutputSpec ¶
type OutputSpec struct {
Links []LinkSpec
}
OutputSpec is the specification of an output.
type PhysicalPlan ¶
type PhysicalPlan struct {
Processors []ProcessorSpec
}
PhysicalPlan is a DAG of processors in distribute framework. Each processor is a node process the task with a pipeline, and receive/pass the result to other processors via input and output links.
func (*PhysicalPlan) AddProcessor ¶
func (p *PhysicalPlan) AddProcessor(processor ProcessorSpec)
AddProcessor adds a node to the DAG.
func (*PhysicalPlan) ToSubtaskMetas ¶
ToSubtaskMetas converts the physical plan to a list of subtask metas.
type PipelineSpec ¶
type PipelineSpec interface {
// ToSubtaskMeta converts the pipeline to a subtask meta
ToSubtaskMeta(PlanCtx) ([]byte, error)
}
PipelineSpec is the specification of an pipeline.
type PlanCtx ¶
type PlanCtx struct {
Ctx context.Context
// integrate with current distribute framework
SessionCtx sessionctx.Context
TaskID int64
TaskKey string
TaskType proto.TaskType
ThreadCnt int
MaxNodeCnt int
Keyspace string
// PreviousSubtaskMetas is subtask metas of previous steps.
// We can remove this field if we find a better way to pass the result between steps.
PreviousSubtaskMetas map[proto.Step][][]byte
GlobalSort bool
NextTaskStep proto.Step
ExecuteNodesCnt int
Store kv.Storage
}
PlanCtx is the context for planning.
type Planner ¶
type Planner struct{}
Planner represents a distribute plan planner.
func (*Planner) Run ¶
func (*Planner) Run(planCtx PlanCtx, plan LogicalPlan, taskMgr *storage.TaskManager) (int64, error)
Run runs the distribute plan.
type ProcessorSpec ¶
type ProcessorSpec struct {
ID int
Input InputSpec
Pipeline PipelineSpec
Output OutputSpec
// We can remove this field if we find a better way to pass the result between steps.
Step proto.Step
}
ProcessorSpec is the specification of a processor. A processor is a node in the DAG. It contains input links from other processors, as well as output links to other processors. It also contains an pipeline which is the actual logic of the processor.