optimizer

func (*Optimizer) Optimize ¶ added in v1.8.0

func (o *Optimizer) Optimize(seq dag.Seq) (dag.Seq, error)

Optimize transforms the DAG by attempting to lift stateless operators from the downstream sequence into the trunk of each data source in the From operator at the entry point of the DAG. Once these paths are lifted, it also attempts to move any candidate filtering operations into the source's pushdown predicate. This should be called before ParallelizeScan(). TBD: we need to do pushdown for search/cut to optimize columnar extraction.

func (*Optimizer) OptimizeDeleter ¶ added in v1.8.0

func (o *Optimizer) OptimizeDeleter(seq dag.Seq, replicas int) (dag.Seq, error)

func (*Optimizer) Parallelize ¶

func (o *Optimizer) Parallelize(seq dag.Seq, n int) (dag.Seq, error)

Parallelize tries to parallelize the DAG by splitting each source path as much as possible of the sequence into n parallel branches.