Documentation
¶
Index ¶
- type RequestPlan
- func (p *RequestPlan) BackprocessSegmenter() *block.Segmenter
- func (p *RequestPlan) ModuleSegmenter(modInitBlock uint64) *block.Segmenter
- func (p *RequestPlan) ReadOutSegmenter(outputModuleInitialBlock uint64) *block.Segmenter
- func (p *RequestPlan) RequiresParallelProcessing() bool
- func (p *RequestPlan) StoresSegmenter() *block.Segmenter
- func (p *RequestPlan) String() string
- func (p *RequestPlan) WriteOutSegmenter() *block.Segmenter
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type RequestPlan ¶
type RequestPlan struct {
// This is simply the ranges that exist and are
// considered in the request. Further process will deal with the
// existence or non-existence of the current partials and full stores.
// And they will decide whether to schedule work or not.
BuildStores *block.Range
// Whether to process the last map stage.
//
// In development mode,
// we only care about processing the stores up to the handoff block,
// which then kicks in the linear mode, which will then output its
// results.
// In production mode, we will want that mapper to be produced
// to generate the ExecOut files, and kick off the ExecOutWalker
// here to output the results.
//
// WriteExecOut will always have a start block on the boundary,
// so the reading process needs to take into account the _start block_
// at which it wants to send the data over. Production of map output
// requires stores to be aligned, so needs to start from previous
// store snapshots.
WriteExecOut *block.Range // Can be nil
// When WriteExecOut produces files, we might want to start reading
// blocks only a bit further down a file (if boundary is at 20,
// the request's start block might be 25). This will instruct
// the output stream to start at that block number.
ReadExecOut *block.Range
// Range that will be produced by the linear pipeline. Could have no end.
LinearPipeline *block.Range
// contains filtered or unexported fields
}
RequestPlan lays out the configuration of the components to accomplish the work of the ParallelProcessor. Different conditions put different constraints on the output of the parallel processor.
func BuildTier1RequestPlan ¶
func BuildTier1RequestPlan(productionMode bool, segmentInterval, lowestInitialBlock, resolvedStartBlock, linearHandoffBlock, exclusiveEndBlock uint64, scheduleStores bool) (*RequestPlan, error)
func (*RequestPlan) BackprocessSegmenter ¶
func (p *RequestPlan) BackprocessSegmenter() *block.Segmenter
func (*RequestPlan) ModuleSegmenter ¶
func (p *RequestPlan) ModuleSegmenter(modInitBlock uint64) *block.Segmenter
func (*RequestPlan) ReadOutSegmenter ¶ added in v1.6.0
func (p *RequestPlan) ReadOutSegmenter(outputModuleInitialBlock uint64) *block.Segmenter
func (*RequestPlan) RequiresParallelProcessing ¶
func (p *RequestPlan) RequiresParallelProcessing() bool
func (*RequestPlan) StoresSegmenter ¶
func (p *RequestPlan) StoresSegmenter() *block.Segmenter
func (*RequestPlan) String ¶
func (p *RequestPlan) String() string
func (*RequestPlan) WriteOutSegmenter ¶
func (p *RequestPlan) WriteOutSegmenter() *block.Segmenter
Click to show internal directories.
Click to hide internal directories.