Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Worker ¶
type Worker struct {
// contains filtered or unexported fields
}
Worker requests tasks from a scheduler and executes them. Task results are sent to other Worker instances or back to the scheduler.
func NewWorker ¶
func NewWorker(params WorkerParams, reg prometheus.Registerer) (*Worker, error)
NewWorker creates a new Worker instance. Use Worker.Service to manage the lifecycle of the Worker.
func (*Worker) RegisterMetrics ¶
func (w *Worker) RegisterMetrics(reg prometheus.Registerer) error
RegisterMetrics registers metrics about w to report to reg.
func (*Worker) RegisterWorkerServer ¶
RegisterWorkerServer registers the wire.Listener of the inner worker as http.Handler on the provided router.
func (*Worker) UnregisterMetrics ¶
func (w *Worker) UnregisterMetrics(reg prometheus.Registerer)
UnregisterMetrics unregisters metrics about w from reg.
type WorkerConfig ¶
type WorkerConfig struct {
WorkerThreads int
SchedulerLookupAddress string
SchedulerLookupInterval time.Duration
}
WorkerConfig represents the configuration for the Worker.
type WorkerParams ¶
type WorkerParams struct {
Logger log.Logger // Logger for optional log messages.
Bucket objstore.Bucket // Bucket to read stored data from.
// DataBucket reads source log objects during LogMerge compaction. When
// Bucket is prefixed with the index-storage prefix (compaction wiring),
// source log objects live at the unprefixed dataobj root and must be read
// through this bucket. Optional; nil falls back to Bucket.
DataBucket objstore.Bucket
Config WorkerConfig // Configuration for the worker.
// Address to advertise to other workers and schedulers. Required.
AdvertiseAddr net.Addr
// Absolute path of the endpoint where the frame handler is registered.
// Used for connecting to scheduler and other workers.
Endpoint string
// ScratchStore holds the temporary data of compaction tasks. Optional; nil
// keeps the data in memory.
ScratchStore scratch.Store
// IndexobjCfg is the builder config for compacted index objects.
IndexobjCfg logsobj.BuilderBaseConfig
// LogsobjCfg is the builder config for compacted log objects.
LogsobjCfg logsobj.BuilderBaseConfig
// IndexMergeObserver is used by compaction to populate output-size
// histograms. Optional; nil disables observation.
IndexMergeObserver executor.IndexMergeObserver
// NewLogMergeObserver returns the LogMerge observer for each worker
// thread. Optional; nil disables observation.
NewLogMergeObserver executor.NewLogMergeObserverFunc
}
WorkerParams holds parameters for constructing a new Worker.
Directories
¶
| Path | Synopsis |
|---|---|
|
Package compactor contains the dataobj compaction planner and worker service skeletons.
|
Package compactor contains the dataobj compaction planner and worker service skeletons. |
|
internal/compactortest
Package compactortest provides object-backed compaction integration fixtures.
|
Package compactortest provides object-backed compaction integration fixtures. |
|
internal
|
|
|
metrictimer
Package metrictimer provides a small helper for timing named operations.
|
Package metrictimer provides a small helper for timing named operations. |
|
obslock
Package obslock provides an observed sync.RWMutex that records how long callers wait to acquire a lock and how long they hold it.
|
Package obslock provides an observed sync.RWMutex that records how long callers wait to acquire a lock and how long they hold it. |
|
proto/ulid
Package ulid provides a ULID implementation.
|
Package ulid provides a ULID implementation. |
|
scheduler
Package scheduler provides an implementation of workflow.Runner that works by scheduling tasks to be executed by a set of workers.
|
Package scheduler provides an implementation of workflow.Runner that works by scheduling tasks to be executed by a set of workers. |
|
scheduler/schedulerstat
Package schedulerstat declares scheduler-recorded per-task observation statistics.
|
Package schedulerstat declares scheduler-recorded per-task observation statistics. |
|
scheduler/wire
Package wire provides the wire protocol for how peers scheduler peers communicate.
|
Package wire provides the wire protocol for how peers scheduler peers communicate. |
|
util/dag
Package dag provides utilities for working with directed acyclic graphs (DAGs).
|
Package dag provides utilities for working with directed acyclic graphs (DAGs). |
|
util/ewma
Package ewma provides an implementation of an exponentially weighted moving average (EWMA) that can be reported as a Prometheus metric.
|
Package ewma provides an implementation of an exponentially weighted moving average (EWMA) that can be reported as a Prometheus metric. |
|
util/queue/fair
Package fair implements a Hierarchical Fair Queue (HFQ), providing balanced service across a hierarchy of queues.
|
Package fair implements a Hierarchical Fair Queue (HFQ), providing balanced service across a hierarchy of queues. |
|
worker
Package worker provides a mechanism to connect to the [scheduler] for executing tasks.
|
Package worker provides a mechanism to connect to the [scheduler] for executing tasks. |
|
worker/workerstat
Package workerstat declares worker-recorded per-task observation statistics.
|
Package workerstat declares worker-recorded per-task observation statistics. |
|
workflow
Package workflow defines how to represent physical plans as distributed workflows.
|
Package workflow defines how to represent physical plans as distributed workflows. |
Click to show internal directories.
Click to hide internal directories.