Documentation
¶
Overview ¶
Package worker provides a mechanism to connect to the [scheduler] for executing tasks.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Config ¶
type Config struct {
// Logger for optional log messages.
Logger log.Logger
// Metrics for the worker's wire connections.
WireMetrics *wire.Metrics
// Bucket to read stored data from.
Bucket objstore.Bucket
// Metastore client to access indexes.
Metastore metastore.Metastore
// Dialer to establish connections to scheduler and remote workers.
Dialer wire.Dialer
// Listener for accepting connections from workers.
Listener wire.Listener
// SchedulerAddress is the address of the remote scheduler to connect to.
// SchedulerAddress is used when there is exactly one scheduler. Use
// SchedulerLookupAddress to discover multiple schedulers by DNS SRV lookup.
SchedulerAddress net.Addr
// SchedulerLookupAddress is the address of the remote scheduler to look up.
// Scheduler addresses are resolved using DNS SRV records at this address.
SchedulerLookupAddress string
// SchedulerLookupInterval is the frequency at which the worker will attempt
// to find new remote schedulers.
SchedulerLookupInterval time.Duration
// BatchSize specifies the maximum number of rows to retrieve in a single
// read call of a task pipeline, or to send in a single message to a peer (sink).
BatchSize int64
// PrefetchBytes controls the number of bytes prefetched when opening a
// data object in scan tasks.
PrefetchBytes int64
// NumThreads is the number of worker threads to spawn. The number of
// threads corresponds to the number of tasks that can be executed
// concurrently.
//
// If NumThreads is set to 0, NumThreads defaults to [runtime.GOMAXPROCS].
NumThreads int
// Absolute path of the endpoint where the frame handler is registered.
// Used for connecting to scheduler and other workers.
Endpoint string
// StreamFilterer is an optional filterer that can filter streams based on their labels.
// When set, streams are filtered before scanning.
StreamFilterer executor.RequestStreamFilterer `yaml:"-"`
}
Config holds configuration options for Worker.
type Worker ¶
type Worker struct {
// contains filtered or unexported fields
}
Worker requests tasks from a set of [scheduler.Scheduler] instances and executes them. Task results are forwarded along streams, which are received by other Worker instances or the scheduler.
func New ¶
New creates a new instance of a worker. Use Worker.Service to manage the lifecycle of the returned worker.
New returns an error if the provided config is invalid.
func (*Worker) RegisterMetrics ¶
func (w *Worker) RegisterMetrics(reg prometheus.Registerer) error
RegisterMetrics registers metrics about s to report to reg.
func (*Worker) UnregisterMetrics ¶
func (w *Worker) UnregisterMetrics(reg prometheus.Registerer)
UnregisterMetrics unregisters metrics about s from reg.
Click to show internal directories.
Click to hide internal directories.