Help us shape the future of Go! Take the 2026 Go Developer Survey

worker

package
v3.7.9 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 20, 2026 License: AGPL-3.0 Imports: 39 Imported by: 0

Documentation

Overview

Package worker provides a mechanism to connect to the [scheduler] for executing tasks.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Config

type Config struct {
	// Logger for optional log messages.
	Logger log.Logger

	// Metrics for the worker's wire connections.
	WireMetrics *wire.Metrics

	// Bucket to read stored data from.
	Bucket objstore.Bucket

	// Metastore client to access indexes.
	Metastore metastore.Metastore

	// Dialer to establish connections to scheduler and remote workers.
	Dialer wire.Dialer

	// Listener for accepting connections from workers.
	Listener wire.Listener

	// SchedulerAddress is the address of the remote scheduler to connect to.
	// SchedulerAddress is used when there is exactly one scheduler. Use
	// SchedulerLookupAddress to discover multiple schedulers by DNS SRV lookup.
	SchedulerAddress net.Addr

	// SchedulerLookupAddress is the address of the remote scheduler to look up.
	// Scheduler addresses are resolved using DNS SRV records at this address.
	SchedulerLookupAddress string

	// SchedulerLookupInterval is the frequency at which the worker will attempt
	// to find new remote schedulers.
	SchedulerLookupInterval time.Duration

	// BatchSize specifies the maximum number of rows to retrieve in a single
	// read call of a task pipeline, or to send in a single message to a peer (sink).
	BatchSize int64

	// PrefetchBytes controls the number of bytes prefetched when opening a
	// data object in scan tasks.
	PrefetchBytes int64

	// NumThreads is the number of worker threads to spawn. The number of
	// threads corresponds to the number of tasks that can be executed
	// concurrently.
	//
	// If NumThreads is set to 0, NumThreads defaults to [runtime.GOMAXPROCS].
	NumThreads int

	// Absolute path of the endpoint where the frame handler is registered.
	// Used for connecting to scheduler and other workers.
	Endpoint string

	// StreamFilterer is an optional filterer that can filter streams based on their labels.
	// When set, streams are filtered before scanning.
	StreamFilterer executor.RequestStreamFilterer `yaml:"-"`
}

Config holds configuration options for Worker.

type Worker

type Worker struct {
	// contains filtered or unexported fields
}

Worker requests tasks from a set of [scheduler.Scheduler] instances and executes them. Task results are forwarded along streams, which are received by other Worker instances or the scheduler.

func New

func New(config Config) (*Worker, error)

New creates a new instance of a worker. Use Worker.Service to manage the lifecycle of the returned worker.

New returns an error if the provided config is invalid.

func (*Worker) RegisterMetrics

func (w *Worker) RegisterMetrics(reg prometheus.Registerer) error

RegisterMetrics registers metrics about s to report to reg.

func (*Worker) Service

func (w *Worker) Service() services.Service

Service returns the service used to manage the lifecycle of the worker.

func (*Worker) UnregisterMetrics

func (w *Worker) UnregisterMetrics(reg prometheus.Registerer)

UnregisterMetrics unregisters metrics about s from reg.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL