worker

package
v0.0.0-...-40cbffd Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 16, 2026 License: Apache-2.0 Imports: 16 Imported by: 0

Documentation

Overview

Package worker provides shared worker pool configuration and execution primitives.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func DefaultQueueSize

func DefaultQueueSize() int

DefaultQueueSize returns the default queue capacity.

func DefaultTaskTimeout

func DefaultTaskTimeout() time.Duration

DefaultTaskTimeout returns the default task timeout.

func DefaultWorkerCount

func DefaultWorkerCount() int

DefaultWorkerCount returns a recommended worker count based on CPU cores.

Types

type BaseTask

type BaseTask struct {
	// contains filtered or unexported fields
}

BaseTask provides a basic implementation of the Task interface

func NewBaseTask

func NewBaseTask(id string, priority int, dependencies []string) *BaseTask

NewBaseTask creates a new BaseTask with the given parameters

func (*BaseTask) Dependencies

func (t *BaseTask) Dependencies() []string

Dependencies returns the list of task IDs this task depends on

func (*BaseTask) ID

func (t *BaseTask) ID() string

ID returns the task's unique identifier

func (*BaseTask) Priority

func (t *BaseTask) Priority() int

Priority returns the task's priority level

func (*BaseTask) SetStatus

func (t *BaseTask) SetStatus(status TaskStatus)

SetStatus updates the task's status

func (*BaseTask) Status

func (t *BaseTask) Status() TaskStatus

Status returns the current status of the task

func (*BaseTask) String

func (t *BaseTask) String() string

String provides a string representation of the task

type ParallelizationStrategy

type ParallelizationStrategy string

ParallelizationStrategy describes how tasks are distributed.

const (
	// StrategyRoundRobin distributes tasks evenly across workers.
	StrategyRoundRobin ParallelizationStrategy = "round-robin"
	// StrategyGrouped keeps related tasks together for cache locality.
	StrategyGrouped ParallelizationStrategy = "grouped"
	// StrategyFIFO processes tasks in first-in/first-out order.
	StrategyFIFO ParallelizationStrategy = "fifo"
)

type Pool

type Pool struct {
	// contains filtered or unexported fields
}

Pool coordinates execution of tasks according to PoolConfig.

func NewPool

func NewPool(cfg *PoolConfig) (*Pool, error)

NewPool creates a pool using the provided configuration.

func (*Pool) Close

func (p *Pool) Close()

Close signals no further tasks will be submitted.

func (*Pool) Errors

func (p *Pool) Errors() <-chan TaskError

Errors returns the channel streaming failed task outcomes.

func (*Pool) Results

func (p *Pool) Results() <-chan TaskResult

Results returns the channel streaming successful task outcomes.

func (*Pool) Start

func (p *Pool) Start(ctx context.Context) error

Start launches worker goroutines bound to the provided context.

func (*Pool) Stop

func (p *Pool) Stop()

Stop cancels workers and closes result channels.

func (*Pool) Submit

func (p *Pool) Submit(task Task) error

Submit queues a task for execution.

type PoolConfig

type PoolConfig struct {
	// Basic pool sizing
	WorkerCount int
	QueueSize   int

	// Timeouts
	TaskTimeout time.Duration

	// Rate limiting
	RateLimit rate.Limit

	// Retry behaviour
	RetryPolicy *RetryPolicy

	// Progress & observability
	ProgressTracking bool
	DetailedLogging  bool

	// Parallelisation hint (may influence task schedulers)
	Strategy ParallelizationStrategy
}

PoolConfig defines the runtime configuration for a worker pool.

func AggressivePoolConfig

func AggressivePoolConfig() *PoolConfig

AggressivePoolConfig preset for stress/integration testing.

func ConservativePoolConfig

func ConservativePoolConfig() *PoolConfig

ConservativePoolConfig preset for low-resource environments.

func DefaultPoolConfig

func DefaultPoolConfig() *PoolConfig

DefaultPoolConfig returns a baseline configuration.

func DistributionPoolConfig

func DistributionPoolConfig() *PoolConfig

DistributionPoolConfig preset for distribution workloads. Uses fixed worker count and reduced rate limit to avoid overwhelming GitLab's Gitaly with concurrent large commits (177+ files per repo).

func FetchPoolConfig

func FetchPoolConfig() *PoolConfig

FetchPoolConfig preset for repository fetching (I/O bound).

func IndexingPoolConfig

func IndexingPoolConfig() *PoolConfig

IndexingPoolConfig preset for indexing tasks (CPU bound).

func LinkingPoolConfig

func LinkingPoolConfig() *PoolConfig

LinkingPoolConfig preset for repository linking operations. Uses conservative worker count and rate limit to avoid overwhelming GitLab with concurrent API requests (tokens, variables, scopes).

func PatchPoolConfig

func PatchPoolConfig() *PoolConfig

PatchPoolConfig preset for patch workloads.

func (*PoolConfig) Validate

func (c *PoolConfig) Validate() error

Validate ensures the pool configuration is coherent.

func (*PoolConfig) WithDefaults

func (c *PoolConfig) WithDefaults() *PoolConfig

WithDefaults returns a copy of the config with default values filled in for zero-value fields.

type ProgressBar

type ProgressBar struct {
	// contains filtered or unexported fields
}

ProgressBar manages the CLI progress bar using bubbles/progress

func NewProgressBar

func NewProgressBar(tracker *ProgressTracker, description string) *ProgressBar

NewProgressBar creates a new CLI progress bar

func (*ProgressBar) Abort

func (pb *ProgressBar) Abort()

Abort aborts the progress bar

func (*ProgressBar) Complete

func (pb *ProgressBar) Complete()

Complete marks the progress bar as complete and waits for it to finish rendering

func (*ProgressBar) Update

func (pb *ProgressBar) Update()

Update updates the progress bar with current tracker state

type ProgressTracker

type ProgressTracker struct {
	// contains filtered or unexported fields
}

ProgressTracker manages the progress of all tasks

func NewProgressTracker

func NewProgressTracker(totalTasks int) *ProgressTracker

NewProgressTracker creates a new ProgressTracker

func (*ProgressTracker) GetCurrentTask

func (p *ProgressTracker) GetCurrentTask() string

GetCurrentTask returns the current task

func (*ProgressTracker) GetETA

func (p *ProgressTracker) GetETA() time.Duration

GetETA calculates the estimated remaining time

func (*ProgressTracker) GetProgress

func (p *ProgressTracker) GetProgress() float64

GetProgress returns the overall progress

func (*ProgressTracker) Increment

func (p *ProgressTracker) Increment(n int64)

Increment marks n tasks as completed

func (*ProgressTracker) SetCurrentTask

func (p *ProgressTracker) SetCurrentTask(taskID string)

SetCurrentTask sets the current task

func (*ProgressTracker) SetTotal

func (p *ProgressTracker) SetTotal(total int64)

SetTotal allows setting/updating the total number of tasks dynamically

func (*ProgressTracker) TaskCompleted

func (p *ProgressTracker) TaskCompleted(taskID string)

TaskCompleted marks a task as completed

func (*ProgressTracker) UpdateTaskProgress

func (p *ProgressTracker) UpdateTaskProgress(taskID string, progress float64)

UpdateTaskProgress updates the progress of a specific task

type Result

type Result interface{}

Result represents the outcome value of a task execution.

type RetryPolicy

type RetryPolicy struct {
	MaxAttempts       int
	InitialBackoff    time.Duration
	MaxBackoff        time.Duration
	BackoffMultiplier float64
	JitterFraction    float64
}

RetryPolicy captures retry/backoff behaviour for tasks.

func DefaultRetryPolicy

func DefaultRetryPolicy() *RetryPolicy

DefaultRetryPolicy returns sensible retry defaults.

func (*RetryPolicy) Validate

func (r *RetryPolicy) Validate() error

Validate ensures retry policy constraints are coherent.

type Task

type Task interface {
	Execute(ctx context.Context) (Result, error)
	ID() string
	Priority() int
	Dependencies() []string
	Status() TaskStatus
	SetStatus(TaskStatus)
}

Task defines the contract for work units processed by the pool.

type TaskError

type TaskError struct {
	Task Task
	Err  error
}

TaskError wraps a failed task outcome.

type TaskQueue

type TaskQueue struct {
	// contains filtered or unexported fields
}

TaskQueue manages tasks with priorities and dependencies

func NewTaskQueue

func NewTaskQueue() *TaskQueue

NewTaskQueue creates a new TaskQueue

func (*TaskQueue) AddTask

func (q *TaskQueue) AddTask(task Task)

AddTask adds a task to the queue

func (*TaskQueue) GetNextTask

func (q *TaskQueue) GetNextTask() Task

GetNextTask returns the next available task with highest priority

func (*TaskQueue) MarkTaskComplete

func (q *TaskQueue) MarkTaskComplete(taskID string)

MarkTaskComplete marks a task as completed and updates dependent tasks

type TaskResult

type TaskResult struct {
	Task  Task
	Value Result
}

TaskResult wraps a successful task outcome.

type TaskRunSummary

type TaskRunSummary struct {
	Results []TaskResult
	Errors  []TaskError
}

TaskRunSummary collects task outcomes.

func RunTasks

func RunTasks(ctx context.Context, cfg *PoolConfig, tasks []Task) (TaskRunSummary, error)

RunTasks is a convenience helper that creates a pool, submits tasks, waits for completion, and aggregates the results.

type TaskStatus

type TaskStatus int

TaskStatus represents the current state of a task

const (
	// StatusPending indicates the task is queued but not yet started.
	StatusPending TaskStatus = iota
	// StatusRunning indicates the task is currently executing.
	StatusRunning
	// StatusCompleted indicates the task finished successfully.
	StatusCompleted
	// StatusFailed indicates the task finished with an error.
	StatusFailed
)

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL