Documentation
¶
Overview ¶
Package worker provides shared worker pool configuration and execution primitives.
Index ¶
- func DefaultQueueSize() int
- func DefaultTaskTimeout() time.Duration
- func DefaultWorkerCount() int
- type BaseTask
- type ParallelizationStrategy
- type Pool
- type PoolConfig
- func AggressivePoolConfig() *PoolConfig
- func ConservativePoolConfig() *PoolConfig
- func DefaultPoolConfig() *PoolConfig
- func DistributionPoolConfig() *PoolConfig
- func FetchPoolConfig() *PoolConfig
- func IndexingPoolConfig() *PoolConfig
- func LinkingPoolConfig() *PoolConfig
- func PatchPoolConfig() *PoolConfig
- type ProgressBar
- type ProgressTracker
- func (p *ProgressTracker) GetCurrentTask() string
- func (p *ProgressTracker) GetETA() time.Duration
- func (p *ProgressTracker) GetProgress() float64
- func (p *ProgressTracker) Increment(n int64)
- func (p *ProgressTracker) SetCurrentTask(taskID string)
- func (p *ProgressTracker) SetTotal(total int64)
- func (p *ProgressTracker) TaskCompleted(taskID string)
- func (p *ProgressTracker) UpdateTaskProgress(taskID string, progress float64)
- type Result
- type RetryPolicy
- type Task
- type TaskError
- type TaskQueue
- type TaskResult
- type TaskRunSummary
- type TaskStatus
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func DefaultQueueSize ¶
func DefaultQueueSize() int
DefaultQueueSize returns the default queue capacity.
func DefaultTaskTimeout ¶
DefaultTaskTimeout returns the default task timeout.
func DefaultWorkerCount ¶
func DefaultWorkerCount() int
DefaultWorkerCount returns a recommended worker count based on CPU cores.
Types ¶
type BaseTask ¶
type BaseTask struct {
// contains filtered or unexported fields
}
BaseTask provides a basic implementation of the Task interface
func NewBaseTask ¶
NewBaseTask creates a new BaseTask with the given parameters
func (*BaseTask) Dependencies ¶
Dependencies returns the list of task IDs this task depends on
func (*BaseTask) SetStatus ¶
func (t *BaseTask) SetStatus(status TaskStatus)
SetStatus updates the task's status
func (*BaseTask) Status ¶
func (t *BaseTask) Status() TaskStatus
Status returns the current status of the task
type ParallelizationStrategy ¶
type ParallelizationStrategy string
ParallelizationStrategy describes how tasks are distributed.
const ( // StrategyRoundRobin distributes tasks evenly across workers. StrategyRoundRobin ParallelizationStrategy = "round-robin" // StrategyGrouped keeps related tasks together for cache locality. StrategyGrouped ParallelizationStrategy = "grouped" // StrategyFIFO processes tasks in first-in/first-out order. StrategyFIFO ParallelizationStrategy = "fifo" )
type Pool ¶
type Pool struct {
// contains filtered or unexported fields
}
Pool coordinates execution of tasks according to PoolConfig.
func NewPool ¶
func NewPool(cfg *PoolConfig) (*Pool, error)
NewPool creates a pool using the provided configuration.
func (*Pool) Results ¶
func (p *Pool) Results() <-chan TaskResult
Results returns the channel streaming successful task outcomes.
type PoolConfig ¶
type PoolConfig struct {
// Basic pool sizing
WorkerCount int
QueueSize int
// Timeouts
TaskTimeout time.Duration
// Rate limiting
RateLimit rate.Limit
// Retry behaviour
RetryPolicy *RetryPolicy
// Progress & observability
ProgressTracking bool
DetailedLogging bool
// Parallelisation hint (may influence task schedulers)
Strategy ParallelizationStrategy
}
PoolConfig defines the runtime configuration for a worker pool.
func AggressivePoolConfig ¶
func AggressivePoolConfig() *PoolConfig
AggressivePoolConfig preset for stress/integration testing.
func ConservativePoolConfig ¶
func ConservativePoolConfig() *PoolConfig
ConservativePoolConfig preset for low-resource environments.
func DefaultPoolConfig ¶
func DefaultPoolConfig() *PoolConfig
DefaultPoolConfig returns a baseline configuration.
func DistributionPoolConfig ¶
func DistributionPoolConfig() *PoolConfig
DistributionPoolConfig preset for distribution workloads. Uses fixed worker count and reduced rate limit to avoid overwhelming GitLab's Gitaly with concurrent large commits (177+ files per repo).
func FetchPoolConfig ¶
func FetchPoolConfig() *PoolConfig
FetchPoolConfig preset for repository fetching (I/O bound).
func IndexingPoolConfig ¶
func IndexingPoolConfig() *PoolConfig
IndexingPoolConfig preset for indexing tasks (CPU bound).
func LinkingPoolConfig ¶
func LinkingPoolConfig() *PoolConfig
LinkingPoolConfig preset for repository linking operations. Uses conservative worker count and rate limit to avoid overwhelming GitLab with concurrent API requests (tokens, variables, scopes).
func PatchPoolConfig ¶
func PatchPoolConfig() *PoolConfig
PatchPoolConfig preset for patch workloads.
func (*PoolConfig) Validate ¶
func (c *PoolConfig) Validate() error
Validate ensures the pool configuration is coherent.
func (*PoolConfig) WithDefaults ¶
func (c *PoolConfig) WithDefaults() *PoolConfig
WithDefaults returns a copy of the config with default values filled in for zero-value fields.
type ProgressBar ¶
type ProgressBar struct {
// contains filtered or unexported fields
}
ProgressBar manages the CLI progress bar using bubbles/progress
func NewProgressBar ¶
func NewProgressBar(tracker *ProgressTracker, description string) *ProgressBar
NewProgressBar creates a new CLI progress bar
func (*ProgressBar) Complete ¶
func (pb *ProgressBar) Complete()
Complete marks the progress bar as complete and waits for it to finish rendering
func (*ProgressBar) Update ¶
func (pb *ProgressBar) Update()
Update updates the progress bar with current tracker state
type ProgressTracker ¶
type ProgressTracker struct {
// contains filtered or unexported fields
}
ProgressTracker manages the progress of all tasks
func NewProgressTracker ¶
func NewProgressTracker(totalTasks int) *ProgressTracker
NewProgressTracker creates a new ProgressTracker
func (*ProgressTracker) GetCurrentTask ¶
func (p *ProgressTracker) GetCurrentTask() string
GetCurrentTask returns the current task
func (*ProgressTracker) GetETA ¶
func (p *ProgressTracker) GetETA() time.Duration
GetETA calculates the estimated remaining time
func (*ProgressTracker) GetProgress ¶
func (p *ProgressTracker) GetProgress() float64
GetProgress returns the overall progress
func (*ProgressTracker) Increment ¶
func (p *ProgressTracker) Increment(n int64)
Increment marks n tasks as completed
func (*ProgressTracker) SetCurrentTask ¶
func (p *ProgressTracker) SetCurrentTask(taskID string)
SetCurrentTask sets the current task
func (*ProgressTracker) SetTotal ¶
func (p *ProgressTracker) SetTotal(total int64)
SetTotal allows setting/updating the total number of tasks dynamically
func (*ProgressTracker) TaskCompleted ¶
func (p *ProgressTracker) TaskCompleted(taskID string)
TaskCompleted marks a task as completed
func (*ProgressTracker) UpdateTaskProgress ¶
func (p *ProgressTracker) UpdateTaskProgress(taskID string, progress float64)
UpdateTaskProgress updates the progress of a specific task
type RetryPolicy ¶
type RetryPolicy struct {
MaxAttempts int
InitialBackoff time.Duration
MaxBackoff time.Duration
BackoffMultiplier float64
JitterFraction float64
}
RetryPolicy captures retry/backoff behaviour for tasks.
func DefaultRetryPolicy ¶
func DefaultRetryPolicy() *RetryPolicy
DefaultRetryPolicy returns sensible retry defaults.
func (*RetryPolicy) Validate ¶
func (r *RetryPolicy) Validate() error
Validate ensures retry policy constraints are coherent.
type Task ¶
type Task interface {
Execute(ctx context.Context) (Result, error)
ID() string
Priority() int
Dependencies() []string
Status() TaskStatus
SetStatus(TaskStatus)
}
Task defines the contract for work units processed by the pool.
type TaskQueue ¶
type TaskQueue struct {
// contains filtered or unexported fields
}
TaskQueue manages tasks with priorities and dependencies
func (*TaskQueue) GetNextTask ¶
GetNextTask returns the next available task with highest priority
func (*TaskQueue) MarkTaskComplete ¶
MarkTaskComplete marks a task as completed and updates dependent tasks
type TaskResult ¶
TaskResult wraps a successful task outcome.
type TaskRunSummary ¶
type TaskRunSummary struct {
Results []TaskResult
Errors []TaskError
}
TaskRunSummary collects task outcomes.
func RunTasks ¶
func RunTasks(ctx context.Context, cfg *PoolConfig, tasks []Task) (TaskRunSummary, error)
RunTasks is a convenience helper that creates a pool, submits tasks, waits for completion, and aggregates the results.
type TaskStatus ¶
type TaskStatus int
TaskStatus represents the current state of a task
const ( // StatusPending indicates the task is queued but not yet started. StatusPending TaskStatus = iota // StatusRunning indicates the task is currently executing. StatusRunning // StatusCompleted indicates the task finished successfully. StatusCompleted // StatusFailed indicates the task finished with an error. StatusFailed )