workers

package
v2.0.0-...-51e8ac7 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 25, 2026 License: MIT Imports: 12 Imported by: 0

Documentation

Overview

Package workers provides a bounded in-process worker pool with retry, tracing, and mandatory observability through webFramework.AddLog.

Workers run outside HTTP request contexts, so each job receives a job-owned webFramework.WebFramework backed by a concurrency-safe BackgroundParser. This ensures that webFramework.AddLog calls (including those from external API calls via handlers.CallAPI) flow into the Splunk transaction pipeline.

Index

Constants

View Source
const CallAPILogEntry string = "ApiCall"

CallAPILogEntry mirrors the v1 handlers.CallAPILogEntry constant so worker jobs can collect API-call log arrays in the finalizer without importing the v1 handlers package directly.

Variables

View Source
var ErrInvalidJob = errors.New("workers: invalid job (empty name or nil handler)")

ErrInvalidJob is returned when a job has an empty name or nil handler.

View Source
var ErrQueueFull = errQueueFull{}

ErrQueueFull is returned when the job queue is full and BlockOnFull is false.

View Source
var ErrShutdown = errShutdown{}

ErrShutdown is returned when Submit is called after Shutdown has begun.

Functions

This section is empty.

Types

type BackgroundParser

type BackgroundParser struct {
	webFramework.FakeParser
	// contains filtered or unexported fields
}

BackgroundParser is a concurrency-safe implementation of the root webFramework.RequestParser interface for worker jobs. It has no HTTP request context; most methods are no-ops inherited from FakeParser.

The critical methods — GetLocal, SetLocal, AddCustomAttributes — are overridden with mutex-protected versions that safely store AddLog entries and forward custom attributes to the TransactionSink for the Splunk transaction pipeline.

This parser satisfies the full root webFramework.RequestParser contract so that webFramework.AddLog, webFramework.CollectLogArrays, and handlers.CallAPI work directly with no adapters or type assertions.

func (*BackgroundParser) AddCustomAttributes

func (p *BackgroundParser) AddCustomAttributes(attr slog.Attr)

AddCustomAttributes adds a log attribute to the transaction sink. This is called by webFramework.CollectLogArrays/CollectLogTags to flush collected log entries into the Splunk transaction pipeline.

func (*BackgroundParser) GetContext

func (p *BackgroundParser) GetContext() context.Context

GetContext returns the job context for tracing propagation.

func (*BackgroundParser) GetLocal

func (p *BackgroundParser) GetLocal(name string) any

GetLocal returns a value from local storage by name (concurrency-safe).

func (*BackgroundParser) GetLocalString

func (p *BackgroundParser) GetLocalString(name string) string

GetLocalString returns a local value as a string (concurrency-safe).

func (*BackgroundParser) GetTraceContext

func (p *BackgroundParser) GetTraceContext() trace.SpanContext

GetTraceContext returns the trace span context from the parser.

func (*BackgroundParser) SetContext

func (p *BackgroundParser) SetContext(ctx context.Context)

SetContext updates the job context.

func (*BackgroundParser) SetLocal

func (p *BackgroundParser) SetLocal(name string, value any)

SetLocal stores a value in local storage by name (concurrency-safe). This is the method called by webFramework.AddLog to store log entries.

func (*BackgroundParser) SetTraceContext

func (p *BackgroundParser) SetTraceContext(spanCtx trace.SpanContext)

SetTraceContext is a no-op for the background parser.

func (*BackgroundParser) StartSpan

func (p *BackgroundParser) StartSpan(name string, opts ...trace.SpanStartOption) (context.Context, trace.Span)

StartSpan returns a no-op span for the background parser.

type Config

type Config struct {
	// WorkerCount is the number of goroutines processing jobs.
	// Default: runtime.NumCPU().
	WorkerCount int

	// QueueSize is the buffered channel capacity.
	// Default: 100.
	QueueSize int

	// BlockOnFull determines whether Submit blocks when the queue is full
	// (true) or returns ErrQueueFull immediately (false).
	// Default: false.
	BlockOnFull bool

	// Clock is the clock source for deterministic testing.
	// If nil, time.Now is used.
	Clock func() time.Time

	// JitterSource is the jitter source for deterministic testing.
	// If nil, a package-level locked random source is used.
	JitterSource func(max int64) int64
}

Config configures an InProcessWorker.

func DefaultConfig

func DefaultConfig() Config

DefaultConfig returns a Config with sensible defaults.

type InProcessWorker

type InProcessWorker struct {
	// contains filtered or unexported fields
}

InProcessWorker is a bounded goroutine pool implementation of Worker.

func NewInProcessWorker

func NewInProcessWorker(config Config) *InProcessWorker

NewInProcessWorker creates a new InProcessWorker with the given configuration.

func (*InProcessWorker) Shutdown

func (w *InProcessWorker) Shutdown(ctx context.Context) error

Shutdown stops accepting new jobs, drains the queue, and waits for in-flight jobs to complete or the context to expire. Shutdown is idempotent; calling it multiple times is safe and returns the same result.

func (*InProcessWorker) Stats

func (w *InProcessWorker) Stats() Stats

Stats returns current worker pool statistics.

func (*InProcessWorker) Submit

func (w *InProcessWorker) Submit(ctx context.Context, job Job) error

Submit enqueues a job for asynchronous execution. Returns an error if the queue is full, the worker is shutting down, or the job is invalid. Only accepted submissions increment the Submitted counter.

The shutdown check and queue send are synchronized under mu to prevent a send on a closed channel when Shutdown runs concurrently.

type Job

type Job struct {
	// Name identifies the job for logging and tracing.
	Name string

	// Handler is the function to execute.
	Handler JobHandler

	// Options configures retry, backoff, and tracing.
	Options JobOptions
}

Job defines a unit of asynchronous work.

type JobContext

type JobContext struct {
	// Context carries cancellation and tracing.
	Context context.Context

	// WebFramework is a job-owned root webFramework.WebFramework with a
	// BackgroundParser that supports webFramework.AddLog calls. This
	// ensures that external API calls and critical business events
	// within jobs are logged to the Splunk transaction pipeline.
	WebFramework webFramework.WebFramework

	// JobName is the name of the job.
	JobName string

	// Attempt is the current attempt number (1-based).
	Attempt int

	// Attributes are tracing attributes for the job.
	Attributes map[string]string
	// contains filtered or unexported fields
}

JobContext provides the execution context for a worker job.

type JobHandler

type JobHandler func(*JobContext) error

JobHandler is the function executed by a worker job. The JobContext provides a context and a job-owned webFramework.WebFramework for mandatory AddLog calls.

type JobOptions

type JobOptions struct {
	// MaxAttempts is the maximum number of execution attempts (including the first).
	// Default: 1 (no retry).
	MaxAttempts int

	// InitialBackoff is the delay before the first retry.
	// Default: 100ms.
	InitialBackoff time.Duration

	// MaxBackoff is the maximum delay between retries.
	// Default: 5s.
	MaxBackoff time.Duration

	// Jitter adds randomness to backoff to prevent thundering herd.
	// Default: true.
	Jitter bool

	// Attributes are tracing attributes for the job span.
	Attributes map[string]string

	// OnFailure is called when all attempts are exhausted.
	// It receives the final error and the total number of attempts.
	OnFailure func(err error, attempts int)

	// PropagateCancel, when true, makes the job context derive from
	// the submit context so cancellation propagates. When false
	// (default), the job context uses context.WithoutCancel so
	// the job outlives the request.
	PropagateCancel bool
}

JobOptions configures job execution behavior.

type ScheduledJob

type ScheduledJob struct {
	// Name identifies the job for logging and tracing.
	Name string

	// Handler is the function executed on each tick.
	Handler JobHandler

	// Interval is the time between successive ticks.
	// Must be > 0.
	Interval time.Duration

	// Options configures retry, backoff, and tracing for each tick.
	// Retry applies within a single tick: if the handler returns an
	// error, it is retried up to MaxAttempts with backoff before
	// the next tick interval begins.
	Options JobOptions
}

ScheduledJob defines a periodic background task run by a Scheduler. Unlike InProcessWorker jobs (which are submitted once and executed asynchronously), a ScheduledJob runs its Handler at a fixed Interval until the Scheduler is shut down.

Each tick of the scheduler creates a fresh JobContext with a BackgroundParser and TransactionSink, providing the same mandatory webFramework.AddLog observability as InProcessWorker jobs.

type Scheduler

type Scheduler struct {
	// contains filtered or unexported fields
}

Scheduler runs periodic background tasks with mandatory webFramework.AddLog observability. Each tick of each scheduled job receives a job-owned webFramework.WebFramework backed by a concurrency-safe BackgroundParser, ensuring that AddLog calls flow into the Splunk transaction pipeline.

The Scheduler is designed for long-running poller loops (e.g. periodic data sync, health checks, cache refresh) that run at fixed intervals. For discrete, one-shot job submission, use InProcessWorker.

The Scheduler is safe for concurrent use. Schedule can be called before or after Start, but jobs only begin ticking after Start is called. Shutdown stops all tickers and waits for in-flight ticks to complete or the context to expire.

func NewScheduler

func NewScheduler() *Scheduler

NewScheduler creates a new Scheduler. Jobs can be registered via Schedule before or after Start is called.

func (*Scheduler) Schedule

func (s *Scheduler) Schedule(job ScheduledJob) error

Schedule registers a periodic job. If the Scheduler has already started, the job begins ticking immediately. If the Scheduler has not started, the job begins ticking when Start is called.

Returns an error if the job name is empty, the handler is nil, the interval is <= 0, or a job with the same name is already registered.

func (*Scheduler) Shutdown

func (s *Scheduler) Shutdown(ctx context.Context) error

Shutdown stops all scheduled jobs, waits for in-flight ticks to complete or the context to expire. Shutdown is idempotent.

func (*Scheduler) Start

func (s *Scheduler) Start()

Start begins ticking all registered jobs. Jobs registered after Start are started immediately by Schedule. Start is idempotent.

func (*Scheduler) Stats

func (s *Scheduler) Stats() map[string]SchedulerStats

Stats returns a map of job name to SchedulerStats for all registered jobs. The stats are a point-in-time snapshot and may be slightly stale for in-flight ticks.

type SchedulerStats

type SchedulerStats struct {
	Ticks     int64
	Succeeded int64
	Failed    int64
	LastRun   time.Time
	LastErr   string
	InFlight  bool
	NextRun   time.Time
}

SchedulerStats holds per-job statistics for a Scheduler.

type Stats

type Stats struct {
	Submitted  int64
	Succeeded  int64
	Failed     int64
	InFlight   int64
	QueueDepth int
	Workers    int
}

Stats holds worker pool statistics.

type TransactionSink

type TransactionSink struct {
	// contains filtered or unexported fields
}

TransactionSink collects AddLog entries emitted during a job attempt and flushes them as a single transaction log entry after the attempt completes. This ensures worker observability even when the job handler does not explicitly collect logs.

func NewTransactionSink

func NewTransactionSink() *TransactionSink

NewTransactionSink creates a new empty TransactionSink.

func (*TransactionSink) Add

func (s *TransactionSink) Add(attr slog.Attr)

Add appends a log attribute to the sink.

func (*TransactionSink) Entries

func (s *TransactionSink) Entries() []slog.Attr

Entries returns a copy of the collected log attributes.

func (*TransactionSink) Reset

func (s *TransactionSink) Reset()

Reset clears the sink.

type Worker

type Worker interface {
	// Submit enqueues a job for asynchronous execution.
	// Returns an error if the queue is full or the worker is shutting down.
	Submit(ctx context.Context, job Job) error

	// Shutdown stops accepting new jobs, drains the queue, and waits
	// for in-flight jobs to complete or the context to expire.
	Shutdown(ctx context.Context) error

	// Stats returns current worker pool statistics.
	Stats() Stats
}

Worker is the interface for submitting and managing background jobs.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL