Documentation
¶
Overview ¶
Package workers provides a bounded in-process worker pool with retry, tracing, and mandatory observability through webFramework.AddLog.
Workers run outside HTTP request contexts, so each job receives a job-owned webFramework.WebFramework backed by a concurrency-safe BackgroundParser. This ensures that webFramework.AddLog calls (including those from external API calls via handlers.CallAPI) flow into the Splunk transaction pipeline.
Index ¶
- Constants
- Variables
- type BackgroundParser
- func (p *BackgroundParser) AddCustomAttributes(attr slog.Attr)
- func (p *BackgroundParser) GetContext() context.Context
- func (p *BackgroundParser) GetLocal(name string) any
- func (p *BackgroundParser) GetLocalString(name string) string
- func (p *BackgroundParser) GetTraceContext() trace.SpanContext
- func (p *BackgroundParser) SetContext(ctx context.Context)
- func (p *BackgroundParser) SetLocal(name string, value any)
- func (p *BackgroundParser) SetTraceContext(spanCtx trace.SpanContext)
- func (p *BackgroundParser) StartSpan(name string, opts ...trace.SpanStartOption) (context.Context, trace.Span)
- type Config
- type InProcessWorker
- type Job
- type JobContext
- type JobHandler
- type JobOptions
- type ScheduledJob
- type Scheduler
- type SchedulerStats
- type Stats
- type TransactionSink
- type Worker
Constants ¶
const CallAPILogEntry string = "ApiCall"
CallAPILogEntry mirrors the v1 handlers.CallAPILogEntry constant so worker jobs can collect API-call log arrays in the finalizer without importing the v1 handlers package directly.
Variables ¶
var ErrInvalidJob = errors.New("workers: invalid job (empty name or nil handler)")
ErrInvalidJob is returned when a job has an empty name or nil handler.
var ErrQueueFull = errQueueFull{}
ErrQueueFull is returned when the job queue is full and BlockOnFull is false.
var ErrShutdown = errShutdown{}
ErrShutdown is returned when Submit is called after Shutdown has begun.
Functions ¶
This section is empty.
Types ¶
type BackgroundParser ¶
type BackgroundParser struct {
webFramework.FakeParser
// contains filtered or unexported fields
}
BackgroundParser is a concurrency-safe implementation of the root webFramework.RequestParser interface for worker jobs. It has no HTTP request context; most methods are no-ops inherited from FakeParser.
The critical methods — GetLocal, SetLocal, AddCustomAttributes — are overridden with mutex-protected versions that safely store AddLog entries and forward custom attributes to the TransactionSink for the Splunk transaction pipeline.
This parser satisfies the full root webFramework.RequestParser contract so that webFramework.AddLog, webFramework.CollectLogArrays, and handlers.CallAPI work directly with no adapters or type assertions.
func (*BackgroundParser) AddCustomAttributes ¶
func (p *BackgroundParser) AddCustomAttributes(attr slog.Attr)
AddCustomAttributes adds a log attribute to the transaction sink. This is called by webFramework.CollectLogArrays/CollectLogTags to flush collected log entries into the Splunk transaction pipeline.
func (*BackgroundParser) GetContext ¶
func (p *BackgroundParser) GetContext() context.Context
GetContext returns the job context for tracing propagation.
func (*BackgroundParser) GetLocal ¶
func (p *BackgroundParser) GetLocal(name string) any
GetLocal returns a value from local storage by name (concurrency-safe).
func (*BackgroundParser) GetLocalString ¶
func (p *BackgroundParser) GetLocalString(name string) string
GetLocalString returns a local value as a string (concurrency-safe).
func (*BackgroundParser) GetTraceContext ¶
func (p *BackgroundParser) GetTraceContext() trace.SpanContext
GetTraceContext returns the trace span context from the parser.
func (*BackgroundParser) SetContext ¶
func (p *BackgroundParser) SetContext(ctx context.Context)
SetContext updates the job context.
func (*BackgroundParser) SetLocal ¶
func (p *BackgroundParser) SetLocal(name string, value any)
SetLocal stores a value in local storage by name (concurrency-safe). This is the method called by webFramework.AddLog to store log entries.
func (*BackgroundParser) SetTraceContext ¶
func (p *BackgroundParser) SetTraceContext(spanCtx trace.SpanContext)
SetTraceContext is a no-op for the background parser.
func (*BackgroundParser) StartSpan ¶
func (p *BackgroundParser) StartSpan(name string, opts ...trace.SpanStartOption) (context.Context, trace.Span)
StartSpan returns a no-op span for the background parser.
type Config ¶
type Config struct {
// WorkerCount is the number of goroutines processing jobs.
// Default: runtime.NumCPU().
WorkerCount int
// QueueSize is the buffered channel capacity.
// Default: 100.
QueueSize int
// BlockOnFull determines whether Submit blocks when the queue is full
// (true) or returns ErrQueueFull immediately (false).
// Default: false.
BlockOnFull bool
// Clock is the clock source for deterministic testing.
// If nil, time.Now is used.
Clock func() time.Time
// JitterSource is the jitter source for deterministic testing.
// If nil, a package-level locked random source is used.
JitterSource func(max int64) int64
}
Config configures an InProcessWorker.
func DefaultConfig ¶
func DefaultConfig() Config
DefaultConfig returns a Config with sensible defaults.
type InProcessWorker ¶
type InProcessWorker struct {
// contains filtered or unexported fields
}
InProcessWorker is a bounded goroutine pool implementation of Worker.
func NewInProcessWorker ¶
func NewInProcessWorker(config Config) *InProcessWorker
NewInProcessWorker creates a new InProcessWorker with the given configuration.
func (*InProcessWorker) Shutdown ¶
func (w *InProcessWorker) Shutdown(ctx context.Context) error
Shutdown stops accepting new jobs, drains the queue, and waits for in-flight jobs to complete or the context to expire. Shutdown is idempotent; calling it multiple times is safe and returns the same result.
func (*InProcessWorker) Stats ¶
func (w *InProcessWorker) Stats() Stats
Stats returns current worker pool statistics.
func (*InProcessWorker) Submit ¶
func (w *InProcessWorker) Submit(ctx context.Context, job Job) error
Submit enqueues a job for asynchronous execution. Returns an error if the queue is full, the worker is shutting down, or the job is invalid. Only accepted submissions increment the Submitted counter.
The shutdown check and queue send are synchronized under mu to prevent a send on a closed channel when Shutdown runs concurrently.
type Job ¶
type Job struct {
// Name identifies the job for logging and tracing.
Name string
// Handler is the function to execute.
Handler JobHandler
// Options configures retry, backoff, and tracing.
Options JobOptions
}
Job defines a unit of asynchronous work.
type JobContext ¶
type JobContext struct {
// Context carries cancellation and tracing.
Context context.Context
// WebFramework is a job-owned root webFramework.WebFramework with a
// BackgroundParser that supports webFramework.AddLog calls. This
// ensures that external API calls and critical business events
// within jobs are logged to the Splunk transaction pipeline.
WebFramework webFramework.WebFramework
// JobName is the name of the job.
JobName string
// Attempt is the current attempt number (1-based).
Attempt int
// Attributes are tracing attributes for the job.
Attributes map[string]string
// contains filtered or unexported fields
}
JobContext provides the execution context for a worker job.
type JobHandler ¶
type JobHandler func(*JobContext) error
JobHandler is the function executed by a worker job. The JobContext provides a context and a job-owned webFramework.WebFramework for mandatory AddLog calls.
type JobOptions ¶
type JobOptions struct {
// MaxAttempts is the maximum number of execution attempts (including the first).
// Default: 1 (no retry).
MaxAttempts int
// InitialBackoff is the delay before the first retry.
// Default: 100ms.
InitialBackoff time.Duration
// MaxBackoff is the maximum delay between retries.
// Default: 5s.
MaxBackoff time.Duration
// Jitter adds randomness to backoff to prevent thundering herd.
// Default: true.
Jitter bool
// Attributes are tracing attributes for the job span.
Attributes map[string]string
// OnFailure is called when all attempts are exhausted.
// It receives the final error and the total number of attempts.
OnFailure func(err error, attempts int)
// PropagateCancel, when true, makes the job context derive from
// the submit context so cancellation propagates. When false
// (default), the job context uses context.WithoutCancel so
// the job outlives the request.
PropagateCancel bool
}
JobOptions configures job execution behavior.
type ScheduledJob ¶
type ScheduledJob struct {
// Name identifies the job for logging and tracing.
Name string
// Handler is the function executed on each tick.
Handler JobHandler
// Interval is the time between successive ticks.
// Must be > 0.
Interval time.Duration
// Options configures retry, backoff, and tracing for each tick.
// Retry applies within a single tick: if the handler returns an
// error, it is retried up to MaxAttempts with backoff before
// the next tick interval begins.
Options JobOptions
}
ScheduledJob defines a periodic background task run by a Scheduler. Unlike InProcessWorker jobs (which are submitted once and executed asynchronously), a ScheduledJob runs its Handler at a fixed Interval until the Scheduler is shut down.
Each tick of the scheduler creates a fresh JobContext with a BackgroundParser and TransactionSink, providing the same mandatory webFramework.AddLog observability as InProcessWorker jobs.
type Scheduler ¶
type Scheduler struct {
// contains filtered or unexported fields
}
Scheduler runs periodic background tasks with mandatory webFramework.AddLog observability. Each tick of each scheduled job receives a job-owned webFramework.WebFramework backed by a concurrency-safe BackgroundParser, ensuring that AddLog calls flow into the Splunk transaction pipeline.
The Scheduler is designed for long-running poller loops (e.g. periodic data sync, health checks, cache refresh) that run at fixed intervals. For discrete, one-shot job submission, use InProcessWorker.
The Scheduler is safe for concurrent use. Schedule can be called before or after Start, but jobs only begin ticking after Start is called. Shutdown stops all tickers and waits for in-flight ticks to complete or the context to expire.
func NewScheduler ¶
func NewScheduler() *Scheduler
NewScheduler creates a new Scheduler. Jobs can be registered via Schedule before or after Start is called.
func (*Scheduler) Schedule ¶
func (s *Scheduler) Schedule(job ScheduledJob) error
Schedule registers a periodic job. If the Scheduler has already started, the job begins ticking immediately. If the Scheduler has not started, the job begins ticking when Start is called.
Returns an error if the job name is empty, the handler is nil, the interval is <= 0, or a job with the same name is already registered.
func (*Scheduler) Shutdown ¶
Shutdown stops all scheduled jobs, waits for in-flight ticks to complete or the context to expire. Shutdown is idempotent.
func (*Scheduler) Start ¶
func (s *Scheduler) Start()
Start begins ticking all registered jobs. Jobs registered after Start are started immediately by Schedule. Start is idempotent.
func (*Scheduler) Stats ¶
func (s *Scheduler) Stats() map[string]SchedulerStats
Stats returns a map of job name to SchedulerStats for all registered jobs. The stats are a point-in-time snapshot and may be slightly stale for in-flight ticks.
type SchedulerStats ¶
type SchedulerStats struct {
Ticks int64
Succeeded int64
Failed int64
LastRun time.Time
LastErr string
InFlight bool
NextRun time.Time
}
SchedulerStats holds per-job statistics for a Scheduler.
type Stats ¶
type Stats struct {
Submitted int64
Succeeded int64
Failed int64
InFlight int64
QueueDepth int
Workers int
}
Stats holds worker pool statistics.
type TransactionSink ¶
type TransactionSink struct {
// contains filtered or unexported fields
}
TransactionSink collects AddLog entries emitted during a job attempt and flushes them as a single transaction log entry after the attempt completes. This ensures worker observability even when the job handler does not explicitly collect logs.
func NewTransactionSink ¶
func NewTransactionSink() *TransactionSink
NewTransactionSink creates a new empty TransactionSink.
func (*TransactionSink) Add ¶
func (s *TransactionSink) Add(attr slog.Attr)
Add appends a log attribute to the sink.
func (*TransactionSink) Entries ¶
func (s *TransactionSink) Entries() []slog.Attr
Entries returns a copy of the collected log attributes.
type Worker ¶
type Worker interface {
// Submit enqueues a job for asynchronous execution.
// Returns an error if the queue is full or the worker is shutting down.
Submit(ctx context.Context, job Job) error
// Shutdown stops accepting new jobs, drains the queue, and waits
// for in-flight jobs to complete or the context to expire.
Shutdown(ctx context.Context) error
// Stats returns current worker pool statistics.
Stats() Stats
}
Worker is the interface for submitting and managing background jobs.