worker

package module
v0.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 30, 2026 License: MIT Imports: 15 Imported by: 0

Documentation

Overview

Package worker implements the claim→heartbeat→execute→complete loop: lease a task (lease TTL + expiry reclaim for exclusivity), heartbeat it while an executor runs, record the outcome, and requeue failures up the requeue ladder; idle polling backs off (cap IdlePollMax, reset on claim/wake).

Viewpoint — Consumer: the pool runtime and the sweepers that depend on claimed work actually finishing; Concern: exclusive claims, liveness (heartbeat, backoff), and outcome recording against the store's single serialized writer; Non-goals: what a task DOES (the executor registry owns that) and where facts live (journal/store own that).

Cross-package map (lifecycle × concern): docs/architecture-understanding/2026-10-23_lifecycle-concern-grid.md

Example

Embedding go-taskqueue in a Go binary: open a store, register executors, run a worker pool, enqueue work. Ctrl-C drains gracefully.

package main

import (
	"context"
	"log"
	"log/slog"
	"os"
	"os/signal"

	"github.com/larsartmann/go-taskqueue/internal/executor"
	"github.com/larsartmann/go-taskqueue/internal/queue"
	"github.com/larsartmann/go-taskqueue/internal/queue/sqlite"
	"github.com/larsartmann/go-taskqueue/internal/task"
	"github.com/larsartmann/go-taskqueue/internal/worker"
)

func main() {
	store, err := sqlite.Open("tasks.db")
	if err != nil {
		log.Fatal(err)
	}
	defer store.Close()

	reg := executor.NewRegistry()
	reg.Register("sh", executor.NewCommandExecutor(""))

	pool := worker.New(store, worker.Config{
		Owner:       "my-app",
		Concurrency: 4,
		Executors:   reg,
	}, slog.Default())

	ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt)
	defer stop()

	if _, err := queue.New(store).Enqueue(ctx, task.New{
		Project: "demo",
		Type:    "sh",
		Payload: []byte(`{"cmd":"go test ./..."}`),
	}); err != nil {
		log.Fatal(err)
	}

	if err := pool.Start(ctx); err != nil {
		log.Fatal(err)
	}
}

Index

Examples

Constants

View Source
const EnvStreakCode = executor.EnvStreakCode

EnvStreakCode marks breaker-burned attempts in the journal: a task.failed fact whose reason carries this code was killed by the environmental-requeue circuit breaker (sustained preflight/gate refusals), not by a judged execution failure. The constant is owned by the executor package (shared classification surface); aliased here for callers that historically reached for worker.EnvStreakCode.

Variables

This section is empty.

Functions

func ExpBackoff

func ExpBackoff(attempt int) time.Duration

ExpBackoff returns an exponential retry delay for the given attempt number.

Types

type Config

type Config struct {
	Owner        string        // lease owner identity (default: auto)
	Concurrency  int           // parallel task executions (default 2)
	PollInterval time.Duration // idle poll gap (default 250ms)
	// IdlePollMax caps the adaptive idle poll gap: consecutive empty
	// claims (ErrNoTaskDue) double the gap up to this cap, and any claim
	// resets it to PollInterval — an idle pool stops paying the full
	// candidate-scan cost at every tick while a fresh task is still
	// claimed within the cap. Default 2s; negative disables the ladder
	// (fixed PollInterval, the historical behavior).
	IdlePollMax time.Duration

	// Wake is the optional claim-wake channel (queue.Waker.Notify from a
	// Waker store): a commit that may have landed a task in PENDING
	// fires it, so an idle loop re-claims in milliseconds instead of
	// waiting out the idle ladder, and the wake resets the ladder to
	// full cadence. The ladder stays the degraded fallback — a nil Wake
	// (or a store without the seam) keeps the pure poll behavior, and
	// the signal is process-local, so same-DB pools in other processes
	// still poll.
	Wake <-chan struct{}

	Lease       time.Duration // claim lease length (default 2m)
	Heartbeat   time.Duration // heartbeat cadence, < Lease (default 30s)
	TaskTimeout time.Duration // per-task execution cap (default 10m)
	Executors   *executor.Registry
	Backoff     func(attempt int) time.Duration // retry backoff (default exp)
	// PreflightBackoff delays re-claiming after a preflight refusal
	// (dirty repo, missing autonomy config). Default 2m. No attempt is
	// burned by preflight refusals.
	PreflightBackoff time.Duration

	// EnvRequeueBurn bounds the environmental-requeue ladder (R3 option
	// (a), docs/planning/2026-10-01_11-19_env-requeue-circuit-breaker.md):
	// after this many consecutive preflight/gate refusals the attempt
	// burns (store.Fail, exponential NotBefore escalation) instead of
	// requeueing again, so a sustained-sick gate cannot churn claims
	// forever. Default 3; negative disables (pure requeue behavior).
	EnvRequeueBurn int

	// Budget is the claim-time spend gate: before a claimed task's
	// executor runs, the hook decides whether the pool may start another
	// paid turn at all. A blocked task is requeued WITHOUT burning an
	// attempt, parked until the returned delay passes (a daily cap
	// resolves at local midnight), so work enqueued before the cap bit
	// cannot burn money after it. The reason rides the task.requeued
	// fact with requeue class "budget". Nil = ungated (historical
	// behavior); budget refusals never ride the environmental
	// streak/burn ladder — an exhausted budget is policy, not a sick
	// environment.
	Budget func(ctx context.Context) (blocked bool, reason string, retryIn time.Duration)

	// DonePreflight is the claim-time done gate: before a claimed task's
	// executor runs, the hook decides whether the task's work is provably
	// already done (footer commits exist, todo item ticked, fix cured…). A
	// done task completes immediately — no agent session, no attempt burn,
	// no review minted — with an AgentResult carrying PreflightDone=reason
	// as the completion detail. The hook must fail open (return false) on
	// its own errors: it augments dispatch, never blocks it. Nil =
	// ungated (historical behavior).
	DonePreflight func(ctx context.Context, t task.Task) (done bool, reason string)
}

Config controls one worker pool.

Example (Budget)

This example arms the claim-time budget gate: the Budget hook runs BEFORE any executor spawns, so an agent pool never spends a paid turn once the day's cap is spent. A blocked claim requeues WITHOUT burning an attempt and parks until the hook's delay (midnight for a daily cap), so work enqueued before the cap bit stays queued instead of costing money after it.

package main

import (
	"context"
	"fmt"
	"time"
)

func main() {
	countEnqueuedToday := func(ctx context.Context) (int, error) {
		// Production: store.CountFacts(ctx, journal.Enqueued, localMidnight)
		return 3, nil //nolint:goerr113 // example stub
	}

	guard := func(ctx context.Context) (blocked bool, reason string, retryIn time.Duration) {
		const dailyCap = 40

		spent, err := countEnqueuedToday(ctx)
		if err != nil {
			return false, "", 0 // fail open: the claim gate is a cost brake, not a correctness gate
		}

		if spent < dailyCap {
			return false, "", 0
		}

		return true, "daily agent budget spent", time.Until(time.Now().AddDate(0, 0, 1).Truncate(24 * time.Hour))
	}

	_ = guard // hand to worker.Config.Budget

	fmt.Println("paid turns stop at the cap; attempts never burn")
}
Output:
paid turns stop at the cap; attempts never burn

type Pool

type Pool struct {
	// contains filtered or unexported fields
}

Pool runs N concurrent claim-execute loops against a store.

func New

func New(store queue.Store, cfg Config, log *slog.Logger) *Pool

New creates a worker pool. Call Start to run it.

func (*Pool) InFlight

func (p *Pool) InFlight() int

InFlight returns how many tasks are currently executing.

func (*Pool) Owner

func (p *Pool) Owner() string

Owner returns the lease owner identity in use (after defaulting).

func (*Pool) Start

func (p *Pool) Start(ctx context.Context) error

Start launches the pool; it blocks the caller until ctx is cancelled, then drains in-flight tasks and returns. Tasks execute under a context that is NOT cancelled by pool shutdown (bounded only by TaskTimeout), so a graceful stop lets agents finish; terminal store writes use that same uncancellable context so a cancelled parent cannot orphan a running task's outcome. Stop waiting with a second Ctrl-C (SIGKILL) if truly urgent.

func (*Pool) Stop

func (p *Pool) Stop()

Stop signals loops to exit after their current task.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL