workflow

package
v0.0.41 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 14, 2026 License: Apache-2.0 Imports: 23 Imported by: 14

README

pkg/workflow

pkg/workflow is the higher-level Go SDK for writing JobDB workers. It builds on pkg/jobdb and runs job workers, task workers, replay, external task waits, and worker loops against any jobdb.WorkflowRuntime.

Use this package when you are writing workflow code. Use pkg/jobdb when you need the lower-level runtime API or runtime implementations.

Core Concepts

  • A JobWorker is the top-level workflow. It orchestrates task calls, waits, child jobs, and final output.
  • A TaskWorker is a reusable unit of work called by a job.
  • An Engine binds workers to a jobdb.WorkflowRuntime.
  • The runtime can be local (sqlite, toy, direct) or remote (runtime/remote) against a running jobdb server.

Quick Start Against A Server

Install and start a server:

npm install -g @colony2/jobdb
jobdb --listen 127.0.0.1:9047 --db jobdb.db

Then run workers against it through the remote runtime adapter:

package main

import (
    "context"
    "log"

    "github.com/colony-2/jobdb/pkg/jobdb"
    remoteruntime "github.com/colony-2/jobdb/pkg/jobdb/runtime/remote"
    "github.com/colony-2/jobdb/pkg/workflow"
)

type DataProcessingJob struct{}

func (DataProcessingJob) Name() string { return "data_processing" }

func (DataProcessingJob) Run(ctx workflow.JobContext, input jobdb.JobData) (jobdb.JobData, error) {
    result, err := ctx.DoTask(jobdb.DefaultRunPolicy(), "validate", input)
    if err != nil {
        return nil, err
    }
    return ctx.DoTask(jobdb.DefaultRunPolicy(), "transform", result)
}

type ValidateTask struct{}

func (ValidateTask) Name() string { return "validate" }

func (ValidateTask) Run(ctx workflow.TaskContext, input jobdb.TaskData) (jobdb.TaskData, error) {
    return input, nil
}

type TransformTask struct{}

func (TransformTask) Name() string { return "transform" }

func (TransformTask) Run(ctx workflow.TaskContext, input jobdb.TaskData) (jobdb.TaskData, error) {
    return input, nil
}

func main() {
    ctx := context.Background()

    runtime, err := remoteruntime.New("http://127.0.0.1:9047", nil)
    if err != nil {
        log.Fatal(err)
    }

    engine, err := workflow.NewEngineBuilder().
        WithRuntime(runtime).
        WithWorkerTenantId("tenant-a").
        PlusWorkers(DataProcessingJob{}, ValidateTask{}, TransformTask{}).
        BuildEngine()
    if err != nil {
        log.Fatal(err)
    }

    go engine.Run(ctx)

    input := jobdb.NewTaskDataOrPanic(map[string]any{"value": 42})
    jobKey, err := engine.SubmitJob(ctx, jobdb.SubmitJob{
        TenantId: "tenant-a",
        JobType:  "data_processing",
        Data:     input,
    })
    if err != nil {
        log.Fatal(err)
    }

    log.Printf("started job: %s", jobKey)
}

Worker Interfaces

type JobWorker interface {
    Name() string
    Run(workflow.JobContext, jobdb.JobData) (jobdb.JobData, error)
}

type TaskWorker interface {
    Name() string
    Run(workflow.TaskContext, jobdb.TaskData) (jobdb.TaskData, error)
}

Jobs use workflow.JobContext:

func (j MyJob) Run(ctx workflow.JobContext, input jobdb.JobData) (jobdb.JobData, error) {
    output, err := ctx.DoTask(jobdb.DefaultRunPolicy(), "task-name", input)
    if err != nil {
        return nil, err
    }

    if err := ctx.AwaitDuration(jobdb.Duration(5 * time.Minute)); err != nil {
        return nil, err
    }

    return output, nil
}

Tasks use workflow.TaskContext:

func (t MyTask) Run(ctx workflow.TaskContext, input jobdb.TaskData) (jobdb.TaskData, error) {
    raw, err := input.GetData()
    if err != nil {
        return nil, err
    }

    var payload MyPayload
    if err := json.Unmarshal(raw, &payload); err != nil {
        return nil, err
    }

    return jobdb.NewTaskData(payload)
}

Engine Configuration

Build an engine over any jobdb.WorkflowRuntime:

engine, err := workflow.NewEngineBuilder().
    WithRuntime(runtime).
    WithWorkerTenantId("tenant-a").
    WithMaxActive(10).
    WithLogger(logger).
    WithAwaitRecycleThreshold(5 * time.Minute).
    PlusWorkers(jobWorker, taskWorkerA, taskWorkerB).
    BuildEngine()

Workers can also be registered after the engine is built:

workset, err := workflow.AsWorkSet(jobWorker, taskWorkerA, taskWorkerB)
if err != nil {
    return err
}
err = engine.RegisterWorkers(workset)

Runtime Choices

Workflow workers can use any runtime implementation:

  • Remote server: remoteruntime.New("http://127.0.0.1:9047", nil).
  • SQLite embedded: sqliteruntime.NewFromConfig(ctx, sqliteruntime.Config{...}).
  • Toy in-memory: toyruntime.New().
  • Direct Postgres: directruntime.NewFromConfig(directruntime.Config{...}).

For server operation and backend flags, see the root README. For runtime package details, see pkg/jobdb/README.md.

Data, Artifacts, Policies, And Errors

The workflow SDK uses pkg/jobdb data types for payloads, artifacts, run policies, schedules, and runtime errors.

Common examples:

data := jobdb.NewTaskDataOrPanic(map[string]any{"n": 1})

policy := jobdb.RunPolicy{
    Retry: jobdb.RetryPolicy{
        InitialInterval:    jobdb.Duration(100 * time.Millisecond),
        BackoffCoefficient: 2,
        MaximumAttempts:    3,
    },
    InvocationTimeout: jobdb.AsDuration(30 * time.Second),
    TotalTimeout:      jobdb.AsDuration(10 * time.Minute),
}

Worker-returned errors are stored as job or task failures. Use jobdb.NewSystemError for infrastructure failures and regular Go errors for application failures.

External Task Completion

Jobs can wait on task capabilities that are completed by external systems. The engine exposes discovery helpers such as FindTasksWaitingForCapability, FindTasksWaiting, and GetWaitingTask; returned TaskHandle values can be completed with Finish.

handles, err := engine.FindTasksWaitingForCapability(ctx, "approval-job", "approve", []string{"tenant-a"})
if err != nil {
    return err
}
for _, handle := range handles {
    output := jobdb.NewTaskDataOrPanic(map[string]any{"approved": true})
    if err := handle.Finish(ctx, output); err != nil {
        return err
    }
}

Replay And Inspection

workflow.ReplayRunRequest and Engine.ReplayJobRun can replay persisted history for inspection and determinism checks. Runtime-level job-run read models come from pkg/jobdb.

Historical replay consumes recorded outcomes and emits their task/job lifecycle events, including failed attempts, with the recorded timestamps. Execution-policy timeouts do not expire historical results or limit inspection; use the caller's context to cancel or bound replay. Recorded timeout failures remain failures. Replay never runs task workers or writes history. Missing outcomes and future waits still produce ReplayCacheMissError.

A job deadline can interrupt orchestration or an in-flight task before a task outcome is recorded. At that boundary, DoTask returns the recorded job timeout without consuming it or emitting task events. The job runner consumes the job outcome and applies the job retry policy. Non-timeout job outcomes in place of task outcomes, and task input mismatches, remain determinism errors.

When changing shared runner timeout, retry, cache, or recovery paths, test both live execution and read-only replay. Final-result assertions alone miss skipped observer events: compare the complete ordered event stream before and after aging fixture timestamps past execution deadlines, including task and job retries ending in success and failure. Also check unchanged chapters, zero task execution, caller cancellation, and incomplete-history cache misses. The regression matrix in replay_history_test.go runs in the normal test suite; it was added after a live recovery scan accidentally skipped historical task events during replay.

Returned timeout errors alone do not cover interruption: they let the worker unwind normally and can produce task/job outcome pairs that a real timer never records. replay_interruption_test.go covers interrupted attempts, late workers, and a task append that commits before cancellation loses its acknowledgement. replay_interruption_integration_test.go generates interrupted histories using real timers through the public run API on toy, SQLite, and remote runtimes, then checks replay event order, unchanged history, and no task re-execution. CI also runs the workflow package with the race detector to check timeout/worker overlap.

Tenant configuration

Both JobContext and TaskContext expose:

snapshot, err := ctx.GetConfiguration(context.Background())
if err != nil {
    return nil, err
}
raw, present := snapshot.Values["build.defaults"]
// Decode raw with json.Unmarshal into an application-owned type when present.
_ = raw
_ = present

The first successful call reads and caches the complete tenant snapshot for the live job attempt. Job and task contexts share it, including task retries and lease renewal. Whole-job retries and resumptions start fresh. Nothing is prefetched; each caller owns its returned map and raw JSON bytes. ResolveSecrets caches values and omissions by name in the same scope. See the cache lifecycle and migration guide. Invocation cancellation and timeouts still apply when passing a background context, and saved contexts cannot read after their invocation ends. Read-only replay rejects live reads. Record settings that control durable workflow branching in normal job input or task results before relying on them during replay. See the configuration contract.

Runtime implementers must add GetConfiguration to WorkflowRuntime and ExecutionLease, and custom job contexts must implement the matching method. Core runtimes must supply Config.Configurations; missing storage fails construction. HTTP servers require jobdb.ConfigurationAdmin as well.

Documentation

Index

Constants

View Source
const (
	JobPrereqComplete = jobdb.JobPrereqComplete
	JobPrereqSuccess  = jobdb.JobPrereqSuccess

	JobStatusReady          = jobdb.JobStatusReady
	JobStatusExpired        = jobdb.JobStatusExpired
	JobStatusPendingJobs    = jobdb.JobStatusPendingJobs
	JobStatusAwaitingFuture = jobdb.JobStatusAwaitingFuture
	JobStatusActive         = jobdb.JobStatusActive
	JobStatusCrashConcern   = jobdb.JobStatusCrashConcern
	JobStatusCancelled      = jobdb.JobStatusCancelled
	JobStatusCompleted      = jobdb.JobStatusCompleted
)
View Source
const (
	DefaultListJobsPageSize = jobdb.DefaultListJobsPageSize
	MaxListJobsPageSize     = jobdb.MaxListJobsPageSize
)
View Source
const (
	TaskAttemptStateSucceeded = jobdb.TaskAttemptStateSucceeded
	TaskAttemptStateFailed    = jobdb.TaskAttemptStateFailed
	TaskAttemptStateReady     = jobdb.TaskAttemptStateReady
	TaskAttemptStateLeased    = jobdb.TaskAttemptStateLeased
	TaskAttemptStateWaiting   = jobdb.TaskAttemptStateWaiting
	TaskAttemptStateRunning   = jobdb.TaskAttemptStateRunning

	TaskOutcomeStatusSucceeded = jobdb.TaskOutcomeStatusSucceeded
	TaskOutcomeStatusFailed    = jobdb.TaskOutcomeStatusFailed

	TaskErrorKindApp     = jobdb.TaskErrorKindApp
	TaskErrorKindSystem  = jobdb.TaskErrorKindSystem
	TaskErrorKindTimeout = jobdb.TaskErrorKindTimeout

	TimeoutScopeInvocation = jobdb.TimeoutScopeInvocation
	TimeoutScopeTotal      = jobdb.TimeoutScopeTotal
)
View Source
const (
	ChapterMetadataNull   = jobdb.ChapterMetadataNull
	ChapterMetadataBool   = jobdb.ChapterMetadataBool
	ChapterMetadataInt    = jobdb.ChapterMetadataInt
	ChapterMetadataDouble = jobdb.ChapterMetadataDouble
	ChapterMetadataString = jobdb.ChapterMetadataString
	ChapterMetadataList   = jobdb.ChapterMetadataList
	ChapterMetadataMap    = jobdb.ChapterMetadataMap
)

Variables

View Source
var (
	ErrArtifactKeyUnavailable   = jobdb.ErrArtifactKeyUnavailable
	ErrChapterNotFound          = jobdb.ErrChapterNotFound
	ErrConflict                 = jobdb.ErrConflict
	ErrExecutionLeaseLost       = jobdb.ErrExecutionLeaseLost
	ErrExistingJobMismatch      = jobdb.ErrExistingJobMismatch
	ErrJobCancelled             = jobdb.ErrJobCancelled
	ErrJobFailed                = jobdb.ErrJobFailed
	ErrJobNotComplete           = jobdb.ErrJobNotComplete
	ErrJobNotFound              = jobdb.ErrJobNotFound
	ErrMissingInputHash         = jobdb.ErrMissingInputHash
	ErrWorkflowNotDeterministic = jobdb.ErrWorkflowNotDeterministic
)
View Source
var ErrReplayShouldNeverMutate = errors.New("replay run should never mutate state")

ErrReplayShouldNeverMutate signals replay attempted to mutate state.

Functions

func AssignArtifactKey

func AssignArtifactKey(art Artifact, key ArtifactKey)

func IsAppError

func IsAppError(err error) bool

func IsConflict

func IsConflict(err error) bool

func IsExecutionLeaseLost

func IsExecutionLeaseLost(err error) bool

func IsExecutionSuspended added in v0.0.40

func IsExecutionSuspended(err error) bool

IsExecutionSuspended recognizes the suspension control error through wrappers. Propagate it to stop task execution. It does not order post-Yield code before Run returns; use JobRunOutcome.Handoff to publish handoff notifications.

func IsExistingJobMismatch

func IsExistingJobMismatch(err error) bool

func IsSystemError

func IsSystemError(err error) bool

func NewSystemError

func NewSystemError(payload SystemErrorPayload) error

func NewTimeoutError

func NewTimeoutError(kind string, after time.Duration, scope string, inputRef *InputReference, retryable bool) error

func SetTaskInputStorageEnabled

func SetTaskInputStorageEnabled(enabled bool)

SetTaskInputStorageEnabled controls whether task inputs are stored in chapters. Enabled by default.

func TaskInputStorageEnabled

func TaskInputStorageEnabled() bool

TaskInputStorageEnabled reports whether task input storage is enabled.

func WaitForJobToComplete

func WaitForJobToComplete(ctx context.Context, timeout time.Duration, jobKey JobKey, engine Engine) error

Types

type AppError

type AppError = jobdb.AppError

type AppErrorOutcome

type AppErrorOutcome = jobdb.AppErrorOutcome

type AppErrorPayload

type AppErrorPayload = jobdb.AppErrorPayload

type ApplicationInputBytes

type ApplicationInputBytes = jobdb.ApplicationInputBytes

type ApplicationOutputBytes

type ApplicationOutputBytes = jobdb.ApplicationOutputBytes

type ApplicationOutputOutcome

type ApplicationOutputOutcome = jobdb.ApplicationOutputOutcome

type Artifact

type Artifact = jobdb.Artifact

func NewArtifact

func NewArtifact(name string, opener func() (io.ReadCloser, int64, error), cleanup func() error) Artifact

func NewArtifactFromBytes

func NewArtifactFromBytes(name string, data []byte) Artifact

func NewArtifactFromFile

func NewArtifactFromFile(name string, filePath string) (Artifact, error)

func NewArtifactFromFileNoCleanup

func NewArtifactFromFileNoCleanup(name string, filePath string) (Artifact, error)

func NewArtifactFromReader

func NewArtifactFromReader(name string, r io.Reader, size int64) Artifact

type ArtifactInfo

type ArtifactInfo = jobdb.ArtifactInfo

type ArtifactKey

type ArtifactKey = jobdb.ArtifactKey

type ArtifactReader

type ArtifactReader = jobdb.ArtifactReader

type ArtifactRef

type ArtifactRef = jobdb.ArtifactRef

type ArtifactUpload

type ArtifactUpload = jobdb.ArtifactUpload

type CancelJob

type CancelJob = jobdb.CancelJob

type CancelJobRequest

type CancelJobRequest = jobdb.CancelJobRequest

type CatchDisposition added in v0.0.36

type CatchDisposition string
const (
	CatchHandled   CatchDisposition = "handled"
	CatchContinue  CatchDisposition = "continue"
	CatchPropagate CatchDisposition = "propagate"
)

type CatchResult added in v0.0.36

type CatchResult struct {
	Disposition CatchDisposition
	Output      TaskData
	Err         error
}

type CatchRule added in v0.0.36

type CatchRule struct {
	ID     string
	Match  func(error) bool
	Handle func(ExecutionContext, error) (CatchResult, error)
}

type Chapter

type Chapter = jobdb.Chapter

type ChapterBody

type ChapterBody = jobdb.ChapterBody

type ChapterMetadata

type ChapterMetadata = jobdb.ChapterMetadata

type ChapterMetadataKind

type ChapterMetadataKind = jobdb.ChapterMetadataKind

type ChapterMetadataValue

type ChapterMetadataValue = jobdb.ChapterMetadataValue

type ChapterOutcome

type ChapterOutcome = jobdb.ChapterOutcome

type ChapterRef

type ChapterRef = jobdb.ChapterRef

type ClientPayloadUpdate added in v0.0.18

type ClientPayloadUpdate = jobdb.ClientPayloadUpdate

type CompleteExecutionRequest

type CompleteExecutionRequest = jobdb.CompleteExecutionRequest

type CompleteTaskIfWaitingRequest

type CompleteTaskIfWaitingRequest = jobdb.CompleteTaskIfWaitingRequest

type ConfigurationSnapshot added in v0.0.33

type ConfigurationSnapshot = jobdb.ConfigurationSnapshot

type Data

type Data = jobdb.Data

type Duration

type Duration = jobdb.Duration

func AsDuration

func AsDuration(t time.Duration) *Duration

type Engine

type Engine interface {
	RegisterWorkers(workset *WorkSet) error
	GetArtifact(tenantId string, key ArtifactKey) (Artifact, error)
	// contains filtered or unexported methods
}

type EngineBuilder

type EngineBuilder struct {
	// contains filtered or unexported fields
}

func NewEngineBuilder

func NewEngineBuilder() *EngineBuilder

func (*EngineBuilder) BuildEngine

func (b *EngineBuilder) BuildEngine() (Engine, error)

func (*EngineBuilder) PlusWorkers

func (e *EngineBuilder) PlusWorkers(jobWorker JobWorker, taskWorkers ...TaskWorker) *EngineBuilder

func (*EngineBuilder) PlusWorkersWithOptions

func (e *EngineBuilder) PlusWorkersWithOptions(jobWorker JobWorker, opts WorkRegistrationOptions, taskWorkers ...TaskWorker) *EngineBuilder

func (*EngineBuilder) WithAwaitRecycleThreshold

func (e *EngineBuilder) WithAwaitRecycleThreshold(d time.Duration) *EngineBuilder

WithAwaitRecycleThreshold configures how far in the future a wait must be before recycling the runner.

func (*EngineBuilder) WithLogger

func (e *EngineBuilder) WithLogger(logger *slog.Logger) *EngineBuilder

func (*EngineBuilder) WithMaxActive

func (e *EngineBuilder) WithMaxActive(maxActive int) *EngineBuilder

func (*EngineBuilder) WithRuntime

func (e *EngineBuilder) WithRuntime(runtime WorkflowRuntime) *EngineBuilder

func (*EngineBuilder) WithWorkerTenantId

func (e *EngineBuilder) WithWorkerTenantId(tenantId string) *EngineBuilder

WithWorkerTenantId configures the tenant polled by the worker loop. Engines with registered workers must set this before BuildEngine.

type EnvelopedTaskData

type EnvelopedTaskData = jobdb.EnvelopedTaskData

type ExecutionChapter added in v0.0.36

type ExecutionChapter = jobdb.ExecutionChapter

type ExecutionContext added in v0.0.36

type ExecutionContext interface {
	Context() context.Context
	GetJobKey() JobKey
	Logger() *slog.Logger
	ExecutionVersions() jobdb.ExecutionVersions
	DoTask(TaskCall, func(context.Context) (TaskData, error)) (TaskData, error)
	DoScope(string, ScopeOptions, func(ExecutionContext) (JobData, error)) (JobData, error)
	AwaitDuration(Duration) error
	AwaitJobs(...string) error
	GetConfiguration(context.Context) (ConfigurationSnapshot, error)
	ResolveSecrets(context.Context, []string) (SecretResolveResult, error)
	ClientPayload() json.RawMessage
	ClientPayloadRevision() int64
	SubmitJob(context.Context, SubmitJob) (JobKey, error)
	SubmitRestartJob(context.Context, SubmitRestartJob) (JobKey, error)
	// Yield reschedules and stops execution. Read JobRunOutcome.Handoff after
	// Run returns for confirmation; post-Yield user code may still be running.
	Yield(context.Context, YieldRequest) error
}

type ExecutionLease

type ExecutionLease = jobdb.ExecutionLease

type ExecutionState added in v0.0.18

type ExecutionState = jobdb.ExecutionState

type ExecutionWorker added in v0.0.36

type ExecutionWorker struct {
	JobType string
	Execute func(ExecutionContext, JobData) (JobData, error)
}

ExecutionWorker selects the versioned invocation/scope engine. It is the temporary registration surface while consumers migrate off JobWorker.Run.

func (ExecutionWorker) Name added in v0.0.36

func (w ExecutionWorker) Name() string

func (ExecutionWorker) Run added in v0.0.36

type FieldName

type FieldName = jobdb.FieldName

type FindTasksWaitingRequest

type FindTasksWaitingRequest struct {
	JobType        string
	TaskType       string
	TenantIds      []string
	MetadataFilter MetadataFilter
	Limit          int
}

type GetJobForRunRequest

type GetJobForRunRequest struct {
	ConsumerVersions map[string]string
	// BeforeRun runs under the validated, renewing lease before application work.
	// It may reschedule the supplied lease to reject execution admission.
	BeforeRun     func(context.Context, ExecutionLease) error
	JobKey        JobKey
	JobWorker     JobWorker
	TaskWorkers   []TaskWorker
	WorkerID      string
	LeaseDuration time.Duration
	Logger        *slog.Logger

	// If the job awaits longer than this, the lease is rescheduled instead of
	// sleeping inline. Zero uses the existing default behavior.
	AwaitThreshold time.Duration
}

type GetJobLeaseRequest

type GetJobLeaseRequest = jobdb.GetJobLeaseRequest

type GetJobRunRequest

type GetJobRunRequest = jobdb.GetJobRunRequest

type GetJobRunResponse

type GetJobRunResponse = jobdb.GetJobRunResponse

type Handoff added in v0.0.41

type Handoff struct {
	NextRoute Route
	Metadata  json.RawMessage
}

Handoff is this Run's acknowledged explicit Yield, not the job's current scheduler state. It is captured before suspension and never inferred from another worker's actions. It does not represent notification delivery.

type InputReference

type InputReference = jobdb.InputReference

type JobAttempt

type JobAttempt = jobdb.JobAttempt

type JobAttemptOutcomeChapter

type JobAttemptOutcomeChapter = jobdb.JobAttemptOutcomeChapter

type JobContext

type JobContext interface {
	// GetConfiguration returns a copy of the first successful tenant snapshot in
	// this live job attempt. Tasks share the same cache; whole-job retries reset it.
	GetConfiguration(context.Context) (ConfigurationSnapshot, error)
	// ResolveSecrets retrieves allowed names under this invocation's live lease.
	// Values and omissions are cached by name for this live job attempt, never
	// recorded. New names refresh all cached available names together; known omissions
	// are not retried. Offline replay rejects reads;
	// use task-level resolution for workflows that must support offline replay.
	ResolveSecrets(context.Context, []string) (SecretResolveResult, error)
	ClientPayload() json.RawMessage
	ClientPayloadRevision() int64
	// Yield reschedules and stops this invocation. It never writes a chapter.
	Yield(context.Context, YieldRequest) error
	GetJobKey() JobKey
	Logger() *slog.Logger
	DoTask(policy RunPolicy, taskType string, data TaskData) (TaskData, error)
	AwaitDuration(waitFor Duration) error
	AwaitJobs(jobIds ...string) error
	SubmitJob(ctx context.Context, submit SubmitJob) (JobKey, error)
	SubmitRestartJob(ctx context.Context, restart SubmitRestartJob) (JobKey, error)
}

type JobData

type JobData = jobdb.TaskData

type JobEndEvent

type JobEndEvent struct {
	JobKey        JobKey
	AttemptNumber int
	Output        JobData
	Err           error
	At            time.Time
}

type JobFailedError

type JobFailedError = jobdb.JobFailedError

type JobHandle

type JobHandle = jobdb.JobHandle

type JobInfo

type JobInfo = jobdb.JobInfo

type JobKey

type JobKey = jobdb.JobKey

type JobPrereqCondition

type JobPrereqCondition = jobdb.JobPrereqCondition

type JobPrerequisite

type JobPrerequisite = jobdb.JobPrerequisite

type JobRunListener

type JobRunListener interface {
	OnJobStart(event JobStartEvent)
	OnTaskStart(event TaskStartEvent)
	OnTaskEnd(event TaskEndEvent)
	OnJobEnd(event JobEndEvent)
}

type JobRunOutcome

type JobRunOutcome struct {
	// Handoff is set only for an explicit Yield acknowledged during this Run.
	// Read it after Run returns; no task callback needs to finish first.
	Handoff       *Handoff
	Status        JobRunStatus
	LeaseAcquired bool

	// Set when Status == COMPLETED.
	Output JobData

	// Set when Status == FAILED.
	JobError error

	// Best-effort job state details, mainly useful when Status == SUSPENDED
	// or NOT_LEASEABLE. For a confirmed handoff, NextRoute is the acknowledged
	// route; these fields are not refreshed from the scheduler.
	JobStatus     *JobStatus
	NextRoute     *Route
	WaitForJobIDs []string
	MissingRoute  *Route
}

type JobRunStatus

type JobRunStatus string
const (
	JobRunNotLeaseable JobRunStatus = "NOT_LEASEABLE"
	JobRunCompleted    JobRunStatus = "COMPLETED"
	JobRunFailed       JobRunStatus = "FAILED"
	JobRunSuspended    JobRunStatus = "SUSPENDED"
)

type JobRunSummary

type JobRunSummary = jobdb.JobRunSummary

type JobRunnable

type JobRunnable struct {
	// contains filtered or unexported fields
}

func GetJobForRun

func GetJobForRun(ctx context.Context, runtime WorkflowRuntime, req GetJobForRunRequest) (*JobRunnable, error)

func GetJobForRunWithLease added in v0.0.22

func GetJobForRunWithLease(ctx context.Context, runtime WorkflowRuntime, lease ExecutionLease, req GetJobForRunRequest) (*JobRunnable, error)

GetJobForRunWithLease prepares one invocation using existing authority. It never acquires work. Validation and renewal occur when Run starts, so a delay between preparation and execution cannot bypass the authority check.

func (*JobRunnable) JobKey

func (r *JobRunnable) JobKey() JobKey

func (*JobRunnable) LeaseAcquired

func (r *JobRunnable) LeaseAcquired() bool

func (*JobRunnable) Outcome

func (r *JobRunnable) Outcome() (JobRunOutcome, bool)

func (*JobRunnable) Run

func (r *JobRunnable) Run(listener JobRunListener) (JobRunOutcome, error)

type JobStart

type JobStart = jobdb.JobStart

type JobStartChapter

type JobStartChapter = jobdb.JobStartChapter

type JobStartEvent

type JobStartEvent struct {
	JobKey        JobKey
	AttemptNumber int
	Input         JobData
	At            time.Time
}

type JobStatus

type JobStatus = jobdb.JobStatus

type JobStore

type JobStore = jobdb.JobStore

type JobSummary

type JobSummary = jobdb.JobSummary

type JobTaskFilter

type JobTaskFilter = jobdb.JobTaskFilter

type JobWorker

type JobWorker interface {
	Name() string
	Run(JobContext, JobData) (JobData, error)
}

type ListChaptersRequest

type ListChaptersRequest = jobdb.ListChaptersRequest

type ListJobsRequest

type ListJobsRequest = jobdb.ListJobsRequest

type ListJobsResponse

type ListJobsResponse = jobdb.ListJobsResponse

type ListScheduleRunsRequest

type ListScheduleRunsRequest = jobdb.ListScheduleRunsRequest

type ListScheduleRunsResponse

type ListScheduleRunsResponse = jobdb.ListScheduleRunsResponse

type ListSchedulesRequest

type ListSchedulesRequest = jobdb.ListSchedulesRequest

type ListSchedulesResponse

type ListSchedulesResponse = jobdb.ListSchedulesResponse

type MetadataFilter

type MetadataFilter = jobdb.MetadataFilter

func Metadata

func Metadata() MetadataFilter

type MetadataPredicate

type MetadataPredicate = jobdb.MetadataPredicate

func MetadataPredicates

func MetadataPredicates(filter MetadataFilter) ([]MetadataPredicate, error)

type NonRetryableError

type NonRetryableError = jobdb.NonRetryableError

type PollWorkRequest

type PollWorkRequest = jobdb.PollWorkRequest

type PutChapterRequest

type PutChapterRequest = jobdb.PutChapterRequest

type ReplayCacheMissError

type ReplayCacheMissError struct {
	JobKey   JobKey
	TaskType string
	Ordinal  int64
	Attempt  int
	Reason   ReplayCacheMissReason
}

ReplayCacheMissError indicates replay could not proceed due to missing cached data.

func (ReplayCacheMissError) Error

func (e ReplayCacheMissError) Error() string

type ReplayCacheMissReason

type ReplayCacheMissReason string
const (
	ReplayCacheMissTaskResultMissing ReplayCacheMissReason = "task_result_missing"
	ReplayCacheMissJobResultMissing  ReplayCacheMissReason = "job_result_missing"
	ReplayCacheMissAwaitNotReady     ReplayCacheMissReason = "await_not_ready"
	ReplayCacheMissAwaitJobsPending  ReplayCacheMissReason = "await_jobs_pending"
)

type ReplayObserver

type ReplayObserver interface {
	OnJobStart(event JobStartEvent)
	OnTaskStart(event TaskStartEvent)
	OnTaskEnd(event TaskEndEvent)
	OnJobEnd(event JobEndEvent)
}

ReplayObserver receives lifecycle events during replay.

type ReplayRunRequest

type ReplayRunRequest struct {
	JobKey   JobKey
	Observer ReplayObserver // optional

	// Optional job worker override for replay-time instrumentation.
	// If nil, the registered job worker is used.
	JobWorker JobWorker
}

ReplayRunRequest describes a cache-only job replay.

type RescheduleExecutionRequest

type RescheduleExecutionRequest = jobdb.RescheduleExecutionRequest

type RestartExtraChapter

type RestartExtraChapter = jobdb.RestartExtraChapter

type RetryPolicy

type RetryPolicy = jobdb.RetryPolicy

type Route added in v0.0.19

type Route = jobdb.Route

type RunPolicy

type RunPolicy = jobdb.RunPolicy

func DefaultRunPolicy

func DefaultRunPolicy() RunPolicy

type RuntimeBuildOptions

type RuntimeBuildOptions struct {
	Logger                *slog.Logger
	MaxActive             int
	AwaitRecycleThreshold time.Duration
	PollTenantId          string
}

type ScheduleFailurePolicy

type ScheduleFailurePolicy = jobdb.ScheduleFailurePolicy

type ScheduleInfo

type ScheduleInfo = jobdb.ScheduleInfo

type ScheduleKey

type ScheduleKey = jobdb.ScheduleKey

type ScheduleMutationRequest

type ScheduleMutationRequest = jobdb.ScheduleMutationRequest

type ScheduleOverlapPolicy

type ScheduleOverlapPolicy = jobdb.ScheduleOverlapPolicy

type ScheduleRunSummary

type ScheduleRunSummary = jobdb.ScheduleRunSummary

type ScheduleSpec

type ScheduleSpec = jobdb.ScheduleSpec

type ScheduleState

type ScheduleState = jobdb.ScheduleState

type ScheduleTarget

type ScheduleTarget = jobdb.ScheduleTarget

type ScheduleTrigger

type ScheduleTrigger = jobdb.ScheduleTrigger

type ScheduleTriggerKind

type ScheduleTriggerKind = jobdb.ScheduleTriggerKind

type ScopeOptions added in v0.0.36

type ScopeOptions struct {
	TotalTimeout   time.Duration
	AttemptTimeout time.Duration
	Retry          RetryPolicy
	Catch          []CatchRule
	// OnRestart runs with fresh consumer state before the selected scope body.
	OnRestart func(json.RawMessage) error
}

type SecretResolveResult added in v0.0.30

type SecretResolveResult = jobdb.SecretResolveResult

type SecretValue added in v0.0.30

type SecretValue = jobdb.SecretValue

type SimpleTaskData

type SimpleTaskData = jobdb.SimpleTaskData

type StoredArtifact

type StoredArtifact = jobdb.StoredArtifact

type SubmitJob

type SubmitJob = jobdb.SubmitJob

type SubmitJobRequest

type SubmitJobRequest = jobdb.SubmitJobRequest

type SubmitRestartJob

type SubmitRestartJob = jobdb.SubmitRestartJob

type SubmitRestartJobRequest

type SubmitRestartJobRequest = jobdb.SubmitRestartJobRequest

type SystemError

type SystemError = jobdb.SystemError

type SystemErrorOutcome

type SystemErrorOutcome = jobdb.SystemErrorOutcome

type SystemErrorPayload

type SystemErrorPayload = jobdb.SystemErrorPayload

type TaskAlternate added in v0.0.23

type TaskAlternate struct {
	TaskType string    `json:"taskType"`
	At       time.Time `json:"at"`
}

TaskAlternate declares another implementation of the same logical task. At is an absolute eligibility time, not a deadline for external completion.

type TaskAlternateAfter added in v0.0.38

type TaskAlternateAfter struct {
	TaskType string        `json:"task_type"`
	After    time.Duration `json:"after"`
}

TaskAlternateAfter makes another implementation eligible after a durable external wait. A locally registered primary still executes immediately. After starts at the first wait and is preserved across recovery. Acquisition of the alternate commits its selection for subsequent task retries. The due time is not a strict cutoff for external answers; normal lease fencing applies.

type TaskAttempt

type TaskAttempt = jobdb.TaskAttempt

type TaskAttemptOutcomeChapter

type TaskAttemptOutcomeChapter = jobdb.TaskAttemptOutcomeChapter

type TaskCall added in v0.0.36

type TaskCall struct {
	Site            string              `json:"site"`
	TaskType        string              `json:"task_type"`
	ContractVersion uint32              `json:"contract_version"`
	Command         json.RawMessage     `json:"command"`
	Policy          RunPolicy           `json:"policy"`
	Catch           []CatchRule         `json:"-"`
	Alternate       *TaskAlternateAfter `json:"alternate,omitempty"`
}

type TaskCompletion

type TaskCompletion struct {
	JobKey JobKey
	Step   int64
	Error  error
}

type TaskContext

type TaskContext struct {
	JobKey JobKey
	// Step is the outcome ordinal for this call to Run. In the versioned
	// engine it remains valid across in-process waits. Suspension or lease loss
	// ends the call; a resumed Run gets a new context and may have a new Step.
	Step int64
	// InvocationID identifies the logical task within JobKey in the versioned
	// engine. It survives task retries and recovery, but changes when an enclosing
	// scope or whole job retries. It is empty for the legacy JobWorker engine.
	InvocationID string
	// AttemptNumber is one-based in the versioned engine. Recovery of unfinished
	// work retains the attempt; a task retry increments it. Legacy contexts use 0.
	AttemptNumber int
	Logger        *slog.Logger
	// contains filtered or unexported fields
}

func NewTaskContext

func NewTaskContext(jobKey JobKey, step int64, logger *slog.Logger, await func(time.Time) error, awaitJobs func(...string) error) TaskContext

NewTaskContext builds a task context with an optional await handler.

func (TaskContext) AwaitDuration

func (tc TaskContext) AwaitDuration(waitFor Duration) error

AwaitDuration pauses task execution for the specified duration. The engine may override this to reschedule work or recycle runners.

func (TaskContext) AwaitJobs

func (tc TaskContext) AwaitJobs(jobIds ...string) error

AwaitJobs waits for the provided job IDs to complete.

func (TaskContext) ClientPayload added in v0.0.18

func (tc TaskContext) ClientPayload() json.RawMessage

func (TaskContext) ClientPayloadRevision added in v0.0.18

func (tc TaskContext) ClientPayloadRevision() int64

func (TaskContext) Context added in v0.0.36

func (tc TaskContext) Context() context.Context

Context is canceled when this execution attempt ends or loses its lease.

func (TaskContext) GetConfiguration added in v0.0.33

func (tc TaskContext) GetConfiguration(ctx context.Context) (ConfigurationSnapshot, error)

GetConfiguration reads the complete tenant configuration using the current invocation's live lease. The first successful read is cached for the live job attempt and shared with other tasks. Each call returns a caller-owned copy. Nothing is prefetched or automatically recorded in workflow history.

func (TaskContext) ResolveSecrets added in v0.0.30

func (tc TaskContext) ResolveSecrets(ctx context.Context, names []string) (SecretResolveResult, error)

ResolveSecrets retrieves allowed names using the current task's live lease. Values and omissions are cached by name for the live job attempt. Requesting a new name refreshes all previously available names together. Known omissions are not retried. Each result is a caller-owned copy. Results are never recorded in workflow history; do not include values in task outputs, artifacts, or errors.

func (TaskContext) SubmitJob added in v0.0.13

func (tc TaskContext) SubmitJob(ctx context.Context, submit SubmitJob) (JobKey, error)

SubmitJob starts a child job using the current job lease.

func (TaskContext) SubmitRestartJob added in v0.0.13

func (tc TaskContext) SubmitRestartJob(ctx context.Context, restart SubmitRestartJob) (JobKey, error)

SubmitRestartJob starts a child restart job using the current job lease.

func (TaskContext) Yield added in v0.0.18

func (tc TaskContext) Yield(ctx context.Context, req YieldRequest) error

Yield reschedules and stops execution. Use JobRunOutcome.Handoff to report the confirmed transition; code following Yield may run after Run returns.

type TaskData

type TaskData = jobdb.TaskData

func ExtractTaskDataResult

func ExtractTaskDataResult(data TaskData) (TaskData, error)

func NewTaskData

func NewTaskData(data any, artifacts ...Artifact) (TaskData, error)

func NewTaskDataOrPanic

func NewTaskDataOrPanic(data any, artifacts ...Artifact) TaskData

func WithTaskOptions added in v0.0.23

func WithTaskOptions(data TaskData, options TaskOptions) TaskData

WithTaskOptions attaches invocation options without changing the input bytes, artifacts, logical task identity, or input hash. It also works through existing JobContext wrappers. Wrappers replacing TaskData must preserve TaskOptionsFor.

type TaskDeterminismMeta

type TaskDeterminismMeta struct {
	Ordinal       int64           `json:"ordinal"`
	TaskType      string          `json:"task_type"`
	WorkerID      string          `json:"worker_id,omitempty"`
	CreatedAt     time.Time       `json:"created_at,omitempty"`
	Attempt       int             `json:"attempt,omitempty"`
	MaxAttempts   int             `json:"max_attempts,omitempty"`
	NextAttemptAt *time.Time      `json:"next_attempt_at,omitempty"`
	BackoffMillis int64           `json:"backoff_ms,omitempty"`
	Retryable     *bool           `json:"retryable,omitempty"`
	InputHash     string          `json:"input_hash,omitempty"`
	InputRef      *InputReference `json:"input_ref,omitempty"`
	RunPolicy     *RunPolicy      `json:"run_policy,omitempty"`
	InputPayload  json.RawMessage `json:"input_payload,omitempty"`
	Version       int             `json:"version,omitempty"`
}

TaskDeterminismMeta exposes deterministic metadata persisted with a task chapter. It is a public, minimal view of the internal chapter metadata used for replay.

type TaskEndEvent

type TaskEndEvent struct {
	InvocationID  string
	JobKey        JobKey
	TaskType      string
	Ordinal       int64
	AttemptNumber int
	Output        TaskData
	Err           error
	At            time.Time
}

type TaskError

type TaskError = jobdb.TaskError

type TaskHandle

type TaskHandle interface {
	JobKey() JobKey
	Data() (TaskData, error)
	Finish(ctx context.Context, taskData TaskData) error
	TaskOrdinalToComplete() int64
	TaskType() string
	CreatedAt() time.Time
	Metadata() json.RawMessage
}

type TaskIO

type TaskIO = jobdb.TaskIO

type TaskInputMismatchError

type TaskInputMismatchError struct {
	TaskType          string
	Ordinal           int64
	CachedInputHash   string
	ComputedInputHash string
	CachedInput       json.RawMessage
	CachedOutput      TaskData
	CachedOutputErr   error
	Meta              TaskDeterminismMeta
}

TaskInputMismatchError is returned when a cached task chapter's input hash does not match the hash of the current DoTask input. It unwraps to ErrWorkflowNotDeterministic so existing callers can continue using errors.Is.

func UnexpectedChapter

func UnexpectedChapter(err error) (TaskInputMismatchError, bool)

UnexpectedChapter extracts TaskInputMismatchError from err, if present.

func (TaskInputMismatchError) CachedInputPayload

func (e TaskInputMismatchError) CachedInputPayload() json.RawMessage

CachedInputPayload returns the persisted task input payload, when task input storage is enabled.

func (TaskInputMismatchError) CachedTaskData

func (e TaskInputMismatchError) CachedTaskData() TaskData

CachedTaskData returns the cached task output (if it could be rehydrated).

func (TaskInputMismatchError) CachedTaskDataErr

func (e TaskInputMismatchError) CachedTaskDataErr() error

CachedTaskDataErr reports any error encountered while rehydrating the cached output.

func (TaskInputMismatchError) ChapterMeta

ChapterMeta exposes the deterministic metadata captured with the cached chapter.

func (TaskInputMismatchError) Error

func (e TaskInputMismatchError) Error() string

func (TaskInputMismatchError) Unwrap

func (e TaskInputMismatchError) Unwrap() error

Unwrap preserves compatibility with errors.Is(err, ErrWorkflowNotDeterministic).

type TaskOptions added in v0.0.23

type TaskOptions struct {
	Alternate *TaskAlternate `json:"alternate,omitempty"`
}

TaskOptions controls dispatch independently of timeout and retry policy. Reconstruct these options from durable workflow state on every replay.

func TaskOptionsFor added in v0.0.23

func TaskOptionsFor(data TaskData) TaskOptions

type TaskOutcome

type TaskOutcome = jobdb.TaskOutcome

type TaskRun

type TaskRun = jobdb.TaskRun

type TaskRuntime

type TaskRuntime = jobdb.TaskRuntime

type TaskStartEvent

type TaskStartEvent struct {
	InvocationID  string
	JobKey        JobKey
	TaskType      string
	Ordinal       int64
	AttemptNumber int
	Input         TaskData
	At            time.Time
}

type TaskWait added in v0.0.18

type TaskWait = jobdb.TaskWait

type TaskWorker

type TaskWorker interface {
	Name() string
	Run(context TaskContext, input TaskData) (TaskData, error)
}

type TimeoutError

type TimeoutError = jobdb.TimeoutError

type TimeoutOutcome

type TimeoutOutcome = jobdb.TimeoutOutcome

type TimeoutPayload

type TimeoutPayload = jobdb.TimeoutPayload

type TriggerScheduleRequest

type TriggerScheduleRequest = jobdb.TriggerScheduleRequest

type UpsertScheduleRequest

type UpsertScheduleRequest = jobdb.UpsertScheduleRequest

type WorkRegistrationOptions

type WorkRegistrationOptions struct {
	ConsumerVersions map[string]string
	MetadataFilter   MetadataFilter
}

type WorkSet

type WorkSet struct {
	JobWorker   JobWorker
	TaskWorkers map[string]TaskWorker
	Options     WorkRegistrationOptions
	// contains filtered or unexported fields
}

func AsWorkSet

func AsWorkSet(jobWorker JobWorker, taskWorkers ...TaskWorker) (*WorkSet, error)

func AsWorkSetWithOptions

func AsWorkSetWithOptions(jobWorker JobWorker, opts WorkRegistrationOptions, taskWorkers ...TaskWorker) (*WorkSet, error)

type Worker

type Worker interface {
	// contains filtered or unexported methods
}

type WorkflowRuntime

type WorkflowRuntime = jobdb.WorkflowRuntime

type YieldRequest added in v0.0.41

type YieldRequest struct {
	Reschedule RescheduleExecutionRequest
	Metadata   json.RawMessage
}

YieldRequest describes an explicit handoff. Metadata is optional consumer JSON returned in JobRunOutcome.Handoff; it is not sent to storage or recorded in task history. Persist anything needed for recovery through Reschedule itself.

Directories

Path Synopsis
internal

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL