Help us shape the future of Go! Take the 2026 Go Developer Survey

workflowexec

package
v1.0.0-rc.12 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 30, 2026 License: Apache-2.0 Imports: 71 Imported by: 0

README

Workflow execution

Workflow owns admission, claims, state transitions and settlement. This package orders the external work performed by selected runners and records its results through workflow.Execution.

  • Executor runs agents and reviews. Its constructor takes the model, scoped credentials, environment, harness and accounting collaborators it needs.
  • Commands runs scripts and tests. Its constructor takes a process runner, progress, repository publication and panic reporting. It needs no model, harness, provider credential or session service.
  • agentenv supplies shared environment policy. The Pi adapter owns Pi format and configuration; execution.Runner owns process/container mechanics.

Follow internal/apptown/work_wiring.go to the bound method, then script.go, tests.go, agent.go or review.go. Script execution claims work, resolves its source, invokes the process, records evidence and then publishes repository work. Tests execute a committed tree and preserve the existing base-failure policy. The different ordering is deliberate; there is no universal runner algorithm.

An agent's or reviewer's outcome is recorded after its paid invocation and publication. A failed recording is retried with the same outcome, under the attempt fence and a bounded backoff (settleProduced); the invocation is never repeated for it. Past the bound the step is an error saying its result could not be recorded; if even that fails, it stays running for startup repair.

Scan/guidance policy remains in Verification; sharing an execution contract does not make every runner one owner.

Shipping claims deploy/preview steps, selects their source, invokes Deployment, and translates the result into evidence and settlement. Deployment owns provider execution and generation guards; a narrow backup requester admits the existing durable follow-up. No model service or App is available to these runners. On an AppTown with a worker (ReleaseBuilds, release_build.go) a landing's deploy step first refuses what would stop the release (no container registry, -private-docker=false, main not on a Docker host, an unknown or mismatched platform), then runs one release-build task through TaskRunner (private Docker, the commit as source, generated build files, no environment or credential), has Deployment publish its images and commit the digest bundle under the step's ownership, settles and releases the task, and only then deploys that bundle, pinned on the generation. A restart ends a deploy step instead of continuing it.

Repositories runs reconciliation and SCM merge-request steps. It resolves Workflow source, invokes SCM/provider operations, then records evidence. The root retains startup hook installation, but supplies no App to either runner.

Calls interpolates nested workflow inputs and invokes Execution.Call. Shared preparation comes from workcommands.Preparation; Workflow owns child admission, parent linkage and subsequent dispatch. No runner registry retains App.

Each paid invocation must record its usage before its answer can count. A failed usage write makes an agent or review step an error, records the incomplete-usage reason in its evidence, and prevents an empty-turn retry or review follow-up. The transcript and parsed output remain available for diagnosis; rerunning a model is not a way to repair missing accounting.

Documentation

Overview

Package workflowexec orchestrates workflow step execution and agent drafting. Workflow owns claims and settlement; this application facade orders preparation and external effects.

Index

Constants

View Source
const AgentKind = "agent/pi"
View Source
const CallKind = "workflow/call"
View Source
const DeployKind = "deploy"
View Source
const MergeRequestKind = "scm/mr"
View Source
const PreviewKind = "preview"
View Source
const ReconcileKind = "reconcile"
View Source
const ReviewKind = "agent/review"
View Source
const ScreenshotKind = "screenshot"

ScreenshotKind is the artifact kind of AppTown's pictures of a preview.

View Source
const ScriptKind = "script"
View Source
const TestsKind = "tests"

Variables

This section is empty.

Functions

func ModelRefusal

func ModelRefusal(selection model.Selection) func(ctx context.Context, projectID ids.ID, spec *schema.Spec, inputsJSON string) (string, error)

ModelRefusal is Work's admission question for a workflow's model steps (agent/pi and agent/review): "" when every one of them has a model, otherwise why not (model.NoModelRefusal). Each step's model is asked for as its execution asks (resolveAgentKnobs; the review's own model): the step's own, else the workflow's default, a catalog pin over both for an agent; then the project's slots and the instance default decide (model.Selection). A request that resolves to nothing is refused; a workflow with no model step is never refused. A selection that cannot be read is an error, not "no model is set up".

func NotContinued

func NotContinued() []string

NotContinued are the step kinds an AppTown restart ends instead of continuing: a deploy step builds, publishes and deploys, and a restart leaves it in error until someone runs it again (design/operations/workers.md, "Releases built on the worker"). Its release-build task, if any, is stopped and released (tasks.UnsettledStepsIn), and its deployment attempt is observed by the deployment owner, never applied again by a resumed step.

func SweepScratchRefs

func SweepScratchRefs(ctx context.Context, db *database.DB, dataDir string) (int, error)

SweepScratchRefs removes, in every project's repository under dataDir, the scratch refs whose step attempt is no longer live: those a crash or a failed removal left. It returns how many it removed, and the first error met; other repositories are still swept.

Types

type AgentTaskDriver

AgentTaskDriver is a worker's task driver, which also reports the agent image it runs agent steps in and the result files a task left (execution.WorkerTasks).

type AgentWorker

type AgentWorker struct {
	Tasks *TaskRunner
}

AgentWorker is where agent steps run when AppTown has a worker: the task runner its script tasks use, whose driver is the worker's (AgentTaskDriver) and whose profile admits the steps' invocations.

type AttemptLinks interface {
	Publish(runID ids.ID, stepID ids.ID, attempt int64, sc trace.SpanContext)
	Forget(runID ids.ID, stepID ids.ID, attempt int64)
}

AttemptLinks lets another owner find a running attempt's span, so an inference request can link to the step it served. Nil means no links.

type Calls

type Calls struct {
	// contains filtered or unexported fields
}

Calls prepares invocation inputs; Workflow owns atomic child admission.

func NewCalls

func NewCalls(db *database.DB, panics interface{ Panic(string, any, []byte) }) *Calls

func (*Calls) Run

func (a *Calls) Run(task *workflow.Execution)

Run executes a workflow/call step: resolve the callee at its current version, create a pinned child run in the parent's context, and return — the child drives itself (pipelines auto-advance); completion records the verdict back on this step.

type CommandConfig

type CommandConfig struct {
	DataDir, Image string
}

type CommandDependencies

type CommandDependencies struct {
	Runner    execution.Runner
	Progress  *progress.Feeds
	Publisher repositoryedit.Publisher
	Branches  *scm.Branches
	// Tasks runs Docker script steps as recorded tasks. Nil keeps the
	// unrecorded path (a runner that dies with AppTown).
	Tasks  *TaskRunner
	Panics interface{ Panic(string, any, []byte) }
	// Docker is asked which containers still write a script's folder.
	Docker *docker.Scope
	// Checks runs a tests step's command, on AppTown's check executor:
	// the worker when one is configured, else this machine. Its zero value
	// runs them through Runner, here.
	Checks CompileCheck
	// Catalog fulfils the app's declared needs (.apptown/app.yaml) as the
	// services a tests step's command runs beside.
	Catalog *dependency.Catalog
	// Now is AppTown's clock; nil is the real one.
	Now func() time.Time
}

type Commands

type Commands struct {
	// contains filtered or unexported fields
}

Commands executes script and test steps. It shares Workflow's execution contract with agent work but requires no model, identity or harness service.

func NewCommands

func NewCommands(db *database.DB, deps CommandDependencies, cfg CommandConfig) *Commands

func (*Commands) RecoverScript

func (a *Commands) RecoverScript(task *workflow.Execution)

RecoverScript continues, in the workflow's dispatch, the task a previous AppTown process recorded for this step's attempt.

func (*Commands) RunScript

func (a *Commands) RunScript(task *workflow.Execution)

RunScript executes a script step as a subprocess in the run's workspace. A Docker script runs as a recorded task (step 2 of the first-slice plan): a AppTown restart adopts it instead of running it again.

func (*Commands) RunTests

func (a *Commands) RunTests(task *workflow.Execution)

RunTests executes a tests step.

type CompileCheck

type CompileCheck struct {
	Run   func(ctx context.Context, req execution.Request) execution.Outcome
	Image string
}

CompileCheck runs one check container and answers with its outcome (verification.CheckRun), in the image the executor's machine has.

type Config

type Config struct {
	DataDir, Image, APIURL string
	// PrivateDocker is apptownd -private-docker: a step declaring private
	// Docker is admitted, here or on a worker, where its attempt records the
	// worker's measured profile. Without it, it is refused.
	PrivateDocker bool
}

type Dependencies

type Dependencies struct {
	Adoption     *adoption.Service
	Integrations *integration.Service
	Secrets      *secret.Service
	Catalog      *dependency.Catalog
	Models       model.Selection
	Capacity     *model.Scheduler
	Progress     *progress.Feeds
	Tokens       *identity.RuntimeTokens
	Docker       *docker.Scope
	Workshop     *agentenv.Workshop
	Instructions *agentenv.Instructions
	Home         piharness.Preparer
	CLI          interface{ Path() (string, error) }
	Process      Process
	Ledger       *usage.Ledger
	Publisher    repositoryedit.Publisher
	Branches     *scm.Branches
	Panics       interface{ Panic(string, any, []byte) }
	// Worker runs agent and review steps when set, and nowhere else
	// (agent_attempt.go, review_worker.go). Nil runs both on AppTown's own
	// Docker.
	Worker *AgentWorker
	// Telemetry; nil means no-op.
	Tracer trace.Tracer
	Meter  metric.Meter
	Links  AttemptLinks
	// Now is AppTown's clock, which stamps the run records this executor
	// writes (artifacts); nil is the real one.
	Now func() time.Time
}

type Executor

type Executor struct {

	// Settle bounds how an agent's or reviewer's finished outcome is
	// recorded again when its write fails (settleProduced). Its zero value
	// is the default bound.
	Settle workflow.Settlement
	// contains filtered or unexported fields
}

func New

func New(db *database.DB, deps Dependencies, cfg Config) *Executor

func (*Executor) BackfillTranscripts

func (a *Executor) BackfillTranscripts(ctx context.Context)

BackfillTranscripts imports, once, the history of agent attempts that recorded none (runs from before transcripts were kept, and attempts the AppTown lost mid-invocation), so every attempt continues through its receipt. The history is the one file in the run's folder whose header names the session (piharness.StepTranscript: regular files only, never the newest file); it is stored and recorded as the attempt's first invocation. An attempt whose session a later attempt continued, or whose file is not there or cannot be kept, is recorded unavailable, with why. Each attempt is recorded once, so running it again imports nothing twice. It runs at startup before recorded work resumes.

func (*Executor) RecoverAgent

func (a *Executor) RecoverAgent(task *workflow.Execution)

RecoverAgent continues, in the workflow's dispatch, the agent step attempt a previous AppTown process ran on the worker: from its newest invocation's recorded state (waiting for it, collecting it, or settling the attempt). An attempt whose step moved on is stopped and released, never settled.

func (*Executor) Review

func (a *Executor) Review(task *workflow.Execution)

Review executes an agent/review step: on AppTown's worker when it has one (runOnWorker), never on its own machine instead.

func (*Executor) Run

func (a *Executor) Run(task *workflow.Execution)

RunAgent executes an agent/pi step. It is a sequence of named pieces — preflight, slot, knobs, steering, prompt, env, sandbox, pi, harvest, record — so a change to one is not a change to all of them.

type Process

type Process interface {
	Run(context.Context, string, execution.Request) execution.Outcome
}

type Recoverer

type Recoverer interface {
	RecoverExecution(context.Context, ids.ID, int64) (*workflow.Execution, error)
}

Recoverer returns the execution of a step attempt a previous process admitted (workflow.Service.RecoverExecution).

type ReleaseBuilds

type ReleaseBuilds struct {
	// Tasks is the task runner whose driver is the worker's
	// (ReleaseTaskDriver).
	Tasks *TaskRunner
	// PrivateDocker is apptownd -private-docker: a release build needs
	// private Docker on the worker.
	PrivateDocker bool
}

ReleaseBuilds is where an AppTown with a worker builds its releases.

type ReleaseData

type ReleaseData interface {
	BeforeRelease(ctx context.Context, projectID ids.ID, deployment string, synthetic bool) (ReleaseWitness, error)
}

ReleaseData is data protection's part in a landing's release: a copy before it, and a witness of each need's data after it.

type ReleaseTaskDriver

ReleaseTaskDriver is the worker's driver as a release build needs it: its agent image (which has Docker's client and buildx), its measured profile, and the images a task left (execution.WorkerTasks).

type ReleaseWitness

type ReleaseWitness interface {
	Complete() bool
	CopyWords() string
	After(ctx context.Context, declared []string) (problems, notes []string)
}

ReleaseWitness is what was read before the release: whether the copy holds all the data, in words, and the check made once it has booted (problems fail the step; notes are what was not compared).

type RelocationRequest

type RelocationRequest struct {
	Attempt tasks.Attempt
	Task    execution.TaskSpec
}

RelocationRequest is the exact persisted admission, read without sending it to a provider. Its Task may contain credentials; it is never journaled/logged.

func RelocationRequests

func RelocationRequests(ctx context.Context, db database.Queryer, dataDir string) ([]RelocationRequest, error)

RelocationRequests reads unreleased local/remote admissions before layout effects. A missing/unreadable request is unknown, not an empty workspace.

func (RelocationRequest) RefuseMovedPaths

func (r RelocationRequest) RefuseMovedPaths(references []repositorylayout.Reference, oldRoot, newRoot string) error

RefuseMovedPaths preserves the exact admitted request. Even a remote task's source repository is a local input for resending its bundle; SharedWorkspace is an opaque backend key and is deliberately never interpreted as a path.

func (RelocationRequest) RefuseWriter

func (r RelocationRequest) RefuseWriter(ctx context.Context, driver execution.TaskDriver, backend string) error

RefuseWriter observes only the originally selected backend. An unavailable backend, ambiguous identity, or private writer without a release receipt does not become evidence that it is safe to change workspace metadata.

type RemoteResults

type RemoteResults interface {
	Collect(ctx context.Context, key, base, message, dest string) (string, error)
}

RemoteResults returns a remote task's commits after base as a bundle written to dest, and the tip it reports ("" when the task changed nothing).

type Repositories

type Repositories struct {
	// contains filtered or unexported fields
}

Repositories adapts repository effects to admitted workflow steps. The provider resolves credentials at use; no transport or App reaches the runner.

func NewRepositories

func NewRepositories(db *database.DB, attachments *scm.RepositoryAttachments, branches *scm.Branches, cfg RepositoryConfig) *Repositories

func (*Repositories) RunReconcile

func (a *Repositories) RunReconcile(task *workflow.Execution)

RunReconcile pins the change's branch as published as the landing's candidate when it already contains the hub's main.

func (*Repositories) RunSCMMR

func (a *Repositories) RunSCMMR(task *workflow.Execution)

RunSCMMR executes an scm/mr step: push the run branch, open an MR, record it as evidence + artifact (the gate's materialization).

type RepositoryConfig

type RepositoryConfig struct {
	DataDir, PublicHost string
	// Now is AppTown's clock, which stamps what a merge request records
	// on a run; nil is the real one.
	Now func() time.Time
}

type Screenshotter

type Screenshotter interface {
	Take(ctx context.Context, base string, pages []string) verification.Capture
}

Screenshotter takes AppTown's pictures of a preview's pages.

type Shipping

type Shipping struct {
	// contains filtered or unexported fields
}

Shipping adapts deployment effects into workflow evidence and settlement. Deployment still owns release admission, generation guards and providers.

func NewShipping

func NewShipping(db *database.DB, d ShippingDependencies, cfg ShippingConfig) *Shipping

func (*Shipping) RunDeploy

func (a *Shipping) RunDeploy(task *workflow.Execution)

handleDeployments: GET /projects/{slug}/deployments RunDeploy executes a deploy step: run the app as the project deployment (stable name + port), replacing any previous release. This is an effect — place it after the acceptance gate in workflows.

func (*Shipping) RunPreview

func (a *Shipping) RunPreview(task *workflow.Execution)

RunPreview executes a preview step: start the app from the run workspace on an ephemeral port; the URL becomes an artifact (the gate's materialization).

type ShippingConfig

type ShippingConfig struct {
	DataDir, Image string
	Routing        deployment.Routing
}

type ShippingDependencies

type ShippingDependencies struct {
	Deployment *deployment.Service
	// Previews boots a run's preview step.
	Previews *preview.Service
	Docker   *docker.Scope
	// Checks runs the landing's compile check where checks run: the
	// worker's executor when one is configured.
	Checks   CompileCheck
	Progress *progress.Feeds
	Backups  interface{ AfterRelease(ids.ID, string) }
	// ReleaseData copies a deployment's data before a landing releases
	// onto it and witnesses it afterwards (dataprotection.BeforeRelease).
	ReleaseData ReleaseData
	// Builds, on an AppTown with a worker, is where its releases are built;
	// nil builds each on its target (docker.PrepareImages), as before.
	Builds *ReleaseBuilds
	// Now is AppTown's clock, which stamps what shipping records on a
	// run; nil is the real one.
	Now func() time.Time
	// Screenshots pictures a run's preview once its boot passes
	// (verification.Screenshots); nil takes none, and the boot's evidence
	// says so.
	Screenshots Screenshotter
}

type TaskProfile

type TaskProfile struct {
	Name    string
	Limits  execution.Limits
	Network execution.NetworkPolicy
}

TaskProfile is an administrator's execution profile: its name and the limits and network policy its tasks run under. Zero limits impose none.

type TaskRunner

type TaskRunner struct {
	Records *tasks.Records
	Driver  execution.TaskDriver
	// Requests is the folder admitted requests are kept in, one folder per
	// operation (<data>/tasks).
	Requests string
	// Profile is the admission every new task carries.
	Profile TaskProfile
	// Backend identifies where tasks run: which Docker daemon, or which
	// worker. Each attempt records it, and a restarted AppTown acts only on
	// attempts recorded for the backend it has now. Empty is the profile's
	// name.
	Backend string
	// Remote is set when tasks run on another machine (a apptown-worker): each
	// is given a published commit instead of a mount, its work comes back as a
	// bundle, and no script step runs on AppTown's own host.
	Remote RemoteResults
	// Now is AppTown's clock, by which a task's deadline and its
	// admission's expiry are written and judged; nil is the real one.
	Now func() time.Time
	// Poll is how often a waiting task checks that its step is still running,
	// and how long to wait before asking an unanswering backend again.
	Poll time.Duration
	// SettleTries is how many times a step's outcome is written before the
	// task is left recorded for a restarted AppTown; zero means three.
	SettleTries int
	// Patience is how long reading a finished task's result keeps asking an
	// unanswering backend; zero is taskGrace.
	Patience time.Duration
	// AbandonAt is a test seam for crash recovery: returning true for a stage
	// ("reserved", "created", "started") abandons the attempt there, as an AppTown that died
	// at that point would.
	AbandonAt func(stage string) bool
	// ScriptWallClock is a test seam: when set, it is every script task's
	// wall clock in place of its step's timeout, so a test reaches a task's
	// deadline without waiting out the shortest a step can declare (a
	// minute). Zero keeps the step's own.
	ScriptWallClock time.Duration
	// contains filtered or unexported fields
}

TaskRunner runs a step's work as a recorded task through a TaskDriver (design/operations/workers.md, "How work travels" and "Recovery"):

  1. reserve the attempt, with its admission and deadline, under an operation key derived from step and attempt;
  2. ensure the task under that key (a retry after a lost reply finds the same task, and a different request under the key is refused), record its identity, then start it;
  3. wait until it exits or its deadline passes, while watching for a recorded stop and for the step to stop being this attempt's;
  4. collect: read the output, commit and publish, settle the step, and only then settle the attempt and remove the task.

The request an attempt is admitted with is kept beside its record, from its reservation until its task's release is confirmed (keepRequest): whatever sends it again, after a lost reply or a restart, sends those bytes, never a request rebuilt from settings that may have changed. A release the backend does not confirm is tried again (ReleaseSettled).

One procedure (execute, then collect) does all of this for a new attempt and for one a previous AppTown process recorded; each step is safe to repeat. An agent step's invocations are driven by the same execute. A lost or failed answer from the backend is observed, never taken as absence, and a task is never started twice for one attempt. A restarted AppTown recovers a recorded task through the workflow's own dispatch (Resume, then RecoverScript), so its run continues when it settles.

func (*TaskRunner) ReleaseSettled

func (r *TaskRunner) ReleaseSettled(ctx context.Context) (int, error)

ReleaseSettled tries again, once each, to release the settled tasks on this AppTown's backend whose release the backend has not confirmed, and records each it confirms. Release is idempotent: a task already gone is released. It returns how many releases were confirmed.

An unreleased row does not prove that what its key names is its task. Rows settled before releases were recorded (migration 97) all arrive here, including attempts refused because another request's task held their key; such a row may have no provider ID. So each release first re-establishes ownership (releaseOwn); the migration does not guess from old details which rows held nothing.

func (*TaskRunner) Resume

func (r *TaskRunner) Resume(ctx context.Context, work Recoverer) ([]ids.ID, error)

Resume sorts the unsettled tasks a previous AppTown process recorded, at startup. A task whose step still belongs to its attempt is returned, for the workflow's dispatch to recover in its stage's order (RecoverScript). A task whose step moved on is stopped and settled now. A task recorded as refused (nothing was created) has its step failed, if still its, and is settled, whatever the backend. A task recorded for another backend than this AppTown's is left as it is, and reported.

func (*TaskRunner) StopStep

func (r *TaskRunner) StopStep(ctx context.Context, stepID ids.ID, attempt int64) (stopped []string, found bool, err error)

StopStep stops a step attempt's tasks that have not settled (an agent's invocations): each stop is recorded before the backend is asked, so the attempt's own runner ends the step. found is false when the attempt has no task. An error names each task whose stop the backend did not confirm; it stays recorded with its stop.

type WorkerRefusal

type WorkerRefusal struct{ Reason string }

WorkerRefusal: the step needs what the worker cannot provide yet, or what it needs could not be read. It is refused before any effect.

func (*WorkerRefusal) Error

func (r *WorkerRefusal) Error() string

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL