scheduledtask

package
v0.8.8 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 30, 2026 License: Apache-2.0 Imports: 22 Imported by: 0

Documentation

Overview

Package scheduledtask runs operator-defined jobs on a cron schedule, in the running container (exec mode) or in a fresh one-off container built from the app's own spec (fresh mode), mirroring internal/backup's Runner/Scheduler split: Runner (this file) does one run and records its outcome, Scheduler (scheduler.go) decides when a run is due. The same Runner backs the cron loop, "run now" and the API trigger, so there is one place that executes a command and records what happened.

Index

Constants

View Source
const (
	HeaderJobRun        = "X-Job-Run"
	HeaderJobName       = "X-Job-Name"
	HeaderJobBackground = "X-Job-Background"
)

Headers sent on every http-mode invocation so the app can tell a job call from user traffic and correlate it with run history.

View Source
const (
	OneShotRunning   = "Running"
	OneShotSucceeded = "Succeeded"
	OneShotFailed    = "Failed"
)

One-shot job states.

Variables

This section is empty.

Functions

func NewRunID added in v0.2.0

func NewRunID() (string, error)

NewRunID mints a run ID (jr_ plus 9 random bytes, URL-safe base64).

func Occurrences added in v0.2.0

func Occurrences(sched *cronexpr.Schedule, loc *time.Location, after, upTo time.Time, limit int) (out []time.Time, truncated bool)

Occurrences returns the cron occurrences in (after, upTo], evaluated on loc's wall clock, oldest first. A wall time skipped by a DST jump runs once at the shifted time; one repeated by a fall-back runs once.

Types

type AppStore

type AppStore interface {
	GetDesiredService(ctx context.Context, name string) (*store.DesiredService, error)
}

AppStore resolves a job's target service.

type AttemptFunc

type AttemptFunc func(ctx context.Context, completion, attempt int) AttemptOutcome

AttemptFunc runs one attempt of completion index (0-based). Each call is expected to record itself in the run history.

type AttemptOutcome

type AttemptOutcome struct {
	RunID  string
	Status string
}

AttemptOutcome is the result of one attempt of one completion.

type HTTPConfigStore

type HTTPConfigStore interface {
	GetScheduledTaskHTTP(ctx context.Context, taskID string) (store.ScheduledTaskHTTP, error)
}

HTTPConfigStore loads the request an http-mode job sends.

type HTTPTarget

type HTTPTarget interface {
	Prepare(ctx context.Context, svc *store.DesiredService) (baseURL string, err error)
}

HTTPTarget wakes an app if it sleeps, waits until it can answer, and returns the base URL its HTTP port is reachable on from the control plane.

type HistoryStore added in v0.2.0

type HistoryStore interface {
	CreateScheduledTaskRun(ctx context.Context, run store.ScheduledTaskRun) error
	FinishScheduledTaskRun(ctx context.Context, run store.ScheduledTaskRun) error
	PruneScheduledTaskRuns(ctx context.Context, taskID string, keep int, outputCutoff time.Time) error
}

HistoryStore persists per-run history. Optional on Runner: nil keeps only the last-run snapshot RunStore writes.

type Limits added in v0.2.0

type Limits struct {
	DefaultTimeout      time.Duration
	MaxRetries          int
	RetryBackoff        time.Duration
	MaxRetryBackoff     time.Duration
	CatchUpLookback     time.Duration
	HistoryKeep         int
	OutputRetention     time.Duration
	MaxPerApp           int
	MaxConcurrentPerApp int
	MaxOutputBytes      int
	BackgroundTimeout   time.Duration
}

Limits are the control-plane-wide knobs for scheduled jobs, all environment-driven so no threshold is hardcoded at a call site.

func DefaultLimits added in v0.2.0

func DefaultLimits() Limits

DefaultLimits is LimitsFromEnv with an empty environment.

func LimitsFromEnv added in v0.2.0

func LimitsFromEnv(lookup func(string) (string, bool)) Limits

LimitsFromEnv reads APP_JOBS_DEFAULT_TIMEOUT, APP_JOBS_MAX_RETRIES, APP_JOBS_RETRY_BACKOFF, APP_JOBS_RETRY_BACKOFF_MAX, APP_JOBS_CATCH_UP_LOOKBACK, APP_JOBS_HISTORY_KEEP, APP_JOBS_OUTPUT_RETENTION, APP_JOBS_MAX_PER_APP, APP_JOBS_MAX_CONCURRENT_PER_APP, APP_JOBS_MAX_OUTPUT_BYTES and APP_JOBS_BACKGROUND_TIMEOUT (how long a background HTTP job is awaited). A value that does not parse or is not positive falls back to its default.

type NodeRuntimeResolver

type NodeRuntimeResolver func(nodeID string) (docker.Runtime, error)

NodeRuntimeResolver resolves the docker.Runtime for a node ID, the same shape internal/api's own resolver has.

type OneShot

type OneShot struct {
	// Sleep waits between retries; nil uses a context-aware timer.
	Sleep func(ctx context.Context, d time.Duration) error
	// Now returns the current time; nil uses time.Now.
	Now func() time.Time
	// contains filtered or unexported fields
}

OneShot drives one job. It needs no controller: Run blocks until the job settles and Snapshot can be read concurrently.

func (*OneShot) Init

func (o *OneShot) Init(spec OneShotSpec)

Init marks the job Running so a snapshot taken before Run starts is valid.

func (*OneShot) Run

func (o *OneShot) Run(ctx context.Context, spec OneShotSpec, attempt AttemptFunc) OneShotSnapshot

Run executes spec with at most spec.Parallelism attempts in flight and returns the final snapshot. A failed attempt is retried after an exponential delay until spec.BackoffLimit failures have been spent.

func (*OneShot) Snapshot

func (o *OneShot) Snapshot() OneShotSnapshot

Snapshot returns a copy of the current state.

type OneShotEntry

type OneShotEntry struct {
	ID     string
	App    string
	TaskID string
	Spec   OneShotSpec
	Job    *OneShot
}

OneShotEntry is one registered job.

type OneShotLimits

type OneShotLimits struct {
	BackoffLimit   int
	Timeout        time.Duration
	MaxCompletions int
	MaxParallelism int
	Keep           int
}

OneShotLimits are the env-driven defaults and ceilings for one-shot jobs.

func OneShotLimitsFromEnv

func OneShotLimitsFromEnv(lookup func(string) (string, bool)) OneShotLimits

OneShotLimitsFromEnv reads APP_JOB_DEFAULT_BACKOFF_LIMIT, APP_JOB_DEFAULT_TIMEOUT, APP_JOB_MAX_COMPLETIONS, APP_JOB_MAX_PARALLELISM and APP_JOB_ONESHOT_KEEP. Unparsable or non-positive values fall back.

type OneShotRegistry

type OneShotRegistry struct {
	// contains filtered or unexported fields
}

OneShotRegistry holds the live and recently finished one-shot jobs of this process, bounded by keep. State is in memory: the per-attempt runs stay in the run history, the job summary does not survive a restart.

func NewOneShotRegistry

func NewOneShotRegistry(keep int) *OneShotRegistry

NewOneShotRegistry returns a registry keeping at most keep finished jobs.

func (*OneShotRegistry) Add

func (g *OneShotRegistry) Add(e *OneShotEntry)

Add registers e and evicts the oldest finished jobs beyond keep.

func (*OneShotRegistry) Get

func (g *OneShotRegistry) Get(id string) (*OneShotEntry, bool)

Get returns the job with id.

type OneShotSnapshot

type OneShotSnapshot struct {
	State          string
	Completions    int
	Succeeded      int
	FailedAttempts int
	Active         int
	RunIDs         []string
	Message        string
	StartedAt      time.Time
	FinishedAt     *time.Time
}

OneShotSnapshot is a point-in-time view of a one-shot job.

type OneShotSpec

type OneShotSpec struct {
	Completions  int
	Parallelism  int
	BackoffLimit int
	BackoffBase  time.Duration
	BackoffMax   time.Duration
}

OneShotSpec describes one run-to-completion job. BackoffLimit is the number of failed attempts tolerated across the whole job before it is declared Failed, as in a Kubernetes Job.

func (*OneShotSpec) Validate

func (s *OneShotSpec) Validate(l OneShotLimits) error

Validate normalizes spec against limits.

type Recoverer added in v0.2.0

type Recoverer interface {
	Cleanup(ctx context.Context, runs []store.ScheduledTaskRun)
}

Recoverer cleans up what a previous process left behind. *Runner satisfies it; optional.

type RunStore

type RunStore interface {
	RecordScheduledTaskRun(ctx context.Context, id string, ranAt time.Time, status, output string) error
}

RunStore persists a run's outcome onto the task's own row.

type Runner

type Runner struct {
	Apps     AppStore
	Runs     RunStore
	History  HistoryStore
	Specs    SpecBuilder
	Resolver NodeRuntimeResolver
	Logger   *slog.Logger
	Limits   Limits
	// Target and HTTPConfigs serve http-mode jobs; Client overrides the
	// HTTP client (nil uses http.DefaultClient).
	Target      HTTPTarget
	HTTPConfigs HTTPConfigStore
	Client      *http.Client
	// NamePrefix namespaces fresh job containers (the brand short name).
	NamePrefix string

	// Now returns the current time; nil falls back to time.Now.
	Now func() time.Time
	// Timeout overrides every attempt's timeout when positive; a test seam.
	Timeout time.Duration
	// Sleep waits between retries; nil uses a context-aware timer.
	Sleep func(ctx context.Context, d time.Duration) error
	// contains filtered or unexported fields
}

Runner runs one job and records the outcome.

func (*Runner) Cleanup added in v0.2.0

func (r *Runner) Cleanup(ctx context.Context, runs []store.ScheduledTaskRun)

Cleanup force-removes the containers of runs a previous process left in flight. Best effort: a node that is unreachable now is simply skipped.

func (*Runner) ContainerName added in v0.2.0

func (r *Runner) ContainerName(service, runID, taskID string) string

ContainerName is the deterministic name of a fresh job container, so a restart can find and remove one the previous process left behind. The run suffix is deliberately not hex, so no service-container parser takes it for a replica.

func (*Runner) Run

func (r *Runner) Run(ctx context.Context, task store.ScheduledTask) error

Run executes task for a manual "run now".

func (*Runner) RunAttempt

func (r *Runner) RunAttempt(ctx context.Context, task store.ScheduledTask) (AttemptOutcome, error)

RunAttempt performs one attempt of task through the same execution path as RunTrigger (history row, redaction, per-app slot) and returns its final status. A full per-app slot comes back as skipped_concurrency so a one-shot waits instead of failing.

func (*Runner) RunTrigger added in v0.2.0

func (r *Runner) RunTrigger(ctx context.Context, task store.ScheduledTask, trig Trigger) error

RunTrigger runs task once. task.ConcurrencyPolicy governs a still in-flight previous run of the same task (forbid skips, replace cancels it), and the per-app cap skips a run when the app already has Limits.MaxConcurrentPerApp jobs running. Retries with backoff happen inside the one run. The returned error is only an infrastructure failure; a failing command is a recorded outcome, not an error.

func (*Runner) Start added in v0.2.0

func (r *Runner) Start(ctx context.Context, task store.ScheduledTask, trigger string) (string, error)

Start records a run's history row synchronously, then runs it in the background, and returns the run ID so a caller can follow it. Without a history store it still runs, with an empty ID. ctx should already be detached from the request that asked.

type ScheduleStore

type ScheduleStore interface {
	ListEnabledScheduledTasks(ctx context.Context) ([]store.ScheduledTask, error)
	ClaimScheduledTaskRun(ctx context.Context, run store.ScheduledTaskRun) (bool, error)
	InterruptRunningScheduledTaskRuns(ctx context.Context, now time.Time) ([]store.ScheduledTaskRun, error)
}

ScheduleStore is the store surface Scheduler needs. *store.DB satisfies it structurally.

type Scheduler

type Scheduler struct {
	Store  ScheduleStore
	Runner TaskRunner
	Logger *slog.Logger
	// Now returns the current time; nil falls back to time.Now.
	Now func() time.Time
	// Grace is how late an occurrence may be and still count as on time;
	// zero means twice the interval passed to Run.
	Grace time.Duration
	// Lookback is the default catch-up window when a job sets none.
	Lookback time.Duration
	// NewID mints run IDs; nil uses NewRunID.
	NewID func() (string, error)
	// contains filtered or unexported fields
}

Scheduler is level-triggered: every tick it derives, from the persisted high-water mark (store.ScheduledTask.LastScheduledFor) and the current time, which cron occurrence is due, claims it atomically in the database and only then runs it. Because the claim is persisted before execution, a restart can neither run an occurrence twice nor lose track of one: an occurrence found overdue is run once (catch_up run_once, inside the lookback), or recorded as missed (skip), never silently dropped.

func NewScheduler

func NewScheduler(st ScheduleStore, runner TaskRunner, logger *slog.Logger) *Scheduler

NewScheduler builds a Scheduler. logger defaults to slog.Default().

func (*Scheduler) Recover added in v0.2.0

func (s *Scheduler) Recover(ctx context.Context, rec Recoverer) error

Recover marks runs a previous process left in flight as interrupted and removes their containers. They are never re-run: re-running an occurrence whose first execution may have done real work is how double runs happen.

func (*Scheduler) Run

func (s *Scheduler) Run(ctx context.Context, interval time.Duration) error

Run recovers interrupted runs, ticks once immediately (so a restart catches up without waiting a full interval), then ticks every interval until ctx is done.

func (*Scheduler) Tick

func (s *Scheduler) Tick(ctx context.Context) error

Tick evaluates every enabled task once. A broken task never blocks the rest; errors are joined.

func (*Scheduler) Wait added in v0.2.0

func (s *Scheduler) Wait()

Wait blocks until every run started by Tick has returned. Tests use it; production never needs to.

type SpecBuilder added in v0.2.0

type SpecBuilder interface {
	BuildJobSpec(ctx context.Context, svc *store.DesiredService, rt docker.Runtime, name string, command []string, res *docker.Resources) (application.JobSpec, error)
	JobSecretValues(ctx context.Context, svc *store.DesiredService, rt docker.Runtime) ([]string, error)
}

SpecBuilder derives a one-off container spec from a service, so a fresh job gets the service's resolved env, volumes and network. Optional: without it fresh mode reports itself unavailable.

type TaskRunner

type TaskRunner interface {
	RunTrigger(ctx context.Context, task store.ScheduledTask, trig Trigger) error
}

TaskRunner runs one claimed occurrence. *Runner satisfies it; a test substitutes a fake.

type Trigger added in v0.2.0

type Trigger struct {
	Kind string
	// RunID is the pre-claimed history row, set by Scheduler; empty makes
	// the runner create one.
	RunID        string
	ScheduledFor *time.Time
	CatchUp      bool
}

Trigger says why a run happens and, for a scheduled one, which cron occurrence it was claimed for.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL