Documentation
¶
Overview ¶
Package scheduledtask runs operator-defined jobs on a cron schedule, in the running container (exec mode) or in a fresh one-off container built from the app's own spec (fresh mode), mirroring internal/backup's Runner/Scheduler split: Runner (this file) does one run and records its outcome, Scheduler (scheduler.go) decides when a run is due. The same Runner backs the cron loop, "run now" and the API trigger, so there is one place that executes a command and records what happened.
Index ¶
- Constants
- func NewRunID() (string, error)
- func Occurrences(sched *cronexpr.Schedule, loc *time.Location, after, upTo time.Time, limit int) (out []time.Time, truncated bool)
- type AppStore
- type AttemptFunc
- type AttemptOutcome
- type HTTPConfigStore
- type HTTPTarget
- type HistoryStore
- type Limits
- type NodeRuntimeResolver
- type OneShot
- type OneShotEntry
- type OneShotLimits
- type OneShotRegistry
- type OneShotSnapshot
- type OneShotSpec
- type Recoverer
- type RunStore
- type Runner
- func (r *Runner) Cleanup(ctx context.Context, runs []store.ScheduledTaskRun)
- func (r *Runner) ContainerName(service, runID, taskID string) string
- func (r *Runner) Run(ctx context.Context, task store.ScheduledTask) error
- func (r *Runner) RunAttempt(ctx context.Context, task store.ScheduledTask) (AttemptOutcome, error)
- func (r *Runner) RunTrigger(ctx context.Context, task store.ScheduledTask, trig Trigger) error
- func (r *Runner) Start(ctx context.Context, task store.ScheduledTask, trigger string) (string, error)
- type ScheduleStore
- type Scheduler
- type SpecBuilder
- type TaskRunner
- type Trigger
Constants ¶
const ( HeaderJobRun = "X-Job-Run" HeaderJobName = "X-Job-Name" HeaderJobBackground = "X-Job-Background" )
Headers sent on every http-mode invocation so the app can tell a job call from user traffic and correlate it with run history.
const ( OneShotRunning = "Running" OneShotSucceeded = "Succeeded" OneShotFailed = "Failed" )
One-shot job states.
Variables ¶
This section is empty.
Functions ¶
func Occurrences ¶ added in v0.2.0
func Occurrences(sched *cronexpr.Schedule, loc *time.Location, after, upTo time.Time, limit int) (out []time.Time, truncated bool)
Occurrences returns the cron occurrences in (after, upTo], evaluated on loc's wall clock, oldest first. A wall time skipped by a DST jump runs once at the shifted time; one repeated by a fall-back runs once.
Types ¶
type AppStore ¶
type AppStore interface {
GetDesiredService(ctx context.Context, name string) (*store.DesiredService, error)
}
AppStore resolves a job's target service.
type AttemptFunc ¶
type AttemptFunc func(ctx context.Context, completion, attempt int) AttemptOutcome
AttemptFunc runs one attempt of completion index (0-based). Each call is expected to record itself in the run history.
type AttemptOutcome ¶
AttemptOutcome is the result of one attempt of one completion.
type HTTPConfigStore ¶
type HTTPConfigStore interface {
GetScheduledTaskHTTP(ctx context.Context, taskID string) (store.ScheduledTaskHTTP, error)
}
HTTPConfigStore loads the request an http-mode job sends.
type HTTPTarget ¶
type HTTPTarget interface {
Prepare(ctx context.Context, svc *store.DesiredService) (baseURL string, err error)
}
HTTPTarget wakes an app if it sleeps, waits until it can answer, and returns the base URL its HTTP port is reachable on from the control plane.
type HistoryStore ¶ added in v0.2.0
type HistoryStore interface {
CreateScheduledTaskRun(ctx context.Context, run store.ScheduledTaskRun) error
FinishScheduledTaskRun(ctx context.Context, run store.ScheduledTaskRun) error
PruneScheduledTaskRuns(ctx context.Context, taskID string, keep int, outputCutoff time.Time) error
}
HistoryStore persists per-run history. Optional on Runner: nil keeps only the last-run snapshot RunStore writes.
type Limits ¶ added in v0.2.0
type Limits struct {
DefaultTimeout time.Duration
MaxRetries int
RetryBackoff time.Duration
MaxRetryBackoff time.Duration
CatchUpLookback time.Duration
HistoryKeep int
OutputRetention time.Duration
MaxPerApp int
MaxConcurrentPerApp int
MaxOutputBytes int
BackgroundTimeout time.Duration
}
Limits are the control-plane-wide knobs for scheduled jobs, all environment-driven so no threshold is hardcoded at a call site.
func DefaultLimits ¶ added in v0.2.0
func DefaultLimits() Limits
DefaultLimits is LimitsFromEnv with an empty environment.
func LimitsFromEnv ¶ added in v0.2.0
LimitsFromEnv reads APP_JOBS_DEFAULT_TIMEOUT, APP_JOBS_MAX_RETRIES, APP_JOBS_RETRY_BACKOFF, APP_JOBS_RETRY_BACKOFF_MAX, APP_JOBS_CATCH_UP_LOOKBACK, APP_JOBS_HISTORY_KEEP, APP_JOBS_OUTPUT_RETENTION, APP_JOBS_MAX_PER_APP, APP_JOBS_MAX_CONCURRENT_PER_APP, APP_JOBS_MAX_OUTPUT_BYTES and APP_JOBS_BACKGROUND_TIMEOUT (how long a background HTTP job is awaited). A value that does not parse or is not positive falls back to its default.
type NodeRuntimeResolver ¶
NodeRuntimeResolver resolves the docker.Runtime for a node ID, the same shape internal/api's own resolver has.
type OneShot ¶
type OneShot struct {
// Sleep waits between retries; nil uses a context-aware timer.
Sleep func(ctx context.Context, d time.Duration) error
// Now returns the current time; nil uses time.Now.
Now func() time.Time
// contains filtered or unexported fields
}
OneShot drives one job. It needs no controller: Run blocks until the job settles and Snapshot can be read concurrently.
func (*OneShot) Init ¶
func (o *OneShot) Init(spec OneShotSpec)
Init marks the job Running so a snapshot taken before Run starts is valid.
func (*OneShot) Run ¶
func (o *OneShot) Run(ctx context.Context, spec OneShotSpec, attempt AttemptFunc) OneShotSnapshot
Run executes spec with at most spec.Parallelism attempts in flight and returns the final snapshot. A failed attempt is retried after an exponential delay until spec.BackoffLimit failures have been spent.
func (*OneShot) Snapshot ¶
func (o *OneShot) Snapshot() OneShotSnapshot
Snapshot returns a copy of the current state.
type OneShotEntry ¶
type OneShotEntry struct {
ID string
App string
TaskID string
Spec OneShotSpec
Job *OneShot
}
OneShotEntry is one registered job.
type OneShotLimits ¶
type OneShotLimits struct {
BackoffLimit int
Timeout time.Duration
MaxCompletions int
MaxParallelism int
Keep int
}
OneShotLimits are the env-driven defaults and ceilings for one-shot jobs.
func OneShotLimitsFromEnv ¶
func OneShotLimitsFromEnv(lookup func(string) (string, bool)) OneShotLimits
OneShotLimitsFromEnv reads APP_JOB_DEFAULT_BACKOFF_LIMIT, APP_JOB_DEFAULT_TIMEOUT, APP_JOB_MAX_COMPLETIONS, APP_JOB_MAX_PARALLELISM and APP_JOB_ONESHOT_KEEP. Unparsable or non-positive values fall back.
type OneShotRegistry ¶
type OneShotRegistry struct {
// contains filtered or unexported fields
}
OneShotRegistry holds the live and recently finished one-shot jobs of this process, bounded by keep. State is in memory: the per-attempt runs stay in the run history, the job summary does not survive a restart.
func NewOneShotRegistry ¶
func NewOneShotRegistry(keep int) *OneShotRegistry
NewOneShotRegistry returns a registry keeping at most keep finished jobs.
func (*OneShotRegistry) Add ¶
func (g *OneShotRegistry) Add(e *OneShotEntry)
Add registers e and evicts the oldest finished jobs beyond keep.
func (*OneShotRegistry) Get ¶
func (g *OneShotRegistry) Get(id string) (*OneShotEntry, bool)
Get returns the job with id.
type OneShotSnapshot ¶
type OneShotSnapshot struct {
State string
Completions int
Succeeded int
FailedAttempts int
Active int
RunIDs []string
Message string
StartedAt time.Time
FinishedAt *time.Time
}
OneShotSnapshot is a point-in-time view of a one-shot job.
type OneShotSpec ¶
type OneShotSpec struct {
Completions int
Parallelism int
BackoffLimit int
BackoffBase time.Duration
BackoffMax time.Duration
}
OneShotSpec describes one run-to-completion job. BackoffLimit is the number of failed attempts tolerated across the whole job before it is declared Failed, as in a Kubernetes Job.
func (*OneShotSpec) Validate ¶
func (s *OneShotSpec) Validate(l OneShotLimits) error
Validate normalizes spec against limits.
type Recoverer ¶ added in v0.2.0
type Recoverer interface {
Cleanup(ctx context.Context, runs []store.ScheduledTaskRun)
}
Recoverer cleans up what a previous process left behind. *Runner satisfies it; optional.
type RunStore ¶
type RunStore interface {
RecordScheduledTaskRun(ctx context.Context, id string, ranAt time.Time, status, output string) error
}
RunStore persists a run's outcome onto the task's own row.
type Runner ¶
type Runner struct {
Apps AppStore
Runs RunStore
History HistoryStore
Specs SpecBuilder
Resolver NodeRuntimeResolver
Logger *slog.Logger
Limits Limits
// Target and HTTPConfigs serve http-mode jobs; Client overrides the
// HTTP client (nil uses http.DefaultClient).
Target HTTPTarget
HTTPConfigs HTTPConfigStore
Client *http.Client
// NamePrefix namespaces fresh job containers (the brand short name).
NamePrefix string
// Now returns the current time; nil falls back to time.Now.
Now func() time.Time
// Timeout overrides every attempt's timeout when positive; a test seam.
Timeout time.Duration
// Sleep waits between retries; nil uses a context-aware timer.
Sleep func(ctx context.Context, d time.Duration) error
// contains filtered or unexported fields
}
Runner runs one job and records the outcome.
func (*Runner) Cleanup ¶ added in v0.2.0
func (r *Runner) Cleanup(ctx context.Context, runs []store.ScheduledTaskRun)
Cleanup force-removes the containers of runs a previous process left in flight. Best effort: a node that is unreachable now is simply skipped.
func (*Runner) ContainerName ¶ added in v0.2.0
ContainerName is the deterministic name of a fresh job container, so a restart can find and remove one the previous process left behind. The run suffix is deliberately not hex, so no service-container parser takes it for a replica.
func (*Runner) RunAttempt ¶
func (r *Runner) RunAttempt(ctx context.Context, task store.ScheduledTask) (AttemptOutcome, error)
RunAttempt performs one attempt of task through the same execution path as RunTrigger (history row, redaction, per-app slot) and returns its final status. A full per-app slot comes back as skipped_concurrency so a one-shot waits instead of failing.
func (*Runner) RunTrigger ¶ added in v0.2.0
RunTrigger runs task once. task.ConcurrencyPolicy governs a still in-flight previous run of the same task (forbid skips, replace cancels it), and the per-app cap skips a run when the app already has Limits.MaxConcurrentPerApp jobs running. Retries with backoff happen inside the one run. The returned error is only an infrastructure failure; a failing command is a recorded outcome, not an error.
func (*Runner) Start ¶ added in v0.2.0
func (r *Runner) Start(ctx context.Context, task store.ScheduledTask, trigger string) (string, error)
Start records a run's history row synchronously, then runs it in the background, and returns the run ID so a caller can follow it. Without a history store it still runs, with an empty ID. ctx should already be detached from the request that asked.
type ScheduleStore ¶
type ScheduleStore interface {
ListEnabledScheduledTasks(ctx context.Context) ([]store.ScheduledTask, error)
ClaimScheduledTaskRun(ctx context.Context, run store.ScheduledTaskRun) (bool, error)
InterruptRunningScheduledTaskRuns(ctx context.Context, now time.Time) ([]store.ScheduledTaskRun, error)
}
ScheduleStore is the store surface Scheduler needs. *store.DB satisfies it structurally.
type Scheduler ¶
type Scheduler struct {
Store ScheduleStore
Runner TaskRunner
Logger *slog.Logger
// Now returns the current time; nil falls back to time.Now.
Now func() time.Time
// Grace is how late an occurrence may be and still count as on time;
// zero means twice the interval passed to Run.
Grace time.Duration
// Lookback is the default catch-up window when a job sets none.
Lookback time.Duration
// NewID mints run IDs; nil uses NewRunID.
NewID func() (string, error)
// contains filtered or unexported fields
}
Scheduler is level-triggered: every tick it derives, from the persisted high-water mark (store.ScheduledTask.LastScheduledFor) and the current time, which cron occurrence is due, claims it atomically in the database and only then runs it. Because the claim is persisted before execution, a restart can neither run an occurrence twice nor lose track of one: an occurrence found overdue is run once (catch_up run_once, inside the lookback), or recorded as missed (skip), never silently dropped.
func NewScheduler ¶
func NewScheduler(st ScheduleStore, runner TaskRunner, logger *slog.Logger) *Scheduler
NewScheduler builds a Scheduler. logger defaults to slog.Default().
func (*Scheduler) Recover ¶ added in v0.2.0
Recover marks runs a previous process left in flight as interrupted and removes their containers. They are never re-run: re-running an occurrence whose first execution may have done real work is how double runs happen.
func (*Scheduler) Run ¶
Run recovers interrupted runs, ticks once immediately (so a restart catches up without waiting a full interval), then ticks every interval until ctx is done.
type SpecBuilder ¶ added in v0.2.0
type SpecBuilder interface {
BuildJobSpec(ctx context.Context, svc *store.DesiredService, rt docker.Runtime, name string, command []string, res *docker.Resources) (application.JobSpec, error)
JobSecretValues(ctx context.Context, svc *store.DesiredService, rt docker.Runtime) ([]string, error)
}
SpecBuilder derives a one-off container spec from a service, so a fresh job gets the service's resolved env, volumes and network. Optional: without it fresh mode reports itself unavailable.
type TaskRunner ¶
type TaskRunner interface {
RunTrigger(ctx context.Context, task store.ScheduledTask, trig Trigger) error
}
TaskRunner runs one claimed occurrence. *Runner satisfies it; a test substitutes a fake.
type Trigger ¶ added in v0.2.0
type Trigger struct {
Kind string
// RunID is the pre-claimed history row, set by Scheduler; empty makes
// the runner create one.
RunID string
ScheduledFor *time.Time
CatchUp bool
}
Trigger says why a run happens and, for a scheduled one, which cron occurrence it was claimed for.