syncjobs

package
v0.7.0-dev.7 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 10, 2026 License: AGPL-3.0 Imports: 13 Imported by: 0

Documentation

Overview

Package syncjobs owns the server-side async sync job machinery (D1, D2): a bounded FIFO dispatcher with an admission lease, the job record registry GET /api/sync/jobs serves, and the sync:done publication seam.

The dispatcher accepts at most MaxJobs top-level entries (queued plus running); overflow is a typed capacity refusal the handler maps to 429, and a same-file dispatch answers the EXISTING job's ids even at cap. Entries run ONE at a time in FIFO order — a queued single behind a season batch waits, visibly — while the syncworker semaphore stays the execution-slot authority (the automatic sync path interleaves there between entries). A season batch (batch.go) is ONE entry holding N item job records.

Locking: mu > activity.Log's lock (Dispatch creates the activity entry under mu so a same-file race cannot observe a half-registered job); the stop registry's callbacks run lock-free (a context cancel). No path takes a foreign lock and then mu.

Index

Constants

View Source
const MaxJobs = 8

MaxJobs is the admission lease: at most this many accepted (queued or running) top-level entries; the next dispatch answers a typed 429.

Variables

View Source
var (
	// ErrCapacity: the admission lease is full — mapped to the wire as 429.
	ErrCapacity = errors.New("sync job capacity reached")
	// ErrShuttingDown: the dispatcher has drained; no new work is accepted.
	ErrShuttingDown = errors.New("sync dispatcher is shutting down")
)

Typed dispatch refusals.

Functions

This section is empty.

Types

type Accepted

type Accepted struct {
	ActivityID string
	JobID      int64
	Existing   bool
}

Accepted is a dispatch answer: the two ids the 202 hands the client, and whether they name a pre-existing live job (same-file dedupe).

type BatchAccepted

type BatchAccepted struct {
	// ActivityID is the batch activity entry's id — the 202 body, the
	// registry filter key, and the stop/cancel address.
	ActivityID string
	// Existing reports the idempotent same-scope answer: a live batch for
	// this (series, season) already exists and its id was returned.
	Existing bool
	// SkippedLive counts items not registered because their subtitle file
	// already has a live job (at most one live job per file — the dedupe
	// invariant; that job's own sync:done settles the file).
	SkippedLive int
}

BatchAccepted is a season dispatch answer.

type BatchInput

type BatchInput struct {
	// Detail is the batch activity entry's aggregate detail line.
	Detail string
	// Items in ordinal order; item i is ordinal i+1.
	Items []ExecInput
	// SeriesID and Season scope the batch: a dispatch for a scope that
	// already has a live batch answers that batch's id (idempotent).
	SeriesID int
	Season   int
}

BatchInput is one season batch's work order: pre-resolved items in ordinal order (the handler enumerates and resolves paths at accept time; the executor re-reads file content at run time) plus the aggregate identity the batch activity entry carries.

type CancelOutcome

type CancelOutcome int

CancelOutcome is Cancel's verdict for the activity endpoints.

const (
	// CancelUnknown: the activity id names no sync job; callers fall through
	// to the legacy activity path.
	CancelUnknown CancelOutcome = iota
	// CancelledQueued: a queued entry was cancelled — settled done(cancelled),
	// slot released IMMEDIATELY (the next 202 enters), no success event.
	CancelledQueued
	// CancelConverted: the delete lost the race to admission and converted to
	// a running-cancel through the just-registered stop entry — 204; the job
	// settles done(cancelled) and releases capacity on worker exit.
	CancelConverted
	// CancelTerminal: the job is already done; dismissal is the caller's
	// (the registry is never touched by a terminal row's dismissal).
	CancelTerminal
)

Cancel verdicts.

type Deps

type Deps struct {
	// Exec runs one job's analysis (synchandlers' audio executor).
	Exec execFunc
	// Log is the activity log: one entry per job, created queued at accept.
	Log *activity.Log
	// Stops is the live stop registry; the admission hook registers each
	// single-file job's stop entry there BEFORE publishing running.
	Stops *activity.StopRegistry
	// PublishDone publishes the sync:done event (EventBus.PublishSyncDone).
	PublishDone func(ev *events.SyncDoneEvent)
	// OnChange runs after every registry transition GET /api/sync/jobs can
	// observe (accept, admission, settle, batch finish, prune), outside mu;
	// the server mints the jobs digest version from it. Optional.
	OnChange func()
	// RegistryCap overrides defaultRegistryCap when positive (tests).
	RegistryCap int
}

Deps wires the dispatcher's collaborators.

type Dispatcher

type Dispatcher struct {
	// contains filtered or unexported fields
}

Dispatcher owns the FIFO, the admission lease, the dedupe map, per-job cancels, and the job registry. One Run loop executes entries in order.

func New

func New(deps Deps) *Dispatcher

New builds a Dispatcher over deps. Call Run exactly once to start it.

func (*Dispatcher) Cancel

func (d *Dispatcher) Cancel(activityID string) CancelOutcome

Cancel routes an activity-id cancellation into the dispatcher. See the CancelOutcome constants for the verdict semantics. A batch activity id addresses the WHOLE batch (the batch is the cancellation unit); its item records are settled by the batch arms, never individually.

func (*Dispatcher) Dispatch

func (d *Dispatcher) Dispatch(in *ExecInput) (Accepted, error)

Dispatch enqueues one single-file job: non-blocking, atomically reserving an admission slot. A live job for the same subtitle file answers its EXISTING ids (even at cap); a full lease answers ErrCapacity (429 on the wire, never auto-retried by the client).

func (*Dispatcher) DispatchBatch

func (d *Dispatcher) DispatchBatch(in *BatchInput) (BatchAccepted, error)

DispatchBatch enqueues one season batch: non-blocking, atomically reserving ONE admission slot for the whole batch. All item records are created NOW (state queued, ordinal set, batch_activity_id riding every record), so a registry read before any sync:done lists the full item set. A live batch for the same (series, season) answers its EXISTING id; a full lease answers ErrCapacity (429 on the wire).

func (*Dispatcher) Jobs

func (d *Dispatcher) Jobs(batchActivityID string) []Job

Jobs snapshots the registry in the total order accepted_at DESC, job_id DESC (numeric), optionally filtered by batch_activity_id. Batch item records live here like any job; the batch itself is not a record — its aggregate is the activity entry.

func (*Dispatcher) Prune

func (d *Dispatcher) Prune(maxAge time.Duration)

Prune removes done records older than maxAge (retention = activity.DefaultPruneAge, one owner: the server's prune ticker drives both). Queued and running records are never evicted, and neither is a done item of a still-live batch — the batch finalizers dereference every item id, and the reload read lists the full item set. Terminal batch entries age out with their items.

func (*Dispatcher) Run

func (d *Dispatcher) Run(ctx context.Context)

Run executes queued entries one at a time in FIFO order until ctx (the server lifetime) dies, then drains: queued jobs settle done(cancelled) without ever executing, and the in-flight worker has already been waited for (its context derives from ctx, so the child is killed and reaped).

type ExecInput

type ExecInput struct {
	SubtitlePath string
	VideoPath    string
	Ref          resolve.FileRef
	DryRun       bool
}

ExecInput is one job's work order, resolved at accept time (the handler validates and resolves before the 202; the executor reads the file at run time, so the analysis sees current content).

type ExecResult

type ExecResult struct {
	Err        error
	Method     string
	Outcome    subflux.JobOutcome
	OffsetMs   int64
	Confidence float64
	Applied    bool
}

ExecResult is the executor's answer for one job. OffsetMs is CUMULATIVE (the stored offset plus this run's correction).

type Job

type Job struct {
	AcceptedAt      time.Time          `json:"accepted_at"`
	StartedAt       *time.Time         `json:"started_at,omitempty"`
	EndedAt         *time.Time         `json:"ended_at,omitempty"`
	BatchActivityID string             `json:"batch_activity_id,omitempty"`
	Method          string             `json:"method,omitempty"`
	Error           string             `json:"error,omitempty"`
	State           JobState           `json:"state"`
	Outcome         subflux.JobOutcome `json:"outcome,omitempty"`
	ActivityID      string             `json:"activity_id"`
	FileRef         resolve.FileRef    `json:"file_ref"`
	JobID           int64              `json:"job_id"`
	OffsetMs        int64              `json:"offset_ms,omitempty"`
	SeriesID        int                `json:"series_id,omitempty"`
	Season          int                `json:"season,omitempty"`
	Ordinal         int                `json:"ordinal,omitempty"`
	Confidence      float64            `json:"confidence,omitempty"`
	Applied         bool               `json:"applied,omitempty"`
	DryRun          bool               `json:"dry_run,omitempty"`
}

Job is one sync job record: the wire shape GET /api/sync/jobs serves. JobID is a NUMERIC process sequence — distinct from the activity log's string ids, which the record carries beside it. The registry is truth independent of the activity ring; a restart drops it (jobs are not persisted — the client re-attaches via this read and finds nothing).

type JobState

type JobState string

JobState is a job's lifecycle position. (Named for the flat TS wire namespace, where a bare State would collide with the activity vocabulary.)

const (
	StateQueued  JobState = "queued"
	StateRunning JobState = "running"
	StateDone    JobState = "done"
)

Job states: queued → running (at the admission hook) → done.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL