Documentation
¶
Overview ¶
Package syncjobs owns the server-side async sync job machinery (D1, D2): a bounded FIFO dispatcher with an admission lease, the job record registry GET /api/sync/jobs serves, and the sync:done publication seam.
The dispatcher accepts at most MaxJobs top-level entries (queued plus running); overflow is a typed capacity refusal the handler maps to 429, and a same-file dispatch answers the EXISTING job's ids even at cap. Entries run ONE at a time in FIFO order — a queued single behind a season batch waits, visibly — while the syncworker semaphore stays the execution-slot authority (the automatic sync path interleaves there between entries). A season batch (batch.go) is ONE entry holding N item job records.
Locking: mu > activity.Log's lock (Dispatch creates the activity entry under mu so a same-file race cannot observe a half-registered job); the stop registry's callbacks run lock-free (a context cancel). No path takes a foreign lock and then mu.
Index ¶
- Constants
- Variables
- type Accepted
- type BatchAccepted
- type BatchInput
- type CancelOutcome
- type Deps
- type Dispatcher
- func (d *Dispatcher) Cancel(activityID string) CancelOutcome
- func (d *Dispatcher) Dispatch(in *ExecInput) (Accepted, error)
- func (d *Dispatcher) DispatchBatch(in *BatchInput) (BatchAccepted, error)
- func (d *Dispatcher) Jobs(batchActivityID string) []Job
- func (d *Dispatcher) Prune(maxAge time.Duration)
- func (d *Dispatcher) Run(ctx context.Context)
- type ExecInput
- type ExecResult
- type Job
- type JobState
Constants ¶
const MaxJobs = 8
MaxJobs is the admission lease: at most this many accepted (queued or running) top-level entries; the next dispatch answers a typed 429.
Variables ¶
var ( // ErrCapacity: the admission lease is full — mapped to the wire as 429. ErrCapacity = errors.New("sync job capacity reached") // ErrShuttingDown: the dispatcher has drained; no new work is accepted. ErrShuttingDown = errors.New("sync dispatcher is shutting down") )
Typed dispatch refusals.
Functions ¶
This section is empty.
Types ¶
type Accepted ¶
Accepted is a dispatch answer: the two ids the 202 hands the client, and whether they name a pre-existing live job (same-file dedupe).
type BatchAccepted ¶
type BatchAccepted struct {
// ActivityID is the batch activity entry's id — the 202 body, the
// registry filter key, and the stop/cancel address.
ActivityID string
// Existing reports the idempotent same-scope answer: a live batch for
// this (series, season) already exists and its id was returned.
Existing bool
// SkippedLive counts items not registered because their subtitle file
// already has a live job (at most one live job per file — the dedupe
// invariant; that job's own sync:done settles the file).
SkippedLive int
}
BatchAccepted is a season dispatch answer.
type BatchInput ¶
type BatchInput struct {
// Detail is the batch activity entry's aggregate detail line.
Detail string
// Items in ordinal order; item i is ordinal i+1.
Items []ExecInput
// SeriesID and Season scope the batch: a dispatch for a scope that
// already has a live batch answers that batch's id (idempotent).
SeriesID int
Season int
}
BatchInput is one season batch's work order: pre-resolved items in ordinal order (the handler enumerates and resolves paths at accept time; the executor re-reads file content at run time) plus the aggregate identity the batch activity entry carries.
type CancelOutcome ¶
type CancelOutcome int
CancelOutcome is Cancel's verdict for the activity endpoints.
const ( // CancelUnknown: the activity id names no sync job; callers fall through // to the legacy activity path. CancelUnknown CancelOutcome = iota // CancelledQueued: a queued entry was cancelled — settled done(cancelled), // slot released IMMEDIATELY (the next 202 enters), no success event. CancelledQueued // CancelConverted: the delete lost the race to admission and converted to // a running-cancel through the just-registered stop entry — 204; the job // settles done(cancelled) and releases capacity on worker exit. CancelConverted // CancelTerminal: the job is already done; dismissal is the caller's // (the registry is never touched by a terminal row's dismissal). CancelTerminal )
Cancel verdicts.
type Deps ¶
type Deps struct {
// Exec runs one job's analysis (synchandlers' audio executor).
Exec execFunc
// Log is the activity log: one entry per job, created queued at accept.
Log *activity.Log
// Stops is the live stop registry; the admission hook registers each
// single-file job's stop entry there BEFORE publishing running.
Stops *activity.StopRegistry
// PublishDone publishes the sync:done event (EventBus.PublishSyncDone).
PublishDone func(ev *events.SyncDoneEvent)
// OnChange runs after every registry transition GET /api/sync/jobs can
// observe (accept, admission, settle, batch finish, prune), outside mu;
// the server mints the jobs digest version from it. Optional.
OnChange func()
// RegistryCap overrides defaultRegistryCap when positive (tests).
RegistryCap int
}
Deps wires the dispatcher's collaborators.
type Dispatcher ¶
type Dispatcher struct {
// contains filtered or unexported fields
}
Dispatcher owns the FIFO, the admission lease, the dedupe map, per-job cancels, and the job registry. One Run loop executes entries in order.
func New ¶
func New(deps Deps) *Dispatcher
New builds a Dispatcher over deps. Call Run exactly once to start it.
func (*Dispatcher) Cancel ¶
func (d *Dispatcher) Cancel(activityID string) CancelOutcome
Cancel routes an activity-id cancellation into the dispatcher. See the CancelOutcome constants for the verdict semantics. A batch activity id addresses the WHOLE batch (the batch is the cancellation unit); its item records are settled by the batch arms, never individually.
func (*Dispatcher) Dispatch ¶
func (d *Dispatcher) Dispatch(in *ExecInput) (Accepted, error)
Dispatch enqueues one single-file job: non-blocking, atomically reserving an admission slot. A live job for the same subtitle file answers its EXISTING ids (even at cap); a full lease answers ErrCapacity (429 on the wire, never auto-retried by the client).
func (*Dispatcher) DispatchBatch ¶
func (d *Dispatcher) DispatchBatch(in *BatchInput) (BatchAccepted, error)
DispatchBatch enqueues one season batch: non-blocking, atomically reserving ONE admission slot for the whole batch. All item records are created NOW (state queued, ordinal set, batch_activity_id riding every record), so a registry read before any sync:done lists the full item set. A live batch for the same (series, season) answers its EXISTING id; a full lease answers ErrCapacity (429 on the wire).
func (*Dispatcher) Jobs ¶
func (d *Dispatcher) Jobs(batchActivityID string) []Job
Jobs snapshots the registry in the total order accepted_at DESC, job_id DESC (numeric), optionally filtered by batch_activity_id. Batch item records live here like any job; the batch itself is not a record — its aggregate is the activity entry.
func (*Dispatcher) Prune ¶
func (d *Dispatcher) Prune(maxAge time.Duration)
Prune removes done records older than maxAge (retention = activity.DefaultPruneAge, one owner: the server's prune ticker drives both). Queued and running records are never evicted, and neither is a done item of a still-live batch — the batch finalizers dereference every item id, and the reload read lists the full item set. Terminal batch entries age out with their items.
func (*Dispatcher) Run ¶
func (d *Dispatcher) Run(ctx context.Context)
Run executes queued entries one at a time in FIFO order until ctx (the server lifetime) dies, then drains: queued jobs settle done(cancelled) without ever executing, and the in-flight worker has already been waited for (its context derives from ctx, so the child is killed and reaped).
type ExecInput ¶
ExecInput is one job's work order, resolved at accept time (the handler validates and resolves before the 202; the executor reads the file at run time, so the analysis sees current content).
type ExecResult ¶
type ExecResult struct {
Err error
Method string
Outcome subflux.JobOutcome
OffsetMs int64
Confidence float64
Applied bool
}
ExecResult is the executor's answer for one job. OffsetMs is CUMULATIVE (the stored offset plus this run's correction).
type Job ¶
type Job struct {
AcceptedAt time.Time `json:"accepted_at"`
StartedAt *time.Time `json:"started_at,omitempty"`
EndedAt *time.Time `json:"ended_at,omitempty"`
BatchActivityID string `json:"batch_activity_id,omitempty"`
Method string `json:"method,omitempty"`
Error string `json:"error,omitempty"`
State JobState `json:"state"`
Outcome subflux.JobOutcome `json:"outcome,omitempty"`
ActivityID string `json:"activity_id"`
FileRef resolve.FileRef `json:"file_ref"`
JobID int64 `json:"job_id"`
OffsetMs int64 `json:"offset_ms,omitempty"`
SeriesID int `json:"series_id,omitempty"`
Season int `json:"season,omitempty"`
Ordinal int `json:"ordinal,omitempty"`
Confidence float64 `json:"confidence,omitempty"`
Applied bool `json:"applied,omitempty"`
DryRun bool `json:"dry_run,omitempty"`
}
Job is one sync job record: the wire shape GET /api/sync/jobs serves. JobID is a NUMERIC process sequence — distinct from the activity log's string ids, which the record carries beside it. The registry is truth independent of the activity ring; a restart drops it (jobs are not persisted — the client re-attaches via this read and finds nothing).