worker

package
v0.0.0-...-4c3468c Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 2, 2026 License: AGPL-3.0 Imports: 10 Imported by: 0

Documentation

Overview

Package worker is the River integration for the RUCSS domain: the rucss_process job, its worker, and a RiverEnqueuer. Phase 6 wires the worker into cmd/wpmgr/main.go's startRiver via the exported Workers/RegisterWorker helpers; this package itself does NOT touch main.go.

Index

Constants

View Source
const RucssQueue = "rucss_process"

RucssQueue is the dedicated River queue for RUCSS computations. A purge is CPU-bound (HTML parse + cascadia matching over the whole stylesheet); a small bounded pool keeps a burst of agent requests from starving other work.

View Source
const RucssSourcePrefix = "rucss-src/"

RucssSourcePrefix is the object-storage key namespace for the temp HTML+CSS source bundles the agent ingest endpoint stashes (perf.rucssBundleKey writes "rucss-src/{tenant}/{site}/{job}.bin"). The sweeper lists + reaps under it.

View Source
const RucssSweepMaxAge = 60 * time.Second

RucssSweepMaxAge is how old a source bundle must be before the backstop sweeper deletes it. The worker reaps its own bundle inline on success/terminal failure within milliseconds; anything still present after this window is an orphan whose job never ran (enqueue failed, River row lost, …). 60s is well past the worker's own delete latency.

View Source
const RucssTimeout = 60 * time.Second

RucssTimeout caps the wall-clock budget for one purge. A large real-world page+stylesheet purges in well under a second; 60s is generous headroom that still lets River reclaim a wedged job.

Variables

This section is empty.

Functions

func Queues

func Queues(maxWorkers int) map[string]river.QueueConfig

Queues returns the River queue config this domain needs, for merging into the startRiver queues map. MaxWorkers is bounded so RUCSS bursts don't starve other queues. Phase 6 merges this into its map[string]river.QueueConfig.

func RegisterSweepWorker

func RegisterSweepWorker(workers *river.Workers, w *RucssSweepWorker)

RegisterSweepWorker adds the backstop sweeper to a River workers registry.

func RegisterWorker

func RegisterWorker(workers *river.Workers, w *Worker)

RegisterWorker adds the RUCSS worker to a River workers registry. Phase 6 calls this from startRiver instead of hand-writing river.AddWorker.

Types

type BundleReaper

type BundleReaper interface {
	ListWithModified(ctx context.Context, prefix string) ([]blobstore.ObjectInfo, error)
	Delete(ctx context.Context, key string) error
}

BundleReaper is the object-storage surface the backstop sweeper needs: list the source-bundle objects with their ages, and delete the stale ones. *blobstore.Store satisfies it.

type CacheReheater

type CacheReheater interface {
	ReheatURL(ctx context.Context, tenantID, siteID uuid.UUID, pageURL string) error
}

CacheReheater re-warms the page cache for a URL whose used-CSS just landed, so the NEXT render is a CP cache HIT (200) that the agent caches as optimized HTML. This is the closing loop that an async RUCSS pipeline needs: the first render is always a 202 (used-CSS not computed yet); without a re-warm the agent's page cache keeps serving that un-optimized render forever. Best-effort — a failure is logged and never fails the job (the organic next-visit path is the backstop). Satisfied by *perf.rucssReheater (agent client + site lookup).

type EventPublisher

type EventPublisher interface {
	Publish(ctx context.Context, ev site.ConnectionEvent) error
}

EventPublisher publishes the rucss.* SSE envelope on the shared tenant bus. *events.Publisher (which satisfies site.EventPublisher) is injected.

type JobLifecycle

type JobLifecycle interface {
	MarkRunning(ctx context.Context, tenantID uuid.UUID, jobID string) error
	MarkDone(ctx context.Context, tenantID uuid.UUID, jobID string, resultID uuid.UUID) error
	MarkFailed(ctx context.Context, tenantID uuid.UUID, jobID, reason string) error
}

JobLifecycle is the worker's view of the job store (running/done/failed). It is satisfied by *repo.Repo (see the adapter used at construction in main / tests). Kept minimal so the worker is unit-testable.

type RiverEnqueuer

type RiverEnqueuer struct {
	// contains filtered or unexported fields
}

RiverEnqueuer enqueues RUCSS jobs onto River.

func NewRiverEnqueuer

func NewRiverEnqueuer(client *river.Client[pgx.Tx]) *RiverEnqueuer

NewRiverEnqueuer builds an enqueuer around the River client.

func (*RiverEnqueuer) EnqueueRucss

func (e *RiverEnqueuer) EnqueueRucss(ctx context.Context, a RucssArgs) error

EnqueueRucss inserts one rucss_process job. Unique opts collapse a burst of identical (structure_hash) requests for a site into a single in-flight job — the worker is idempotent (the service dedups + upserts) so a stray duplicate is harmless, but de-duping at enqueue keeps the queue shallow.

type RucssArgs

type RucssArgs struct {
	TenantID      uuid.UUID `json:"tenant_id"`
	SiteID        uuid.UUID `json:"site_id"`
	JobID         string    `json:"job_id"`
	StructureHash string    `json:"structure_hash"`
	URL           string    `json:"url,omitempty"`
	// SourceKey locates the HTML+CSS bundle the worker fetches (e.g. an S3 key).
	SourceKey string `json:"source_key,omitempty"`
	// Safelist is the per-site css_rucss_include_selectors at enqueue time.
	Safelist []string `json:"safelist,omitempty"`
	// Reheat is true when this job was produced by a CP post-compute re-warm
	// self-fetch (rather than an operator/organic render). The worker does NOT
	// re-trigger another reheat for a reheat-originated job — this terminates the
	// loop if the page's structure_hash drifts between renders.
	Reheat bool `json:"reheat,omitempty"`
}

RucssArgs is the River job payload for one RUCSS computation. It carries only IDs + the small scalars the worker needs; the (large) HTML/CSS source bytes are fetched by a SourceFetcher from the location the enqueuer recorded (typically a temp object the Phase-6 ingest endpoint uploaded), so the River row stays tiny.

func (RucssArgs) InsertOpts

func (RucssArgs) InsertOpts() river.InsertOpts

InsertOpts pins the job to the dedicated RUCSS queue.

func (RucssArgs) Kind

func (RucssArgs) Kind() string

Kind implements river.JobArgs.

type RucssSweepArgs

type RucssSweepArgs struct{}

RucssSweepArgs is the periodic backstop-sweep job payload.

func (RucssSweepArgs) Kind

func (RucssSweepArgs) Kind() string

Kind implements river.JobArgs.

type RucssSweepWorker

type RucssSweepWorker struct {
	river.WorkerDefaults[RucssSweepArgs]
	// contains filtered or unexported fields
}

RucssSweepWorker is the backstop that deletes orphaned RUCSS source bundles (page HTML) that the inline per-job delete never reached because the job never ran. It is the safety net for the "never retain page HTML in object storage" invariant; an equivalent bucket lifecycle rule (expire objects under "rucss-src/" after ~1 minute) would also satisfy the requirement and is the recommended alternative on managed S3/GCS — this River periodic exists so the guarantee holds even on backends without lifecycle support (SeaweedFS/MinIO).

func NewRucssSweepWorker

func NewRucssSweepWorker(reaper BundleReaper, maxAge time.Duration, logger *slog.Logger) *RucssSweepWorker

NewRucssSweepWorker builds the backstop sweeper. reaper may be nil (degraded env) — Work is then a no-op. maxAge <= 0 defaults to RucssSweepMaxAge.

func (*RucssSweepWorker) Work

Work lists every object under rucss-src/ and deletes the ones older than maxAge. It always returns nil: a transient list/delete error is logged and retried on the next periodic tick (no point failing the job — it is a best-effort reaper, not a correctness-critical path).

type Source

type Source struct {
	HTML []byte
	CSS  []byte
}

Source is the resolved HTML+CSS for a RUCSS job.

type SourceDeleter

type SourceDeleter interface {
	DeleteSource(ctx context.Context, args RucssArgs) error
}

SourceDeleter removes the temp HTML+CSS bundle once the worker has consumed it (success) or the job has failed terminally and will NOT be retried. The page HTML in that bundle must not be retained in object storage past the single computation it feeds (security: source page output is sensitive). Optional — when nil the worker relies on the backstop lifecycle/sweeper to reap orphans. Phase 6's *perf.rucssSourceFetcher satisfies both this and SourceFetcher.

type SourceFetcher

type SourceFetcher interface {
	Fetch(ctx context.Context, args RucssArgs) (Source, error)
}

SourceFetcher resolves the HTML+CSS bundle for a job from wherever the enqueuer stashed it (object storage, an inline cache, …). Phase 6 supplies the concrete implementation; the worker depends only on this interface so it can be unit-tested with a fake.

type Worker

type Worker struct {
	river.WorkerDefaults[RucssArgs]
	// contains filtered or unexported fields
}

Worker runs one RUCSS computation: fetch source -> service.ComputeOrGetCached -> record done/failed -> publish rucss.completed / rucss.failed.

func NewWorker

func NewWorker(svc *service.Service, src SourceFetcher, jobs JobLifecycle, events EventPublisher, reheat CacheReheater, logger *slog.Logger) *Worker

NewWorker builds the RUCSS worker. src/jobs/events may be nil in degraded environments; the worker handles nil dependencies gracefully (a nil source fetcher yields a recorded failure, never a panic). If src ALSO implements SourceDeleter (Phase 6's fetcher does), the worker reaps the temp source bundle after it has been consumed or has failed terminally. reheat may be nil (degraded env / agent client unwired) — the worker then skips the post-compute re-warm and relies on the organic next-visit backstop.

func (*Worker) Timeout

func (w *Worker) Timeout(*river.Job[RucssArgs]) time.Duration

Timeout overrides River's default per-job context deadline. See RucssTimeout.

func (*Worker) Work

func (w *Worker) Work(ctx context.Context, job *river.Job[RucssArgs]) error

Work runs one RUCSS computation.

Failure model (mirrors backup.SqlInspectLegacyWorker): a recorded terminal failure (engine kept-all, store unwired, bad job) returns nil so River does NOT retry; a transient infra error (source fetch / store write transport) returns the error so River retries with backoff. The job NEVER panics.

Source-bundle lifecycle (FIX 1): the agent-posted HTML+CSS bundle the ingest endpoint stashed at a.SourceKey is sensitive page output and MUST NOT persist in object storage past the single computation it feeds. We delete it on the terminal paths only — the SUCCESS path and a TERMINAL failure AFTER the fetch — because on those paths River will NOT retry and the bundle is spent. We deliberately do NOT delete on a transient/retryable error at or before the fetch (River retries and must be able to re-fetch the bundle). Delete is idempotent, so a race with the backstop sweeper is harmless. A backstop periodic sweeper (RucssSweepArgs) reaps orphans for jobs that never run.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL