integrity

type OrphanSidecarSweeper ¶ added in v0.1.4

type OrphanSidecarSweeper struct {
	// contains filtered or unexported fields
}

OrphanSidecarSweeper walks `outputDir/transcoded/` on a cadence (configured via `cfg.Integrity.OrphanSidecarSweepIntervalSec`) and unlinks `.flac` files whose absolute path is NOT present in the current `track_variants.sidecar_path` snapshot. The forward half of the operator-triggered `bridge upscale --gc` sweep, which `VariantWatcher` (in variants.go) does NOT cover — that type handles the REVERSE direction (rows whose sidecar file disappeared on disk).

**Disabled by default** — opt-in via a non-zero interval. The existing operator workflow of "run `--gc` manually when storage gets tight" stays correct; this knob exists for the hands-off-operator profile.

**Snapshot-then-walk** (NOT walk-then-snapshot): the sweeper takes the `track_variants.sidecar_path` projection BEFORE the filesystem walk so a concurrent `UpsertVariant` writer cannot produce a false-positive orphan (under the reverse order, the new sidecar lands on disk BEFORE the new row is in the snapshot — and the sweeper would unlink the file behind a row that hasn't yet rolled into its view). SQLite WAL mode gives every SELECT a consistent snapshot natively, so `AllSidecarPaths` is safe to call without an explicit transaction wrapper.

**Chunked walking**: at most `gcChunkSize` files are stat'd / unlinked per tick. Operators on libraries with hundreds of thousands of variants get steady progress across multiple ticks rather than one long-running pass that competes with library scanning. Per-tick wall-clock stays bounded.

**Walk pointer survives across ticks**: when a tick hits the chunk cap, the next tick's walk picks up from where the prior tick stopped via filename-relative-ordering — `filepath.Walk` is deterministic in lexical order, so we can track the `lastProcessedPath` cursor and skip-until on the next pass. Pre-cursor the sweeper would re-walk the same first 100 files every tick forever on libraries with >100 sidecars.

Threading: one long-lived goroutine spun up by Start; stops on ctx cancellation OR the stopFn closing the done channel. Mirrors `VariantWatcher` exactly so cmd/bridge's wiring + shutdown ordering treats both watchers symmetrically.

func NewOrphanSidecarSweeper(lister SidecarLister, outputDir string, interval time.Duration) *OrphanSidecarSweeper

func (s *OrphanSidecarSweeper) SetOnTickComplete(fn func(unlinked int))

func (s *OrphanSidecarSweeper) Start(ctx context.Context) (stopFn func())

type PublishFunc ¶

type PublishFunc func(paths []string, variantIDs []string)

PublishFunc is the domain-specific publish callback fired once per sweep that observed at least one missing sidecar. The integrity package keeps the wire-shape construction at the cmd/bridge wiring layer so it can build the typed api.UpscaleDeletedEvent without an upward import cycle — the integrity package only knows "I observed these disappearances", not the broker's serialization concerns.

`paths` and `variantIDs` are positional but NOT zipped 1:1: callers treat them as the set of paths affected AND the set of variantIDs that disappeared somewhere in those paths. Same semantic the api.UpscaleDeletedEvent struct already documents.

type SidecarLister ¶ added in v0.1.4

type SidecarLister interface {
	AllSidecarPaths(ctx context.Context) (map[string]struct{}, error)
}

SidecarLister is the integrity-package-local read surface for `track_variants.sidecar_path` projection. `manifest.Store` will be wired via a thin adapter in cmd/bridge; the explicit interface lets tests inject fakes without spinning a real SQLite store.

Returns a SET of sidecar paths (`map[string]struct{}` for O(1) lookup against thousands of filesystem entries during a walk). Bare `[]string` was rejected: per-file lookup against a slice is O(n) and a 50k-variant library would O(n²)-walk on every tick.

type VariantDeleter ¶

type VariantDeleter interface {
	DeleteVariant(sourcePath, variantID string) error
}

VariantDeleter removes one variant row by (source_path, variant_id). The Store's DeleteVariant transactionally bumps `tracks.indexed_at` so iOS delta-sync observes the removal on the next manifest fetch. Per-row error tolerance: a Watcher tick logs and continues on per-row failure, but still publishes the events for the rows that DID delete.

type VariantLister ¶

type VariantLister interface {
	AllVariants() ([]VariantSnapshot, error)
}

VariantLister enumerates every row in the track_variants table. The watcher consumes the snapshot per tick and stats each sidecar path. Mirrors the manifest.Store method shape; cmd/bridge wires the adapter.

type VariantSnapshot ¶

type VariantSnapshot struct {
	SourcePath  string
	VariantID   string
	SidecarPath string
}

VariantSnapshot is the integrity-package-local projection of one track_variants row. Mirrors `api.VariantSummary` but stays internal to integrity so the package doesn't import internal/api (which would create an upward dependency cycle — api consumes integrity events via the broker, not the other way round).

type VariantWatcher ¶

type VariantWatcher struct {
	// contains filtered or unexported fields
}

VariantWatcher walks the track_variants table on a cadence configurable via cfg.Integrity.VariantSweepInterval (default 1 h) and reconciles rows whose sidecar file no longer exists on disk. Reasons a sidecar might disappear outside the bridge: operator `rm -rf <DataDir>/transcoded/`, backup software with eager retention, disk-image rebuild that preserved the SQLite DB but not the sidecar tree.

On every miss, the row is removed via the supplied Deleter (bumps `tracks.indexed_at` so iOS delta-sync sees the disappearance) AND a single batched `upscale.deleted` SSE event is published per tick — iOS reconciles immediately without waiting for a manifest re-sync.

Threading: one long-lived goroutine spun up by Start; stops on the supplied ctx's cancellation. Time.NewTicker is reset on every tick (we use a manual select loop) so the first sweep fires immediately at boot — closes the "operator deleted variant files while the bridge was down" case without waiting for the first interval to elapse.

func NewVariantWatcher(lister VariantLister, deleter VariantDeleter, publish PublishFunc, interval time.Duration) *VariantWatcher

func (w *VariantWatcher) SetOnTickComplete(fn func(deletedCount int))

func (w *VariantWatcher) Start(ctx context.Context) (stopFn func())