Documentation
¶
Overview ¶
Package transcode owns offline PCM-upscaling: building sox(1) invocations, choosing variant identifiers + sidecar paths, and the worker pool the CLI subcommand and (in Phase 2.5) the HTTP `POST /v1/upscale` handler share.
**Why shell out to sox** instead of cgo + libSoXr: keeping the bridge build pure-Go preserves `make build-all` cross-compilation to darwin/linux/windows × amd64/arm64 from a single host. SoXr is the same engine via a different surface; perceived audio quality is identical at the `-v` ("very high") preset we use here. The cost is a runtime dependency the operator installs once (`apt install sox` / `brew install sox` / `choco install sox`).
Bridge `serve` and the `bridge upscale` CLI both probe for `sox` on PATH at startup; if missing, the CLI exits with an install hint and `serve` logs an error then disables the feature in- memory (graceful degradation — the rest of the server keeps running).
**Bit-exact mission preserved**: transcoding is offline; the bridge never modulates bytes in flight. The sidecar `.flac` is a real on-disk file the bridge then serves bit-exact via the same `serveFile` path that the original source uses.
Index ¶
Constants ¶
const VariantSchemaVersion = "v2"
VariantSchemaVersion is bumped when the on-disk sidecar layout or the SoX command shape changes in a way that makes prior sidecars semantically different from what a fresh run would produce. Today's only producer is `bridge upscale --quality very-high`. A future change of resampler preset (e.g. switching to a min-phase filter) bumps this so the iOS picker can distinguish "v1 upscale" from "v2 upscale" if they ever coexist during a transition.
**v2 (this version)**: `-G` (gain-guard) added to SoxArgs. Sox pre-scans the input for the headroom needed to prevent clipping during the rate-conversion + dither pipeline and applies a matching attenuation. Fires only when the source has 0 dBFS peaks (most modern pop masters); the typical attenuation is well under 1 dB. The audio CONTENT shifts under guard, so the schema bump produces a fresh VariantID — operators run `bridge upscale` once after upgrade and the iOS client picks up the new guard-clean variants automatically. Pre-v2 sidecars stay served by their existing track_variants rows until the next `bridge upscale --gc` pass cleans them up.
Variables ¶
var ErrPoolClosed = errors.New("transcode pool is closed")
ErrPoolClosed is returned when Enqueue is called after Stop. Production consumers shouldn't see this — `bridge serve` only stops the pool during graceful shutdown, when no new requests are accepted. Tests use it.
var ErrQueueFull = errors.New("transcode pool queue is full")
ErrQueueFull is returned by Enqueue when the pending-job channel is at capacity. HTTP callers map this to a 503 `queue_full` response so the iOS toast can say "Queue full — wait for current conversions to finish, then try again."
var ErrSoxMissing = errors.New("sox binary not found on PATH")
ErrSoxMissing is returned by PrecheckSox when `sox` isn't on PATH. Test for it via errors.Is so callers can surface a targeted install-hint message vs the generic "something went wrong" path.
Functions ¶
func CreatedAtNow ¶
func CreatedAtNow() int64
CreatedAtNow returns wall-clock UTC nanoseconds — the value callers stamp into `track_variants.created_at` on insert. Centralised here so test stubs can override (today: a const, good enough).
func PickTargetRate ¶
PickTargetRate decides the output sample rate from the source rate when the operator passed `--target-rate auto`. Picks the highest integer-ratio target within the 44.1/48 family that stays at or below 192 kHz — the sweet spot for modern DACs' integer-ratio oversampling filters:
44100 → 176400 (4× — two octaves above source) 88200 → 176400 (2× — one octave above source) 176400 → 0 (skip; source is already at the auto target) 48000 → 192000 (4×) 96000 → 192000 (2×) 192000 → 0 (skip) other → 0 (don't auto-pick a target for unfamiliar rates)
Returns 0 when no auto target makes sense; callers treat 0 as "skip this track". Operators with DACs that prefer a higher rate (Chord M-Scaler at 705.6) override via explicit `--target- rate 705600`.
func PrecheckSox ¶
func PrecheckSox() error
PrecheckSox returns nil if the `sox` binary is on PATH and reports a non-error stderr to `--version`. Returns ErrSoxMissing if the binary can't be located, or a generic error if invocation fails for any other reason. Called by both `bridge upscale` (CLI entry point) and the `bridge serve` startup gate (Phase 2.5).
**Bounded by a 2 s timeout** (CodeRabbit second-pass on PR #108) so a wedge from a broken PATH wrapper or a hung sox process can't deadlock startup. `bridge serve` runs this before opening the listen socket; without the timeout, every service-manager restart with a misbehaving sox installation would block forever instead of degrading cleanly.
func ResolveTargetRate ¶
ResolveTargetRate converts the user-facing `--target-rate` flag (string, possibly "auto") into the integer Hz value to feed into SoxArgs. Returns (0, nil) when the resolution is "skip this source" (auto + already at/above target, or a deliberate integer at/below the source). Returns (>0, nil) when there's a real target. Returns (0, err) on a malformed flag.
func RunSox ¶
RunSox executes one JobSpec, writes the sidecar to disk, and returns the on-disk size on success. Atomic-rename pattern: the sox output goes to `<sidecar>.tmp`, then we `os.Rename` to the final path on success — a crash mid-conversion leaves at most a `.tmp` file behind (cleaned up by `bridge upscale --gc`).
The `sox` binary is located via PATH. Caller is expected to have already probed for it via PrecheckSox; we don't repeat the probe here so a worker-pool body doesn't pay the LookPath cost per iteration.
**Cancellation**: ctx is plumbed via `exec.CommandContext`. A SIGINT to the CLI / SIGTERM to `bridge serve` cancels the outer context, exec.CommandContext SIGKILLs the in-flight sox process, and we clean up the partial `.tmp`. Without this a half-converted album could hang the operator's terminal until the largest file finishes (Gemini bot review on PR #108).
Output directory created if missing — `bridge upscale` only guarantees DataDir exists, not the `transcoded` subdir.
Types ¶
type JobSpec ¶
type JobSpec struct {
SourceAbsPath string
SourceLibraryRel string
SourceMTimeNS int64
SourceSize int64
SourceSampleRate int // Hz; 0 if unknown (won't be used in target-rate selection)
TargetSampleRate int // Hz; e.g. 176400 / 192000
TargetBits int // 16/24/32
Quality Quality
OutputDir string // <dataDir>/transcoded
}
JobSpec describes one source-to-sidecar conversion. Constructed by the CLI / handler from a (Track, target params) pair; passed to RunSox.
`SourceAbsPath` is the absolute filesystem path to read from; `SourceLibraryRel` is the library-relative wire form that lands in the `track_variants.source_path` column (matches the manifest `Track.Path` field that iOS uses to construct download URLs). Keeping both fields here avoids a re-resolve at write time.
func (*JobSpec) FreshnessFromFile ¶
FreshnessFromFile populates the SourceMTimeNS / SourceSize fields on a JobSpec by stat-ing SourceAbsPath. Used by the CLI to capture "what version of the source did we convert from" at the moment of conversion, which the variant-resolve path later uses to detect drift.
func (JobSpec) SidecarPath ¶
SidecarPath returns the absolute filesystem path the converted FLAC will land at. Filename pattern:
<sha256(SourceLibraryRel)[:16]>-<VariantID>.flac
The variantID suffix is load-bearing: hashing only the source path would let two runs at different target rates overwrite each other (the user's first call with --target-rate 176400 followed by a second with --target-rate 192000 would clobber the first). Including the variantID guarantees multiple variants of the same source coexist safely on disk.
16 hex chars = 64 bits of entropy. Collisions are scoped to a single user's SourceLibraryRel namespace (typically <100k tracks) so 64 bits is far more than enough; truncating buys ~48 chars of Windows MAX_PATH headroom for deeply-nested OutputDirs.
Migration: existing 64-char-hash sidecars on disk stay served by their existing track_variants.sidecar_path rows (the absolute path stored in the DB still points at them). Only newly-transcoded variants get the 16-char form. Zero-friction upgrade.
func (JobSpec) SoxArgs ¶
SoxArgs builds the argv for the sox invocation. Returns the exact slice exec.Command will receive (including the leading input/output sentinels) and a JSON-friendly settings string for `track_variants.sox_settings` (forensic record of what produced this sidecar). Pure function — no I/O — so unit tests can pin the exact command shape without invoking sox.
Quality preset → `rate` flag mapping:
QualityVeryHigh → "-v" (very high; ~95 dB stopband, 32-bit internal) QualityHigh → "-h" QualityMedium → "-m"
`dither -s` is shaped TPDF dither when truncating the 32-bit internal float to the 24/32-bit integer FLAC output. Required for audible transparency at 24-bit and benign at 32-bit (the dither noise is below the LSB).
`-G` (gain-guard, leading global option) instructs sox to pre- scan the input for the headroom needed to prevent clipping through the rate-conversion + dither pipeline and apply a matching attenuation. Required for upscaling 0 dBFS-mastered material (most modern pop / loudness-war masters): without it, intersample peaks the rate filter reconstructs above 0 dBFS would clip on the integer FLAC output. The attenuation is computed from the actual peak — typically well under 1 dB and inaudible — and only fires when the source has the headroom problem. Cost is one extra peak-scan pass; invisible against the rate-conversion CPU dominating the run.
func (JobSpec) VariantID ¶
VariantID returns the opaque identifier that uniquely names this JobSpec's output variant. Convention:
upscaled-<schemaVersion>-<targetRate>-<targetBits>
e.g. `upscaled-v2-176400-24`. iOS keys on the `upscaled-` prefix to slot the variant into the share-level "prefer upscaled" resolution. Future variant kinds (e.g. PCM→DSD synthesis) get their own prefix.
type Pool ¶
type Pool struct {
// contains filtered or unexported fields
}
Pool is the long-lived worker pool that hosts SoX conversions for the v1.2 PCM upscaling feature. Two consumers in production:
- `bridge serve`: instantiates one Pool at startup (only when `cfg.Upscale.Enabled == true` AND the sox-on-PATH probe passes), attaches it to the api.Server, and feeds it with `POST /v1/upscale` requests.
- `bridge upscale` CLI: instantiates its own per-invocation Pool with worker count from `--workers`. Same primitive, different lifetime.
The Pool's pending-job channel is bounded (`QueueCap`); enqueues are non-blocking via `select` + default → return ErrQueueFull instead of waiting. The HTTP handler maps that to a `503 queue_full` response so a user spamming "Generate" on a 50k- track library bounces against a clean rejection rather than exhausting memory.
**Dedup keyed on (source_path, variant_id)**: a duplicate enqueue while a job is already queued or running is a silent no-op. The hash set is mutex-guarded; reads happen on the HTTP handler goroutine, writes on the worker goroutines and the enqueue site. Same lock covers both — contention is bounded by the worker count.
func NewPool ¶
NewPool constructs a Pool sized at `workers` parallel sox processes with a `queueCap`-bounded pending-job channel. Starts `workers` goroutines immediately; they live until Stop is called. `store` is the SQLite-backed manifest store the completion path writes the row into.
Caller is expected to have already verified sox is on PATH via PrecheckSox — the worker doesn't repeat the probe per job.
func (*Pool) Enqueue ¶
Enqueue submits a JobSpec to the worker pool. Non-blocking: if the queue is full, returns ErrQueueFull immediately. Dedup is silent — a duplicate (same source_path + variant_id) returns nil without taking a slot. After Stop, returns ErrPoolClosed.
**Race-safe vs Stop**: pre-fix, Stop() did `close(p.jobs)` concurrently with an Enqueue holding `inflight[dedup]` but not yet at the channel send — sending on a closed channel panics (Gemini high + Qodo bug 1 + CodeRabbit echo on PR #109). Fix: the channel-send branch runs INSIDE `p.mu` and Stop() takes the same mutex before close. The mutex serialises the two operations cheaply (the send is non-blocking via the select + default, so the lock window stays in microseconds), and the re-checked `closed` flag inside the lock catches a Stop that won the race for the mutex but hadn't yet closed the channel.
func (*Pool) SetOnStateChange ¶
func (p *Pool) SetOnStateChange(fn func())
SetOnStateChange wires (or rewires) the callback fired after every observable state transition. nil disables notification (back-compat for tests / non-broker deployments). Called once during cmd/bridge wiring after the broker is up; subsequent calls are race-safe but in practice this is set-once.
func (*Pool) Stats ¶
Stats returns the current snapshot. Safe to call concurrently with Enqueue / worker activity.
func (*Pool) Stop ¶
func (p *Pool) Stop()
Stop signals the workers to drain the queue and exit. Blocks until every in-flight conversion completes (either through success, failure, or sox-process kill on the cancelled context). The Pool can't be reused after Stop.
Idempotent: calling Stop twice is safe (the underlying CancelFunc is itself idempotent).
**Acquires `p.mu` before closing the channel** so any in-flight Enqueue completes its channel send (or its dedup-rollback) before close() lands. Without this, a concurrent Enqueue could pass its `closed` re-check, attempt to send, and hit a "send on closed channel" panic. See the comment on Enqueue for the full race trace.
type PoolStats ¶
type PoolStats struct {
Workers int
QueueCap int
QueueLen int
Inflight int
Enqueued uint64
Done uint64
Failed uint64
}
Stats returns a snapshot of pool counters for /v1/health observability or tests. Numbers are monotonic per process lifetime; restarting bridge serve resets them.
Source Files