boltstore

package
v0.1.151 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 19, 2026 License: GPL-2.0, GPL-3.0 Imports: 21 Imported by: 0

Documentation

Overview

Package boltstore is the bbolt-backed core store for subflux's search, subtitle, scan, sync-offset, and poll domains. It fully implements the composite api.Store interface.

Naming

The package lives at internal/boltstore rather than internal/store for a historical reason: during the SQLite-to-bbolt rewrite both engines existed side by side (the differential-parity oracle imported both), so the new engine could not take the internal/store path. The SQLite engine is gone; internal/store now holds only the engine-agnostic leaves this package builds on (store/kv: codec, key encoders, index helpers; store/storetest: the api.Store contract suite). boltstore is the permanent home of the engine.

Ownership

The core store OWNS the shared *bbolt.DB handle: Open opens it and Close closes it. The auth store (internal/authstore) shares the same handle via authstore.New and never closes it.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func IsDiskFullError

func IsDiskFullError(err error) bool

IsDiskFullError reports whether err indicates the underlying storage is full (ENOSPC) or otherwise unable to accept writes. The server uses this to decide whether to raise a persistent alert instead of crash-looping.

Types

type DB

type DB struct {
	// contains filtered or unexported fields
}

DB is the bbolt-backed core store. It owns the *bbolt.DB handle that the auth store shares.

func Open

func Open(path string) (*DB, error)

Open opens (creating if necessary) the bbolt database at path with owner-only (0o600) permissions and a bounded file-lock timeout. The returned *DB owns the handle; the caller must Close it to release the file lock.

A held file lock (a second opener of the same file) fails fast within openTimeout rather than blocking indefinitely (Requirement 13.2).

Open's ordering is explicit (migrate.go owns steps 1-3):

  1. Read both schema-version stamps STRICTLY (missing, malformed, older, current, and newer are distinguished; a populated file with a missing or malformed stamp, or any stamp newer than this build, refuses to open).
  2. Validate the registered migration ladders (a malformed registry fails the open loudly before any user data is touched).
  3. Run any pending forward migration steps, snapshotting the file first. When both stamps already equal this build's versions — every open but the first after an upgrade — this is a no-op fast path.
  4. Bootstrap EVERY core and auth bucket (CreateBucketIfNotExists) and re-stamp the current versions. Bucket bootstrap deliberately runs AFTER the ladder so it cannot pre-create buckets a migration step expects to create or rename.

The core store is the single bucket-schema owner: it creates every primary and index bucket from coreBuckets+authBuckets (the auth-domain key builders live in internal/authstore, but the buckets are owned here), so the auth store only ever shares the already-bootstrapped handle.

The schema versions are written UNCONDITIONALLY to the current value, not only-when-absent. The migration runner has already refused a newer stamp and advanced an older one through the ladder, so the only values that reach the write are absent (fresh file) or equal (re-open / just-migrated). Setting them to the current value in every case is correct and avoids a read-modify-write branch: after a successful Open the file is, by definition, a current-build file.

If migration or bootstrap fails, Open closes the handle before returning so a failed Open never leaks the OS file lock.

func (*DB) BackedOffProviders

func (d *DB) BackedOffProviders(_ context.Context, mediaType api.MediaType, mediaID, language string, maxAttempts int) ([]api.ProviderID, error)

BackedOffProviders returns the providers that should be skipped for a triple due to adaptive backoff. A provider is backed off when its failure count reaches maxAttempts OR its next_retry is in the future; when maxAttempts is zero or negative the threshold check is disabled and only the next_retry check applies. Providers with no recorded row for the triple are absent from the scan, so they are never backed off (no row means immediately eligible).

Rows with an empty provider component are skipped, matching the old store's `provider != ”` filter.

func (*DB) BackupInto

func (d *DB) BackupInto(ctx context.Context, dest string) error

BackupInto writes a single consistent hot snapshot of the entire bbolt file (the core domain plus the auth buckets — one engine, one file, one backup) to dest. It runs bbolt's tx.WriteTo inside a read (View) transaction, so the snapshot is a point-in-time-consistent copy that never captures a torn or mid-commit state, even while the live store keeps serving writes (MVCC readers do not block the single writer).

dest must be an absolute, traversal-free path (the server builds it under the configured backup directory). The server names artifacts "subflux-<ts>.bolt"; the matching prune glob in internal/server/backup.go is "subflux-*.bolt", and the leading dash excludes the live "subflux.bolt" from pruning.

WriteTo is a copy, not a compaction

tx.WriteTo copies every live AND free page, so the snapshot is the same size as the live file and is NOT defragmented. Reclaiming disk after heavy churn is a SEPARATE, explicit bbolt.Compact step (see the file-growth runbook, spec task 12.4); it is deliberately not done here so a routine backup stays a cheap, read-only streaming copy.

Atomic write

The snapshot streams into an atomicfile.PendingFile in dest's directory, is fsynced and closed, then renamed onto dest with a parent-directory fsync (via atomicfile). A crash mid-backup therefore leaves either a complete snapshot at dest or nothing at dest — never a partial "subflux-<ts>.bolt" that pruning would later treat as a valid backup.

Restore procedure (summary; full runbook is spec task 12.4)

Backups are plain bbolt files. To restore: stop the subflux container (so it releases the exclusive file lock), copy a chosen snapshot over /config/subflux.bolt, then restart the container. Point Kopia (or any snapshotting backup agent) at the backup snapshot DIRECTORY, never at the live mmap'd /config/subflux.bolt — copying the live file out from under the writer can capture a torn page; these WriteTo snapshots are the consistent artifact to archive.

func (*DB) BoltDB

func (d *DB) BoltDB() *bbolt.DB

BoltDB exposes the underlying *bbolt.DB handle so the auth store (internal/authstore) can share the same file. The auth store NEVER closes this handle — the core store owns it exclusively.

func (*DB) CleanupDrift

func (d *DB) CleanupDrift(_ context.Context, drift api.ConfigDrift) error

CleanupDrift clears adaptive-backoff state after a config change, mirroring the old SQLite CleanupDrift (Requirements 7.7, 7.8). The drift describes what the new config dropped relative to the old one:

  • AdaptiveDisabled: adaptive search was turned off, so EVERY search_attempts row is cleared. This short-circuits the per-language/provider cleanup (the old store returned early after the blanket DELETE), since clearing everything subsumes it.
  • RemovedLanguages / RemovedProviders: a language or provider left the config, so the rows whose attempt-key carries that language or provider component are cleared (the language and provider are components of the `mt 0x00 mid 0x00 lang 0x00 provider` key).

All deletes route through the deleteAttempt chokepoint, so the attempts counter stays consistent. The whole operation runs in one bbolt Update: config-drift cleanup is a hot-reload-time event over a small bucket, not a per-item hot path, so it does not need the bounded batching that DeleteStateByPaths / ReconcileState use. Keys are collected before deletion (bbolt skips the next key if you delete during cursor iteration). The operation is idempotent: re-running it finds the matching rows already gone.

func (*DB) ClearManualLock

func (d *DB) ClearManualLock(_ context.Context, mediaType api.MediaType, mediaID, language string, variant api.Variant) error

ClearManualLock removes the quad's manual lock so automated scans and upgrades resume; an empty variant clears the locks of EVERY variant of the language (the CLI/API "unlock this item+language" default). It is NON-destructive: it flips each manual row's flag to auto (manual=false) and rewrites the row, preserving its id, path, score, provider, release_name, and media_imported, so the rows stay visible to GetState and DownloadedRefs (Requirement 4.3). This mirrors the old SQLite `UPDATE subtitle_state SET manual = 0 WHERE ... AND manual = 1`, which was a flag flip, not a delete.

Each flipped row routes through the putState chokepoint (the record carries its own quad), so the ix_state_quad projection is rewritten with manual=false in the same transaction (the row keeps its id, so its other index entries are unchanged and the downloads counter is not double-counted). A quad with no manual row is a no-op.

func (*DB) ClearScanCycleStart added in v0.1.148

func (d *DB) ClearScanCycleStart(_ context.Context) error

ClearScanCycleStart removes the cycle-start mark. Idempotent.

func (*DB) Close

func (d *DB) Close(_ context.Context) error

Close closes the underlying bbolt handle. The core store owns the handle, so this is the single place it is closed; the auth store's Close never touches it. The context satisfies the api.Store contract (it bounds caller shutdown time, e.g. a SIGTERM grace period); bbolt's own Close takes no context. Close is safe to call on a zero DB.

func (*DB) CurrentScore

func (d *DB) CurrentScore(_ context.Context, mediaType api.MediaType, mediaID, language string, variant api.Variant) (score int, mediaImported time.Time, found bool, err error)

CurrentScore returns the highest auto-row (manual=false) score for the quad, that winning row's media_imported, and a found flag that is false when the quad has no auto row. It mirrors the old SQLite `SELECT score, media_imported FROM subtitle_state WHERE ... AND manual = 0 ORDER BY score DESC LIMIT 1` (Requirement 3.4), refined per variant: manual rows are ignored, an fr/forced row never answers for fr/standard, and no auto row means (0, zero-time, false, nil) with no error.

The ix_state_quad projection carries the score but not media_imported, so the winning row is dereferenced via the shared collectStateRows helper (which fails closed on a decode error). On a score tie the first row in quad-scan order (ascending surrogate id) wins, a deterministic choice the contract leaves open.

func (*DB) DeleteStateByPaths

func (d *DB) DeleteStateByPaths(_ context.Context, videoPaths []string) (api.CleanupResult, error)

DeleteStateByPaths removes every subtitle_state row backed by any of the given video paths and cleans the resulting orphans, mirroring the old SQLite DeleteStateByPaths (Requirement 7.6). A single video file backs SEVERAL state rows (an auto row plus manual rows across languages), so for each path it:

  • prefix-scans ix_state_video to collect every row's surrogate id,
  • deletes each row through the deleteState chokepoint (which removes the ix_state_quad / ix_state_imported / ix_state_video entries and decrements the downloads counter in the same tx),
  • clears every affected language triple's search_attempts backoff via clearTripleBackoff,
  • deletes the subtitle_files and scan_state rows for any affected media item left with no remaining subtitle_state row (the "media left with no state" condition, matching the old cleanOrphanedCoverage).

It returns the non-empty subtitle file paths of the deleted rows so the caller can remove them from disk as a fallback (the arr usually deletes them already). Work is split into bounded Update transactions; the operation is idempotent, so re-running it on the same paths is a safe no-op.

func (*DB) DeleteSubtitleFile

func (d *DB) DeleteSubtitleFile(_ context.Context, mediaType api.MediaType, mediaID, language string, variant api.Variant, source api.SubtitleSource, path string) error

DeleteSubtitleFile removes the single subtitle_files row identified by its full composite key, mirroring the old SQLite DELETE on the full primary key (Requirement 15.7). It is idempotent: deleting an absent row is a no-op, and the TotalSubtitleFiles counter only moves when a row actually existed.

func (*DB) DownloadedRefs

func (d *DB) DownloadedRefs(_ context.Context, mediaType api.MediaType, mediaID, language string) ([]api.DownloadedRef, error)

DownloadedRefs returns every distinct (release_name, provider) pair already downloaded for the language, across BOTH auto and manual rows of EVERY variant, excluding rows whose release_name is empty. It mirrors the old SQLite `SELECT DISTINCT release_name, provider FROM subtitle_state WHERE ... AND release_name <> ”` (Requirement 3.5): the manual-search popup uses it to mark every previously-saved subtitle as already on disk, and the popup lists all variants of the language, so this read is deliberately language-scoped (empty-variant scan). An empty release_name can never match a search result's non-empty ReleaseName, so it is dropped (matching the legacy WHERE clause).

release_name lives on the primary row, not in the ix_state_quad projection (which carries only manual/score/provider), so this walks the language via the shared collectStateRows helper, which dereferences each primary and fails closed on a decode error. Distinctness is preserved with first-seen ordering over the scan (ascending (variant, surrogate id)).

func (*DB) GetBackoffByPrefix

func (d *DB) GetBackoffByPrefix(_ context.Context, mediaType api.MediaType, mediaIDPrefix string) ([]api.BackoffEntry, error)

GetBackoffByPrefix returns the backed-off provider rows for one media type, optionally narrowed to media ids that start with mediaIDPrefix, ordered by media id then ascending next_retry. It prefix-scans the search_attempts primary bucket on `mediaType 0x00 mediaIDPrefix` (an empty prefix returns every row for the type) and then sorts by (media_id, next_retry), because no single index orders by media id then next_retry. Rows with an empty provider component are excluded, matching the old store's `WHERE media_type = ? AND provider != ” ... ORDER BY media_id, next_retry ASC`.

The prefix is a media-id starts-with match (LIKE 'prefix%'): querying "tt1" intentionally returns both "tt1" and "tt12", unlike the exact triple scans which use a trailing separator for component-boundary isolation.

func (*DB) GetBackoffItems

func (d *DB) GetBackoffItems(_ context.Context) ([]api.BackoffEntry, error)

GetBackoffItems returns every backed-off provider row ordered by ascending next_retry, then by primary key for a deterministic tie order. It scans the primary bucket and sorts in memory: the bucket holds only currently backed-off (triple, provider) pairs, so the sort input is small and bounded, and this listing is a rare introspection call (CLI `subflux backoff`, the backoff API). Rows with an empty provider component are excluded, matching the old store's `WHERE provider != ” ORDER BY next_retry ASC`.

func (*DB) GetManualLocks

func (d *DB) GetManualLocks(_ context.Context) ([]api.ManualLockEntry, error)

GetManualLocks returns one entry per manually locked quad (a quad with at least one manual row), each carrying its manual-row count, ordered by media_type then media_id. It mirrors the old SQLite `SELECT media_type, media_id, language, COUNT(*) ... WHERE manual = 1 GROUP BY media_type, media_id, language ORDER BY media_type, media_id` (Requirement 15.5), refined per variant: an fr/forced lock and an fr/standard lock list as separate entries.

It is served entirely from the ix_state_quad projection: the manual flag lives in the projection value and the quad components in the key, so no primary subtitle_state row is dereferenced. Walking ix_state_quad visits entries in (mt, mid, lang, variant, id) byte order, so accumulating per quad yields groups already ordered by (media_type, media_id) — a deterministic refinement of the old ORDER BY. As a lock-bearing read it FAILS CLOSED: an undecodable projection aborts the read rather than silently dropping a lock.

func (*DB) GetPollTimestamp

func (d *DB) GetPollTimestamp(_ context.Context, key api.PollKey) (time.Time, error)

GetPollTimestamp returns the last poll cursor stored for an arr source, or the zero time with no error when the key has no stored value (Requirement 6.3), matching the old SQLite not-found-means-zero behaviour. A non-canonical key is rejected (mirrors the old store's key.Valid guard), and a stored value that cannot be parsed as RFC3339Nano is surfaced as an error rather than silently treated as the zero time.

func (*DB) GetScanStates

func (d *DB) GetScanStates(_ context.Context, mediaType api.MediaType, mediaIDPrefix string) ([]api.ScanStateRow, error)

GetScanStates returns the scan_state rows for a media type and an optional media-id prefix, ordered by media_id. It mirrors the old SQLite GetScanStates (Requirement 5.2), which builds its filter with txutil.AppendPrefixFilter: an empty prefix returns every row of the media type, and a non-empty prefix is a byte-PREFIX match (media_id LIKE 'prefix%'). The bbolt cursor yields keys in byte order, so within a fixed media type the rows already come out ordered by media_id, matching the SQL ORDER BY for free. scanned_at is rendered with scanTimeLayout to match the SQLite string shape. scan_state is a derived bucket, so an undecodable value is skipped with a warning (Requirement 13.4).

func (*DB) GetState

func (d *DB) GetState(_ context.Context, q *api.StateQuery) ([]api.StateEntry, error)

GetState returns subtitle-state rows matching the query, most-recently- imported first. It mirrors the old SQLite GetState (Requirement 8.4, 15.1, 15.2, 15.3):

  • Filters by media_type, language (both carried in the ix_state_quad key) and provider (carried in the primary row); zero-value fields mean no filter.
  • Title search is a case-insensitive CONTAINS match in which the user's %, _, and \ are treated literally (see asciiContainsFold), matching the old `LIKE ... ESCAPE '\'`.
  • Orders by media_imported DESC, then surrogate-id DESC. This is exactly a REVERSE walk of ix_state_imported (be64(media_imported) 0x00 be64(id)), so the ordering — including the id tiebreak on equal media_imported — comes from the index, not an in-memory sort.
  • Applies the numeric Offset (skip that many matches) for shallow callers and caps the result at Limit, defaulting to defaultQueryLimit (1000) when Limit <= 0. Walking the index and skipping matched rows is the keyset- friendly path: a page costs O(offset + limit) index steps regardless of table size, with no full primary scan or sort.

The record is self-contained (it carries its quad), so the page walk dereferences and decodes ONLY the rows it visits — a page costs O(offset + limit) index steps plus that many primary decodes, independent of table size. A row whose primary cannot be decoded is skipped with a warning (subtitle_state is a derived bucket the next scan rebuilds; this is not a lock-bearing read).

func (*DB) GetSubtitleFiles

func (d *DB) GetSubtitleFiles(_ context.Context, mediaType api.MediaType, mediaIDPrefix string) ([]api.SubtitleEntry, error)

GetSubtitleFiles returns the subtitle_files rows for a media type and an optional media-id filter, ordered by media_id, language, variant, source (the natural byte order of the composite key, so the cursor yields it for free). It mirrors the old SQLite GetSubtitleFiles, including the prefix semantics and the subtitle_state join (Requirement 5.2):

  • an empty filter returns every row of the media type,
  • a filter NOT ending in "-" is an EXACT media-id match,
  • a filter ending in "-" is a media-id PREFIX match (e.g. "tvdb-111-").

The MediaID/Language/Variant/Source/Path fields come from the KEY (no value decode), and only Codec comes from the value; OffsetMs is joined from sync_offsets. Score and VideoPath reproduce the old `LEFT JOIN subtitle_state ... AND s.manual = 0 AND f.source != 'embedded'` with `CASE WHEN f.source = 'embedded' THEN 0` and the COALESCE defaults: an embedded row always reports score 0 and an empty video_path; an external row takes them from the triple's auto subtitle_state row (see autoStateInfo). An undecodable value is skipped (derived bucket).

func (*DB) GetSyncOffset

func (d *DB) GetSyncOffset(_ context.Context, path string) (int64, error)

GetSyncOffset returns the stored sync offset (in milliseconds) for a subtitle path, or 0 with no error when the path has no stored offset (Requirement 6.1), matching the old SQLite store's not-found-means-zero behaviour. The value is the be64(offset_ms) written by SetSyncOffset, decoded and reinterpreted back to int64 (bit-preserving, so negatives round-trip).

func (*DB) HistoryMediaIDs

func (d *DB) HistoryMediaIDs(_ context.Context, mediaType api.MediaType, mediaIDPrefix string) ([]string, error)

HistoryMediaIDs returns the distinct media ids that have download history for the given media_type, optionally filtered to those whose id starts with mediaIDPrefix. It mirrors the old SQLite `SELECT DISTINCT media_id FROM subtitle_state WHERE media_type = ? [AND media_id LIKE escaped(prefix)||'%' ESCAPE '\']` (Requirement 8.4): the prefix match is case-insensitive (ASCII) with the user's %/_/\ treated literally (asciiHasPrefixFold).

The media_type and media_id live in the ix_state_quad key, so the distinct set is answered key-only via a bounded skip-scan: seek to the media-type prefix, emit the id under the cursor, then leapfrog directly past that id's remaining rows (its 0x00 separator bumped to 0x01 seeks to the first key of the NEXT id, including ids that textually extend this one). Cost is one seek per distinct id instead of one visit per row, ids arrive in ascending byte order, and no dedup map is needed. Measured at the 52k-episode reference shape: ~8.9x faster for the later-sorting media type (movies) and at parity in the single-language worst case (see query_scale_bench_test.go).

func (*DB) IsManuallyLocked

func (d *DB) IsManuallyLocked(_ context.Context, mediaType api.MediaType, mediaID, language string, variant api.Variant) (bool, error)

IsManuallyLocked reports whether the quad has at least one manual row, so it should be excluded from all automated actions. An empty variant asks whether ANY variant of the language is locked (the language-level summary the manual search popup shows). It mirrors the old SQLite `SELECT EXISTS(... WHERE ... AND manual = 1)` (Requirement 4.2), refined per variant.

The answer comes purely from the ix_state_quad projection's manual flag via walkStateProjection, so no primary subtitle_state row is dereferenced (Requirement 18.3). As a lock-bearing read it fails closed: if the projection cannot be read the quad is reported locked AND the error is returned, so a decode fault can never silently unlock an item (Requirement 13.4).

func (*DB) LastScanTime

func (d *DB) LastScanTime(_ context.Context) (string, error)

LastScanTime returns the most recent scanned_at, formatted with scanTimeLayout in UTC, or the empty string when nothing has been scanned. It mirrors the old SQLite `SELECT MAX(scanned_at)` returning NULL -> "" semantics (Requirement 5.6). The newest entry is the last key of the ix_scan_at index (keys sort chronologically), so this is an O(1) cursor.Last() plus a decode of the timestamp embedded in the index key.

func (*DB) ManualDownloadCount

func (d *DB) ManualDownloadCount(_ context.Context, mediaType api.MediaType, mediaID, language string, variant api.Variant) (int, error)

ManualDownloadCount returns how many manual rows exist for the quad (exact variant), mirroring the old SQLite `SELECT COUNT(*) ... WHERE ... AND manual = 1` (Requirement 15.6). Like IsManuallyLocked it is served purely from the ix_state_quad projection's manual flag via walkStateProjection, with no primary dereference (Requirement 18.3).

func (*DB) ManualSubtitlePaths

func (d *DB) ManualSubtitlePaths(_ context.Context, mediaType api.MediaType, mediaID, language string, variant api.Variant) ([]string, error)

ManualSubtitlePaths returns the subtitle file paths from every manual row for the quad — or every variant of the language when variant is empty — excluding rows with an empty path, mirroring the old SQLite `SELECT path ... WHERE ... AND manual = 1 AND path != ”` (Requirement 15.6). maybeRevertManualLock uses it (exact variant) to check which manual files still exist on disk.

The path lives on the primary row, not in the ix_state_quad projection (which carries only manual/score/provider), so this walks the quad via the shared collectStateRows helper, which dereferences each primary and fails closed on a decode error.

func (*DB) NextManualNumber

func (d *DB) NextManualNumber(_ context.Context, mediaType api.MediaType, mediaID, language string, variant api.Variant) int

NextManualNumber returns the next manual subtitle ordinal for the quad: one greater than the highest ordinal currently encoded in ANY of the quad's row paths, or 1 when no row carries an ordinal (Requirement 4.4). Ordinal discovery deliberately ignores the Manual flag: a top-pick manual download records as an AUTO row (manual=false, auto-mode semantics) yet occupies a numbered path, so a manual-only scan would hand the next download the same number and the atomic write would overwrite the top pick's file. Rows the app numbered are what reserve ordinals, however they are flagged; plain auto rows (movie.fr.srt) have no trailing ordinal and contribute nothing. Sequences stay per variant: movie.fr.1.srt (standard) and movie.fr.forced.1.srt (forced) advance independently, matching the variant-aware manual file naming.

The ordinal lives on the primary path, so this walks the quad via collectStateRows and parses each row's ordinal with the shared parseManualOrdinal helper (rows whose path has no trailing ordinal, including empty paths, contribute nothing, matching the legacy CAST of a non-numeric suffix to 0). The contract has no error channel, so a read fault falls back to ManualDownloadCount + 1, and to 1 if that also fails, matching the old store's degraded path (the count-based fallback cannot see auto-row ordinals; it only runs on a primary decode fault).

func (*DB) RecentlyScanned

func (d *DB) RecentlyScanned(_ context.Context, cutoff time.Time) (map[string]bool, error)

RecentlyScanned returns the set of media ids scanned at or AFTER cutoff (inclusive), across every media type, mirroring the old SQLite `WHERE scanned_at >= ?` (Requirement 5.4). It seeks the ix_scan_at index to be64(cutoff.UnixNano()) and walks forward: a row whose scanned_at equals the cutoff produces an index key be64(cutoff) 0x00 primary, which sorts at or after the 8-byte seek key, so the cutoff itself is included while the instant just before it is excluded. The media id is recovered from the dereferenced primary key (mt 0x00 mid); like the SQL query it keys the result set by media id alone, independent of media type.

func (*DB) ReconcileState

func (d *DB) ReconcileState(ctx context.Context) (api.ReconcileResult, error)

ReconcileState reconciles subtitle_state against the filesystem in three branches, group-aware by quad (media_type, media_id, language, variant), mirroring the corrected three-way semantics of the old store (Requirements 7.1-7.5):

  • VIDEO GONE: every row whose video file no longer exists is deleted along with its orphaned subtitle file, the language's backoff, and any scan_state for a media item left with no state. This fan-out is delegated to DeleteStateByPaths (task 6.1), which already prefix-scans ix_state_video, routes deletes through the deleteState chokepoint, clears backoff, and cleans orphaned coverage in bounded transactions.
  • SUBTITLE GONE, A SIBLING STILL PRESENT: when a row's subtitle file is gone but its video exists AND at least one other row for the SAME quad still has its subtitle on disk, only that one row is deleted. The remaining rows and any manual lock are PRESERVED, backoff is NOT cleared, and no row is reset (Requirement 7.2).
  • ALL SUBTITLES FOR A QUAD GONE: when every row for a quad has lost its subtitle (video still present), the auto rows are reset in place (clear path/score/provider/release_name, bump media_imported to now) so the next scan re-searches, the manual rows are deleted, and the language's backoff is cleared (Requirement 7.3). Counted once per quad in ResetCount. Grouping by quad keeps the branches variant-precise: deleting the last forced subtitle resets only the forced rows, never the standard ones.

The read phase snapshots the rows under short View transactions and performs all filesystem stats with no transaction open; the mutation phase runs in bounded Update batches (Requirement 7.4). Every operation is idempotent, so an interrupted pass converges on re-run (Requirement 7.5): a deleted row stays gone, and a reset auto row has an empty sub_path that classifies as skip on the next pass, so media_imported is not bumped again.

func (*DB) RecordNoResult

func (d *DB) RecordNoResult(_ context.Context, mediaType api.MediaType, mediaID, language string, providerName api.ProviderID,
	bp api.BackoffParams,
) error

RecordNoResult records a failed provider search attempt for a triple and recomputes its backoff window, all in one write transaction. It reads the prior attempt (if any) to obtain the failure count, increments it, computes the new next_retry from the BackoffParams, and writes the row through the putAttempt chokepoint, which maintains the attempts counter in the same transaction.

A brand-new row starts at failures=1 with next_retry = now + InitialDelay, matching the old SQLite INSERT branch (which used the full InitialDelay duration without the integer-second truncation of the upsert path).

func (*DB) RecordScanState

func (d *DB) RecordScanState(_ context.Context, rec *api.ScanRecord) error

RecordScanState upserts the scan-state row for a media item, keyed by (media_type, media_id), and refreshes its ix_scan_at recency entry, in one write transaction. It mirrors the old SQLite upsert (Requirement 5.3): a re-record overwrites title/season/episode/audio_lang and stamps scanned_at to now (UTC). Because scanned_at changes on every record, the putScanState chokepoint deletes the stale ix_scan_at entry and adds the fresh one in the same tx, so the recency index always reflects the latest scan.

func (*DB) RecordSubtitleFiles

func (d *DB) RecordSubtitleFiles(_ context.Context, mediaType api.MediaType, mediaID string, files []api.SubtitleFile) (bool, error)

RecordSubtitleFiles diff-syncs the subtitle_files rows for one media item against the set discovered on disk, in a single write transaction. It mirrors the old SQLite store's diff-based RecordSubtitleFiles (Requirement 5.1):

  • rows present on disk but absent from the store are inserted,
  • rows whose codec changed are updated in place,
  • rows no longer on disk are deleted,
  • it reports changed=true iff at least one insert, update, or delete happened (a set identical to what is already stored returns false).

Duplicate entries in files collapse (last codec wins) because they share a subFileKey, matching the old store's map build. Inserts and deletes move the maintained TotalSubtitleFiles counter through the chokepoints; a codec-only update does not (the row already existed).

func (*DB) SaveDownload

func (d *DB) SaveDownload(_ context.Context, rec *api.DownloadRecord) error

SaveDownload records (or upgrades) a subtitle download in one write transaction, preserving the old SQLite store's observable behaviour while keying state by the (media_type, media_id, language, variant) quad:

  • It first clears ALL adaptive-backoff rows for the language triple (success clears backoff; backoff has no variant dimension), each through the deleteAttempt chokepoint so the attempts counter stays consistent (Requirement 3.3).
  • For an AUTO download (Meta.Manual false), it updates every existing auto row for the QUAD in place, preserving each row's original media_imported and surrogate id (Requirement 3.1); if no auto row exists for the quad it inserts a fresh one with media_imported = now (Requirement 3.2). An fr/forced download therefore never overwrites the fr/standard auto row.
  • For a MANUAL download (Meta.Manual true), it always appends a new manual row (the manual row is the quad's automation lock), storing rec.Path verbatim so the row's ordinal equals the ordinal the handler already encoded in the filename (Requirement 4.5); the ordinal is authoritative in rec.Path and is never re-derived from the bucket here.

An empty rec.Variant is normalized to VariantStandard, so legacy callers that predate the variant dimension keep writing standard rows.

All mutations route through the putState / deleteAttempt index-maintenance chokepoints, so the secondary indexes and the maintained meta counters are updated in the same all-or-nothing transaction as the primary writes.

func (*DB) ScanCycleStart added in v0.1.148

func (d *DB) ScanCycleStart(_ context.Context) (time.Time, error)

ScanCycleStart returns the persisted cycle-start mark, or the zero time with no error when no mark is stored. An unparsable stored value is surfaced as an error rather than silently treated as absent.

func (*DB) SetPollTimestamp

func (d *DB) SetPollTimestamp(_ context.Context, key api.PollKey, t time.Time) error

SetPollTimestamp stores the last poll cursor for an arr source, formatted with RFC3339Nano so GetPollTimestamp recovers it with full precision (Requirement 6.2). A non-canonical key is rejected before any write (mirrors the old store's key.Valid guard), preventing a typo from silently creating a new cursor row and forcing a full history re-fetch. A later set overwrites the prior cursor.

func (*DB) SetScanCycleStart added in v0.1.148

func (d *DB) SetScanCycleStart(_ context.Context, t time.Time) error

SetScanCycleStart persists the cycle-start mark (RFC3339Nano), overwriting any prior mark.

func (*DB) SetSyncOffset

func (d *DB) SetSyncOffset(_ context.Context, path string, offsetMs int64) error

SetSyncOffset stores the cumulative sync offset (in milliseconds) for a subtitle file, keyed by its bare path in the dedicated sync_offsets bucket (Requirement 6.1). The value is be64(offset_ms): a single fixed-width put, no JSON, since sync_offsets has no secondary index and no projection. The int64 is reinterpreted as a uint64 for encoding, which is bit-preserving, so a negative offset (subtitle ahead of audio) round-trips exactly through GetSyncOffset. A later set for the same path overwrites the prior value.

func (*DB) Stats

func (d *DB) Stats(_ context.Context) (downloads, attempts int, err error)

Stats returns the maintained O(1) download and attempt counters from the meta bucket, mirroring the old SQLite `COUNT(*) FROM subtitle_state` and `COUNT(*) FROM search_attempts` without scanning either bucket (Requirements 18.1, 18.4). The counters are moved inside the same Update as every row insert/delete via the index-maintenance chokepoints, so they track row existence exactly like COUNT(*).

func (*DB) StoreFileStats

func (d *DB) StoreFileStats() (fileBytes, freelistBytes int64)

StoreFileStats returns the current database file size and reclaimable freelist bytes. These are cheap reads: file size from os.Stat, freelist from bbolt's in-memory DB.Stats() (no transaction required for the freelist stat). The caller (typically the metrics collector on a periodic timer or at scrape time) uses these to set the subflux_store_file_bytes and subflux_store_freelist_bytes gauges.

If the store is not open or a stat error occurs, it returns zeros without error (metrics should degrade gracefully, not block scrapes).

func (*DB) TotalSubtitleFiles

func (d *DB) TotalSubtitleFiles(_ context.Context) (int, error)

TotalSubtitleFiles returns the total subtitle_files row count from the O(1) maintained meta counter (readFileCount), never a full-bucket scan (Requirement 18.1). The counter is moved inside the same transaction as every insert/delete by the putSubtitleFile / deleteSubtitleFile chokepoints, so it equals COUNT(*) of the bucket.

func (*DB) UpsertSubtitleFile

func (d *DB) UpsertSubtitleFile(_ context.Context, mediaType api.MediaType, mediaID string, f *api.SubtitleFile) error

UpsertSubtitleFile inserts or updates a single subtitle_files row, mirroring the old SQLite `INSERT ... ON CONFLICT DO UPDATE SET codec, updated_at` (Requirement 15.7). The write routes through putSubtitleFile so the TotalSubtitleFiles counter is maintained.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL