worker

package
v0.0.49 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 25, 2026 License: MIT Imports: 60 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var ErrInvalidScanOptions = errors.New("exactly one of FilePath, FileID, or BookID must be set")

ErrInvalidScanOptions is returned when ScanOptions validation fails.

Functions

func ComputeBulkFingerprint added in v0.0.21 

func ComputeBulkFingerprint(fileIDs []int, fileHashes []string) string

ComputeBulkFingerprint computes a composite fingerprint hash from sorted file IDs and their individual hashes.

func DeduplicateFilenames added in v0.0.21 

func DeduplicateFilenames(names map[int]string) map[int]string

DeduplicateFilenames takes a map of fileID -> filename and appends (2), (3), etc. for duplicates.

func EnsureHashGenerationJob added in v0.0.29 

func EnsureHashGenerationJob(ctx context.Context, jobService *jobs.Service, libraryID int) error

EnsureHashGenerationJob checks whether a pending or in-progress hash generation job already exists for the given library and creates one only if none is found. Safe to call from the scan job at the end of each scan.

Types

type AliasLister added in v0.0.43 

type AliasLister interface {
	ListPersonAliases(ctx context.Context, personID int) ([]string, error)
	ListGenreAliases(ctx context.Context, genreID int) ([]string, error)
	ListTagAliases(ctx context.Context, tagID int) ([]string, error)
	ListSeriesAliases(ctx context.Context, seriesID int) ([]string, error)
	ListPublisherAliases(ctx context.Context, publisherID int) ([]string, error)
}

AliasLister fetches alias names for resolved resources so the cache can pre-populate entries for every name variant (canonical + aliases).

func NewAliasServiceAdapter added in v0.0.43 

func NewAliasServiceAdapter(svc *aliases.Service) AliasLister

type GenreFinder added in v0.0.11 

type GenreFinder interface {
	FindOrCreateGenre(ctx context.Context, name string, libraryID int) (*models.Genre, error)
}

GenreFinder is an interface for finding or creating genres.

type Monitor added in v0.0.20 

type Monitor struct {
	// contains filtered or unexported fields
}

Monitor watches library paths for filesystem changes and triggers targeted rescans using a debounce pattern inspired by Jellyfin's FileRefresher.

func (*Monitor) IgnorePath added in v0.0.20 

func (m *Monitor) IgnorePath(path string)

IgnorePath temporarily suppresses filesystem events for the given path. Used by the scanner to prevent its own writes (covers, sidecars, file moves) from triggering redundant rescans. The path is ignored for 2× the debounce delay.

func (*Monitor) RefreshWatches added in v0.0.20 

func (m *Monitor) RefreshWatches()

RefreshWatches signals the monitor to reload library paths and update watches. Call this after libraries are created, updated, or deleted.

type PersonFinder added in v0.0.11 

type PersonFinder interface {
	FindOrCreatePerson(ctx context.Context, name string, libraryID int) (*models.Person, error)
}

PersonFinder is an interface for finding or creating persons.

type PublisherFinder added in v0.0.11 

type PublisherFinder interface {
	FindOrCreatePublisher(ctx context.Context, name string, libraryID int) (*models.Publisher, error)
}

PublisherFinder is an interface for finding or creating publishers.

type RelationshipUpdates added in v0.0.11 

type RelationshipUpdates struct {
	Authors    []*models.Author
	Narrators  []*models.Narrator
	BookGenres []*models.BookGenre
	BookTags   []*models.BookTag
	BookSeries []*models.BookSeries

	// Delete flags indicate which relationships should be cleared before inserting new ones
	DeleteAuthors   bool
	DeleteNarrators bool
	DeleteGenres    bool
	DeleteTags      bool
	DeleteSeries    bool

	// FileID is required when DeleteNarrators is true (narrators are per-file)
	FileID int
}

RelationshipUpdates holds all relationship data to be updated for a book. This enables bulk inserts for better scan performance. When used in parallel processing, callers should use ScanCache.LockBook to prevent race conditions.

type ScanCache added in v0.0.11 

type ScanCache struct {
	// contains filtered or unexported fields
}

ScanCache provides thread-safe caching for entity lookups during parallel file processing. It uses sync.Map for cache storage and per-key mutexes to prevent duplicate concurrent DB calls.

func NewScanCache added in v0.0.11 

func NewScanCache() *ScanCache

NewScanCache creates a new ScanCache.

func (*ScanCache) AddKnownFile added in v0.0.29 

func (c *ScanCache) AddKnownFile(f *models.File)

AddKnownFile adds a file to the known-files cache at its new path. Used after move reconciliation updates a file's filepath so the parallel processing loop treats the new path as already known (skipping it instead of creating a duplicate). NewScanCache guarantees the map is initialized, so no nil check is needed.

func (*ScanCache) GenreCount added in v0.0.11 

func (c *ScanCache) GenreCount() int

GenreCount returns the number of unique genres in the cache.

func (*ScanCache) GetKnownFile added in v0.0.18 

func (c *ScanCache) GetKnownFile(path string) *models.File

GetKnownFile returns a known file by path, or nil if not found.

func (*ScanCache) GetOrCreateGenre added in v0.0.11 

func (c *ScanCache) GetOrCreateGenre(ctx context.Context, name string, libraryID int, svc GenreFinder) (*models.Genre, error)

GetOrCreateGenre retrieves a genre from the cache or calls the service to find/create one. It uses per-key locking to prevent duplicate concurrent DB calls for the same genre.

func (*ScanCache) GetOrCreatePerson added in v0.0.11 

func (c *ScanCache) GetOrCreatePerson(ctx context.Context, name string, libraryID int, svc PersonFinder) (*models.Person, error)

GetOrCreatePerson retrieves a person from the cache or calls the service to find/create one. It uses per-key locking to prevent duplicate concurrent DB calls for the same person.

func (*ScanCache) GetOrCreatePublisher added in v0.0.11 

func (c *ScanCache) GetOrCreatePublisher(ctx context.Context, name string, libraryID int, svc PublisherFinder) (*models.Publisher, error)

GetOrCreatePublisher retrieves a publisher from the cache or calls the service to find/create one. It uses per-key locking to prevent duplicate concurrent DB calls for the same publisher.

func (*ScanCache) GetOrCreateSeries added in v0.0.11 

func (c *ScanCache) GetOrCreateSeries(ctx context.Context, name string, libraryID int, nameSource string, svc SeriesFinder) (*models.Series, error)

GetOrCreateSeries retrieves a series from the cache or calls the service to find/create one. It uses per-key locking to prevent duplicate concurrent DB calls for the same series.

func (*ScanCache) GetOrCreateTag added in v0.0.11 

func (c *ScanCache) GetOrCreateTag(ctx context.Context, name string, libraryID int, svc TagFinder) (*models.Tag, error)

GetOrCreateTag retrieves a tag from the cache or calls the service to find/create one. It uses per-key locking to prevent duplicate concurrent DB calls for the same tag.

func (*ScanCache) IsMovedOrphan added in v0.0.29 

func (c *ScanCache) IsMovedOrphan(id int) bool

IsMovedOrphan returns true if the given file ID was matched by the move reconciliation phase and should be skipped by orphan cleanup.

func (*ScanCache) LibraryRootPaths added in v0.0.29 

func (c *ScanCache) LibraryRootPaths() []string

LibraryRootPaths returns the cached library root paths. Returns nil if SetLibraryRootPaths was not called for this scan.

func (*ScanCache) LoadKnownFiles added in v0.0.18 

func (c *ScanCache) LoadKnownFiles(files []*models.File)

LoadKnownFiles populates the known files cache from a list of existing files. Replaces any previously-set knownFiles map (e.g. from NewScanCache).

func (*ScanCache) LockBook added in v0.0.11 

func (c *ScanCache) LockBook(bookID int) func()

LockBook acquires a lock for the given book ID to prevent concurrent relationship updates. Multiple files belonging to the same book may be processed in parallel, so we need to serialize their relationship updates (authors, series, genres, tags) to avoid race conditions. Returns an unlock function that must be called when done.

func (*ScanCache) LockBookPath added in v0.0.11 

func (c *ScanCache) LockBookPath(path string, libraryID int) func()

LockBookPath acquires a lock for the given book path to prevent concurrent book creation. Multiple files in the same directory would have the same book path, so we need to serialize their book creation to avoid unique constraint violations. Returns an unlock function that must be called when done.

func (*ScanCache) MovedBookIDs added in v0.0.29 

func (c *ScanCache) MovedBookIDs() map[int]struct{}

MovedBookIDs returns the set of book IDs whose files were matched as moves. Returns nil if reconciliation did not run or found no matches.

func (*ScanCache) PersonCount added in v0.0.11 

func (c *ScanCache) PersonCount() int

PersonCount returns the number of unique persons in the cache.

func (*ScanCache) PublisherCount added in v0.0.11 

func (c *ScanCache) PublisherCount() int

PublisherCount returns the number of unique publishers in the cache.

func (*ScanCache) SeriesCount added in v0.0.11 

func (c *ScanCache) SeriesCount() int

SeriesCount returns the number of unique series in the cache.

func (*ScanCache) SetAliasLister added in v0.0.43 

func (c *ScanCache) SetAliasLister(lister AliasLister)

SetAliasLister configures alias lookup so that cache misses pre-populate entries for the canonical name and every alias of the resolved resource.

func (*ScanCache) SetLibraryRootPaths added in v0.0.29 

func (c *ScanCache) SetLibraryRootPaths(paths []string)

SetLibraryRootPaths caches the library's root paths so downstream helpers (currently syncBookFilepathAfterMove) can check "is this a library root?" without a per-call DB lookup.

func (*ScanCache) SetMovedBookIDs added in v0.0.29 

func (c *ScanCache) SetMovedBookIDs(ids map[int]struct{})

SetMovedBookIDs stores the set of book IDs whose files were matched by the move reconciliation phase. The scan loop reads this after processing to ensure organize_file_structure runs on the moved books.

func (*ScanCache) SetMovedOrphanIDs added in v0.0.29 

func (c *ScanCache) SetMovedOrphanIDs(ids map[int]struct{})

SetMovedOrphanIDs stores the set of file IDs identified as moved orphans. Must be called before the parallel worker pool starts (not thread-safe for writes).

func (*ScanCache) TagCount added in v0.0.11 

func (c *ScanCache) TagCount() int

TagCount returns the number of unique tags in the cache.

type ScanOptions 

type ScanOptions struct {
	// Entry points (mutually exclusive - exactly one must be set)
	FilePath string // Batch scan: discover/create by path
	FileID   int    // Single file resync: file already in DB
	BookID   int    // Book resync: scan all files in book

	// Context (required for FilePath mode)
	LibraryID int

	// Behavior
	ForceRefresh  bool // Bypass priority checks, overwrite all metadata
	SkipPlugins   bool // Skip enricher plugins, use only file-embedded metadata
	Reset         bool // Wipe all metadata before scanning (reset to file-only state)
	BookResetDone bool // Book-level wipe already done by scanBook (skip in scanFileByID)

	// Logging (optional, for batch scan job context)
	JobLog *joblogs.JobLogger
}

ScanOptions configures a scan operation.

Entry points are mutually exclusive - exactly one of FilePath, FileID, or BookID must be set:

  • FilePath: Batch scan mode - discover or create file/book records by path. Requires LibraryID to be set.
  • FileID: Single file resync - file already exists in DB. If the file no longer exists on disk, it will be deleted from the database.
  • BookID: Book resync - scan all files belonging to the book. If the book has no files, it will be deleted.

type ScanResult 

type ScanResult struct {
	// For single file scans
	File        *models.File // The scanned/updated file (nil if deleted)
	Book        *models.Book // The parent book (nil if deleted)
	FileCreated bool         // True if file was newly created (FilePath mode only)
	FileDeleted bool         // True if file was deleted (no longer on disk)
	BookDeleted bool         // True if book was also deleted (was last file)

	// For book scans (multiple files)
	Files []*ScanResult // Results for each file in the book (BookID mode only)
}

ScanResult contains the results of a scan operation.

For single file scans (FilePath or FileID mode), the File and Book fields contain the scanned/updated records. FileCreated indicates whether a new file record was created (only possible in FilePath mode). FileDeleted and BookDeleted indicate whether records were removed because the file no longer exists on disk.

For book scans (BookID mode), the Files slice contains the results for each individual file in the book. The top-level Book field contains the updated book record (unless BookDeleted is true).

type SeriesFinder added in v0.0.11 

type SeriesFinder interface {
	FindOrCreateSeries(ctx context.Context, name string, libraryID int, nameSource string) (*models.Series, error)
}

SeriesFinder is an interface for finding or creating series.

type TagFinder added in v0.0.11 

type TagFinder interface {
	FindOrCreateTag(ctx context.Context, name string, libraryID int) (*models.Tag, error)
}

TagFinder is an interface for finding or creating tags.

type Worker 

type Worker struct {
	// contains filtered or unexported fields
}

func New 

func New(cfg *config.Config, db *bun.DB, pm *plugins.Manager, broker *events.Broker, dlCache *downloadcache.Cache) *Worker

func (*Worker) ProcessBulkDownloadJob added in v0.0.21 

func (w *Worker) ProcessBulkDownloadJob(ctx context.Context, job *models.Job, jobLog *joblogs.JobLogger) error

ProcessBulkDownloadJob generates metadata-injected files and creates a zip archive.

func (*Worker) ProcessHashGenerationJob added in v0.0.29 

func (w *Worker) ProcessHashGenerationJob(ctx context.Context, job *models.Job, jobLog *joblogs.JobLogger) error

ProcessHashGenerationJob computes and stores sha256 fingerprints for all files in a library that don't yet have one. Per-file errors are logged and skipped so a single unreadable file cannot fail the whole job.

The job loops until ListFilesMissingAlgorithm returns only IDs we've already attempted. Files added to the library while the job is running (either by a monitor batch in parallel, or because a spurious rescan invalidated an existing fingerprint) get picked up in a subsequent pass instead of being orphaned by job dedup in EnsureHashGenerationJob.

Error handling is split into "permanent" vs "transient": a file that's missing on disk, unreadable, or gone from the DB is marked as attempted and never retried within this job run. DB errors on Insert/Retrieve are treated as transient — the ID is left unmarked so the next pass tries again. maxPasses still bounds total retries.

func (*Worker) ProcessRecomputeReviewJob added in v0.0.39 

func (w *Worker) ProcessRecomputeReviewJob(ctx context.Context, job *models.Job, _ *joblogs.JobLogger) error

func (*Worker) ProcessScanJob 

func (w *Worker) ProcessScanJob(ctx context.Context, job *models.Job, jobLog *joblogs.JobLogger) error

func (*Worker) RefreshMonitorWatches added in v0.0.20 

func (w *Worker) RefreshMonitorWatches()

RefreshMonitorWatches signals the filesystem monitor to reload library paths. Safe to call even if the monitor is disabled (no-op).

func (*Worker) Scan 

func (w *Worker) Scan(ctx context.Context, opts books.ScanOptions) (*books.ScanResult, error)

Scan implements the books.Scanner interface. It converts books.ScanOptions to worker.ScanOptions, calls the internal scanInternal method, and converts the result back to books.ScanResult.

func (*Worker) Shutdown 

func (w *Worker) Shutdown()

func (*Worker) Start 

func (w *Worker) Start()

func (*Worker) UpdateBookRelationships added in v0.0.11 

func (w *Worker) UpdateBookRelationships(ctx context.Context, bookID int, updates RelationshipUpdates) error

UpdateBookRelationships updates all book relationships using the books service methods. It first deletes existing relationships (if flagged) then bulk inserts new ones. This method should be called with appropriate locking when used in parallel processing.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.