mediadb

package
v2.7.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 8, 2025 License: GPL-3.0 Imports: 26 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	IndexingStatusRunning   = "running"
	IndexingStatusPending   = "pending"
	IndexingStatusCompleted = "completed"
	IndexingStatusFailed    = "failed"
	IndexingStatusCancelled = "cancelled"
)

Indexing status constants

View Source 
const (
	DBConfigLastGeneratedAt    = "LastGeneratedAt"
	DBConfigOptimizationStatus = "OptimizationStatus"
	DBConfigOptimizationStep   = "OptimizationStep"
	DBConfigIndexingStatus     = "IndexingStatus"
	DBConfigLastIndexedSystem  = "LastIndexedSystem"
	DBConfigIndexingSystems    = "IndexingSystems"
)

Variables

View Source 
var ErrNullSQL = errors.New("MediaDB is not connected")

Functions

func BuildTagFilterSQL added in v2.7.0 

func BuildTagFilterSQL(filters []database.TagFilter) (clauses []string, args []any)

BuildTagFilterSQL constructs SQL WHERE clauses and arguments for tag filtering using a hybrid strategy optimized for SQLite performance:

  • AND filters: INTERSECT pattern
  • NOT filters: NOT EXISTS pattern
  • OR filters: EXISTS with OR conditions

Returns a slice of WHERE clause strings and corresponding arguments. Clauses should be joined with " AND " and appended to the main query's WHERE conditions.

Types

type BatchInserter added in v2.7.0 

type BatchInserter struct {
	// contains filtered or unexported fields
}

BatchInserter manages batched multi-row inserts for a specific table

func NewBatchInserter added in v2.7.0 

func NewBatchInserter(
	ctx context.Context,
	tx *sql.Tx,
	tableName string,
	columns []string,
	batchSize int,
) (*BatchInserter, error)

NewBatchInserter creates a batch inserter for the given table

func NewBatchInserterWithOptions added in v2.7.0 

func NewBatchInserterWithOptions(
	ctx context.Context,
	tx *sql.Tx,
	tableName string,
	columns []string,
	batchSize int,
	orIgnore bool,
) (*BatchInserter, error)

NewBatchInserterWithOptions creates a batch inserter with OR IGNORE option

func (*BatchInserter) Add added in v2.7.0 

func (b *BatchInserter) Add(values ...any) error

Add appends a row to the current batch and auto-flushes when the batch size is reached. Dependency ordering ensures parent tables are flushed before children, preventing FK violations.

func (*BatchInserter) Close added in v2.7.0 

func (b *BatchInserter) Close() error

Close flushes remaining items and closes the statement

func (*BatchInserter) Flush added in v2.7.0 

func (b *BatchInserter) Flush() error

Flush executes the current batch and resets the buffer

func (*BatchInserter) SetDependencies added in v2.7.0 

func (b *BatchInserter) SetDependencies(deps ...*BatchInserter)

SetDependencies sets the parent batch inserters that must flush before this one

type MediaDB 

type MediaDB struct {
	// contains filtered or unexported fields
}

func OpenMediaDB 

func OpenMediaDB(ctx context.Context, pl platforms.Platform) (*MediaDB, error)

func (*MediaDB) Allocate 

func (db *MediaDB) Allocate() error

func (*MediaDB) Analyze added in v2.7.0 

func (db *MediaDB) Analyze() error

func (*MediaDB) BackgroundOperationDone added in v2.7.1 

func (db *MediaDB) BackgroundOperationDone()

BackgroundOperationDone decrements the background operations counter. This should be called when an operation started with TrackBackgroundOperation completes.

func (*MediaDB) BeginTransaction 

func (db *MediaDB) BeginTransaction(batchEnabled bool) error

func (*MediaDB) CheckForDuplicateMediaTitles added in v2.7.0 

func (db *MediaDB) CheckForDuplicateMediaTitles() ([]string, error)

CheckForDuplicateMediaTitles returns any MediaTitle records that have duplicate (SystemDBID, Slug) combinations. Used in tests to validate data integrity after selective updates.

func (*MediaDB) Close 

func (db *MediaDB) Close() error

func (*MediaDB) CommitTransaction 

func (db *MediaDB) CommitTransaction() error

func (*MediaDB) CreateIndexes added in v2.7.0 

func (*MediaDB) CreateIndexes() error

func (*MediaDB) Exists 

func (db *MediaDB) Exists() bool

func (*MediaDB) FindMedia 

func (db *MediaDB) FindMedia(row database.Media) (database.Media, error)

func (*MediaDB) FindMediaTag 

func (db *MediaDB) FindMediaTag(row database.MediaTag) (database.MediaTag, error)

func (*MediaDB) FindMediaTitle 

func (db *MediaDB) FindMediaTitle(row *database.MediaTitle) (database.MediaTitle, error)

func (*MediaDB) FindOrInsertMedia 

func (db *MediaDB) FindOrInsertMedia(row database.Media) (database.Media, error)

func (*MediaDB) FindOrInsertMediaTag 

func (db *MediaDB) FindOrInsertMediaTag(row database.MediaTag) (database.MediaTag, error)

func (*MediaDB) FindOrInsertMediaTitle 

func (db *MediaDB) FindOrInsertMediaTitle(row *database.MediaTitle) (database.MediaTitle, error)

func (*MediaDB) FindOrInsertSystem 

func (db *MediaDB) FindOrInsertSystem(row database.System) (database.System, error)

func (*MediaDB) FindOrInsertTag 

func (db *MediaDB) FindOrInsertTag(row database.Tag) (database.Tag, error)

func (*MediaDB) FindOrInsertTagType 

func (db *MediaDB) FindOrInsertTagType(row database.TagType) (database.TagType, error)

func (*MediaDB) FindSystem 

func (db *MediaDB) FindSystem(row database.System) (database.System, error)

func (*MediaDB) FindSystemBySystemID added in v2.7.0 

func (db *MediaDB) FindSystemBySystemID(systemID string) (database.System, error)

func (*MediaDB) FindTag 

func (db *MediaDB) FindTag(row database.Tag) (database.Tag, error)

func (*MediaDB) FindTagType 

func (db *MediaDB) FindTagType(row database.TagType) (database.TagType, error)

func (*MediaDB) GetAllMedia added in v2.7.0 

func (db *MediaDB) GetAllMedia() ([]database.Media, error)

func (*MediaDB) GetAllMediaTitles added in v2.7.0 

func (db *MediaDB) GetAllMediaTitles() ([]database.MediaTitle, error)

func (*MediaDB) GetAllSystems added in v2.7.0 

func (db *MediaDB) GetAllSystems() ([]database.System, error)

func (*MediaDB) GetAllTagTypes added in v2.7.0 

func (db *MediaDB) GetAllTagTypes() ([]database.TagType, error)

func (*MediaDB) GetAllTags added in v2.7.0 

func (db *MediaDB) GetAllTags() ([]database.Tag, error)

func (*MediaDB) GetAllUsedTags added in v2.7.0 

func (db *MediaDB) GetAllUsedTags(ctx context.Context) ([]database.TagInfo, error)

GetAllUsedTags returns all tags that are actually in use (have media associated) This is optimized for the "all systems" case and avoids expensive system filtering

func (*MediaDB) GetCachedSlugResolution added in v2.7.0 

func (db *MediaDB) GetCachedSlugResolution(
	ctx context.Context, systemID, slug string, tagFilters []database.TagFilter,
) (mediaDBID int64, strategy string, found bool)

GetCachedSlugResolution retrieves a cached slug resolution result. Returns the MediaDBID, strategy name, and true if found; otherwise returns 0, "", false.

func (*MediaDB) GetCachedStats added in v2.7.0 

func (db *MediaDB) GetCachedStats(ctx context.Context, query *database.MediaQuery) (MediaStats, bool)

GetCachedStats returns cached statistics for the given media query, if available. Returns the stats and true if found, or empty stats and false if not cached.

func (*MediaDB) GetDBPath 

func (db *MediaDB) GetDBPath() string

func (*MediaDB) GetIndexingStatus added in v2.7.0 

func (db *MediaDB) GetIndexingStatus() (string, error)

func (*MediaDB) GetIndexingSystems added in v2.7.0 

func (db *MediaDB) GetIndexingSystems() ([]string, error)

func (*MediaDB) GetLastGenerated 

func (db *MediaDB) GetLastGenerated() (time.Time, error)

func (*MediaDB) GetLastIndexedSystem added in v2.7.0 

func (db *MediaDB) GetLastIndexedSystem() (string, error)

func (*MediaDB) GetLaunchCommandForMedia added in v2.7.0 

func (db *MediaDB) GetLaunchCommandForMedia(ctx context.Context, systemID, path string) (string, error)

GetLaunchCommandForMedia generates a title-based launch command for the given media.

func (*MediaDB) GetMaxMediaID added in v2.7.0 

func (db *MediaDB) GetMaxMediaID() (int64, error)

func (*MediaDB) GetMaxMediaTagID added in v2.7.0 

func (db *MediaDB) GetMaxMediaTagID() (int64, error)

func (*MediaDB) GetMaxSystemID added in v2.7.0 

func (db *MediaDB) GetMaxSystemID() (int64, error)

GetMax*ID methods for resume functionality

func (*MediaDB) GetMaxTagID added in v2.7.0 

func (db *MediaDB) GetMaxTagID() (int64, error)

func (*MediaDB) GetMaxTagTypeID added in v2.7.0 

func (db *MediaDB) GetMaxTagTypeID() (int64, error)

func (*MediaDB) GetMaxTitleID added in v2.7.0 

func (db *MediaDB) GetMaxTitleID() (int64, error)

func (*MediaDB) GetMediaByDBID added in v2.7.0 

func (db *MediaDB) GetMediaByDBID(ctx context.Context, mediaDBID int64) (database.SearchResultWithCursor, error)

GetMediaByDBID retrieves a single SearchResultWithCursor by Media DBID. This is used to reconstruct the full result from a cached Media DBID.

func (*MediaDB) GetMediaWithFullPath added in v2.7.0 

func (db *MediaDB) GetMediaWithFullPath() ([]database.MediaWithFullPath, error)

GetMediaWithFullPath retrieves all media with their associated title and system information using JOIN queries. This eliminates the need for nested loops to match media with titles and systems.

func (*MediaDB) GetMediaWithFullPathExcluding added in v2.7.0 

func (db *MediaDB) GetMediaWithFullPathExcluding(excludeSystemIDs []string) ([]database.MediaWithFullPath, error)

GetMediaWithFullPathExcluding retrieves all media with their associated title and system information, excluding those belonging to systems in the excludeSystemIDs list. This is optimized for selective indexing to avoid loading data for systems being reindexed.

func (*MediaDB) GetOptimizationStatus added in v2.7.0 

func (db *MediaDB) GetOptimizationStatus() (string, error)

func (*MediaDB) GetOptimizationStep added in v2.7.0 

func (db *MediaDB) GetOptimizationStep() (string, error)

func (*MediaDB) GetSystemTagsCached added in v2.7.0 

func (db *MediaDB) GetSystemTagsCached(ctx context.Context, systems []systemdefs.System) ([]database.TagInfo, error)

GetSystemTagsCached retrieves tags for specific systems using the cache table Falls back to the optimized subquery approach if cache is empty

func (*MediaDB) GetSystemsExcluding added in v2.7.0 

func (db *MediaDB) GetSystemsExcluding(excludeSystemIDs []string) ([]database.System, error)

GetSystemsExcluding retrieves all systems except those in the excludeSystemIDs list. This is optimized for selective indexing to avoid loading data for systems being reindexed.

func (*MediaDB) GetTags added in v2.7.0 

func (db *MediaDB) GetTags(ctx context.Context, systems []systemdefs.System) ([]database.TagInfo, error)

func (*MediaDB) GetTitlesWithPreFilter added in v2.7.0 

func (db *MediaDB) GetTitlesWithPreFilter(
	ctx context.Context, systemID string, minLength, maxLength, minWordCount, maxWordCount int,
) ([]database.MediaTitle, error)

GetTitlesWithPreFilter retrieves media titles filtered by slug length and word count ranges. This dramatically reduces the candidate set for fuzzy matching by using indexed pre-filter columns. Uses the composite index idx_media_prefilter (SlugLength, SlugWordCount) for efficient range queries.

func (*MediaDB) GetTitlesWithSystems added in v2.7.0 

func (db *MediaDB) GetTitlesWithSystems() ([]database.TitleWithSystem, error)

GetTitlesWithSystems retrieves all media titles with their associated system IDs using a JOIN query. This is more efficient than fetching titles and systems separately and matching them in application code.

func (*MediaDB) GetTitlesWithSystemsExcluding added in v2.7.0 

func (db *MediaDB) GetTitlesWithSystemsExcluding(excludeSystemIDs []string) ([]database.TitleWithSystem, error)

GetTitlesWithSystemsExcluding retrieves all media titles with their associated system IDs, excluding those belonging to systems in the excludeSystemIDs list. This is optimized for selective indexing to avoid loading data for systems being reindexed.

func (*MediaDB) GetTotalMediaCount added in v2.7.0 

func (db *MediaDB) GetTotalMediaCount() (int, error)

GetTotalMediaCount returns the total number of media entries in the database.

func (*MediaDB) IndexedSystems 

func (db *MediaDB) IndexedSystems() ([]string, error)

IndexedSystems returns all systems indexed in the media database.

func (*MediaDB) InsertMedia 

func (db *MediaDB) InsertMedia(row database.Media) (database.Media, error)

func (*MediaDB) InsertMediaTag 

func (db *MediaDB) InsertMediaTag(row database.MediaTag) (database.MediaTag, error)

func (*MediaDB) InsertMediaTitle 

func (db *MediaDB) InsertMediaTitle(row *database.MediaTitle) (database.MediaTitle, error)

func (*MediaDB) InsertSystem 

func (db *MediaDB) InsertSystem(row database.System) (database.System, error)

func (*MediaDB) InsertTag 

func (db *MediaDB) InsertTag(row database.Tag) (database.Tag, error)

func (*MediaDB) InsertTagType 

func (db *MediaDB) InsertTagType(row database.TagType) (database.TagType, error)

InsertTagType inserts a new TagType into the database.

func (*MediaDB) InvalidateCountCache added in v2.7.0 

func (db *MediaDB) InvalidateCountCache() error

InvalidateCountCache clears all cached media counts. This should be called after any operation that changes the media database content.

func (*MediaDB) InvalidateSlugCache added in v2.7.0 

func (db *MediaDB) InvalidateSlugCache(ctx context.Context) error

InvalidateSlugCache clears all cached slug resolutions. This should be called after any operation that changes the media database content.

func (*MediaDB) InvalidateSlugCacheForSystems added in v2.7.0 

func (db *MediaDB) InvalidateSlugCacheForSystems(ctx context.Context, systemIDs []string) error

InvalidateSlugCacheForSystems clears cached slug resolutions for specific systems. This is used during selective system reindexing to avoid invalidating the entire cache.

func (*MediaDB) InvalidateSystemTagsCache added in v2.7.0 

func (db *MediaDB) InvalidateSystemTagsCache(ctx context.Context, systems []systemdefs.System) error

InvalidateSystemTagsCache removes cache entries for specific systems Useful for incremental cache updates when only certain systems change If no systems are provided, this is a no-op and returns success.

func (*MediaDB) MigrateUp 

func (db *MediaDB) MigrateUp() error

func (*MediaDB) Open 

func (db *MediaDB) Open() error

func (*MediaDB) PopulateSystemTagsCache added in v2.7.0 

func (db *MediaDB) PopulateSystemTagsCache(ctx context.Context) error

PopulateSystemTagsCache rebuilds the cache table for fast tag lookups by system This should be called after media indexing completes

func (*MediaDB) PopulateSystemTagsCacheForSystems added in v2.7.0 

func (db *MediaDB) PopulateSystemTagsCacheForSystems(ctx context.Context, systems []systemdefs.System) error

PopulateSystemTagsCacheForSystems rebuilds cache for specific systems only Used for incremental cache updates after individual system changes

func (*MediaDB) RandomGame 

func (db *MediaDB) RandomGame(systems []systemdefs.System) (database.SearchResult, error)

RandomGame returns a random game from specified systems.

func (*MediaDB) RandomGameWithQuery added in v2.7.0 

func (db *MediaDB) RandomGameWithQuery(query *database.MediaQuery) (database.SearchResult, error)

RandomGameWithQuery returns a random game matching the specified MediaQuery.

func (*MediaDB) RollbackTransaction 

func (db *MediaDB) RollbackTransaction() error

RollbackTransaction rolls back the current transaction and cleans up resources

func (*MediaDB) RunBackgroundOptimization added in v2.7.0 

func (db *MediaDB) RunBackgroundOptimization(statusCallback func(optimizing bool))

RunBackgroundOptimization performs database optimization operations in the background. This includes creating indexes, running ANALYZE, and vacuuming the database. It can be safely interrupted and resumed later.

func (*MediaDB) SearchMediaBySecondarySlug added in v2.7.0 

func (db *MediaDB) SearchMediaBySecondarySlug(
	ctx context.Context, systemID string, secondarySlug string, tags []database.TagFilter,
) ([]database.SearchResultWithCursor, error)

func (*MediaDB) SearchMediaBySlug added in v2.7.0 

func (db *MediaDB) SearchMediaBySlug(
	ctx context.Context, systemID string, slug string, tags []database.TagFilter,
) ([]database.SearchResultWithCursor, error)

func (*MediaDB) SearchMediaBySlugIn added in v2.7.0 

func (db *MediaDB) SearchMediaBySlugIn(
	ctx context.Context, systemID string, slugList []string, tags []database.TagFilter,
) ([]database.SearchResultWithCursor, error)

SearchMediaBySlugIn searches for media items matching any of the provided slugs using an IN clause. This is optimized for searching multiple slug candidates in a single query.

func (*MediaDB) SearchMediaBySlugPrefix added in v2.7.0 

func (db *MediaDB) SearchMediaBySlugPrefix(
	ctx context.Context, systemID string, slugPrefix string, tags []database.TagFilter,
) ([]database.SearchResultWithCursor, error)

func (*MediaDB) SearchMediaPathExact 

func (db *MediaDB) SearchMediaPathExact(systems []systemdefs.System, query string) ([]database.SearchResult, error)

SearchMediaPathExact returns indexed names matching an exact query (case-insensitive).

func (*MediaDB) SearchMediaPathGlob 

func (db *MediaDB) SearchMediaPathGlob(systems []systemdefs.System, query string) ([]database.SearchResult, error)

func (*MediaDB) SearchMediaWithFilters added in v2.7.0 

func (db *MediaDB) SearchMediaWithFilters(
	ctx context.Context,
	filters *database.SearchFilters,
) ([]database.SearchResultWithCursor, error)

func (*MediaDB) SetCachedSlugResolution added in v2.7.0 

func (db *MediaDB) SetCachedSlugResolution(
	ctx context.Context, systemID, slug string, tagFilters []database.TagFilter, mediaDBID int64, strategy string,
) error

SetCachedSlugResolution stores a successful slug resolution in the cache.

func (*MediaDB) SetCachedStats added in v2.7.0 

func (db *MediaDB) SetCachedStats(ctx context.Context, query *database.MediaQuery, stats MediaStats) error

SetCachedStats stores statistics for the given media query in the cache.

func (*MediaDB) SetDBPathForTesting added in v2.7.0 

func (db *MediaDB) SetDBPathForTesting(dbPath string)

SetDBPathForTesting explicitly sets the DB path so test memory DBs can reload.

func (*MediaDB) SetIndexingStatus added in v2.7.0 

func (db *MediaDB) SetIndexingStatus(status string) error

func (*MediaDB) SetIndexingSystems added in v2.7.0 

func (db *MediaDB) SetIndexingSystems(systemIDs []string) error

func (*MediaDB) SetLastIndexedSystem added in v2.7.0 

func (db *MediaDB) SetLastIndexedSystem(systemID string) error

func (*MediaDB) SetOptimizationStatus added in v2.7.0 

func (db *MediaDB) SetOptimizationStatus(status string) error

func (*MediaDB) SetOptimizationStep added in v2.7.0 

func (db *MediaDB) SetOptimizationStep(step string) error

func (*MediaDB) SetSQLForTesting 

func (db *MediaDB) SetSQLForTesting(ctx context.Context, sqlDB *sql.DB, platform platforms.Platform) error

SetSQLForTesting allows injection of a sql.DB instance for testing purposes. This method should only be used in tests to set up in-memory databases.

func (*MediaDB) SystemIndexed 

func (db *MediaDB) SystemIndexed(system *systemdefs.System) bool

SystemIndexed returns true if a specific system is indexed in the media database.

func (*MediaDB) TrackBackgroundOperation added in v2.7.1 

func (db *MediaDB) TrackBackgroundOperation()

TrackBackgroundOperation increments the background operations counter. Call BackgroundOperationDone when the operation completes. This allows external code (like the indexing goroutine) to be tracked.

func (*MediaDB) Truncate 

func (db *MediaDB) Truncate() error

func (*MediaDB) TruncateSystems added in v2.7.0 

func (db *MediaDB) TruncateSystems(systemIDs []string) error

func (*MediaDB) UnsafeGetSQLDb 

func (db *MediaDB) UnsafeGetSQLDb() *sql.DB

func (*MediaDB) UpdateLastGenerated 

func (db *MediaDB) UpdateLastGenerated() error

func (*MediaDB) Vacuum 

func (db *MediaDB) Vacuum() error

func (*MediaDB) WaitForBackgroundOperations added in v2.7.0 

func (db *MediaDB) WaitForBackgroundOperations()

WaitForBackgroundOperations waits for all background operations to complete. This should be called before closing the database to ensure clean shutdown.

type MediaStats added in v2.7.0 

type MediaStats struct {
	Count   int
	MinDBID int64
	MaxDBID int64
}

MediaStats represents cached statistics for a media query

type PreFilterQuery added in v2.7.0 

type PreFilterQuery struct {
	MinLength    int
	MaxLength    int
	MinWordCount int
	MaxWordCount int
}

PreFilterQuery represents pre-filter parameters for efficient fuzzy matching candidate reduction.

type SlugMetadata added in v2.7.0 

type SlugMetadata struct {
	Slug          string
	SecondarySlug string
	SlugLength    int
	SlugWordCount int
}

SlugMetadata contains pre-filter columns computed from a slug. This metadata enables efficient fuzzy matching by reducing the candidate set before applying expensive string similarity algorithms.

func GenerateSlugWithMetadata added in v2.7.0 

func GenerateSlugWithMetadata(mediaType slugs.MediaType, input string) SlugMetadata

GenerateSlugWithMetadata normalizes input and computes all prefilter metadata. CRITICAL: Uses SlugifyWithTokens to ensure metadata matches the actual slug.

The metadata is computed from the EXACT tokens extracted during the 14-stage normalization pipeline, not from re-tokenization. This ensures consistency between the slug and its metadata.

The mediaType parameter should match the MediaType of the system being indexed, ensuring consistent slugification between indexing and resolution.

If the title contains a secondary title separator (: or -), the secondary slug is also extracted and stored separately to enable efficient secondary title matching.

Example: 

metadata := GenerateSlugWithMetadata(slugs.MediaTypeGame, "The Legend of Zelda: Ocarina of Time")
metadata.Slug         → "legendofzeldaocarinaoftime"
metadata.SlugLength   → 26 (character count)
metadata.SlugWordCount → 6 (token count: legend, of, zelda, ocarina, of, time)
metadata.SecondarySlug → "ocarinaoftime" (secondary title after colon)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.