scraper

package
v2.15.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 1, 2026 License: GPL-3.0 Imports: 3 Imported by: 0

Documentation

Overview

Package scraper defines the metadata scraper types and generic run loop.

Concrete scrapers (gamelist.xml, ScreenScraper, TheGamesDB, etc.) call RunScraper from their [platforms.Scraper].Scrape callback, passing their record-specific load/match/map functions directly.

The sentinel tag pattern (scraper.<id>:scraped on the Media record) ensures that a crashed mid-write run is safely retried on the next invocation.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func RunScraper

func RunScraper(fn func(chan<- ScrapeUpdate) error) <-chan ScrapeUpdate

RunScraper creates a ScrapeUpdate channel, calls fn (which must start its own goroutine and close the channel when done), and returns the read end. If fn returns an error synchronously, a terminal FatalErr update is emitted and the channel is closed.

func RunTagInfo added in v2.14.1

func RunTagInfo(scraperID, runID string) database.TagInfo

RunTagInfo returns the per-run marker written to media rows completed during a persisted scraper operation.

func SentinelTagInfo

func SentinelTagInfo(scraperID string) database.TagInfo

SentinelTagInfo returns the sentinel TagInfo for the given scraper ID. Scrape callbacks write this after a successful record write to mark it done.

Types

type MapResult

type MapResult struct {
	MediaTags  []database.TagInfo
	TitleTags  []database.TagInfo
	TitleProps []database.MediaProperty
	MediaProps []database.MediaProperty
}

MapResult holds the tag and property writes produced by a MapToDB call.

type MatchResult

type MatchResult struct {
	MediaDBID      int64
	MediaTitleDBID int64
}

MatchResult is the output of a successful Match call. Both IDs must be positive database IDs; invalid IDs are treated as an implementation error and skipped by the generic run loop before any writes are attempted.

type ScrapeOptions

type ScrapeOptions struct {
	// Pauser pauses scrape work while another foreground activity needs the system.
	Pauser *syncutil.Pauser

	// RunID identifies a persisted scraper operation across restarts.
	RunID string

	// Systems limits scraping to these system IDs. Nil or empty means all systems.
	Systems []string

	// Force re-processes records that already have a sentinel tag.
	Force bool
}

ScrapeOptions configures a scrape run.

type ScrapeSystem

type ScrapeSystem struct {
	ID       string
	ROMPaths []string
	DBID     int64
}

ScrapeSystem carries the DB identity and filesystem paths needed by the scrape loop and concrete scraper implementations.

type ScrapeUpdate

type ScrapeUpdate struct {
	Err         error
	FatalErr    error
	SystemID    string
	Processed   int
	Total       int
	Matched     int
	Skipped     int
	TotalSteps  int
	CurrentStep int
	Done        bool
}

ScrapeUpdate is one progress event emitted on the channel returned by Scrape.

Directories

Path Synopsis
Package gamelistxml implements a scraper that reads EmulationStation gamelist.xml files to enrich the Zaparoo MediaDB with developer, publisher, genre, rating, year, artwork paths, and descriptions.
Package gamelistxml implements a scraper that reads EmulationStation gamelist.xml files to enrich the Zaparoo MediaDB with developer, publisher, genre, rating, year, artwork paths, and descriptions.
Package localmedia imports EmulationStation-style media folder artwork.
Package localmedia imports EmulationStation-style media folder artwork.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL