scraper

package
v2.14.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 8, 2026 License: GPL-3.0 Imports: 3 Imported by: 0

Documentation

Overview

Package scraper defines the metadata scraper types and generic run loop.

Concrete scrapers (gamelist.xml, ScreenScraper, TheGamesDB, etc.) call RunScraper from their [platforms.Scraper].Scrape callback, passing their record-specific load/match/map functions directly.

The sentinel tag pattern (scraper.<id>:scraped on the Media record) ensures that a crashed mid-write run is safely retried on the next invocation.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func RunScraper

func RunScraper(fn func(chan<- ScrapeUpdate) error) <-chan ScrapeUpdate

RunScraper creates a ScrapeUpdate channel, calls fn (which must start its own goroutine and close the channel when done), and returns the read end. If fn returns an error synchronously, a terminal FatalErr update is emitted and the channel is closed.

func SentinelTagInfo

func SentinelTagInfo(scraperID string) database.TagInfo

SentinelTagInfo returns the sentinel TagInfo for the given scraper ID. Scrape callbacks write this after a successful record write to mark it done.

Types

type MapResult

type MapResult struct {
	MediaTags  []database.TagInfo
	TitleTags  []database.TagInfo
	TitleProps []database.MediaProperty
	MediaProps []database.MediaProperty
}

MapResult holds the tag and property writes produced by a MapToDB call.

type MatchResult

type MatchResult struct {
	MediaDBID      int64
	MediaTitleDBID int64
}

MatchResult is the output of a successful Match call. Both IDs must be positive database IDs; invalid IDs are treated as an implementation error and skipped by the generic run loop before any writes are attempted.

type ScrapeOptions

type ScrapeOptions struct {
	// Pauser pauses scrape work while another foreground activity needs the system.
	Pauser *syncutil.Pauser

	// Systems limits scraping to these system IDs. Nil or empty means all systems.
	Systems []string

	// Force re-processes records that already have a sentinel tag.
	Force bool
}

ScrapeOptions configures a scrape run.

type ScrapeSystem

type ScrapeSystem struct {
	ID       string
	ROMPaths []string
	DBID     int64
}

ScrapeSystem carries the DB identity and filesystem paths needed by the scrape loop and concrete scraper implementations.

type ScrapeUpdate

type ScrapeUpdate struct {
	Err         error
	FatalErr    error
	SystemID    string
	Processed   int
	Total       int
	Matched     int
	Skipped     int
	TotalSteps  int
	CurrentStep int
	Done        bool
}

ScrapeUpdate is one progress event emitted on the channel returned by Scrape.

Directories

Path Synopsis
Package gamelistxml implements a scraper that reads EmulationStation gamelist.xml files to enrich the Zaparoo MediaDB with developer, publisher, genre, rating, year, artwork paths, and descriptions.
Package gamelistxml implements a scraper that reads EmulationStation gamelist.xml files to enrich the Zaparoo MediaDB with developer, publisher, genre, rating, year, artwork paths, and descriptions.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL