scan

package
v0.10.0-dev.74 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 11, 2026 License: AGPL-3.0 Imports: 14 Imported by: 0

Documentation

Overview

Package scan runs one loop per configured source: the whole passes and change batches that bring a source's library into statedb through its pass protocol (design 5). It is the one owner of the source:<id> and source_overlap:<id> alerts, so no other reader can disagree about a source's health.

Index

Constants

This section is empty.

Variables

View Source
var Codes = slices.Concat([]struct {
	Err  error
	Code string
}{
	{ErrScanCapacity, "scan_capacity"},
}, status.StartCodes)

Codes is every sentinel Start returns, status's passed on, with the code api answers for it. Start's *mediasource.SourceError is typed, not a sentinel, so api lists its codes through Owner.Typed. The element type is unnamed, so scan never imports api's wirespec.

View Source
var ErrScanCapacity = errors.New("scan: too many queued requests for this source or owner")

ErrScanCapacity answers a request needing a new queue slot while its source holds 16 or its owner 4 there.

Functions

This section is empty.

Types

type Deps

type Deps struct {
	Store    passes
	Status   jobs
	Logger   *slog.Logger
	Schedule mediasource.Schedule
}

Deps is what a Scanner runs over. Logger nil is slog.Default().

type Request

type Request struct {
	Root   *mediasource.NativeRef
	Key    *status.JobKey
	Source media.SourceID
	Owner  int64
}

Request is a manual scan: Root nil asks a whole pass of Source, else a refetch of that one root. A refetch whose read fails while the source answers its probe ends its job Failed with the *SourceError; the root's retry continues with no job attached.

type Scanner

type Scanner struct {
	// contains filtered or unexported fields
}

Scanner owns one loop per source of its Set.

func New

func New(d Deps, set sources) *Scanner

New builds a Scanner over set; Run starts its loops. A nil Store, Status or set, or a schedule interval that is not positive, is a wiring error and panics.

func (*Scanner) Collectors

func (s *Scanner) Collectors() []measure.Metric

Collectors declares scan_last_complete_timestamp_seconds{source}, the end of each source's latest complete whole pass, seeded with its loop's start and deleted when Reload removes the source, and scan_whole_interval_seconds; app registers them through obs.

func (*Scanner) PassNow

func (s *Scanner) PassNow()

PassNow makes every loop's scheduled whole pass due at once, with its SystemOwner Scan job; a loop already walking runs one more whole pass after the current one, and calls while a pass is pending coalesce.

func (*Scanner) Reload

func (s *Scanner) Reload(set sources, changed, rekeyed []media.SourceID)

Reload hands Run a rebuilt Set and returns once it is applied; the latest call wins. changed names every added source and every source whose entry differs outside its kind's Secret fields; rekeyed every source whose entry differs only in them. Removed and changed sources' queued requests end Interrupted; apply owns what each loop does.

func (*Scanner) Run

func (s *Scanner) Run(ctx context.Context) error

Run owns one loop per source until ctx ends, then waits for each loop's pass to finish and returns nil. A second call is an error.

func (*Scanner) SetSchedule

func (s *Scanner) SetSchedule(sched mediasource.Schedule)

SetSchedule hands Run a new scan section; the latest call wins. Every loop keeps its cursor, queue, retry set and backoff, and re-arms its next change check and its scheduled whole pass from the new intervals at once. A schedule whose interval is not positive is refused with ERROR, the old one kept.

func (*Scanner) Start

func (s *Scanner) Start(ctx context.Context, r Request) (status.Job, error)

Start records a Queued job and hands it to the source's loop, which runs it with the next pass of its scope. In order: a keyed request already recorded returns its job, whatever has happened to its source since; an unknown source is a *SourceError with UnknownSource, a root of a kind the source does not serve Unsupported; then, under the source's admission lock: ErrScanCapacity, writing nothing; Begin, whose replay of a racing duplicate returns that job unqueued and whose error queues nothing; and the insertion. status's errors pass through.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL