Documentation
¶
Overview ¶
Package scan runs one loop per configured source: the whole passes and change batches that bring a source's library into statedb through its pass protocol (design 5). It is the one owner of the source:<id> and source_overlap:<id> alerts, so no other reader can disagree about a source's health.
Index ¶
- Variables
- type Deps
- type Request
- type Scanner
- func (s *Scanner) Collectors() []measure.Metric
- func (s *Scanner) PassNow()
- func (s *Scanner) Reload(set sources, changed, rekeyed []media.SourceID)
- func (s *Scanner) Run(ctx context.Context) error
- func (s *Scanner) SetSchedule(sched mediasource.Schedule)
- func (s *Scanner) Start(ctx context.Context, r Request) (status.Job, error)
Constants ¶
This section is empty.
Variables ¶
var Codes = slices.Concat([]struct { Err error Code string }{ {ErrScanCapacity, "scan_capacity"}, }, status.StartCodes)
Codes is every sentinel Start returns, status's passed on, with the code api answers for it. Start's *mediasource.SourceError is typed, not a sentinel, so api lists its codes through Owner.Typed. The element type is unnamed, so scan never imports api's wirespec.
var ErrScanCapacity = errors.New("scan: too many queued requests for this source or owner")
ErrScanCapacity answers a request needing a new queue slot while its source holds 16 or its owner 4 there.
Functions ¶
This section is empty.
Types ¶
type Deps ¶
type Deps struct {
Store passes
Status jobs
Logger *slog.Logger
Schedule mediasource.Schedule
}
Deps is what a Scanner runs over. Logger nil is slog.Default().
type Request ¶
type Request struct {
Root *mediasource.NativeRef
Key *status.JobKey
Source media.SourceID
Owner int64
}
Request is a manual scan: Root nil asks a whole pass of Source, else a refetch of that one root. A refetch whose read fails while the source answers its probe ends its job Failed with the *SourceError; the root's retry continues with no job attached.
type Scanner ¶
type Scanner struct {
// contains filtered or unexported fields
}
Scanner owns one loop per source of its Set.
func New ¶
New builds a Scanner over set; Run starts its loops. A nil Store, Status or set, or a schedule interval that is not positive, is a wiring error and panics.
func (*Scanner) Collectors ¶
Collectors declares scan_last_complete_timestamp_seconds{source}, the end of each source's latest complete whole pass, seeded with its loop's start and deleted when Reload removes the source, and scan_whole_interval_seconds; app registers them through obs.
func (*Scanner) PassNow ¶
func (s *Scanner) PassNow()
PassNow makes every loop's scheduled whole pass due at once, with its SystemOwner Scan job; a loop already walking runs one more whole pass after the current one, and calls while a pass is pending coalesce.
func (*Scanner) Reload ¶
Reload hands Run a rebuilt Set and returns once it is applied; the latest call wins. changed names every added source and every source whose entry differs outside its kind's Secret fields; rekeyed every source whose entry differs only in them. Removed and changed sources' queued requests end Interrupted; apply owns what each loop does.
func (*Scanner) Run ¶
Run owns one loop per source until ctx ends, then waits for each loop's pass to finish and returns nil. A second call is an error.
func (*Scanner) SetSchedule ¶
func (s *Scanner) SetSchedule(sched mediasource.Schedule)
SetSchedule hands Run a new scan section; the latest call wins. Every loop keeps its cursor, queue, retry set and backoff, and re-arms its next change check and its scheduled whole pass from the new intervals at once. A schedule whose interval is not positive is refused with ERROR, the old one kept.
func (*Scanner) Start ¶
Start records a Queued job and hands it to the source's loop, which runs it with the next pass of its scope. In order: a keyed request already recorded returns its job, whatever has happened to its source since; an unknown source is a *SourceError with UnknownSource, a root of a kind the source does not serve Unsupported; then, under the source's admission lock: ErrScanCapacity, writing nothing; Begin, whose replay of a racing duplicate returns that job unqueued and whose error queues nothing; and the insertion. status's errors pass through.