availability

package
v0.3.8 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 11, 2026 License: GPL-3.0 Imports: 25 Imported by: 0

README

Availability monitoring

Ether observes recent releases and reuses successful resolver health checks. It never removes catalogue/search results because evidence is missing. Movie dates and episode air dates still use the catalogue's existing release eligibility; quality, CAM acceptance, identities, blocklists and episode targeting use the resolver's existing policies rather than a second rules engine.

Owner controls

Sources → Availability provides monitoring, selective validation, participating feeds, debrid participation, per-type release modes (MovieMode, ShowMode, falling back to Mode), preference for checked recommendations and optional newly available shelves. Monitoring and validation start disabled. Presentation preferences start enabled and can benefit from ordinary playback observations.

Default daily background allowances (UTC, shared across nodes):

  • Per indexer: 10% (Gentle 5%; Unlimited up to 100 − reservePercent) of its daily API limit: the owner's, else the one it reports (Newznab apilimits), else 1,000 (the strictest private tier), so 100 feed and search requests by default. The day's remaining head polls are reserved, so gap pages and title searches cannot starve the feeds; a feed whose allowance cannot fit 30-minute polls in half of it is polled less often. Requests to one indexer are at least Deps.Pace (3 s) apart.
  • In total, the sum of those (or the owner's custom total).
  • 100 validation API requests, 20 candidate checks and 20 descriptor grabs.
  • 512 MiB of conservative data reservations. NNTP BODY reservations use the provider's maximum article size, so actual transferred bytes can be much lower.

An allowance of zero disables that work. Foreground requests still have their normal account limits. Every feed page, provider API request and source read goes through admission before network work. Ordinary playback does not use the background allowance. Shared worker contexts retain the admission hook.

Sticky release gate and bounded retry schedule

Newznab/Torznab recent-release pages use an empty t=search request, following the incremental model documented by Radarr. Built-in sites list their latest page (a keywordless search, as Prowlarr does); Stremio addons and plugin sources are search-only. One page describes up to 100 releases. Head polls default to every 30 minutes. A sync reads from the head back to the previous head: one page usually, otherwise only the pages the gap needs (at most MaxPages per sync), resuming an unfinished gap every minute while its allowance has room. A first sync starts at the head; catch-up (optional) also walks recent history. Failed requests do not advance the cursor.

In-window titles without an open gate row are checked via an indexed due-retry table (availability_title_checks_due) anchored to the movie's digital/physical release date (falling back to theatrical) or the episode's air date on the bounded ladder [0, 30m, 1h, 90m, 2h, 150m, 3h, 4h, 8h] and then every 24h until the evidence window ends. At most probeBatch (10) due titles are searched per minute, ordered by priority (unchecked first-look, watchlisted, and post-date titles first), asking one source at a time and stopping at the first acceptable release. On an idle server with no due rows, the scheduler performs one indexed due-retry query returning zero rows and no catalogue scan or set rebuild.

Once a title's gate (availability_gate, keyed by (policy, rating_key)) opens under a policy fingerprint, it never closes: Invalidate retracts that specific release's evidence in availability_releases and console evidence so playback/search move on to alternative releases, and time passing never re-hides the title. Only a policy, profile, or source-generation change starts a fresh gate policy fingerprint.

Because opening is irreversible, opening is strict against pre-release fakes:

  • Verified evidence opens the gate immediately: a successful background health check that read real video (VerifyFeedCandidate), a definitive cached debrid copy (CacheCheck), or successful ordinary playback (Observe).
  • Unverified feed/search matches open the gate immediately only after the title's official date has passed (movie digital/physical release date, episode air date). Before that date, an unverified profile-accepted match leaves the gate shut and queues the candidate in availability_releases for background verification within the validation allowance.

Matching scans local title identities and resolves episodes locally; it does not call TMDB for every feed item. Conflicting identifiers and ambiguous title matches are rejected. A season pack only contributes known, already-aired episodes, and its health check is targeted to the chosen episode using the existing resolver.

Library browse, rows, facets, episode counts and search evaluate the sticky gate directly inside their existing SQL queries via an indexed EXISTS lookup on availability_gate(policy, rating_key) combined with request-clock window bounds — zero extra round trips, zero Go-side filtering of title lists, and zero in-memory hidden-set recomputation. Durable release verdicts live in availability_verdicts (pruned with feed items and capped at 60,000 rows) so policy changes reuse or re-evaluate retained observations cleanly.

Recommendation priority and newly available shelves still require fresh (6-hour) checked evidence (verifiedKeys), distinct from sticky browse visibility.

Operations

The existing cluster leader runs one scheduler; cursors, sticky gate rows, observations and atomic usage counters are durable in the shared database. Hot settings are respected by admission. Checks pause while local playback is active. Each cycle is bounded, cancellation propagates, transient outcomes back off, and descriptors expire after 30 days with a hard retention cap of 30,000 observations. Signed release locators are sealed using the config store; owner status never exposes them.

Documentation

Overview

Package availability discovers releases incrementally and spends a bounded background allowance validating useful candidates. Recent releases require sticky, profile-aware evidence for discovery; older catalogue entries need no background search.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func EvidenceWindow added in v0.2.9

func EvidenceWindow(t core.Title, now time.Time) bool

EvidenceWindow is a proof window, not a delay: an eligible release unlocks a title immediately. Older catalogue entries do not need a scan.

func WithDiscoveryMemo added in v0.3.0

func WithDiscoveryMemo(ctx context.Context) context.Context

WithDiscoveryMemo shares one availability snapshot for the rest of this request. Home authorizes the same titles many times, and each check was cloning config and feed settings.

Types

type Catalog

type Catalog interface {
	ReleaseMatches(context.Context, []core.Release) (map[string][]core.Title, error)
	CachedTitle(context.Context, core.RatingKey) (*core.Title, error)
	CachedTitles(context.Context, []core.RatingKey) ([]core.Title, error)
}

type Deps

type Deps struct {
	DB         *sql.DB
	Config     func() config.Config
	Feeds      Feeds
	Catalog    Catalog
	Validator  Validator
	Seal       func(string) (string, error)
	Open       func(string) (string, error)
	CacheCheck func(context.Context, []string) (map[string]bool, bool)
	// Search is a budgeted id search across indexers and configured addons.
	Search           func(context.Context, core.SearchRequest) ([]core.Release, error)
	Watchlist        func(context.Context) (map[core.RatingKey]bool, error)
	Active           func() bool
	Now              func() time.Time
	Logger           *slog.Logger
	ConfigGeneration func() uint64
	// Pace is the least time between two background requests to one
	// indexer (zero: no spacing), so feeds, gap pages and title searches
	// never burst at an indexer.
	Pace time.Duration
	// Async and Clustered are retained for Deps compatibility; availability_gate
	// is shared SQL state with no background hidden-set recomputation.
	Async     bool
	Clustered bool
}

type FeedStatus

type FeedStatus struct {
	core.ReleaseFeed
	Selected    bool    `json:"selected"`
	LastAttempt int64   `json:"lastAttempt"`
	LastSync    int64   `json:"lastSync"`
	Gap         bool    `json:"gap"`
	Message     string  `json:"message"`
	Matched     int     `json:"matched"`
	Queued      bool    `json:"queued"`
	Progress    float64 `json:"progress,omitempty"`
	ETASeconds  int64   `json:"etaSeconds,omitempty"`
}

type Feeds

type Feeds interface {
	FeedSources() []core.ReleaseFeed
	Recent(context.Context, string, int, int) ([]core.Release, error)
}

type HiddenPublish

type HiddenPublish struct {
	Added, Removed int
	Seq            int64
	Hold           time.Duration
}

HiddenPublish reports one publishHidden: the membership rows it wrote (Added inserts + Removed closes), the resulting seq, and how long its transaction held the writer.

type NewRelease added in v0.3.0

type NewRelease struct {
	Key        string `json:"key"`
	Title      string `json:"title"`
	Year       int    `json:"year,omitempty"`
	Poster     string `json:"poster,omitempty"`
	Theatrical int64  `json:"theatrical,omitempty"`
	Digital    int64  `json:"digital,omitempty"`
	Code       string `json:"code"`
	Source     string `json:"source,omitempty"`
	NextCheck  int64  `json:"nextCheck,omitempty"`
	Reason     string `json:"reason,omitempty"`
}

NewRelease is one movie inside the evidence window, for the owner list.

type Observation

type Observation struct {
	Key           core.RatingKey `json:"key"`
	Title         string         `json:"title"`
	Release       string         `json:"release"`
	Protocol      string         `json:"protocol"`
	State         string         `json:"state"`
	Reason        string         `json:"reason"`
	FirstSeen     int64          `json:"firstSeen"`
	FirstVerified int64          `json:"firstVerified"`
	VerifiedAt    int64          `json:"verifiedAt"`
	NextCheck     int64          `json:"nextCheck"`
}

type Service

type Service struct {
	// contains filtered or unexported fields
}

func New

func New(d Deps) (*Service, error)

func (*Service) Candidates

func (s *Service) Candidates(ctx context.Context, t *core.Title) []core.Release

Candidates serves remembered descriptors locally; playback still applies its exact viewer/device policy and validates the selected source.

func (*Service) Close added in v0.3.3

func (s *Service) Close()

Close stops background verdict writers and waits for them.

func (*Service) Cycle

func (s *Service) Cycle(ctx context.Context) error

func (*Service) DigitalDiscovery added in v0.3.4

func (s *Service) DigitalDiscovery(ctx context.Context) (bool, bool)

DigitalDiscovery reports the discovery policy for the current request.

func (*Service) EpisodesSettledBefore

func (s *Service) EpisodesSettledBefore(ctx context.Context) time.Time

EpisodesSettledBefore is the release time at or before which FilterDiscovery keeps an episode without looking for evidence, at the clock the request's later FilterDiscovery calls use.

func (*Service) Explain added in v0.3.0

func (s *Service) Explain(ctx context.Context, key core.RatingKey) (TitleEvidence, error)

func (*Service) FilterDiscovery added in v0.2.9

func (s *Service) FilterDiscovery(ctx context.Context, items []core.Title) []core.Title

A short local memo keeps repeated artwork/shelf checks inexpensive while browse/rows/search query availability_gate directly in SQL. Opening a gate clears this memo immediately.

func (*Service) FilterDiscoveryCurrent added in v0.3.0

func (s *Service) FilterDiscoveryCurrent(ctx context.Context, items []core.Title) []core.Title

FilterDiscoveryCurrent bypasses node-local visibility and resolver memos for one bounded policy preflight batch. It performs no provider lookup.

func (*Service) Handler

func (s *Service) Handler() http.Handler

Handler is mounted only inside the authenticated owner API.

func (*Service) HiddenEpisodes added in v0.3.4

func (s *Service) HiddenEpisodes(ctx context.Context) []core.RatingKey

HiddenEpisodes reports currently gated first-day episodes.

func (*Service) HiddenMovies added in v0.2.9

func (s *Service) HiddenMovies(ctx context.Context) []core.RatingKey

HiddenMovies reports currently gated recent movies for the owner console. Viewer browse/rows/search evaluate availability_gate directly in SQL.

func (*Service) HiddenShows added in v0.2.9

func (s *Service) HiddenShows(ctx context.Context) []core.RatingKey

HiddenShows reports currently gated first-day series.

func (*Service) Hubs

func (s *Service) Hubs(ctx context.Context, limit int) []core.Hub

func (*Service) Invalidate

func (s *Service) Invalidate(ctx context.Context, id string)

Invalidate retracts positive evidence on that release when playback or the owner invalidates a source so ranking/search move on to alternatives. It never closes availability_gate once a title's gate has opened.

func (*Service) JudgedOnFacts

func (s *Service) JudgedOnFacts(ctx context.Context) func(core.Title) bool

JudgedOnFacts returns, for one request, whether FilterDiscovery judges a title on its facts: one inside the evidence window, whose releases are matched against its names, genres, keywords, countries and directors. A caller holding partial titles passes those in full.

func (*Service) NewReleases added in v0.3.0

func (s *Service) NewReleases(ctx context.Context) ([]NewRelease, error)

func (*Service) Observe

func (s *Service) Observe(ctx context.Context, t *core.Title, r core.Release)

Observe reuses an actual successful resolver health check. Optimistic starts and prepared sessions without health evidence do not call this method.

func (*Service) PagingFilter added in v0.3.4

func (s *Service) PagingFilter(ctx context.Context, f core.Filter) (core.Filter, error)

PagingFilter attaches the sticky availability_gate policy and request-clock evidence window bounds to f with zero SQL round trips in steady state. On legacy read-only databases without availability_gate, it falls back to any published availability_hidden_current seqs without issuing writes.

func (*Service) Prioritize

func (s *Service) Prioritize(ctx context.Context, items []core.Title) []core.Title

func (*Service) Prune

func (s *Service) Prune(ctx context.Context) (int64, error)

Prune removes expired observations, retired hidden generations, and aged completed title-check records during scheduled maintenance.

func (*Service) PublishHiddenSet

func (s *Service) PublishHiddenSet(ctx context.Context, kind string, keys []core.RatingKey) (HiddenPublish, error)

PublishHiddenSet publishes keys as kind's hidden set under the current policy. Retained for the library-scale performance harness.

func (*Service) RecordSearch added in v0.2.9

func (s *Service) RecordSearch(ctx context.Context, t *core.Title, rs []core.Release)

RecordSearch retains descriptors from an actual requested search. It does not initiate searches, grab media or claim health. Post-date accepted matches open availability_gate immediately; pre-date matches queue verification.

func (*Service) Run

func (s *Service) Run(ctx context.Context)

func (*Service) Status

func (s *Service) Status(ctx context.Context) (Status, error)

func (*Service) Summary added in v0.3.0

func (s *Service) Summary(ctx context.Context) (Summary, error)

type Status

type Status struct {
	Mode                  string           `json:"mode"`
	Message               string           `json:"message"`
	Feeds                 []FeedStatus     `json:"feeds"`
	Usage                 map[string]int64 `json:"usage"`
	Counts                map[string]int   `json:"counts"`
	Recent                []Observation    `json:"recent"`
	Day                   string           `json:"day"`
	Ceilings              map[string]int64 `json:"ceilings,omitempty"`
	Effort                string           `json:"effort"`
	EffectiveFeedRequests int              `json:"effectiveFeedRequests"`
	EffectivePollMinutes  int              `json:"effectivePollMinutes"`
	EffectiveMaxPages     int              `json:"effectiveMaxPages"`
	OverAllowance         bool             `json:"overAllowance"`
}

type Summary added in v0.3.0

type Summary struct {
	Ready       int   `json:"ready"`
	Waiting     int   `json:"waiting"`
	LastChecked int64 `json:"lastChecked,omitempty"`
}

Summary is the quiet status line for "When they can play": how many in-window movies are ready, how many are still waiting, and when Ether last looked.

type TitleEvidence added in v0.3.0

type TitleEvidence struct {
	Code      string `json:"code"`
	CheckedAt int64  `json:"checkedAt,omitempty"`
	NextCheck int64  `json:"nextCheck,omitempty"`
	Reason    string `json:"reason,omitempty"`
}

TitleEvidence is why a movie is shown or hidden. Codes are for the owner console; they are not sentences.

type Validator

type Validator interface {
	FeedCandidate(context.Context, *core.Title, core.Release) (core.Release, string, error)
	VerifyFeedCandidate(context.Context, *core.Title, core.Release) error
}

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL