provider

package
v0.10.0-dev.64 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 10, 2026 License: AGPL-3.0 Imports: 27 Imported by: 0

Documentation

Overview

Package provider is the boundary between subflux and the subtitle sites it searches. An adapter sees only its Client, whose one HTTP path pins every request to the entry's Hosts, retries a GET or HEAD on a transient failure only, and classifies every non-2xx answer into a *Failure; Build wraps the adapter so every error leaving it is a redacted *Failure and every answer is bounded before the engine ranks it or makes a handle.

Index

Constants

View Source
const (
	MaxAnswer   = 500
	MaxRefToken = 1 << 10
	MaxText     = 512
	MaxCovers   = 64
)

The wrapper's answer bounds: candidates past MaxAnswer are dropped, a candidate whose Ref token is empty or over MaxRefToken bytes or with more than MaxCovers entries is dropped whole, and Release and Title are sanitized to one line of at most MaxText bytes.

View Source
const MaxBody = 10 << 20

MaxBody is the ceiling on every body the Client reads.

Variables

This section is empty.

Functions

func Flags

func Flags(flagged language.Flags, texts ...string) language.Flags

Flags is a result's flags: each one the provider sets, or that texts (its release name, file name or comment) say as language.ReadFlags reads them.

func PayloadName

func PayloadName(resp *http.Response) string

PayloadName is the file name an answer gives its body: the Content-Disposition filename, else the last segment of the answered URL's path, as one line of at most MaxText bytes with no directory part.

func ReadAll

func ReadAll(resp *http.Response, limit int64) ([]byte, error)

ReadAll reads resp's body; the caller closes it. A body past limit (at most MaxBody) fails as Permanent with CauseTooLarge and is never truncated; a read error is Transient with its cause.

func WirePairs

func WirePairs(labelTag ...string) []language.WirePair

WirePairs builds an adapter's language table from alternating labels and tags ("English", "en", ...), for language.MustProviderMap; it panics on an odd count or a tag language.Parse refuses, since both are typos in a package-level table.

Types

type Adapter

type Adapter interface {
	Search(ctx context.Context, q Query) ([]Candidate, error)
	Fetch(ctx context.Context, ref Ref) (Payload, error)
}

Adapter is what Build returns. Search returns every candidate it read; when a later page failed it returns them with that page's *Failure, so the funnel observes the failure and no no-result evidence is booked from a partial answer.

func Build

func Build(e Entry, cfg Config) (Adapter, error)

Build checks cfg against e, makes e's Client, calls e.New and wraps the adapter. Every error the wrapped Search, Fetch or Test returns leaves it as a *Failure (any error that is none Transient) whose Msg is one line of at most 256 bytes with cfg's Secret values redacted, but a *Skipped returned before any request, which passes through; every Search answer is bounded as MaxAnswer's block says. The wrapped adapter implements Tester exactly when e.New's does.

type Candidate

type Candidate struct {
	Ref     Ref
	Lang    language.Tag
	Release string
	Title   string
	Covers  []media.Numbering
	Year    int
	FPS     float64
	Flags   language.Flags
	// MatchedBy is how the provider matched the query.
	MatchedBy rank.MatchedBy
	// Unconfirmed marks a file found for the query's media that the adapter
	// cannot place on its episode; search offers it for manual search only.
	Unconfirmed bool
	// MachineTranslated: the provider says a machine or an AI made it.
	MachineTranslated bool
}

Candidate is one result. Title is the provider's name for the media it matched ("" when it matched by id) and Year its year (0 when none); Covers is the numbering the provider claims, nil for none. An adapter sets Covers with MatchedBy ID only when a published mapping placed the file on those episodes, since search then reads them instead of the name.

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client is the only HTTP path an adapter has. Build makes it from the entry's Hosts.

func (Client) Do

func (c Client) Do(req *http.Request) (*http.Response, error)

Do sends req and answers a 2xx, or a 304 to a request carrying If-None-Match or If-Modified-Since; any other answer is closed and returned as a *Failure. A GET or HEAD is retried on a Transient failure, 3 attempts from 2 seconds; any other method is sent once, so a quota-spending POST never spends twice. An Exception entry's 403, 429 or 503 is RateLimited for at least an hour and never retried.

func (Client) HTTPClient

func (c Client) HTTPClient() *http.Client

HTTPClient is a new http.Client over the Client's pinned round tripper, for a library that takes one. Every hop is pinned whatever CheckRedirect the caller sets; its Timeout is 3 minutes.

type Config

type Config struct {
	Values   formschema.Values
	ID       ProviderID
	Priority int
	Enabled  bool
}

Config is one provider's settings section.

type Entry

type Entry struct {
	Languages language.ProviderMap
	// Quirk, nil for none, sees each non-2xx answer with at most 4 KiB of its
	// body and returns a *Failure, or nil for the default classification.
	Quirk func(*http.Response, []byte) error
	New   func(Client, formschema.Values) (Adapter, error)
	// Data, nil for none, takes the engine's report for data the entry
	// downloads: the entry calls report with that data's condition at once
	// and when each refresh of it ends, and a later Data call replaces the
	// earlier report.
	Data func(report func(text string, active bool))
	ID   ProviderID
	// Summary is one English line of 1 to 120 bytes on what the provider
	// covers and whether it needs an account.
	Summary string
	// UserAgent, "" for subflux's own, replaces it on every request of an
	// Exception whose ruling allows a browser agent; Build refuses it on any
	// other entry.
	UserAgent string
	// Hosts is every host name the adapter may reach, exact names only.
	Hosts  []string
	Schema formschema.Schema
	// Exception marks a scraping adapter the user named, which the Client
	// holds to robots.txt, 4 requests a call and an hour's back-off on a 403,
	// 429 or 503.
	Exception bool
}

Entry is one row of the provider table.

type Failure

type Failure struct {
	Host       string
	Msg        string        // one line, at most 256 bytes, secrets redacted once the wrapper returns it
	RetryAfter time.Duration // RateLimited only; 0: none given
	Status     int           // 0: no HTTP answer
	Cause      httpclient.Cause
	Kind       FailureKind
	// contains filtered or unexported fields
}

Failure is the boundary's one classification of a provider failure.

func Fail

func Fail(k FailureKind, msg string) *Failure

Fail is how an adapter reports a fault a 200 body carries; the wrapper redacts its Msg like every other failure's.

func (*Failure) Error

func (f *Failure) Error() string

func (*Failure) IsTransient

func (f *Failure) IsTransient() bool

IsTransient lets httpclient.Do, inside the Client, retry a GET or HEAD on a Transient failure only.

func (*Failure) Unwrap

func (f *Failure) Unwrap() error

Unwrap answers the context error that ended the call, or nil.

type FailureKind

type FailureKind uint8

FailureKind is what the funnel routes a failure on.

const (
	Auth FailureKind = iota + 1
	RateLimited
	Transient
	Permanent
	NotFound
)

The failure kinds, from 1, so the zero kind is none.

func (FailureKind) String

func (k FailureKind) String() string

type Payload

type Payload struct {
	Name  string
	Bytes []byte
}

Payload is one fetched subtitle or archive.

type ProviderID

type ProviderID string //nolint:revive // design 5's name; provider.ID would read as a provider's own id field

ProviderID names one provider in settings, state and the wire.

type Query

type Query struct {
	Release   release.Parsed
	Titles    []string
	Langs     []language.Tag
	External  media.ExternalIDs
	Hash      VideoHash
	Numbering media.Numbering
	Year      int
	Absolute  int
	Kind      media.Kind
}

Query is built once per run. For an episode External holds the series' ids; Year is the root's (0 unknown); Titles are the root's title then its alternative titles; Langs are this provider's tags from the query union.

type Ref

type Ref struct {
	// contains filtered or unexported fields
}

Ref is the adapter's own token for one download. It is held server-side and resolves only on the adapter's Hosts.

func NewRef

func NewRef(token string) Ref

NewRef wraps an adapter's token.

func (Ref) Token

func (r Ref) Token() string

Token is the adapter's token.

type Skipped

type Skipped struct{ Cause string } //nolint:errname // design 5 names it Skipped: a search not sent, not a fault

Skipped is Search's error when the adapter sends no request because the query lacks what it searches by. Cause is one English line naming what it lacks. The wrapper passes it through when no request was sent and makes it Permanent otherwise.

func (*Skipped) Error

func (s *Skipped) Error() string

type Tester

type Tester interface {
	Test(ctx context.Context) error
}

Tester is implemented by a credentialed adapter: the cheapest call that proves its credential.

type VideoHash

type VideoHash struct {
	Size   int64
	OSHash uint64
}

VideoHash is the OpenSubtitles hash of the video; zero is unknown.

Directories

Path Synopsis
Package animemap is what the anime adapters share: animap's anime map, read through mapread and refreshed on demand once per process, its alert, the landing lookups over it, and the one reader of what an anime release's file name claims.
Package animemap is what the anime adapters share: animap's anime map, read through mapread and refreshed on demand once per process, its alert, the landing lookups over it, and the one reader of what an anime release's file name claims.
Package animetosho is the AnimeTosho provider: the subtitle attachments of the anime releases AnimeTosho's feed indexes, searched by AniDB episode or anime through the anime map, else by title.
Package animetosho is the AnimeTosho provider: the subtitle attachments of the anime releases AnimeTosho's feed indexes, searched by AniDB episode or anime through the anime map, else by title.
Package archive extracts a subtitle from a ZIP or RAR container.
Package archive extracts a subtitle from a ZIP or RAR container.
Package betaseries is the BetaSeries provider: English and French TV subtitles from the BetaSeries API by the series' TVDB id, downloaded through www.betaseries.com's /srt links.
Package betaseries is the BetaSeries provider: English and French TV subtitles from the BetaSeries API by the series' TVDB id, downloaded through www.betaseries.com's /srt links.
Package gestdown is the Gestdown provider: Addic7ed's TV subtitles through Gestdown's REST API (https://gestdown.readme.io/reference), found by the series' TVDB id, one season and language per request.
Package gestdown is the Gestdown provider: Addic7ed's TV subtitles through Gestdown's REST API (https://gestdown.readme.io/reference), found by the series' TVDB id, one season and language per request.
Package hdbits is the HDBits.org provider: its JSON API finds a title's torrents by TVDB season or IMDb id and lists each torrent's subtitles, and getdox.php serves a subtitle or season-pack archive.
Package hdbits is the HDBits.org provider: its JSON API finds a title's torrents by TVDB season or IMDb id and lists each torrent's subtitles, and getdox.php serves a subtitle or season-pack archive.
internal
dialhook
Package dialhook carries a TLS dial on a request context, so a test can serve a provider's real hosts in memory while the provider Client still runs its host pin first.
Package dialhook carries a TLS dial on a request context, so a test can serve a provider's real hosts in memory while the provider Client still runs its host pin first.
Package opensubtitles is the OpenSubtitles.com provider.
Package opensubtitles is the OpenSubtitles.com provider.
Package providers is the table of every provider subflux can search.
Package providers is the table of every provider subflux can search.
Package providertest serves provider fakes on their real host names, in memory and over TLS, for tests of code that sends requests through a provider Client.
Package providertest serves provider fakes on their real host names, in memory and over TLS, for tests of code that sends requests through a provider Client.
Package subdl is the SubDL provider: its REST API at api.subdl.com, by IMDb or TMDB id or by title, with downloads from dl.subdl.com.
Package subdl is the SubDL provider: its REST API at api.subdl.com, by IMDb or TMDB id or by title, with downloads from dl.subdl.com.
Package subsource is the SubSource provider: its REST API at api.subsource.net, which finds a title by IMDb id (falling back to its text search) and lists that title's subtitles per language.
Package subsource is the SubSource provider: its REST API at api.subsource.net, which finds a title by IMDb id (falling back to its text search) and lists that title's subtitles per language.
Package yifysubtitles is the YIFY Subtitles provider: movie subtitles read from yifysubtitles.ch's pages by IMDb id.
Package yifysubtitles is the YIFY Subtitles provider: movie subtitles read from yifysubtitles.ch's pages by IMDb id.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL