Documentation
¶
Overview ¶
Package provider is the boundary between subflux and the subtitle sites it searches. An adapter sees only its Client, whose one HTTP path pins every request to the entry's Hosts, retries a GET or HEAD on a transient failure only, and classifies every non-2xx answer into a *Failure; Build wraps the adapter so every error leaving it is a redacted *Failure and every answer is bounded before the engine ranks it or makes a handle.
Index ¶
- Constants
- func Flags(flagged language.Flags, texts ...string) language.Flags
- func PayloadName(resp *http.Response) string
- func ReadAll(resp *http.Response, limit int64) ([]byte, error)
- func WirePairs(labelTag ...string) []language.WirePair
- type Adapter
- type Candidate
- type Client
- type Config
- type Entry
- type Failure
- type FailureKind
- type Payload
- type ProviderID
- type Query
- type Ref
- type Skipped
- type Tester
- type VideoHash
Constants ¶
const ( MaxAnswer = 500 MaxRefToken = 1 << 10 MaxText = 512 MaxCovers = 64 )
The wrapper's answer bounds: candidates past MaxAnswer are dropped, a candidate whose Ref token is empty or over MaxRefToken bytes or with more than MaxCovers entries is dropped whole, and Release and Title are sanitized to one line of at most MaxText bytes.
const MaxBody = 10 << 20
MaxBody is the ceiling on every body the Client reads.
Variables ¶
This section is empty.
Functions ¶
func Flags ¶
Flags is a result's flags: each one the provider sets, or that texts (its release name, file name or comment) say as language.ReadFlags reads them.
func PayloadName ¶
PayloadName is the file name an answer gives its body: the Content-Disposition filename, else the last segment of the answered URL's path, as one line of at most MaxText bytes with no directory part.
func ReadAll ¶
ReadAll reads resp's body; the caller closes it. A body past limit (at most MaxBody) fails as Permanent with CauseTooLarge and is never truncated; a read error is Transient with its cause.
Types ¶
type Adapter ¶
type Adapter interface {
Search(ctx context.Context, q Query) ([]Candidate, error)
Fetch(ctx context.Context, ref Ref) (Payload, error)
}
Adapter is what Build returns. Search returns every candidate it read; when a later page failed it returns them with that page's *Failure, so the funnel observes the failure and no no-result evidence is booked from a partial answer.
func Build ¶
Build checks cfg against e, makes e's Client, calls e.New and wraps the adapter. Every error the wrapped Search, Fetch or Test returns leaves it as a *Failure (any error that is none Transient) whose Msg is one line of at most 256 bytes with cfg's Secret values redacted, but a *Skipped returned before any request, which passes through; every Search answer is bounded as MaxAnswer's block says. The wrapped adapter implements Tester exactly when e.New's does.
type Candidate ¶
type Candidate struct {
Ref Ref
Lang language.Tag
Release string
Title string
Covers []media.Numbering
Year int
FPS float64
Flags language.Flags
// MatchedBy is how the provider matched the query.
MatchedBy rank.MatchedBy
// Unconfirmed marks a file found for the query's media that the adapter
// cannot place on its episode; search offers it for manual search only.
Unconfirmed bool
// MachineTranslated: the provider says a machine or an AI made it.
MachineTranslated bool
}
Candidate is one result. Title is the provider's name for the media it matched ("" when it matched by id) and Year its year (0 when none); Covers is the numbering the provider claims, nil for none. An adapter sets Covers with MatchedBy ID only when a published mapping placed the file on those episodes, since search then reads them instead of the name.
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client is the only HTTP path an adapter has. Build makes it from the entry's Hosts.
func (Client) Do ¶
Do sends req and answers a 2xx, or a 304 to a request carrying If-None-Match or If-Modified-Since; any other answer is closed and returned as a *Failure. A GET or HEAD is retried on a Transient failure, 3 attempts from 2 seconds; any other method is sent once, so a quota-spending POST never spends twice. An Exception entry's 403, 429 or 503 is RateLimited for at least an hour and never retried.
func (Client) HTTPClient ¶
HTTPClient is a new http.Client over the Client's pinned round tripper, for a library that takes one. Every hop is pinned whatever CheckRedirect the caller sets; its Timeout is 3 minutes.
type Config ¶
type Config struct {
Values formschema.Values
ID ProviderID
Priority int
Enabled bool
}
Config is one provider's settings section.
type Entry ¶
type Entry struct {
Languages language.ProviderMap
// Quirk, nil for none, sees each non-2xx answer with at most 4 KiB of its
// body and returns a *Failure, or nil for the default classification.
Quirk func(*http.Response, []byte) error
New func(Client, formschema.Values) (Adapter, error)
// Data, nil for none, takes the engine's report for data the entry
// downloads: the entry calls report with that data's condition at once
// and when each refresh of it ends, and a later Data call replaces the
// earlier report.
Data func(report func(text string, active bool))
ID ProviderID
// Summary is one English line of 1 to 120 bytes on what the provider
// covers and whether it needs an account.
Summary string
// UserAgent, "" for subflux's own, replaces it on every request of an
// Exception whose ruling allows a browser agent; Build refuses it on any
// other entry.
UserAgent string
// Hosts is every host name the adapter may reach, exact names only.
Hosts []string
Schema formschema.Schema
// Exception marks a scraping adapter the user named, which the Client
// holds to robots.txt, 4 requests a call and an hour's back-off on a 403,
// 429 or 503.
Exception bool
}
Entry is one row of the provider table.
type Failure ¶
type Failure struct {
Host string
Msg string // one line, at most 256 bytes, secrets redacted once the wrapper returns it
RetryAfter time.Duration // RateLimited only; 0: none given
Status int // 0: no HTTP answer
Cause httpclient.Cause
Kind FailureKind
// contains filtered or unexported fields
}
Failure is the boundary's one classification of a provider failure.
func Fail ¶
func Fail(k FailureKind, msg string) *Failure
Fail is how an adapter reports a fault a 200 body carries; the wrapper redacts its Msg like every other failure's.
func (*Failure) IsTransient ¶
IsTransient lets httpclient.Do, inside the Client, retry a GET or HEAD on a Transient failure only.
type FailureKind ¶
type FailureKind uint8
FailureKind is what the funnel routes a failure on.
const ( Auth FailureKind = iota + 1 RateLimited Transient Permanent NotFound )
The failure kinds, from 1, so the zero kind is none.
func (FailureKind) String ¶
func (k FailureKind) String() string
type ProviderID ¶
type ProviderID string //nolint:revive // design 5's name; provider.ID would read as a provider's own id field
ProviderID names one provider in settings, state and the wire.
type Query ¶
type Query struct {
Release release.Parsed
Titles []string
Langs []language.Tag
External media.ExternalIDs
Hash VideoHash
Numbering media.Numbering
Year int
Absolute int
Kind media.Kind
}
Query is built once per run. For an episode External holds the series' ids; Year is the root's (0 unknown); Titles are the root's title then its alternative titles; Langs are this provider's tags from the query union.
type Ref ¶
type Ref struct {
// contains filtered or unexported fields
}
Ref is the adapter's own token for one download. It is held server-side and resolves only on the adapter's Hosts.
type Skipped ¶
type Skipped struct{ Cause string } //nolint:errname // design 5 names it Skipped: a search not sent, not a fault
Skipped is Search's error when the adapter sends no request because the query lacks what it searches by. Cause is one English line naming what it lacks. The wrapper passes it through when no request was sent and makes it Permanent otherwise.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package animemap is what the anime adapters share: animap's anime map, read through mapread and refreshed on demand once per process, its alert, the landing lookups over it, and the one reader of what an anime release's file name claims.
|
Package animemap is what the anime adapters share: animap's anime map, read through mapread and refreshed on demand once per process, its alert, the landing lookups over it, and the one reader of what an anime release's file name claims. |
|
Package animetosho is the AnimeTosho provider: the subtitle attachments of the anime releases AnimeTosho's feed indexes, searched by AniDB episode or anime through the anime map, else by title.
|
Package animetosho is the AnimeTosho provider: the subtitle attachments of the anime releases AnimeTosho's feed indexes, searched by AniDB episode or anime through the anime map, else by title. |
|
Package archive extracts a subtitle from a ZIP or RAR container.
|
Package archive extracts a subtitle from a ZIP or RAR container. |
|
Package betaseries is the BetaSeries provider: English and French TV subtitles from the BetaSeries API by the series' TVDB id, downloaded through www.betaseries.com's /srt links.
|
Package betaseries is the BetaSeries provider: English and French TV subtitles from the BetaSeries API by the series' TVDB id, downloaded through www.betaseries.com's /srt links. |
|
Package gestdown is the Gestdown provider: Addic7ed's TV subtitles through Gestdown's REST API (https://gestdown.readme.io/reference), found by the series' TVDB id, one season and language per request.
|
Package gestdown is the Gestdown provider: Addic7ed's TV subtitles through Gestdown's REST API (https://gestdown.readme.io/reference), found by the series' TVDB id, one season and language per request. |
|
Package hdbits is the HDBits.org provider: its JSON API finds a title's torrents by TVDB season or IMDb id and lists each torrent's subtitles, and getdox.php serves a subtitle or season-pack archive.
|
Package hdbits is the HDBits.org provider: its JSON API finds a title's torrents by TVDB season or IMDb id and lists each torrent's subtitles, and getdox.php serves a subtitle or season-pack archive. |
|
internal
|
|
|
dialhook
Package dialhook carries a TLS dial on a request context, so a test can serve a provider's real hosts in memory while the provider Client still runs its host pin first.
|
Package dialhook carries a TLS dial on a request context, so a test can serve a provider's real hosts in memory while the provider Client still runs its host pin first. |
|
Package opensubtitles is the OpenSubtitles.com provider.
|
Package opensubtitles is the OpenSubtitles.com provider. |
|
Package providers is the table of every provider subflux can search.
|
Package providers is the table of every provider subflux can search. |
|
Package providertest serves provider fakes on their real host names, in memory and over TLS, for tests of code that sends requests through a provider Client.
|
Package providertest serves provider fakes on their real host names, in memory and over TLS, for tests of code that sends requests through a provider Client. |
|
Package subdl is the SubDL provider: its REST API at api.subdl.com, by IMDb or TMDB id or by title, with downloads from dl.subdl.com.
|
Package subdl is the SubDL provider: its REST API at api.subdl.com, by IMDb or TMDB id or by title, with downloads from dl.subdl.com. |
|
Package subsource is the SubSource provider: its REST API at api.subsource.net, which finds a title by IMDb id (falling back to its text search) and lists that title's subtitles per language.
|
Package subsource is the SubSource provider: its REST API at api.subsource.net, which finds a title by IMDb id (falling back to its text search) and lists that title's subtitles per language. |
|
Package yifysubtitles is the YIFY Subtitles provider: movie subtitles read from yifysubtitles.ch's pages by IMDb id.
|
Package yifysubtitles is the YIFY Subtitles provider: movie subtitles read from yifysubtitles.ch's pages by IMDb id. |