provider

package
v0.9.0-dev.30 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 18, 2026 License: AGPL-3.0 Imports: 21 Imported by: 0

Documentation

Overview

Package provider contains the provider registry and retry wrappers.

Index

Constants

View Source
const DownloadRetryAttempts = 3

DownloadRetryAttempts is the maximum number of download attempts per provider.

View Source
const DownloadRetryInitBackoff = 2 * time.Second

DownloadRetryInitBackoff is the initial backoff duration between download retries.

View Source
const ErrProviderInit registryErrorKind = iota + 1

ErrProviderInit indicates a provider factory returned an error.

View Source
const HTTPTimeoutExtended = 30 * time.Second

HTTPTimeoutExtended is the timeout for heavier provider APIs that may return larger payloads or have slower backends (opensubtitles, subsource, subdl).

View Source
const HTTPTimeoutStandard = 15 * time.Second

HTTPTimeoutStandard is the default timeout for lightweight provider APIs (hdbits, betaseries, gestdown, animetosho, yifysubtitles).

Variables

This section is empty.

Functions

func ClearCaches added in v0.4.0

func ClearCaches(providers []Provider)

ClearCaches is typically called at scan completion to free memory.

func ExtractAndValidate

func ExtractAndValidate(data []byte, want epmarker.Target) ([]byte, error)

ExtractAndValidate turns a provider's download body into usable subtitle bytes: extract from an archive when there is one, fall back to the raw body when nothing here could read it, and validate the result as subtitle content.

The two failure paths report different things on purpose. When the archive opened and had no subtitle for this target, its own refusal is returned, because subtitlefile.Validate can only say "detected zip archive" — which reads as "archives are unsupported" for a payload this code unpacked and inspected member by member. Validate is the answer only when nothing opened the data, where the raw bytes really are the whole story.

Nothing here converts encodings. Validate judges a decoded VIEW so a UTF-16 subtitle is recognised as text, and the bytes returned are the bytes that arrived, because whether a conversion is PERSISTED belongs to post_processing.normalize_utf8 alone.

Every refusal from an archive that DID open is marked permanent on the way out. Those errors wrap a cause so a caller can reach it with errors.Is, and one of the causes a truncated member produces is io.ErrUnexpectedEOF, which httpx.IsTransient reads as a reason to re-download. An unreadable or wrong-episode archive is a property of the bytes, so a retry fetches the same bytes and fails identically while delaying the fall-through to the next candidate. IsPermanent is consulted before every transient test, and httpx.Permanent adds no text and keeps the chain, so neither the operator's log line nor errors.Is changes.

func NewHTTPClient

func NewHTTPClient(timeout time.Duration) *http.Client

NewHTTPClient returns a *http.Client preconfigured with the SSRF-safe transport, User-Agent injection, redirect policy, the hard response-body ceiling (maxResponseBodyBytes), and the given timeout.

All providers share these defaults: any redirect to a private IP, link-local address, or HTTP-downgrade is rejected; private/link-local DNS resolutions are also blocked at the transport layer. Per-provider tuning (auth headers, caching, rate limiting) layers on top of the *http.Client returned here.

Use this factory instead of constructing &http.Client{} ad-hoc; consistency of SSRF/redirect policy across providers is a security property the factory enforces.

func NewHTTPClientNoClientTimeout

func NewHTTPClientNoClientTimeout(allowedPorts ...uint16) *http.Client

NewHTTPClientNoClientTimeout omits the client-level Timeout for providers (e.g. anidb) that fetch large bodies with per-request context.WithTimeout, where a client-level Timeout would clip mid-stream. allowedPorts overrides ssrf's default {443} dial-port allowlist — pass it only for a known public endpoint on a non-standard port (e.g. the AniDB HTTP API on 9001); omit it for the https/443 default.

func NormalizeSettings added in v0.1.148

func NormalizeSettings(fields []subflux.ProviderSchemaField, raw map[string]any) map[string]any

NormalizeSettings returns a copy of the raw settings map with every declared-but-absent field filled from its schema Default, coerced per the declared Type (P14). The registry applies this before invoking a factory, which makes the providerEntries declaration the SINGLE source of a setting's default: factories read the normalized map through FromMap / the typed accessors and never re-encode a default of their own (the use_hash dual-encoding this replaces drifted exactly once already). Undeclared keys pass through untouched, and a declared field the user DID set is never rewritten.

func SettingBool

func SettingBool(settings map[string]any, key SettingKey, def bool) bool

SettingBool accepts native bool or string "true"/"false"; returns def if the key is missing or the value is not a recognized boolean.

func SettingFloat

func SettingFloat(settings map[string]any, key SettingKey, def float64) float64

SettingFloat accepts native float64/int/int64 or numeric string; returns def if the key is missing or the value is not convertible.

func SettingInt

func SettingInt(settings map[string]any, key SettingKey, def int) int

SettingInt accepts native int/int64/float64 (whole YAML numeric) or numeric string; returns def if missing, non-convertible, or a non-whole float64.

func SettingString

func SettingString(settings map[string]any, key SettingKey) string

SettingString returns "" if the key is missing or the value is not a string.

func TargetOf added in v0.4.35

func TargetOf(sub *subflux.Subtitle) epmarker.Target

TargetOf returns the episode a subtitle download is looking for: the episode the subtitle names, or epmarker.Any when it names none, which is what a movie and a provider that reports no numbering both arrive with.

It exists so the wire-to-target conversion happens once. Season and Episode are adjacent ints of the same type, so a transposed conversion would compile and then quietly extract the wrong episode; here it is written once and tested.

Types

type CacheClearer added in v0.4.0

type CacheClearer interface {
	ClearCache()
}

CacheClearer is declared here, at the consumer, rather than in a shared contract package: this package is the only one that asks the question. One method, because clearing is all it asks — the providers that hold a download cache (season-pack zips) offer it, the eight that hold nothing do not, and ClearCaches below decides per provider at runtime.

EXPORTED against the unexported-by-default rule, and the reason is a real guard rather than convenience: a provider opts in from its own package with `var _ provider.CacheClearer = (*Provider)(nil)`, and that assertion is the only compile-time check on the opt-in. The dispatch here is a type assertion, so a renamed or dropped ClearCache would leave a provider silently never freeing its cache, with nothing red anywhere.

type CredentialChecker added in v0.6.0

type CredentialChecker interface {
	CheckCredentials(ctx context.Context) error
}

CredentialChecker is implemented by a provider whose settings carry credentials it can validate without performing a search.

A rejected credential is *subflux.AuthError; every other failure means the check did not complete, and the operator's remedy differs between the two.

type FactoryFunc

type FactoryFunc func(ctx context.Context, settings map[string]any) (Provider, error)

FactoryFunc creates a provider from config settings. The context parameter enables cancellation during provider initialization (e.g. credential validation, API pings) and respects shutdown signals.

type Provider added in v0.4.0

type Provider interface {
	Name() subflux.ProviderID

	Search(ctx context.Context, req *subflux.SearchRequest) ([]subflux.Subtitle, error)

	// Download fetches the subtitle content for the given search result.
	//
	// A subtitle the upstream no longer holds is reported as an error
	// wrapping subflux.ErrSubtitleAbsent, never as a successful download of zero
	// bytes: absence and a truncated or corrupt payload are different
	// operator problems, and returning nil bytes with a nil error made them
	// read the same. Implementations therefore never return (nil, nil) —
	// every return either carries subtitle content or names why it does not.
	Download(ctx context.Context, sub *subflux.Subtitle) ([]byte, error)
}

Provider is what a subtitle source must offer to be registered here: the name config and logs identify it by, a search over one media item, and a download of one result. THREE methods, against a denominator of five — the exported methods the nine implementations offer between them. Those three are the intersection: every implementation has all three, and the two that are not universal (CountShowSubtitles on opensubtitles, ClearCache on hdbits) are declared as their own one-method interfaces in cache.go and discovered by type assertion. Nothing in this contract is optional, which is why the optional pair is not in it.

Declared in this package because this package is the consumer: FactoryFunc below returns one, Registry.LoadAll builds them, and wrapRetry wraps one. EXPORTED against the unexported-by-default rule because the nine Factory functions in the subpackages name it in their return type, and providers.go types them as FactoryFunc — the name has to cross the package boundary for the registration table to exist at all.

Returning an interface from those factories is the stdlib registry idiom, not a lapse from "return structs": image.RegisterFormat takes constructors of a uniform signature returning image.Image, and image/png exports Decode with no concrete alternative (ratified as C14). This is also the only interface in subflux with real polymorphism — nine implementations plus the retry wrapper, where every other one has exactly one.

func WrapRetryAll

func WrapRetryAll(providers []Provider, maxAttempts int, initBackoff time.Duration) []Provider

WrapRetryAll wraps each provider with download retry logic.

type Registry

type Registry struct {
	// contains filtered or unexported fields
}

Registry holds provider factories keyed by name.

func NewRegistry

func NewRegistry() *Registry

NewRegistry creates an empty provider registry.

func (*Registry) CheckCredentials added in v0.6.0

func (r *Registry) CheckCredentials(ctx context.Context, name subflux.ProviderID, settings map[string]any) error

CheckCredentials builds the named provider from settings and asks it whether its credentials are accepted. A nil error means they are.

Settings are normalized exactly as LoadAll normalizes them, so the check runs against the values a save would activate rather than a second interpretation of the same map. A factory failure is reported as *subflux.AuthError: every factory here fails only on a missing required credential, so the remedy is the operator's blank field rather than the network.

func (*Registry) CredentialCheck added in v0.6.0

func (r *Registry) CredentialCheck(name subflux.ProviderID) bool

CredentialCheck reports whether the named provider offers a credential check. Schema data: the settings UI renders a test control for the providers it answers true for, and the endpoint refuses a request naming any other.

func (*Registry) LoadAll

func (r *Registry) LoadAll(ctx context.Context, providers map[subflux.ProviderID]subflux.ProviderCfg) ([]Provider, error)

LoadAll creates all enabled providers from config. Providers are loaded in parallel for reduced startup latency when factories perform network validation. Results are sorted by name for deterministic ordering. Unknown provider names are skipped with a warning (a typo in config should not prevent all other providers from loading). Providers are returned unwrapped; download retry wrapping (WrapRetryAll) is applied by the composition root (main.go wiring).

LoadAll reads only cfg.Enabled and cfg.Settings from each entry. Other ProviderCfg fields (if added) will not affect provider loading.

Zero providers loading is NOT an error: a config with no enabled acquisition providers is a valid "embedded detection and coverage only" setup. The counts are WARN-logged so operators can still distinguish a typo (unknown>0) from a deliberate all-disabled state.

func (*Registry) Normalize added in v0.6.6

func (r *Registry) Normalize(name subflux.ProviderID, raw map[string]any) map[string]any

Normalize fills the named provider's absent settings from their schema defaults. It is the one normalization the factories, the credential check and the provider gate all read, so the three compare the same values.

func (*Registry) ProviderNames

func (r *Registry) ProviderNames() []subflux.ProviderID

ProviderNames returns all registered provider names in sorted order.

func (*Registry) Register

func (r *Registry) Register(name subflux.ProviderID, f FactoryFunc)

Register adds a provider factory to the registry.

func (*Registry) RegisterCredentialCheck added in v0.6.0

func (r *Registry) RegisterCredentialCheck(name subflux.ProviderID, offered bool)

RegisterCredentialCheck records whether a provider's implementation can validate its credentials without searching. Declared at the registration table rather than derived here, because deriving it would mean building the provider, which needs the credentials the schema is rendered to collect.

func (*Registry) RegisterSchema

func (r *Registry) RegisterSchema(name subflux.ProviderID, label string, fields []subflux.ProviderSchemaField)

RegisterSchema adds UI metadata for a provider.

func (*Registry) Schema

Schema returns the label and settings fields for a provider.

The fields are cloned: the registry is built once at boot and read by the settings-UI handler on every request, so a caller sorting or rewriting the returned slice in place would be editing what every later request renders. One level of clone is the whole copy — ProviderSchemaField holds five strings and a bool and no reference type, so the elements share nothing further.

type SettingKey

type SettingKey string

SettingKey is a typed constant for provider setting keys, preventing typos and enabling IDE navigation.

const (
	KeyPassword SettingKey = "password"

	KeyIncludeAI       SettingKey = "include_ai_translated"
	KeyIncludeMT       SettingKey = "include_machine_translated"
	KeyAniDBClientKey  SettingKey = "anidb_client_key"
	KeyMode            SettingKey = "mode"
	KeyErrorMessage    SettingKey = "error_message"
	KeyDownloadError   SettingKey = "download_error"
	KeySubtitleContent SettingKey = "subtitle_content"
	KeyIncludeHash     SettingKey = "include_hash"
	KeyHearingImpaired SettingKey = "hearing_impaired"
	KeyForced          SettingKey = "forced"
	KeyResultCount     SettingKey = "result_count"
	KeyFlakyRate       SettingKey = "flaky_rate"
	KeyDelayMs         SettingKey = "delay_ms"
)

keyAPIKey and its siblings enumerate the provider setting keys.

type SettingReporter added in v0.6.6

type SettingReporter interface {
	SettingVerdict() (setting string, refusal error)
	ForgetSettingVerdict()
}

SettingReporter is implemented by a provider that keeps working whatever the upstream answers about one OPTIONAL setting. SettingVerdict names the setting and the upstream's refusal, nil for an acceptance, or returns ("", nil) until the upstream answers; ForgetSettingVerdict drops the answer, so the next call asks the upstream again.

type Settings

type Settings struct {
	Custom   map[string]any // provider-specific extras
	APIKey   string
	Username string
	Password string
	Passkey  string
	Token    string
	UseHash  bool
}

Settings provides typed access to common provider configuration fields. Every common field has migrated here (FromMap gives compile-time safety); Custom carries only genuinely provider-specific extras.

func FromMap

func FromMap(settings map[string]any) Settings

FromMap constructs a Settings from the raw settings map, extracting common fields with type-safe accessors and preserving remaining entries in Custom for provider-specific use.

type ShowSubtitleCounter added in v0.4.0

type ShowSubtitleCounter interface {
	CountShowSubtitles(ctx context.Context, q subflux.ShowSubtitleQuery) (int, error)
}

ShowSubtitleCounter is the optional show-level count a provider may support: how many subtitles exist for a show+language without naming an episode. ONE method, and it is declared here rather than beside Provider because it is not part of the provider contract — only OpenSubtitles implements it, and the search engine discovers that by type assertion.

Directories

Path Synopsis
Package anidb provides TVDB → AniDB mapping using the community-maintained anime-list.xml from https://github.com/Anime-Lists/anime-lists.
Package anidb provides TVDB → AniDB mapping using the community-maintained anime-list.xml from https://github.com/Anime-Lists/anime-lists.
Package animetosho implements the AnimeTosho subtitle provider.
Package animetosho implements the AnimeTosho subtitle provider.
Package archive extracts a subtitle from a ZIP or RAR container.
Package archive extracts a subtitle from a ZIP or RAR container.
Package betaseries implements the BetaSeries subtitle provider.
Package betaseries implements the BetaSeries subtitle provider.
Package classify provides subtitle classification (forced/HI) and language resolution utilities consumed by provider sub-packages.
Package classify provides subtitle classification (forced/HI) and language resolution utilities consumed by provider sub-packages.
Package gestdown implements the Gestdown subtitle provider.
Package gestdown implements the Gestdown subtitle provider.
Package hdbits implements the HDBits.org subtitle provider.
Package hdbits implements the HDBits.org subtitle provider.
Package opensubtitles implements the OpenSubtitles.com REST API provider.
Package opensubtitles implements the OpenSubtitles.com REST API provider.
Package subdl implements the SubDL subtitle provider.
Package subdl implements the SubDL subtitle provider.
Package subsource implements the SubSource subtitle provider.
Package subsource implements the SubSource subtitle provider.
Package synthetic provides a subtitle provider that answers from generated data instead of an upstream service.
Package synthetic provides a subtitle provider that answers from generated data instead of an upstream service.
Package yifysubtitles implements the YIFY Subtitles provider.
Package yifysubtitles implements the YIFY Subtitles provider.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL