Help us shape the future of Go! Take the 2026 Go Developer Survey

newznab

package
v0.3.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 3, 2026 License: GPL-3.0 Imports: 32 Imported by: 0

Documentation

Overview

Package newznab implements core.Indexer against Newznab-compatible Usenet indexers, with per-indexer daily budgets and a result cache.

Index

Constants

View Source 
const MaxRateLimitPause = maxRateLimitPause

MaxRateLimitPause is the shared cap on a service-requested pause, exported so HTTP handlers cap Retry-After exactly like this client instead of duplicating a literal bound.

Variables

This section is empty.

Functions

func DiscoverProwlarr 

func DiscoverProwlarr(ctx context.Context, cfg config.Prowlarr, opts Options) ([]config.Indexer, error)

DiscoverProwlarr imports enabled Usenet (Newznab) and torrent (Torznab) indexers; torrent indexers get Protocol "torrent" and stream through a debrid provider. Each receives a stable source-scoped ID and its own Ether hit/grab budget, cache and reputation. Prowlarr's /{id}/api alias is intentionally used so ordinary Newznab URL construction and all existing budget enforcement apply unchanged.

func InfoHashFromMagnet 

func InfoHashFromMagnet(magnet string) string

InfoHashFromMagnet extracts the v1 info hash from a magnet URI's xt=urn:btih: parameter; "" when absent.

func MagnetFor 

func MagnetFor(infoHash, name string, trackers []string) string

MagnetFor builds a magnet URI for an info hash.

func NormalizeInfoHash 

func NormalizeInfoHash(s string) string

NormalizeInfoHash returns a lower-case 40-character hex BitTorrent v1 info hash, converting a 32-character base32 hash; "" when s is neither.

func ProwlarrIDPrefix added in v0.2.34 

func ProwlarrIDPrefix(cfg config.Prowlarr) string

ProwlarrIDPrefix returns the ID prefix of indexers imported from cfg's Prowlarr instance ("prowlarr-<scope>-"), or "" when the URL is not a usable base URL (same validation as discovery).

func Test 

func Test(ctx context.Context, cfg config.Indexer, policy config.ClientIdentity) error

Test verifies an indexer config for the setup wizard: caps plus one search, with human-readable errors. policy is the client identity the probe presents, so the probe matches the traffic the indexer will see.

func TestWithOptions 

func TestWithOptions(ctx context.Context, cfg config.Indexer, opts Options) error

TestWithOptions is Test with injectable transport/clock (tests).

Types

type APIError 

type APIError struct {
	Code        int
	Description string
	Budget      bool // daily limit / rate limited; unwraps to core.ErrBudgetExhausted
}

APIError is a typed Newznab error document (<error code=".." description=".."/>) or an HTTP-level failure translated into the same shape.

func (*APIError) Error 

func (e *APIError) Error() string

func (*APIError) Unwrap 

func (e *APIError) Unwrap() error

type Backend 

type Backend interface {
	Search(ctx context.Context, req core.SearchRequest, reserve func(context.Context, string) error) ([]core.Release, error)
	FetchNZB(ctx context.Context, r core.Release, reserve func(context.Context, string) error) ([]byte, error)
	ResolveTorrent(ctx context.Context, r core.Release, reserve func(context.Context, string) error) (core.Release, error)
}

Backend searches a site that is not a Newznab API, such as a built-in site run from an owner-installed definition. The Client keeps budgets, the result cache and events around it: a backend calls reserve with "hit" or "grab" before each request that counts against a daily budget.

type Caps 

type Caps struct {
	ServerTitle  string   `json:"serverTitle,omitempty"`
	ServerType   string   `json:"serverType,omitempty"`
	LimitDefault int      `json:"limitDefault,omitempty"`
	LimitMax     int      `json:"limitMax,omitempty"`
	SearchAvail  bool     `json:"searchAvailable"`
	SearchParams []string `json:"searchParams,omitempty"`
	MovieAvail   bool     `json:"movieSearchAvailable"`
	MovieParams  []string `json:"movieParams,omitempty"`
	TVAvail      bool     `json:"tvSearchAvailable"`
	TVParams     []string `json:"tvParams,omitempty"`
	Categories   []int    `json:"categories,omitempty"`
}

Caps is a parsed Newznab capabilities document.

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client is one configured Newznab indexer. It implements core.Indexer.

func New 

func New(cfg config.Indexer, db *sql.DB, opts Options) *Client

New builds a Client for one indexer. db may be nil (budgets and cache then live in memory).

func NewClient 

func NewClient(cfg config.Indexer, db *sql.DB, opts Options) *Client

NewClient is an alias for New.

func (*Client) Caps 

func (c *Client) Caps(ctx context.Context) (*Caps, error)

Caps returns the indexer's capabilities, cached for 24h per endpoint (URL and API key): an indexer edited to point elsewhere fetches anew, and a response from the old endpoint still in flight is not kept.

func (*Client) Config 

func (c *Client) Config() config.Indexer

Config returns the indexer's current config.

func (*Client) Exhausted 

func (c *Client) Exhausted(ctx context.Context) bool

Exhausted reports whether either daily budget is spent.

func (*Client) FetchNZB 

func (c *Client) FetchNZB(ctx context.Context, r core.Release) ([]byte, error)

FetchNZB implements core.Indexer. Every call returns a slice of its own, which the caller may keep or modify.

func (*Client) Name 

func (c *Client) Name() string

Name implements core.Indexer.

func (*Client) Remaining 

func (c *Client) Remaining(ctx context.Context) (Remaining, error)

Remaining returns today's budget state.

func (*Client) ResolveTorrent 

func (c *Client) ResolveTorrent(ctx context.Context, r core.Release) (core.Release, error)

ResolveTorrent fills a torrent release's InfoHash and MagnetURL by downloading its .torrent link (counting as a grab) when the indexer only supplied a download URL. Releases that already carry both are returned unchanged without a request.

func (*Client) Search 

func (c *Client) Search(ctx context.Context, req core.SearchRequest) ([]core.Release, error)

Search implements core.Indexer.

func (*Client) SearchRescue added in v0.2.4 

func (c *Client) SearchRescue(ctx context.Context, req core.SearchRequest) ([]core.Release, error)

SearchRescue runs broader keyword, alias, OriginalTitle, miniseries and season-pack queries when the primary Search results had no playable candidate. Results are cached under a separate rescue key and respect the indexer's daily budget and rate-limit state.

func (*Client) SearchWith added in v0.2.9 

func (c *Client) SearchWith(ctx context.Context, req core.SearchRequest, arrived func([]core.Release)) ([]core.Release, error)

SearchWith delivers complete items while the HTTP body is still arriving. Cache/backend answers use the same callback; each ID is emitted once even when fallback queries overlap. The cache only receives complete responses.

func (*Client) SetCacheTTL 

func (c *Client) SetCacheTTL(d time.Duration)

SetCacheTTL changes the result-cache TTL (called on config reload). A negative value disables the cache.

func (*Client) SetClientIdentity added in v0.3.2 

func (c *Client) SetClientIdentity(p config.ClientIdentity)

SetClientIdentity resolves the owner's policy into the four identities this indexer presents, honoring the indexer's own override.

func (*Client) UpdateConfig 

func (c *Client) UpdateConfig(cfg config.Indexer)

UpdateConfig swaps the indexer config (called on config reload).

type Event 

type Event struct {
	Indexer         string
	Kind            string
	Success         bool
	BudgetExhausted bool
	Results         int
	Duration        time.Duration
	AverageAge      time.Duration
	At              time.Time
}

Event records one indexer's observed operation; no URLs or keys are exposed.

type Manager 

type Manager struct {
	// contains filtered or unexported fields
}

Manager holds all configured indexers and spreads searches across them within their budgets. It implements core.Indexer so the resolver can treat the fleet as one indexer; FetchNZB routes by the release's Indexer name.

func NewManager 

func NewManager(db *sql.DB, cfgs []config.Indexer, opts Options) *Manager

NewManager builds a Manager from indexer configs.

func (*Manager) ApplyConfig 

func (m *Manager) ApplyConfig(cfg config.Config)

ApplyConfig reloads indexers and the search-cache TTL from a full config snapshot; subscribe it to config.Store changes.

func (*Manager) CachedReleases added in v0.2.9 

func (m *Manager) CachedReleases(ctx context.Context) ([]core.Release, error)

CachedReleases reuses a bounded set of already-paid-for search/feed results. It performs no requests, capability discovery, cache refresh, or downloads.

func (*Manager) Clients 

func (m *Manager) Clients() []*Client

Clients returns the enabled and disabled clients in priority order.

func (*Manager) FeedSources added in v0.2.8 

func (m *Manager) FeedSources() []core.ReleaseFeed

FeedSources reports capabilities without making a request. Built-in sites remain searchable but are not treated as paginated feeds without that contract.

func (*Manager) FetchNZB 

func (m *Manager) FetchNZB(ctx context.Context, r core.Release) ([]byte, error)

FetchNZB implements core.Indexer by routing to the originating client.

func (*Manager) HasTorrentIndexer added in v0.2.20 

func (m *Manager) HasTorrentIndexer() bool

HasTorrentIndexer reports whether an enabled indexer returns torrents (Torznab, a Stremio addon or a torrent built-in site).

func (*Manager) Indexers 

func (m *Manager) Indexers() []core.Indexer

Indexers returns the clients as core.Indexer values.

func (*Manager) MissingSources added in v0.2.20 

func (m *Manager) MissingSources() error

MissingSources reports core.ErrNoSources when no indexer is enabled: a search would return nothing whatever the title.

func (*Manager) Name 

func (m *Manager) Name() string

Name implements core.Indexer.

func (*Manager) Prune 

func (m *Manager) Prune(ctx context.Context) (int64, error)

Prune deletes budget rows older than budgetRetentionDays and any expired search-cache rows (normally removed on write, but an idle indexer never writes). It is meant to be called periodically.

func (*Manager) PurgeLegacyCache added in v0.2.4 

func (m *Manager) PurgeLegacyCache(ctx context.Context) (int64, error)

PurgeLegacyCache deletes search-cache rows written before cached links were tokenized (they contain indexer API keys) and returns how many it removed. Call it once no node running an older release writes the cache.

func (*Manager) Recent added in v0.2.8 

func (m *Manager) Recent(ctx context.Context, id string, offset, limit int) ([]core.Release, error)

Recent fetches one page, charging the same account-wide limits as Search. It never invokes title searches, capability discovery, or implicit retries.

func (*Manager) Reload 

func (m *Manager) Reload(cfgs []config.Indexer)

Reload replaces the indexer set (call on config change). Existing clients with an unchanged ID keep their in-memory caches.

func (*Manager) Remaining 

func (m *Manager) Remaining(ctx context.Context) ([]Remaining, error)

Remaining returns budget state for every indexer.

func (*Manager) ResolveTorrent 

func (m *Manager) ResolveTorrent(ctx context.Context, r core.Release) (core.Release, error)

ResolveTorrent implements the debrid indexer's torrent resolution by routing to the release's originating indexer.

func (*Manager) Search 

func (m *Manager) Search(ctx context.Context, req core.SearchRequest) ([]core.Release, error)

Search implements core.Indexer: it queries enabled indexers in parallel (up to MaxParallel), letting each client check its cache before budgets, merges by priority order and de-duplicates by normalised name + size.

func (*Manager) SearchRescue added in v0.2.4 

func (m *Manager) SearchRescue(ctx context.Context, req core.SearchRequest) ([]core.Release, error)

SearchRescue runs a second-pass rescue fan-out across all enabled indexers when the primary Search results had no playable candidate.

func (*Manager) SearchRescueWith added in v0.2.4 

func (m *Manager) SearchRescueWith(ctx context.Context, req core.SearchRequest, arrived func([]core.Release)) ([]core.Release, error)

SearchRescueWith is SearchRescue with per-indexer early arrival callbacks.

func (*Manager) SearchTimings 

func (m *Manager) SearchTimings() []SearchTiming

SearchTimings returns a copy of the most recent per-indexer search timings.

func (*Manager) SearchWith added in v0.2.4 

func (m *Manager) SearchWith(ctx context.Context, req core.SearchRequest, arrived func([]core.Release)) ([]core.Release, error)

SearchWith is Search, also handing each indexer's results to arrived as soon as that indexer answers (concurrently, from its own goroutine), so callers can start follow-up work, such as debrid cache checks, before the slowest indexer is done.

func (*Manager) Subscribe 

func (m *Manager) Subscribe(fn func(Event)) func()

type Options 

type Options struct {
	HTTPClient     *http.Client
	Logger         *slog.Logger
	SearchCacheTTL time.Duration // <0 disables cache, 0 = default 6h
	NZBDir         string        // directory for downloaded NZBs; "" keeps them in memory only
	MaxNZBBytes    int64
	Now            func() time.Time // injected clock (tests)
	MaxParallel    int              // Manager fan-out; 0 dispatches every configured indexer
	// ClientIdentity is the owner's policy for the client Ether presents to
	// indexers; the zero value keeps the Ether identity.
	ClientIdentity config.ClientIdentity
	// Backend builds the search backend for an indexer whose Kind is not
	// Newznab (built-in sites, internal/cardigann). Nil disables them.
	Backend func(config.Indexer) (Backend, error)
}

Options tunes a Client or Manager. Zero values get sensible defaults.

type ProwlarrDiscovery added in v0.2.34 

type ProwlarrDiscovery struct {
	Indexers               []config.Indexer `json:"indexers"`
	SkippedUnknownProtocol int              `json:"skippedUnknownProtocol"`
	Warnings               []string         `json:"warnings,omitempty"`
	// PresentIDs are the scoped IDs of every enabled indexer the source
	// reported, including ones filtered out below (unknown protocol or,
	// when tags are set, a tag the owner did not choose). It is diagnostic
	// only. Tag sync decides what to stop using from Indexers, never from
	// this list, and an empty tag list stays additive.
	PresentIDs []string `json:"presentIds,omitempty"`
}

ProwlarrDiscovery is the result of a successful Prowlarr discovery: the importable indexers plus what was left out and why, so a preview can report skips instead of dropping them silently (same pattern as the quality-profile import's Skipped/Warnings).

func DiscoverProwlarrReport added in v0.2.34 

func DiscoverProwlarrReport(ctx context.Context, cfg config.Prowlarr, opts Options) (ProwlarrDiscovery, error)

DiscoverProwlarrReport is DiscoverProwlarr plus a count (and warning) for enabled indexers skipped because Prowlarr reported a protocol Ether cannot use.

It only reads Prowlarr's indexer list (GET /api/v1/indexer). It does not search or download, so indexer hit and grab quotas are not spent. cfg.Tags empty keeps every enabled Usenet and torrent indexer. With tags set, an indexer is kept only when its tags field shares one of those ids.

type ProwlarrTag added in v0.3.0 

type ProwlarrTag struct {
	ID    int    `json:"id"`
	Label string `json:"label"`
}

ProwlarrTag is one tag from Prowlarr's /api/v1/tag.

func ListProwlarrTags added in v0.3.0 

func ListProwlarrTags(ctx context.Context, cfg config.Prowlarr, opts Options) ([]ProwlarrTag, error)

ListProwlarrTags reads Prowlarr's tag catalogue (GET /api/v1/tag) so the owner can pick tags. It does not search or download.

type RateLimitError added in v0.2.17 

type RateLimitError struct {
	RetryAfter string
	Err        error
}

RateLimitError is a backend's rate-limit refusal (HTTP 429). The Client pauses the indexer until RetryAfter, a Retry-After header value ("" for the default pause), exactly as for a Newznab 429.

func (*RateLimitError) Error added in v0.2.17 

func (e *RateLimitError) Error() string

func (*RateLimitError) Unwrap added in v0.2.17 

func (e *RateLimitError) Unwrap() []error

type Remaining 

type Remaining struct {
	Indexer        string `json:"indexer"`
	Day            string `json:"day"`
	HitsUsed       int    `json:"hitsUsed"`
	HitsLimit      int    `json:"hitsLimit"`
	HitsRemaining  int    `json:"hitsRemaining"`
	GrabsUsed      int    `json:"grabsUsed"`
	GrabsLimit     int    `json:"grabsLimit"`
	GrabsRemaining int    `json:"grabsRemaining"`
	RetryAt        int64  `json:"retryAt,omitempty"` // service-reported throttling; automatic retry (Unix seconds)
	// Enabled and Torrent let the dashboard warn when nothing can be
	// searched, or when debrid has no torrent indexer to search.
	Enabled bool `json:"enabled"`
	Torrent bool `json:"torrent"` // Torznab, a Stremio addon or a torrent built-in site
}

Remaining reports one indexer's budget state for today (UTC). A remaining value of -1 means there is no personal cap; service limits still apply.

type SearchTiming 

type SearchTiming struct {
	Indexer  string
	Duration time.Duration
	Err      string
}

SearchTiming records the most recent aggregate search's per-indexer cost.

type TorrentMeta 

type TorrentMeta struct {
	InfoHash string
	Name     string
	Trackers []string
}

TorrentMeta is what a .torrent file contributes to a release.

func ParseTorrent 

func ParseTorrent(data []byte) (TorrentMeta, error)

ParseTorrent computes the info hash (SHA-1 of the bencoded info dictionary) and reads the name and trackers of a .torrent file.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.