newznab

package
v0.2.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 21, 2026 License: GPL-3.0 Imports: 27 Imported by: 0

Documentation

Overview

Package newznab implements core.Indexer against Newznab-compatible Usenet indexers, with per-indexer daily budgets and a result cache.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func DiscoverProwlarr 

func DiscoverProwlarr(ctx context.Context, cfg config.Prowlarr, opts Options) ([]config.Indexer, error)

DiscoverProwlarr imports enabled Usenet (Newznab) and torrent (Torznab) indexers; torrent indexers get Protocol "torrent" and stream through a debrid provider. Each receives a stable source-scoped ID and its own Ether hit/grab budget, cache and reputation. Prowlarr's /{id}/api alias is intentionally used so ordinary Newznab URL construction and all existing budget enforcement apply unchanged.

func InfoHashFromMagnet 

func InfoHashFromMagnet(magnet string) string

InfoHashFromMagnet extracts the v1 info hash from a magnet URI's xt=urn:btih: parameter; "" when absent.

func MagnetFor 

func MagnetFor(infoHash, name string, trackers []string) string

MagnetFor builds a magnet URI for an info hash.

func NormalizeInfoHash 

func NormalizeInfoHash(s string) string

NormalizeInfoHash returns a lower-case 40-character hex BitTorrent v1 info hash, converting a 32-character base32 hash; "" when s is neither.

func Test 

func Test(ctx context.Context, cfg config.Indexer) error

Test verifies an indexer config for the setup wizard: caps plus one search, with human-readable errors.

func TestWithOptions 

func TestWithOptions(ctx context.Context, cfg config.Indexer, opts Options) error

TestWithOptions is Test with injectable transport/clock (tests).

Types

type APIError 

type APIError struct {
	Code        int
	Description string
	Budget      bool // daily limit / rate limited; unwraps to core.ErrBudgetExhausted
}

APIError is a typed Newznab error document (<error code=".." description=".."/>) or an HTTP-level failure translated into the same shape.

func (*APIError) Error 

func (e *APIError) Error() string

func (*APIError) Unwrap 

func (e *APIError) Unwrap() error

type Backend 

type Backend interface {
	Search(ctx context.Context, req core.SearchRequest, reserve func(context.Context, string) error) ([]core.Release, error)
	FetchNZB(ctx context.Context, r core.Release, reserve func(context.Context, string) error) ([]byte, error)
	ResolveTorrent(ctx context.Context, r core.Release, reserve func(context.Context, string) error) (core.Release, error)
}

Backend searches a site that is not a Newznab API, such as a built-in site run from an owner-installed definition. The Client keeps budgets, the result cache and events around it: a backend calls reserve with "hit" or "grab" before each request that counts against a daily budget.

type Caps 

type Caps struct {
	ServerTitle  string   `json:"serverTitle,omitempty"`
	ServerType   string   `json:"serverType,omitempty"`
	LimitDefault int      `json:"limitDefault,omitempty"`
	LimitMax     int      `json:"limitMax,omitempty"`
	SearchAvail  bool     `json:"searchAvailable"`
	SearchParams []string `json:"searchParams,omitempty"`
	MovieAvail   bool     `json:"movieSearchAvailable"`
	MovieParams  []string `json:"movieParams,omitempty"`
	TVAvail      bool     `json:"tvSearchAvailable"`
	TVParams     []string `json:"tvParams,omitempty"`
	Categories   []int    `json:"categories,omitempty"`
}

Caps is a parsed Newznab capabilities document.

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client is one configured Newznab indexer. It implements core.Indexer.

func New 

func New(cfg config.Indexer, db *sql.DB, opts Options) *Client

New builds a Client for one indexer. db may be nil (budgets and cache then live in memory).

func NewClient 

func NewClient(cfg config.Indexer, db *sql.DB, opts Options) *Client

NewClient is an alias for New.

func (*Client) Caps 

func (c *Client) Caps(ctx context.Context) (*Caps, error)

Caps returns the indexer's capabilities, cached for 24h per endpoint (URL and API key): an indexer edited to point elsewhere fetches anew, and a response from the old endpoint still in flight is not kept.

func (*Client) Config 

func (c *Client) Config() config.Indexer

Config returns the indexer's current config.

func (*Client) Exhausted 

func (c *Client) Exhausted(ctx context.Context) bool

Exhausted reports whether either daily budget is spent.

func (*Client) FetchNZB 

func (c *Client) FetchNZB(ctx context.Context, r core.Release) ([]byte, error)

FetchNZB implements core.Indexer.

func (*Client) Name 

func (c *Client) Name() string

Name implements core.Indexer.

func (*Client) Remaining 

func (c *Client) Remaining(ctx context.Context) (Remaining, error)

Remaining returns today's budget state.

func (*Client) ResolveTorrent 

func (c *Client) ResolveTorrent(ctx context.Context, r core.Release) (core.Release, error)

ResolveTorrent fills a torrent release's InfoHash and MagnetURL by downloading its .torrent link (counting as a grab) when the indexer only supplied a download URL. Releases that already carry both are returned unchanged without a request.

func (*Client) Search 

func (c *Client) Search(ctx context.Context, req core.SearchRequest) ([]core.Release, error)

Search implements core.Indexer.

func (*Client) SearchRescue added in v0.2.4 

func (c *Client) SearchRescue(ctx context.Context, req core.SearchRequest) ([]core.Release, error)

SearchRescue runs broader keyword, alias, OriginalTitle, miniseries and season-pack queries when the primary Search results had no playable candidate. Results are cached under a separate rescue key and respect the indexer's daily budget and rate-limit state.

func (*Client) SetCacheTTL 

func (c *Client) SetCacheTTL(d time.Duration)

SetCacheTTL changes the result-cache TTL (called on config reload). A negative value disables the cache.

func (*Client) UpdateConfig 

func (c *Client) UpdateConfig(cfg config.Indexer)

UpdateConfig swaps the indexer config (called on config reload).

type Event 

type Event struct {
	Indexer         string
	Kind            string
	Success         bool
	BudgetExhausted bool
	Results         int
	Duration        time.Duration
	AverageAge      time.Duration
	At              time.Time
}

Event records one indexer's observed operation; no URLs or keys are exposed.

type Manager 

type Manager struct {
	// contains filtered or unexported fields
}

Manager holds all configured indexers and spreads searches across them within their budgets. It implements core.Indexer so the resolver can treat the fleet as one indexer; FetchNZB routes by the release's Indexer name.

func NewManager 

func NewManager(db *sql.DB, cfgs []config.Indexer, opts Options) *Manager

NewManager builds a Manager from indexer configs.

func (*Manager) ApplyConfig 

func (m *Manager) ApplyConfig(cfg config.Config)

ApplyConfig reloads indexers and the search-cache TTL from a full config snapshot; subscribe it to config.Store changes.

func (*Manager) Clients 

func (m *Manager) Clients() []*Client

Clients returns the enabled and disabled clients in priority order.

func (*Manager) FetchNZB 

func (m *Manager) FetchNZB(ctx context.Context, r core.Release) ([]byte, error)

FetchNZB implements core.Indexer by routing to the originating client.

func (*Manager) Indexers 

func (m *Manager) Indexers() []core.Indexer

Indexers returns the clients as core.Indexer values.

func (*Manager) Name 

func (m *Manager) Name() string

Name implements core.Indexer.

func (*Manager) Prune 

func (m *Manager) Prune(ctx context.Context) (int64, error)

Prune deletes budget rows older than budgetRetentionDays and any expired search-cache rows (normally removed on write, but an idle indexer never writes). It is meant to be called periodically.

func (*Manager) PurgeLegacyCache added in v0.2.4 

func (m *Manager) PurgeLegacyCache(ctx context.Context) (int64, error)

PurgeLegacyCache deletes search-cache rows written before cached links were tokenized (they contain indexer API keys) and returns how many it removed. Call it once no node running an older release writes the cache.

func (*Manager) Reload 

func (m *Manager) Reload(cfgs []config.Indexer)

Reload replaces the indexer set (call on config change). Existing clients with an unchanged ID keep their in-memory caches.

func (*Manager) Remaining 

func (m *Manager) Remaining(ctx context.Context) ([]Remaining, error)

Remaining returns budget state for every indexer.

func (*Manager) ResolveTorrent 

func (m *Manager) ResolveTorrent(ctx context.Context, r core.Release) (core.Release, error)

ResolveTorrent implements the debrid indexer's torrent resolution by routing to the release's originating indexer.

func (*Manager) Search 

func (m *Manager) Search(ctx context.Context, req core.SearchRequest) ([]core.Release, error)

Search implements core.Indexer: it queries enabled indexers in parallel (up to MaxParallel), letting each client check its cache before budgets, merges by priority order and de-duplicates by normalised name + size.

func (*Manager) SearchRescue added in v0.2.4 

func (m *Manager) SearchRescue(ctx context.Context, req core.SearchRequest) ([]core.Release, error)

SearchRescue runs a second-pass rescue fan-out across all enabled indexers when the primary Search results had no playable candidate.

func (*Manager) SearchRescueWith added in v0.2.4 

func (m *Manager) SearchRescueWith(ctx context.Context, req core.SearchRequest, arrived func([]core.Release)) ([]core.Release, error)

SearchRescueWith is SearchRescue with per-indexer early arrival callbacks.

func (*Manager) SearchTimings 

func (m *Manager) SearchTimings() []SearchTiming

SearchTimings returns a copy of the most recent per-indexer search timings.

func (*Manager) SearchWith added in v0.2.4 

func (m *Manager) SearchWith(ctx context.Context, req core.SearchRequest, arrived func([]core.Release)) ([]core.Release, error)

SearchWith is Search, also handing each indexer's results to arrived as soon as that indexer answers (concurrently, from its own goroutine), so callers can start follow-up work, such as debrid cache checks, before the slowest indexer is done.

func (*Manager) Subscribe 

func (m *Manager) Subscribe(fn func(Event)) func()

type Options 

type Options struct {
	HTTPClient     *http.Client
	Logger         *slog.Logger
	SearchCacheTTL time.Duration // <0 disables cache, 0 = default 6h
	NZBDir         string        // directory for downloaded NZBs; "" keeps them in memory only
	MaxNZBBytes    int64
	Now            func() time.Time // injected clock (tests)
	MaxParallel    int              // Manager fan-out; 0 = 4
	// Backend builds the search backend for an indexer whose Kind is not
	// Newznab (built-in sites, internal/cardigann). Nil disables them.
	Backend func(config.Indexer) (Backend, error)
}

Options tunes a Client or Manager. Zero values get sensible defaults.

type Remaining 

type Remaining struct {
	Indexer        string `json:"indexer"`
	Day            string `json:"day"`
	HitsUsed       int    `json:"hitsUsed"`
	HitsLimit      int    `json:"hitsLimit"`
	HitsRemaining  int    `json:"hitsRemaining"`
	GrabsUsed      int    `json:"grabsUsed"`
	GrabsLimit     int    `json:"grabsLimit"`
	GrabsRemaining int    `json:"grabsRemaining"`
	RetryAt        int64  `json:"retryAt,omitempty"` // service-reported throttling; automatic retry (Unix seconds)
}

Remaining reports one indexer's budget state for today (UTC). A remaining value of -1 means there is no personal cap; service limits still apply.

type SearchTiming 

type SearchTiming struct {
	Indexer  string
	Duration time.Duration
	Err      string
}

SearchTiming records the most recent aggregate search's per-indexer cost.

type TorrentMeta 

type TorrentMeta struct {
	InfoHash string
	Name     string
	Trackers []string
}

TorrentMeta is what a .torrent file contributes to a release.

func ParseTorrent 

func ParseTorrent(data []byte) (TorrentMeta, error)

ParseTorrent computes the info hash (SHA-1 of the bencoded info dictionary) and reads the name and trackers of a .torrent file.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.