proxypool

package
v0.0.5 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 19, 2026 License: MIT Imports: 22 Imported by: 0

Documentation

Overview

Package proxypool implements the proxy pool service.

Responsibilities:

  • Storage of manual and list-sourced proxies
  • Health checks with immediate culling of dead list-sourced entries
  • Per-provider health memory (a proxy region-locked by one provider may still work for others)
  • Best-proxy selection given plugin location preferences

Index

Constants

View Source
const (
	SourceStatusIdle     = "idle"
	SourceStatusFetching = "fetching"
	SourceStatusChecking = "checking"
)

Source lifecycle statuses reported to the dashboard.

View Source
const ManualSource = "manual"

ManualSource marks user-registered proxies.

Variables

View Source
var ErrProxyPoolExhausted = errors.New("proxy pool exhausted")

ErrProxyPoolExhausted reports that every pooled proxy failed for one request; the wrapped cause is the last attempt's error.

Functions

func DetectExitCountry

func DetectExitCountry(ctx context.Context, proxyURL string) (string, error)

DetectExitCountry resolves the country a proxyegresses from by querying the detection endpoint through the proxy. It reports ground truth for the exit IP; list-supplied country metadata is only a seed value. A probe failure returns an error and the caller keeps the stored value.

func ListSource

func ListSource(typeKey string) string

ListSource builds the source tag for a list plugin.

func NormalizeCountryCode

func NormalizeCountryCode(raw string) string

NormalizeCountryCode canonicalizes a country code to uppercase ISO 3166-1 alpha-2. Alpha-3 input maps through the ISO table; anything else passes through uppercased so selection never mismatches on case alone.

func ParseProxyMode

func ParseProxyMode(providerConfig map[string]any) (mode string, ids []string)

ParseProxyMode reads the provider-level proxy config from ProviderInstance.Config. Unknown shapes fall back to disabled.

func ResolveProxy

func ResolveProxy(s *Service, mode string, ids []string, location string, forceOnMismatch bool, serverCountry, providerType string) (proxyID, proxyURL string, err error)

ResolveProxy picks the pool proxy for one provider call and reports it as (id, url). It returns ("", "", nil) when the call goes direct: proxying disabled, or auto mode with no usable proxy pooled. A nil proxy is a legitimate direct route, never a silent fallback: every explicit proxy demand that cannot be satisfied (manual pick, forced exit with an empty pool) fails loudly instead.

The plugin's preferred location is a preference, never a gate: matching proxies lead the ranking, others follow, and the retry layer falls through them when the upstream rejects one.

func TransportFor

func TransportFor(proxyURL string, timeout time.Duration) (*http.Transport, error)

TransportFor builds an http.Transport routing through the given proxy. Supports http, https, socks4 (with 4a remote DNS) and socks5.

Types

type Service

type Service struct {

	// CheckURL is the lightweight endpoint used for generic health checks.
	CheckURL string
	// DialTimeout bounds a single health-check dial.
	DialTimeout time.Duration
	// contains filtered or unexported fields
}

Service manages the proxy pool.

func New

func New(database *db.DB) *Service

New constructs the proxy pool service. List-sourced DB rows that were never verified alive are culled: under the streaming pipeline only verified proxies may occupy the bucket.

func (*Service) AddManual

func (s *Service) AddManual(rawURL, country string) (*models.Proxy, error)

AddManual registers a user-provided proxy URL.

func (*Service) BeginFetch added in v0.0.5

func (s *Service) BeginFetch(sourceKey string)

BeginFetch marks a source as fetching. Pair with RefreshSource on success or FailFetch on error.

func (*Service) Check

func (s *Service) Check(ctx context.Context, id string) (*models.Proxy, error)

Check probes one proxy through the generic check URL and records the outcome. Dead list-sourced proxies are culled immediately; manual entries are kept but marked not alive.

func (*Service) CheckAll

func (s *Service) CheckAll(ctx context.Context)

CheckAll probes every pooled proxy with a bounded worker pool. Geo detection rides along inside Check; both run checkConcurrency-wide.

func (*Service) Delete

func (s *Service) Delete(id string) error

Delete removes a proxy by ID.

func (*Service) FailFetch added in v0.0.5

func (s *Service) FailFetch(sourceKey string, err error)

FailFetch records a fetch error and returns the source to idle.

func (*Service) Get

func (s *Service) Get(id string) (*models.Proxy, error)

Get returns one proxy.

func (*Service) List

func (s *Service) List() ([]*models.Proxy, error)

List returns all proxies, manual first, then by latency.

func (*Service) PoolTotals added in v0.0.5

func (s *Service) PoolTotals() (total, alive int, err error)

PoolTotals reports the dashboard header counters: total is every proxy the pool knows about (manual entries plus the last fetched totals of all list sources), alive is the DB-level verified count.

func (*Service) RecordOutcome

func (s *Service) RecordOutcome(proxyID, providerType string, ok bool, latencyMs int64)

RecordOutcome stores per-provider health after a real routed request.

func (*Service) RefreshSource added in v0.0.5

func (s *Service) RefreshSource(sourceKey string, candidates []models.ProxyCandidate) (int, error)

RefreshSource stores the fetched list in RAM and enqueues the next check window. Only verified proxies are persisted; nothing unverified touches the DB. Candidates start flowing to workers as soon as the list arrives.

func (*Service) Select

func (s *Service) Select(prefs models.ProxyPreferences, providerType string) *models.Proxy

Select returns the best proxy for a provider given plugin preferences. Ranking: the freshest known-good proxy for this provider (sticky — repeat traffic keeps one working exit instead of re-walking the pool) > location match > known-good > latency. A proxy known-bad for this provider is demoted, never excluded: it stays a fallback when nothing better is pooled.

func (*Service) SelectManual

func (s *Service) SelectManual(ids []string, providerType string) *models.Proxy

SelectManual returns the first alive proxy from the given IDs. Proxies known-bad for this provider sink to the end of the order instead of being skipped: an explicit manual pick is still honored when nothing better answers.

func (*Service) SourceInfos added in v0.0.5

func (s *Service) SourceInfos(keys []string) []SourceInfo

SourceInfos snapshots the runtime state of the given source keys. Alive is the DB-level count of verified proxies per source, so the per-source numbers sum to the pool-wide totals the dashboard header reports. Fetch totals fall back to the persisted last-fetch meta when the RAM pipeline has not seen a fetch since startup.

func (*Service) SourceProxies added in v0.0.5

func (s *Service) SourceProxies(sourceKey string, offset, limit int) ([]*models.Proxy, int, error)

SourceProxies returns alive list-sourced proxies of one source, fastest first, paginated. The second return value is the total alive count.

func (*Service) StartWorkers added in v0.0.5

func (s *Service) StartWorkers(ctx context.Context)

StartWorkers launches the streaming check pipeline. Call once at startup; the workers stop with ctx.

type SourceInfo added in v0.0.5

type SourceInfo struct {
	Key         string    `json:"key"`
	Status      string    `json:"status"`
	Total       int       `json:"total"`
	Checked     int       `json:"checked"`
	Alive       int       `json:"alive"`
	Pending     int       `json:"pending"`
	LastFetchAt time.Time `json:"last_fetch_at,omitempty"`
	LastError   string    `json:"last_error,omitempty"`
}

SourceInfo is the dashboard-facing snapshot of one source.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL