Documentation
¶
Overview ¶
Package proxypool implements the proxy pool service.
Responsibilities:
- Storage of manual and list-sourced proxies
- Health checks with immediate culling of dead list-sourced entries
- Per-provider health memory (a proxy region-locked by one provider may still work for others)
- Best-proxy selection given plugin location preferences
Index ¶
- Constants
- Variables
- func DetectExitCountry(ctx context.Context, proxyURL string) (string, error)
- func ListSource(typeKey string) string
- func NormalizeCountryCode(raw string) string
- func ParseProxyMode(providerConfig map[string]any) (mode string, ids []string)
- func ResolveProxy(s *Service, mode string, ids []string, location string, forceOnMismatch bool, ...) (proxyID, proxyURL string, err error)
- func TransportFor(proxyURL string, timeout time.Duration) (*http.Transport, error)
- type Service
- func (s *Service) AddManual(rawURL, country string) (*models.Proxy, error)
- func (s *Service) BeginFetch(sourceKey string)
- func (s *Service) Check(ctx context.Context, id string) (*models.Proxy, error)
- func (s *Service) CheckAll(ctx context.Context)
- func (s *Service) Delete(id string) error
- func (s *Service) FailFetch(sourceKey string, err error)
- func (s *Service) Get(id string) (*models.Proxy, error)
- func (s *Service) List() ([]*models.Proxy, error)
- func (s *Service) PoolTotals() (total, alive int, err error)
- func (s *Service) RecordOutcome(proxyID, providerType string, ok bool, latencyMs int64)
- func (s *Service) RefreshSource(sourceKey string, candidates []models.ProxyCandidate) (int, error)
- func (s *Service) Select(prefs models.ProxyPreferences, providerType string) *models.Proxy
- func (s *Service) SelectManual(ids []string, providerType string) *models.Proxy
- func (s *Service) SourceInfos(keys []string) []SourceInfo
- func (s *Service) SourceProxies(sourceKey string, offset, limit int) ([]*models.Proxy, int, error)
- func (s *Service) StartWorkers(ctx context.Context)
- type SourceInfo
Constants ¶
const ( SourceStatusIdle = "idle" SourceStatusFetching = "fetching" SourceStatusChecking = "checking" )
Source lifecycle statuses reported to the dashboard.
const ManualSource = "manual"
ManualSource marks user-registered proxies.
Variables ¶
var ErrProxyPoolExhausted = errors.New("proxy pool exhausted")
ErrProxyPoolExhausted reports that every pooled proxy failed for one request; the wrapped cause is the last attempt's error.
Functions ¶
func DetectExitCountry ¶
DetectExitCountry resolves the country a proxyegresses from by querying the detection endpoint through the proxy. It reports ground truth for the exit IP; list-supplied country metadata is only a seed value. A probe failure returns an error and the caller keeps the stored value.
func ListSource ¶
ListSource builds the source tag for a list plugin.
func NormalizeCountryCode ¶
NormalizeCountryCode canonicalizes a country code to uppercase ISO 3166-1 alpha-2. Alpha-3 input maps through the ISO table; anything else passes through uppercased so selection never mismatches on case alone.
func ParseProxyMode ¶
ParseProxyMode reads the provider-level proxy config from ProviderInstance.Config. Unknown shapes fall back to disabled.
func ResolveProxy ¶
func ResolveProxy(s *Service, mode string, ids []string, location string, forceOnMismatch bool, serverCountry, providerType string) (proxyID, proxyURL string, err error)
ResolveProxy picks the pool proxy for one provider call and reports it as (id, url). It returns ("", "", nil) when the call goes direct: proxying disabled, or auto mode with no usable proxy pooled. A nil proxy is a legitimate direct route, never a silent fallback: every explicit proxy demand that cannot be satisfied (manual pick, forced exit with an empty pool) fails loudly instead.
The plugin's preferred location is a preference, never a gate: matching proxies lead the ranking, others follow, and the plugin's geo-rotation falls through them when the upstream rejects one.
Types ¶
type Service ¶
type Service struct {
// CheckURL is the lightweight endpoint used for generic health checks.
CheckURL string
// DialTimeout bounds a single health-check dial.
DialTimeout time.Duration
// contains filtered or unexported fields
}
Service manages the proxy pool.
func New ¶
New constructs the proxy pool service. List-sourced DB rows that were never verified alive are culled: under the streaming pipeline only verified proxies may occupy the bucket.
func (*Service) BeginFetch ¶ added in v0.0.5
BeginFetch marks a source as fetching. Pair with RefreshSource on success or FailFetch on error.
func (*Service) Check ¶
Check probes one proxy through the generic check URL and records the outcome. Dead list-sourced proxies are culled immediately; manual entries are kept but marked not alive.
func (*Service) CheckAll ¶
CheckAll probes every pooled proxy with a bounded worker pool. Geo detection rides along inside Check; both run checkConcurrency-wide.
func (*Service) FailFetch ¶ added in v0.0.5
FailFetch records a fetch error and returns the source to idle.
func (*Service) PoolTotals ¶ added in v0.0.5
PoolTotals reports the dashboard header counters: total is every proxy the pool knows about (manual entries plus the last fetched totals of all list sources), alive is the DB-level verified count.
func (*Service) RecordOutcome ¶
RecordOutcome stores per-provider health after a real routed request.
func (*Service) RefreshSource ¶ added in v0.0.5
RefreshSource stores the fetched list in RAM and enqueues the next check window. Only verified proxies are persisted; nothing unverified touches the DB. Candidates start flowing to workers as soon as the list arrives.
func (*Service) Select ¶
Select returns the best proxy for a provider given plugin preferences. Ranking: the freshest known-good proxy for this provider (sticky — repeat traffic keeps one working exit instead of re-walking the pool) > location match > known-good > latency. A proxy known-bad for this provider is demoted, never excluded: it stays a fallback when nothing better is pooled.
func (*Service) SelectManual ¶
SelectManual returns the first alive proxy from the given IDs. Proxies known-bad for this provider sink to the end of the order instead of being skipped: an explicit manual pick is still honored when nothing better answers.
func (*Service) SourceInfos ¶ added in v0.0.5
func (s *Service) SourceInfos(keys []string) []SourceInfo
SourceInfos snapshots the runtime state of the given source keys. Alive is the DB-level count of verified proxies per source, so the per-source numbers sum to the pool-wide totals the dashboard header reports. Fetch totals fall back to the persisted last-fetch meta when the RAM pipeline has not seen a fetch since startup.
func (*Service) SourceProxies ¶ added in v0.0.5
SourceProxies returns alive list-sourced proxies of one source, fastest first, paginated. The second return value is the total alive count.
func (*Service) StartWorkers ¶ added in v0.0.5
StartWorkers launches the streaming check pipeline. Call once at startup; the workers stop with ctx.
type SourceInfo ¶ added in v0.0.5
type SourceInfo struct {
Key string `json:"key"`
Status string `json:"status"`
Total int `json:"total"`
Checked int `json:"checked"`
Alive int `json:"alive"`
Pending int `json:"pending"`
LastFetchAt time.Time `json:"last_fetch_at,omitempty"`
LastError string `json:"last_error,omitempty"`
}
SourceInfo is the dashboard-facing snapshot of one source.