Documentation
¶
Overview ¶
Package proxypool implements the outbound proxy pool.
Design: two-stage probe (CONNECT handshake, download speed), one live state per proxy-provider pair (rate limit, block), demand-driven fetch. Presence in the bucket means the proxy answered the last probe; dead proxies are deleted, never flagged.
Index ¶
- Constants
- Variables
- func DetectExitCountry(ctx context.Context, proxyURL string) (string, error)
- func ListSource(typeKey string) string
- func NormalizeCountryCode(raw string) string
- func TransportFor(proxyURL string, timeout time.Duration) (*http.Transport, error)
- type Pick
- type Service
- func (s *Service) AddCandidates(ctx context.Context, sourceKey string, candidates []models.ProxyCandidate) error
- func (s *Service) AddManual(rawURL, location string) (*models.Proxy, error)
- func (s *Service) Choose(whitelist []string, mode string, ids []string, provider string) (string, string, error)
- func (s *Service) Delete(id string) error
- func (s *Service) Get(id string) (*models.Proxy, error)
- func (s *Service) List() ([]*models.Proxy, error)
- func (s *Service) NeedsSearch() bool
- func (s *Service) NoteDemand(regions []string)
- func (s *Service) PoolTotals() (int, error)
- func (s *Service) Rank(whitelist []string, mode string, ids []string, provider string) ([]Pick, error)
- func (s *Service) RecordBlocked(proxyID, provider, reason string)
- func (s *Service) RecordRateLimit(proxyID, provider string, resetsAt time.Time)
- func (s *Service) RotateAll(ctx context.Context) error
- func (s *Service) SetConfig(minSpeedKbps int64, maxPerLocation int)
- func (s *Service) SetSourceDone(sourceKey string, total int)
- func (s *Service) SetSourceFailed(sourceKey string, err error)
- func (s *Service) SetSourceFetching(sourceKey string)
- func (s *Service) SourceInfos(keys []string) []SourceInfo
- func (s *Service) SourceProxies(sourceKey string, offset, limit int) ([]*models.Proxy, int, error)
- func (s *Service) UpdateOne(ctx context.Context, id string) (*models.Proxy, error)
- type SourceInfo
Constants ¶
const ( SourceStatusIdle = "idle" SourceStatusFetching = "fetching" SourceStatusAdding = "adding" )
Source lifecycle statuses reported to the dashboard.
const ManualSource = "manual"
ManualSource marks user-registered proxies.
Variables ¶
var DefaultRegions = []string{"US", "DE", "NL", "GB", "FR", "CA"}
DefaultRegions seeds demand so the pool fills even before the first whitelisted request arrives.
var ErrBusy = errors.New("proxypool: rotation already in progress")
ErrBusy reports a rotation already in progress. Rotation runs under a semaphore of size one: a tick or a manual refresh arriving while another pass runs is skipped, never queued.
Functions ¶
func DetectExitCountry ¶
DetectExitCountry resolves the country a proxyegresses from by querying the detection endpoint through the proxy. It reports ground truth for the exit IP; list-supplied country metadata is only a seed value. A probe failure returns an error and the caller keeps the stored value.
func ListSource ¶
ListSource builds the source tag for a list plugin.
func NormalizeCountryCode ¶
NormalizeCountryCode canonicalizes a country code to uppercase ISO 3166-1 alpha-2. Alpha-3 input maps through the ISO table; anything else passes through uppercased so selection never mismatches on case alone.
Types ¶
type Service ¶
type Service struct {
// CheckURL is the speed test download endpoint.
CheckURL string
// HandshakeHost is the CONNECT tunnel target (host:port).
HandshakeHost string
// contains filtered or unexported fields
}
Service manages the proxy pool.
func New ¶
New constructs the proxy pool service with default pool settings. Tune with SetConfig once RouterConfiguration is loaded.
func (*Service) AddCandidates ¶ added in v0.0.7
func (s *Service) AddCandidates(ctx context.Context, sourceKey string, candidates []models.ProxyCandidate) error
AddCandidates adds fetched list candidates under the rotation semaphore. Candidates outside demand are skipped before probing; coverage proceeds in chunks while shortfall persists, capped at one window per fetch, so a full pool costs zero probes. A concurrent rotation is skipped with ErrBusy, never queued.
func (*Service) AddManual ¶
AddManual registers a user-provided proxy URL. An existing URL is re-probed instead of duplicated. Dead entries are skipped; slow entries are skipped once their location is over the cap.
func (*Service) Choose ¶ added in v0.0.7
func (s *Service) Choose(whitelist []string, mode string, ids []string, provider string) (string, string, error)
Choose returns the fastest usable proxy, or ("", "", nil) for direct.
func (*Service) NeedsSearch ¶ added in v0.0.7
NeedsSearch reports whether any demanded region lacks fast proxies. True means the next scheduled fetch should run; false pauses it.
func (*Service) NoteDemand ¶ added in v0.0.7
NoteDemand records whitelisted regions as demanded. Throttled to one bucket write per region per hour; reads stay in the bucket.
func (*Service) PoolTotals ¶ added in v0.0.5
PoolTotals reports the pooled proxy count.
func (*Service) Rank ¶ added in v0.0.7
func (s *Service) Rank(whitelist []string, mode string, ids []string, provider string) ([]Pick, error)
Rank returns the ordered proxy list for a provider call: whitelist and pair-state filtered, fastest first. Disabled mode returns nil (direct). Manual mode follows the ids order and fails loudly when nothing usable is pooled. An empty whitelist allows any location.
func (*Service) RecordBlocked ¶ added in v0.0.7
RecordBlocked blocks a proxy for a provider after a geo-block answer. Blocks never expire on their own; the pair dies with the proxy.
func (*Service) RecordRateLimit ¶ added in v0.0.7
RecordRateLimit records an upstream rate-limit for a proxy-provider pair. The pair is skipped until resetsAt passes.
func (*Service) RotateAll ¶ added in v0.0.7
RotateAll re-probes every pooled proxy and trims each location to the fastest N. It runs under a semaphore of size one: a concurrent rotation is skipped with ErrBusy, never queued.
func (*Service) SetConfig ¶ added in v0.0.7
SetConfig installs the pool settings from RouterConfiguration.
func (*Service) SetSourceDone ¶ added in v0.0.7
SetSourceDone returns a source to idle without an error, keeping the fetched total. Used when the adds succeeded but a later step did not run.
func (*Service) SetSourceFailed ¶ added in v0.0.7
SetSourceFailed records a source fetch error for the dashboard.
func (*Service) SetSourceFetching ¶ added in v0.0.7
SetSourceFetching marks a source fetch in progress for the dashboard.
func (*Service) SourceInfos ¶ added in v0.0.5
func (s *Service) SourceInfos(keys []string) []SourceInfo
SourceInfos snapshots the runtime state of the given source keys.
func (*Service) SourceProxies ¶ added in v0.0.5
SourceProxies returns pooled proxies of one source, fastest first.
type SourceInfo ¶ added in v0.0.5
type SourceInfo struct {
Key string `json:"key"`
Status string `json:"status"`
Total int `json:"total"`
Pooled int `json:"pooled"`
LastFetchAt time.Time `json:"last_fetch_at,omitempty"`
LastError string `json:"last_error,omitempty"`
}
SourceInfo is the dashboard-facing snapshot of one source.