proxypool

package
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 27, 2026 License: MIT Imports: 23 Imported by: 0

Documentation

Overview

Package proxypool implements the outbound proxy pool.

Design: two-stage probe (CONNECT handshake, download speed), one live state per proxy-provider pair (rate limit, block), demand-driven fetch. Presence in the bucket means the proxy answered the last probe; dead proxies are deleted, never flagged.

Index

Constants

View Source
const (
	SourceStatusIdle     = "idle"
	SourceStatusFetching = "fetching"
	SourceStatusAdding   = "adding"
	SourceStatusRotating = "rotating"
)

Source lifecycle statuses reported to the dashboard.

View Source
const ManualSource = "manual"

ManualSource marks user-registered proxies.

Variables

View Source
var DefaultRegions = []string{"US", "DE", "NL", "GB", "FR", "CA"}

DefaultRegions seeds demand so the pool fills even before the first whitelisted request arrives.

View Source
var ErrBusy = errors.New("proxypool: rotation already in progress")

ErrBusy reports a rotation already in progress. Rotation runs under a semaphore of size one: a tick or a manual refresh arriving while another pass runs is skipped, never queued.

View Source
var ErrNoProxies = errors.New("proxypool: no usable proxy")

ErrNoProxies reports a settled pool with no usable pick: nothing is running that could add one, so waiting longer is pointless.

Functions

func DetectExitCountry

func DetectExitCountry(ctx context.Context, proxyURL string) (string, error)

DetectExitCountry resolves the country a proxyegresses from by querying the detection endpoint through the proxy. It reports ground truth for the exit IP; list-supplied country metadata is only a seed value. A probe failure returns an error and the caller keeps the stored value.

func ListSource

func ListSource(typeKey string) string

ListSource builds the source tag for a list plugin.

func NormalizeCountryCode

func NormalizeCountryCode(raw string) string

NormalizeCountryCode canonicalizes a country code to uppercase ISO 3166-1 alpha-2. Alpha-3 input maps through the ISO table; anything else passes through uppercased so selection never mismatches on case alone.

func TransportFor

func TransportFor(proxyURL string, timeout time.Duration) (*http.Transport, error)

TransportFor builds an http.Transport routing through the given proxy. Supports http, https, socks4 (with 4a remote DNS) and socks5.

Types

type Pick added in v0.0.7

type Pick struct {
	ID       string
	URL      string
	Location string
}

Pick is one ranked proxy candidate.

type Service

type Service struct {

	// CheckURL is the speed test download endpoint.
	CheckURL string
	// HandshakeHost is the CONNECT tunnel target (host:port).
	HandshakeHost string
	// contains filtered or unexported fields
}

Service manages the proxy pool.

func New

func New(database *db.DB) *Service

New constructs the proxy pool service with default pool settings. Tune with SetConfig once RouterConfiguration is loaded.

func (*Service) AddCandidates added in v0.0.7

func (s *Service) AddCandidates(ctx context.Context, sourceKey string, candidates []models.ProxyCandidate) error

AddCandidates adds fetched list candidates under the rotation semaphore. Candidates outside demand are skipped before probing; coverage proceeds in chunks while shortfall persists, capped at one window per fetch, so a full pool costs zero probes. A concurrent rotation is skipped with ErrBusy, never queued. The caller owns the final source transition: status stays at adding on return.

func (*Service) AddManual

func (s *Service) AddManual(rawURL, location string) (*models.Proxy, error)

AddManual registers a user-provided proxy URL. An existing URL is re-probed instead of duplicated. Dead entries are skipped; slow entries are skipped once their location is over the cap.

func (*Service) Checking added in v0.1.0

func (s *Service) Checking() bool

Checking reports whether probe work is running right now (a rotation or candidate adds). Dashboard polling keys off it.

func (*Service) Choose added in v0.0.7

func (s *Service) Choose(whitelist []string, mode string, ids []string, provider string) (string, string, error)

Choose returns the fastest usable proxy, or ("", "", nil) for direct.

func (*Service) Clear added in v0.2.0

func (s *Service) Clear() error

Clear removes all pooled proxies.

func (*Service) Delete

func (s *Service) Delete(id string) error

Delete removes a proxy.

func (*Service) Get

func (s *Service) Get(id string) (*models.Proxy, error)

Get returns one proxy.

func (*Service) List

func (s *Service) List() ([]*models.Proxy, error)

List returns all proxies, manual first, then fastest first.

func (*Service) NeedsSearch added in v0.0.7

func (s *Service) NeedsSearch() bool

NeedsSearch reports whether any demanded region lacks fast proxies. True means the next scheduled fetch should run; false pauses it.

func (*Service) NoteDemand added in v0.0.7

func (s *Service) NoteDemand(regions []string)

NoteDemand records whitelisted regions as demanded. Throttled to one bucket write per region per hour; reads stay in the bucket.

func (*Service) PoolTotals added in v0.0.5

func (s *Service) PoolTotals() (int, error)

PoolTotals reports the pooled proxy count.

func (*Service) Rank added in v0.0.7

func (s *Service) Rank(whitelist []string, mode string, ids []string, provider string) ([]Pick, error)

Rank returns the ordered proxy list for a provider call: whitelist filtered, fastest first. Limit filtering lives downstream of ranking, in the exhausted store. Disabled mode returns nil (direct). Manual mode follows the ids order and fails loudly when nothing usable is pooled. An empty whitelist allows any location.

func (*Service) RankWait added in v0.1.0

func (s *Service) RankWait(ctx context.Context, whitelist []string, ids []string, provider string) ([]Pick, error)

RankWait is Rank for auto mode with waiting: empty picks while probe work runs or a fetch is still due block until a pick appears, the pool settles empty (ErrNoProxies), or the context aborts. Providers wait for ready or no-proxies instead of silently going direct.

func (*Service) RekeySource added in v0.1.2

func (s *Service) RekeySource(oldSource, newSource string) error

RekeySource moves pooled proxies and fetch metadata from one source tag to another. Used once when source keys change scheme: rows keep serving under the new key instead of orphaning.

func (*Service) RotateAll added in v0.0.7

func (s *Service) RotateAll(ctx context.Context) error

RotateAll re-probes every pooled proxy and trims each location to the fastest N. It runs under a semaphore of size one: a concurrent rotation is skipped with ErrBusy, never queued.

func (*Service) SetConfig added in v0.0.7

func (s *Service) SetConfig(minSpeedKbps int64, maxPerLocation int)

SetConfig installs the pool settings from RouterConfiguration.

func (*Service) SetSourceDone added in v0.0.7

func (s *Service) SetSourceDone(sourceKey string, total int)

SetSourceDone returns a source to idle without an error, keeping the fetched total. Used when the adds succeeded but a later step did not run.

func (*Service) SetSourceFailed added in v0.0.7

func (s *Service) SetSourceFailed(sourceKey string, err error)

SetSourceFailed records a source fetch error for the dashboard.

func (*Service) SetSourceFetching added in v0.0.7

func (s *Service) SetSourceFetching(sourceKey string)

SetSourceFetching marks a source fetch in progress for the dashboard.

func (*Service) SetSourceRotating added in v0.1.0

func (s *Service) SetSourceRotating(sourceKey string)

SetSourceRotating marks a source pool check in progress for the dashboard.

func (*Service) SourceInfos added in v0.0.5

func (s *Service) SourceInfos(keys []string) []SourceInfo

SourceInfos snapshots the runtime state of the given source keys. Status copies take the lock; bucket reads run unlocked so slow storage never stalls ranking waiters.

func (*Service) SourceProxies added in v0.0.5

func (s *Service) SourceProxies(sourceKey string, offset, limit int) ([]*models.Proxy, int, error)

SourceProxies returns pooled proxies of one source, fastest first.

func (*Service) StoredSources added in v0.1.2

func (s *Service) StoredSources() ([]string, error)

StoredSources lists distinct source tags present in pooled rows. Used once to find legacy keys needing a scheme migration. Stale fetch metadata without pooled rows needs no migration: offsets restart from zero, which only refetches one window.

func (*Service) UpdateOne added in v0.0.7

func (s *Service) UpdateOne(ctx context.Context, id string) (*models.Proxy, error)

UpdateOne re-probes one pooled proxy by ID.

type SourceInfo added in v0.0.5

type SourceInfo struct {
	Key         string    `json:"key"`
	Name        string    `json:"name"`
	Status      string    `json:"status"`
	Total       int       `json:"total"`
	Pooled      int       `json:"pooled"`
	LastFetchAt time.Time `json:"last_fetch_at,omitempty"`
	LastError   string    `json:"last_error,omitempty"`
}

SourceInfo is the dashboard-facing snapshot of one source.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL