client

package
v1.2.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 1, 2026 License: MIT Imports: 27 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var DefaultHostPoolConfig = HostPoolConfig{
	MaxConnsPerHost:     100,
	MaxIdleConnsPerHost: 100,
	IdleTimeout:         120 * time.Second,
}

Functions

func ApplyEvasionHeaders added in v1.1.0 

func ApplyEvasionHeaders(req *http.Request)

func CacheKey added in v1.2.0 

func CacheKey(provider, apiKey string) string

CacheKey hashes (provider, apiKey) into the storage key. Exported so the validator can pre-compute the lookup if needed.

func CompressionEnabled added in v1.2.1 

func CompressionEnabled() bool

CompressionEnabled reports the current default.

func EnableHTTP3 added in v1.2.1 

func EnableHTTP3(on bool)

EnableHTTP3 records the --http3 flag. It must be called before the shared client is constructed (validate.go does this ahead of the runner/factory setup) because transports are built once.

func GetRandomUserAgent 

func GetRandomUserAgent() string

func HTTP3Enabled added in v1.2.1 

func HTTP3Enabled() bool

HTTP3Enabled reports whether the HTTP/3 flag is on.

func SetCompressionEnabled added in v1.2.1 

func SetCompressionEnabled(on bool)

SetCompressionEnabled flips whether appendEntry produces compressed lines. Existing plain lines remain valid; load handles both formats transparently.

func WarmConnections added in v1.1.0 

func WarmConnections(client *http.Client, domains []string)

func WarmDNS added in v1.0.9 

func WarmDNS(domains []string)

Types

type ConnectionPoolManager added in v1.0.9 

type ConnectionPoolManager struct {
	// contains filtered or unexported fields
}

func NewConnectionPoolManager added in v1.0.9 

func NewConnectionPoolManager(config HostPoolConfig) *ConnectionPoolManager

func (*ConnectionPoolManager) BuildTransport added in v1.0.9 

func (cpm *ConnectionPoolManager) BuildTransport(dialer *net.Dialer, proxyFunc func(*http.Request) (*url.URL, error)) *http.Transport

type HostPoolConfig added in v1.0.9 

type HostPoolConfig struct {
	MaxConnsPerHost     int
	MaxIdleConnsPerHost int
	IdleTimeout         time.Duration
}

type LayeredCache added in v1.2.0 

type LayeredCache struct {
	// contains filtered or unexported fields
}

LayeredCache combines an in-memory ValidationCache (per-process) with a cross-run PersistentCache. Reads consult the in-memory cache first, then fall back to the persistent one (promoting hits into the in-memory cache). Writes populate both layers.

func NewLayeredCache added in v1.2.0 

func NewLayeredCache(mem *ValidationCache, disk *PersistentCache) *LayeredCache

NewLayeredCache builds a layered cache. Either argument may be nil; the other layer still functions. If both are nil the returned cache is a no-op.

func (*LayeredCache) Get added in v1.2.0 

func (l *LayeredCache) Get(provider, apiKey string) (*models.ValidationResult, bool)

Get returns a fresh result from whichever layer has it. mem is consulted first since it is faster; misses are promoted into mem for subsequent reads.

func (*LayeredCache) Set added in v1.2.0 

func (l *LayeredCache) Set(provider, apiKey string, result *models.ValidationResult)

Set writes through to both layers with the cache's default TTL.

func (*LayeredCache) SetWithTTL added in v1.2.1 

func (l *LayeredCache) SetWithTTL(provider, apiKey string, result *models.ValidationResult, ttl time.Duration)

SetWithTTL writes through to both layers with a per-call TTL override. ttl <= 0 means "use each layer's default".

func (*LayeredCache) Stats added in v1.2.0 

func (l *LayeredCache) Stats() (mh, mm, dh, dm int64)

Stats returns (mem_hits, mem_misses, disk_hits, disk_misses).

type PersistentCache added in v1.2.0 

type PersistentCache struct {
	// contains filtered or unexported fields
}

PersistentCache is a file-backed positive validation cache.

Entries are keyed on sha256(provider + "|" + apiKey) and recorded with the validation result, the timestamp it was observed, and a per-entry TTL. The store is append-only on write and rebuilt into an in-memory map on load so reads are O(1) and the file grows predictably. Unlike ValidationCache this survives across processes and runs.

Default location: ~/.kunji/pos_cache.jsonl

func NewDisabledPersistentCache added in v1.2.0 

func NewDisabledPersistentCache() *PersistentCache

NewDisabledPersistentCache returns a cache that never records or returns hits. Used when --no-cache is set.

func NewPersistentCache added in v1.2.0 

func NewPersistentCache(path string, ttl time.Duration) (*PersistentCache, error)

NewPersistentCache opens (or creates) the JSONL cache at path. ttl defines how long a positive result stays fresh; once expired the entry is treated as a miss but the file is left intact (it will be evicted on the next save).

func (*PersistentCache) Enabled added in v1.2.0 

func (c *PersistentCache) Enabled() bool

Enabled reports whether the cache is actively storing entries.

func (*PersistentCache) Get added in v1.2.0 

func (c *PersistentCache) Get(provider, apiKey string) (*models.ValidationResult, bool)

Get returns a cached result if present and still fresh.

func (*PersistentCache) Set added in v1.2.0 

func (c *PersistentCache) Set(provider, apiKey string, result *models.ValidationResult)

Set records a fresh positive result with the cache's default TTL.

func (*PersistentCache) SetWithTTL added in v1.2.1 

func (c *PersistentCache) SetWithTTL(provider, apiKey string, result *models.ValidationResult, ttl time.Duration)

SetWithTTL records a fresh positive result with a per-call TTL override. ttl <= 0 falls back to the cache's default. Validators use this to honor per-provider cache_ttl_seconds declared in their YAML schema.

func (*PersistentCache) Size added in v1.2.0 

func (c *PersistentCache) Size() int

Size returns the number of live (non-expired) entries in memory.

func (*PersistentCache) Stats added in v1.2.0 

func (c *PersistentCache) Stats() (hits, misses int64)

Stats returns cumulative hit/miss counters since process start.

type ProxyRotator 

type ProxyRotator struct {
	// contains filtered or unexported fields
}

func NewHTTPClient 

func NewHTTPClient(proxyStr string, timeoutSecs int) (*http.Client, *ProxyRotator, error)

func NewProxyRotator 

func NewProxyRotator(proxyInput string) (*ProxyRotator, error)

func (*ProxyRotator) EgressIPs added in v1.2.1 

func (pr *ProxyRotator) EgressIPs() map[string]string

EgressIPs maps each alive proxy URL to its last observed exit address.

func (*ProxyRotator) Fastest added in v1.2.1 

func (pr *ProxyRotator) Fastest() (proxy, egress string, latency time.Duration, ok bool)

Fastest reports the lowest-latency alive proxy from the last scoring run.

func (*ProxyRotator) FilterDeadProxies added in v1.0.9 

func (pr *ProxyRotator) FilterDeadProxies(timeoutSecs int) int

func (*ProxyRotator) GetProxy 

func (pr *ProxyRotator) GetProxy(req *http.Request) (*url.URL, error)

type RateLimiterManager added in v1.0.9 

type RateLimiterManager struct {
	// contains filtered or unexported fields
}

func NewRateLimiterManager added in v1.0.9 

func NewRateLimiterManager(r rate.Limit, b int) *RateLimiterManager

func (*RateLimiterManager) ReportResult added in v1.0.9 

func (rm *RateLimiterManager) ReportResult(provider string, statusCode int)

func (*RateLimiterManager) ReportResultWithRetry added in v1.2.0 

func (rm *RateLimiterManager) ReportResultWithRetry(provider string, statusCode int, retryAfterSecs int)

ReportResultWithRetry behaves like ReportResult but, when the server returns a 429 with a Retry-After hint, extends the per-provider backoff window past the advertised wait time. retryAfterSecs is in seconds; 0 means "use the default exponential schedule".

func (*RateLimiterManager) SetGlobalLimit added in v1.2.0 

func (rm *RateLimiterManager) SetGlobalLimit(rps int)

SetGlobalLimit installs a single shared token bucket that caps aggregate outbound requests across all providers. rps=0 disables the global ceiling.

func (*RateLimiterManager) SetLimit added in v1.0.9 

func (rm *RateLimiterManager) SetLimit(provider string, r rate.Limit, b int)

func (*RateLimiterManager) Stats added in v1.0.9 

func (rm *RateLimiterManager) Stats() map[string]struct {
	Requests int64
	Rate429s int64
	Limit    float64
}

func (*RateLimiterManager) Wait added in v1.0.9 

func (rm *RateLimiterManager) Wait(ctx context.Context, provider string) error

type ResultCache added in v1.2.0 

type ResultCache interface {
	Get(provider, apiKey string) (*models.ValidationResult, bool)
	Set(provider, apiKey string, result *models.ValidationResult)
	SetWithTTL(provider, apiKey string, result *models.ValidationResult, ttl time.Duration)
}

ResultCache is the contract validators rely on for positive-cache reads and writes. Both *ValidationCache and *LayeredCache satisfy it.

type ValidationCache added in v1.0.9 

type ValidationCache struct {
	// contains filtered or unexported fields
}

func NewValidationCache added in v1.0.9 

func NewValidationCache(ttl time.Duration, maxSize int) *ValidationCache

func (*ValidationCache) Get added in v1.0.9 

func (c *ValidationCache) Get(provider, apiKey string) (*models.ValidationResult, bool)

func (*ValidationCache) Set added in v1.0.9 

func (c *ValidationCache) Set(provider, apiKey string, result *models.ValidationResult)

func (*ValidationCache) SetWithTTL added in v1.2.1 

func (c *ValidationCache) SetWithTTL(provider, apiKey string, result *models.ValidationResult, ttl time.Duration)

SetWithTTL stores result with the given TTL. ttl <= 0 falls back to the cache's default. Used by validators to honor per-provider cache_ttl_seconds declared in their YAML schema.

func (*ValidationCache) Stats added in v1.0.9 

func (c *ValidationCache) Stats() (hits, misses, size int)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.