client

package
v1.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 5, 2026 License: MIT Imports: 21 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var DefaultHostPoolConfig = HostPoolConfig{
	MaxConnsPerHost:     100,
	MaxIdleConnsPerHost: 100,
	IdleTimeout:         120 * time.Second,
}

Functions

func ApplyEvasionHeaders added in v1.1.0 

func ApplyEvasionHeaders(req *http.Request)

func CacheKey added in v1.2.0 

func CacheKey(provider, apiKey string) string

CacheKey hashes (provider, apiKey) into the storage key. Exported so the validator can pre-compute the lookup if needed.

func GetRandomUserAgent 

func GetRandomUserAgent() string

func WarmConnections added in v1.1.0 

func WarmConnections(client *http.Client, domains []string)

func WarmDNS added in v1.0.9 

func WarmDNS(domains []string)

Types

type ConnectionPoolManager added in v1.0.9 

type ConnectionPoolManager struct {
	// contains filtered or unexported fields
}

func NewConnectionPoolManager added in v1.0.9 

func NewConnectionPoolManager(config HostPoolConfig) *ConnectionPoolManager

func (*ConnectionPoolManager) BuildTransport added in v1.0.9 

func (cpm *ConnectionPoolManager) BuildTransport(dialer *net.Dialer, proxyFunc func(*http.Request) (*url.URL, error)) *http.Transport

func (*ConnectionPoolManager) RecordH2Multiplex added in v1.0.9 

func (cpm *ConnectionPoolManager) RecordH2Multiplex(host string)

func (*ConnectionPoolManager) RecordRequest added in v1.0.9 

func (cpm *ConnectionPoolManager) RecordRequest(host string, latency time.Duration, statusCode int)

func (*ConnectionPoolManager) Stats added in v1.0.9 

func (cpm *ConnectionPoolManager) Stats() []HostPoolStats

type HostPoolConfig added in v1.0.9 

type HostPoolConfig struct {
	MaxConnsPerHost     int
	MaxIdleConnsPerHost int
	IdleTimeout         time.Duration
}

type HostPoolStats added in v1.0.9 

type HostPoolStats struct {
	Host            string
	ActiveConns     int64
	IdleConns       int64
	TotalRequests   int64
	TotalFailures   int64
	H2Multiplexed   int64
	AvgResponseTime time.Duration
}

type LayeredCache added in v1.2.0 

type LayeredCache struct {
	// contains filtered or unexported fields
}

LayeredCache combines an in-memory ValidationCache (per-process) with a cross-run PersistentCache. Reads consult the in-memory cache first, then fall back to the persistent one (promoting hits into the in-memory cache). Writes populate both layers.

func NewLayeredCache added in v1.2.0 

func NewLayeredCache(mem *ValidationCache, disk *PersistentCache) *LayeredCache

NewLayeredCache builds a layered cache. Either argument may be nil; the other layer still functions. If both are nil the returned cache is a no-op.

func (*LayeredCache) Get added in v1.2.0 

func (l *LayeredCache) Get(provider, apiKey string) (*models.ValidationResult, bool)

Get returns a fresh result from whichever layer has it. mem is consulted first since it is faster; misses are promoted into mem for subsequent reads.

func (*LayeredCache) Set added in v1.2.0 

func (l *LayeredCache) Set(provider, apiKey string, result *models.ValidationResult)

Set writes through to both layers.

func (*LayeredCache) Stats added in v1.2.0 

func (l *LayeredCache) Stats() (mh, mm, dh, dm int64)

Stats returns (mem_hits, mem_misses, disk_hits, disk_misses).

type PersistentCache added in v1.2.0 

type PersistentCache struct {
	// contains filtered or unexported fields
}

PersistentCache is a file-backed positive validation cache.

Entries are keyed on sha256(provider + "|" + apiKey) and recorded with the validation result, the timestamp it was observed, and a per-entry TTL. The store is append-only on write and rebuilt into an in-memory map on load so reads are O(1) and the file grows predictably. Unlike ValidationCache this survives across processes and runs.

Default location: ~/.kunji/pos_cache.jsonl

func NewDisabledPersistentCache added in v1.2.0 

func NewDisabledPersistentCache() *PersistentCache

NewDisabledPersistentCache returns a cache that never records or returns hits. Used when --no-cache is set.

func NewPersistentCache added in v1.2.0 

func NewPersistentCache(path string, ttl time.Duration) (*PersistentCache, error)

NewPersistentCache opens (or creates) the JSONL cache at path. ttl defines how long a positive result stays fresh; once expired the entry is treated as a miss but the file is left intact (it will be evicted on the next save).

func (*PersistentCache) Enabled added in v1.2.0 

func (c *PersistentCache) Enabled() bool

Enabled reports whether the cache is actively storing entries.

func (*PersistentCache) Get added in v1.2.0 

func (c *PersistentCache) Get(provider, apiKey string) (*models.ValidationResult, bool)

Get returns a cached result if present and still fresh.

func (*PersistentCache) Set added in v1.2.0 

func (c *PersistentCache) Set(provider, apiKey string, result *models.ValidationResult)

Set records a fresh positive result. Errors writing to disk are returned but the in-memory entry is still updated — the next save() call will retry.

func (*PersistentCache) Size added in v1.2.0 

func (c *PersistentCache) Size() int

Size returns the number of live (non-expired) entries in memory.

func (*PersistentCache) Stats added in v1.2.0 

func (c *PersistentCache) Stats() (hits, misses int64)

Stats returns cumulative hit/miss counters since process start.

type ProxyRotator 

type ProxyRotator struct {
	// contains filtered or unexported fields
}

func NewHTTPClient 

func NewHTTPClient(proxyStr string, timeoutSecs int) (*http.Client, *ProxyRotator, error)

func NewProxyRotator 

func NewProxyRotator(proxyInput string) (*ProxyRotator, error)

func (*ProxyRotator) FilterDeadProxies added in v1.0.9 

func (pr *ProxyRotator) FilterDeadProxies(timeoutSecs int) int

func (*ProxyRotator) GetProxy 

func (pr *ProxyRotator) GetProxy(req *http.Request) (*url.URL, error)

func (*ProxyRotator) ReportFailure added in v1.0.9 

func (pr *ProxyRotator) ReportFailure(pxy *url.URL)

type RateLimiterManager added in v1.0.9 

type RateLimiterManager struct {
	// contains filtered or unexported fields
}

func NewRateLimiterManager added in v1.0.9 

func NewRateLimiterManager(r rate.Limit, b int) *RateLimiterManager

func (*RateLimiterManager) ReportResult added in v1.0.9 

func (rm *RateLimiterManager) ReportResult(provider string, statusCode int)

func (*RateLimiterManager) ReportResultWithRetry added in v1.2.0 

func (rm *RateLimiterManager) ReportResultWithRetry(provider string, statusCode int, retryAfterSecs int)

ReportResultWithRetry behaves like ReportResult but, when the server returns a 429 with a Retry-After hint, extends the per-provider backoff window past the advertised wait time. retryAfterSecs is in seconds; 0 means "use the default exponential schedule".

func (*RateLimiterManager) SetGlobalLimit added in v1.2.0 

func (rm *RateLimiterManager) SetGlobalLimit(rps int)

SetGlobalLimit installs a single shared token bucket that caps aggregate outbound requests across all providers. rps=0 disables the global ceiling.

func (*RateLimiterManager) SetLimit added in v1.0.9 

func (rm *RateLimiterManager) SetLimit(provider string, r rate.Limit, b int)

func (*RateLimiterManager) Stats added in v1.0.9 

func (rm *RateLimiterManager) Stats() map[string]struct {
	Requests int64
	Rate429s int64
	Limit    float64
}

func (*RateLimiterManager) Wait added in v1.0.9 

func (rm *RateLimiterManager) Wait(ctx context.Context, provider string) error

type ResultCache added in v1.2.0 

type ResultCache interface {
	Get(provider, apiKey string) (*models.ValidationResult, bool)
	Set(provider, apiKey string, result *models.ValidationResult)
}

ResultCache is the contract validators rely on for positive-cache reads and writes. Both *ValidationCache and *LayeredCache satisfy it.

type ValidationCache added in v1.0.9 

type ValidationCache struct {
	// contains filtered or unexported fields
}

func NewValidationCache added in v1.0.9 

func NewValidationCache(ttl time.Duration, maxSize int) *ValidationCache

func (*ValidationCache) Get added in v1.0.9 

func (c *ValidationCache) Get(provider, apiKey string) (*models.ValidationResult, bool)

func (*ValidationCache) Set added in v1.0.9 

func (c *ValidationCache) Set(provider, apiKey string, result *models.ValidationResult)

func (*ValidationCache) Stats added in v1.0.9 

func (c *ValidationCache) Stats() (hits, misses, size int)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.