httpclientpool

package
v3.11.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 6, 2026 License: MIT Imports: 32 Imported by: 9

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var (
	ErrRebuildURL = errors.New("could not rebuild request URL")
)

Functions

func Get 

func Get(options *types.Options, configuration *Configuration, host string) (*retryablehttp.Client, error)

Get creates or gets a client for the protocol based on custom configuration. The host parameter scopes the client to a specific target, enabling per-host connection reuse with keep-alive. Pass an empty string for non-scanning uses.

func GetConnectionStats added in v3.10.0 

func GetConnectionStats() (newConns, reused int64)

GetConnectionStats returns the current connection statistics.

NOTE: counters are package-global and accumulate across in-process scans. Callers running multiple SDK/embedded executions in the same process should invoke ResetConnectionStats() at the start of each run to avoid reporting totals that mix results from earlier runs.

func GetPerHostRateLimiter added in v3.10.0 

func GetPerHostRateLimiter(options *types.Options, hostname string) (*ratelimit.Limiter, error)

GetPerHostRateLimiter gets or creates a rate limiter for a specific host Returns nil if per-host rate limiting is not enabled

func GetRawHTTP 

func GetRawHTTP(options *protocols.ExecutorOptions) *rawhttp.Client

GetRawHTTP returns the rawhttp request client

func RecordHTTPToHTTPSPortMismatch added in v3.10.0 

func RecordHTTPToHTTPSPortMismatch(options *types.Options, hostname string)

RecordHTTPToHTTPSPortMismatch records that a host:port requires HTTPS

func RecordPerHostRateLimitRequest added in v3.10.0 

func RecordPerHostRateLimitRequest(options *types.Options, hostname string)

RecordPerHostRateLimitRequest records a request for pps stats calculation

func ResetConnectionStats added in v3.10.0 

func ResetConnectionStats()

ResetConnectionStats clears the package-global new/reused connection counters (both the global totals and the per-host breakdown). Intended to be called at the start of an execution to scope the metrics to a single run.

func SendRawRequest added in v3.2.8 

func SendRawRequest(client *rawhttp.Client, opts *RawHttpRequestOpts) (*http.Response, error)

SendRawRequest sends a raw http request with the provided options and returns http response

Types

type Configuration 

type Configuration struct {
	// Threads contains the threads for the client
	Threads int
	// MaxRedirects is the maximum number of redirects to follow
	MaxRedirects int
	// NoTimeout disables http request timeout for context based usage
	NoTimeout bool
	// DisableCookie disables cookie reuse for the http client (cookiejar impl)
	DisableCookie bool
	// FollowRedirects specifies the redirects flow
	RedirectFlow RedirectFlow
	// Connection defines custom connection configuration
	Connection *ConnectionConfiguration
	// ResponseHeaderTimeout is the timeout for response body to be read from the server
	ResponseHeaderTimeout time.Duration
}

Configuration contains the custom configuration options for a client

func (*Configuration) Clone added in v3.3.2 

func (c *Configuration) Clone() *Configuration

func (*Configuration) HasStandardOptions 

func (c *Configuration) HasStandardOptions() bool

HasStandardOptions checks whether the configuration requires custom settings

func (*Configuration) Hash 

func (c *Configuration) Hash() string

Hash returns the hash of the configuration to allow client pooling

type ConnectionConfiguration 

type ConnectionConfiguration struct {
	// DisableKeepAlive of the connection
	DisableKeepAlive bool
	// CustomMaxTimeout is the custom timeout for the connection
	// This overrides all other timeouts and is used for accurate time based fuzzing.
	CustomMaxTimeout time.Duration
	// contains filtered or unexported fields
}

ConnectionConfiguration contains the custom configuration options for a connection

func (*ConnectionConfiguration) GetCookieJar 

func (cc *ConnectionConfiguration) GetCookieJar() *cookiejar.Jar

func (*ConnectionConfiguration) HasCookieJar 

func (cc *ConnectionConfiguration) HasCookieJar() bool

func (*ConnectionConfiguration) SetCookieJar 

func (cc *ConnectionConfiguration) SetCookieJar(cookiejar *cookiejar.Jar)

type ConnectionStats added in v3.10.0 

type ConnectionStats struct {
	New    atomic.Int64
	Reused atomic.Int64
}

ConnectionStats tracks HTTP connection reuse across the scan.

type HTTPToHTTPSPortStats added in v3.10.0 

type HTTPToHTTPSPortStats struct {
	TotalDetections  uint64
	TotalCorrections uint64
	TrackedPorts     int
}

HTTPToHTTPSPortStats contains statistics about the HTTP-to-HTTPS port tracker

type HTTPToHTTPSPortTracker added in v3.10.0 

type HTTPToHTTPSPortTracker struct {
	// contains filtered or unexported fields
}

HTTPToHTTPSPortTracker tracks host:port combinations that require HTTPS This is used to automatically detect and correct cases where HTTP requests are sent to HTTPS ports (detected via 400 error with specific message).

NOTE: detection and correction apply to the standard net/http (retryablehttp) request path only. Unsafe/raw (rawhttp) requests bypass this scheme rewrite.

func GetHTTPToHTTPSPortTracker added in v3.10.0 

func GetHTTPToHTTPSPortTracker(options *types.Options) *HTTPToHTTPSPortTracker

GetHTTPToHTTPSPortTracker gets or creates the HTTP-to-HTTPS port tracker

func NewHTTPToHTTPSPortTracker added in v3.10.0 

func NewHTTPToHTTPSPortTracker() *HTTPToHTTPSPortTracker

NewHTTPToHTTPSPortTracker creates a new HTTP-to-HTTPS port tracker

func (*HTTPToHTTPSPortTracker) Evict added in v3.10.0 

func (t *HTTPToHTTPSPortTracker) Evict(hostPort string)

Evict removes a host:port from the tracker. It is used to self-heal a false positive: when an http->https correction is applied but the https request then fails, the original http scheme is retried and, if the entry was wrong, evicted so subsequent requests (including those from unrelated templates) against the same host:port are not silently broken.

func (*HTTPToHTTPSPortTracker) PrintStats added in v3.10.0 

func (t *HTTPToHTTPSPortTracker) PrintStats()

PrintStats prints statistics about the tracker

func (*HTTPToHTTPSPortTracker) RecordCorrection added in v3.10.0 

func (t *HTTPToHTTPSPortTracker) RecordCorrection()

RecordCorrection records that an HTTP->HTTPS correction was actually applied

func (*HTTPToHTTPSPortTracker) RecordHTTPToHTTPSPort added in v3.10.0 

func (t *HTTPToHTTPSPortTracker) RecordHTTPToHTTPSPort(hostPort string)

RecordHTTPToHTTPSPort records that a host:port requires HTTPS

func (*HTTPToHTTPSPortTracker) RequiresHTTPS added in v3.10.0 

func (t *HTTPToHTTPSPortTracker) RequiresHTTPS(hostPort string) bool

RequiresHTTPS checks if a host:port requires HTTPS

func (*HTTPToHTTPSPortTracker) Stats added in v3.10.0 

func (t *HTTPToHTTPSPortTracker) Stats() HTTPToHTTPSPortStats

Stats returns statistics about the tracker

type PerHostConnStat added in v3.10.0 

type PerHostConnStat struct {
	Host   string
	New    int64
	Reused int64
}

PerHostConnStat is a point-in-time snapshot of a single host's connection reuse.

func GetPerHostConnectionStats added in v3.10.0 

func GetPerHostConnectionStats() []PerHostConnStat

GetPerHostConnectionStats returns a snapshot of per-host connection reuse.

type PerHostRateLimitPool added in v3.10.0 

type PerHostRateLimitPool struct {
	// contains filtered or unexported fields
}

func NewPerHostRateLimitPool added in v3.10.0 

func NewPerHostRateLimitPool(size int, maxIdleTime, maxLifetime time.Duration, options *types.Options) *PerHostRateLimitPool

func (*PerHostRateLimitPool) Cap added in v3.10.0 

func (p *PerHostRateLimitPool) Cap() int

func (*PerHostRateLimitPool) Close added in v3.10.0 

func (p *PerHostRateLimitPool) Close()

func (*PerHostRateLimitPool) EvictAll added in v3.10.0 

func (p *PerHostRateLimitPool) EvictAll()

func (*PerHostRateLimitPool) EvictHost added in v3.10.0 

func (p *PerHostRateLimitPool) EvictHost(host string) bool

func (*PerHostRateLimitPool) GetAllRateLimitInfo added in v3.10.0 

func (p *PerHostRateLimitPool) GetAllRateLimitInfo() []*RateLimitInfo

func (*PerHostRateLimitPool) GetLimiterForHost added in v3.10.0 

func (p *PerHostRateLimitPool) GetLimiterForHost(host string) (*ratelimit.Limiter, bool)

func (*PerHostRateLimitPool) GetOrCreate added in v3.10.0 

func (p *PerHostRateLimitPool) GetOrCreate(
	host string,
) (*ratelimit.Limiter, error)

func (*PerHostRateLimitPool) GetRateLimitInfo added in v3.10.0 

func (p *PerHostRateLimitPool) GetRateLimitInfo(host string) *RateLimitInfo

func (*PerHostRateLimitPool) ListAllLimiters added in v3.10.0 

func (p *PerHostRateLimitPool) ListAllLimiters() []string

func (*PerHostRateLimitPool) PrintPerHostPPSStats added in v3.10.0 

func (p *PerHostRateLimitPool) PrintPerHostPPSStats()

PrintPerHostPPSStats prints requests per second for each host

func (*PerHostRateLimitPool) PrintStats added in v3.10.0 

func (p *PerHostRateLimitPool) PrintStats()

func (*PerHostRateLimitPool) RecordRequest added in v3.10.0 

func (p *PerHostRateLimitPool) RecordRequest(host string)

RecordRequest records a request timestamp for a host to calculate pps

func (*PerHostRateLimitPool) Resize added in v3.10.0 

func (p *PerHostRateLimitPool) Resize(size int) int

func (*PerHostRateLimitPool) Size added in v3.10.0 

func (p *PerHostRateLimitPool) Size() int

func (*PerHostRateLimitPool) Stats added in v3.10.0 

func (p *PerHostRateLimitPool) Stats() RateLimitPoolStats

type RateLimitInfo added in v3.10.0 

type RateLimitInfo struct {
	Host        string
	CreatedAt   time.Time
	AccessCount uint64
	Age         time.Duration
}

type RateLimitPoolStats added in v3.10.0 

type RateLimitPoolStats struct {
	Hits      uint64
	Misses    uint64
	Evictions uint64
	Size      int
}

type RawHttpRequestOpts added in v3.2.8 

type RawHttpRequestOpts struct {
	// Method is the http method to use
	Method string
	// URL is the url to request
	URL string
	// Path is request path to use
	Path string
	// Headers is the headers to use
	Headers map[string][]string
	// Body is the body to use
	Body io.Reader
	// Options is more client related options
	Options *rawhttp.Options
}

RawHttpRequestOpts is a configuration for raw http request

type RedirectFlow 

type RedirectFlow uint8
const (
	DontFollowRedirect RedirectFlow = iota
	FollowSameHostRedirect
	FollowAllRedirect
	FollowSameSchemeRedirect
)

type WithCustomTimeout added in v3.2.8 

type WithCustomTimeout struct {
	Timeout time.Duration
}

WithCustomTimeout is a configuration for custom timeout

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.