fetcher

package
v0.4.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 10, 2026 License: MIT Imports: 11 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func IsRetryable

func IsRetryable(statusCode int) bool

IsRetryable returns true for HTTP status codes that should be retried.

func RetryAfterDuration

func RetryAfterDuration(headerValue string) time.Duration

RetryAfterDuration parses the Retry-After header value. Returns 0 if absent or unparseable. Caps at 60s.

Types

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client wraps HTTP downloading with timeouts.

func NewClient

func NewClient() *Client

NewClient creates a fetcher client with a tuned transport. This is the single constructor for all HTTP clients — transport tuning in this function applies globally (Pool workers, re-downloads, etc.).

func (*Client) DownloadFull

func (c *Client) DownloadFull(ctx context.Context, url string) (*DownloadResponse, error)

DownloadFull fetches a URL and returns body, status, headers.

func (*Client) DownloadWithRetry

func (c *Client) DownloadWithRetry(ctx context.Context, url string) ([]byte, error)

DownloadWithRetry downloads a URL with retry logic per spec. Max 3 retries; 429 respects Retry-After; 4xx (except 429) not retried.

type DownloadResponse

type DownloadResponse struct {
	Body       []byte
	StatusCode int
	Headers    http.Header
}

DownloadResult holds response details for retry decisions.

type DownloadResult

type DownloadResult struct {
	Task  DownloadTask
	Data  []byte
	Error error
}

DownloadResult holds the outcome of a download.

type DownloadTask

type DownloadTask struct {
	Name     string
	Version  string
	URL      string
	Shasum   string // SHA-1 hex digest, empty to skip verification
	DistType string
}

DownloadTask represents a single package to download.

type Pool

type Pool struct {
	OnProgress func(completed, total int, name string)
	OnStart    func(workerID int, name, version string)
	OnFinish   func(workerID int, name, version string)
	// contains filtered or unexported fields
}

Pool manages parallel downloads.

func NewPool

func NewPool(workers int) *Pool

NewPool creates a download pool with the given worker count.

func (*Pool) Download

func (p *Pool) Download(ctx context.Context, tasks []DownloadTask, resultBuf int) <-chan DownloadResult

Download downloads all tasks in parallel, returning a channel of results. resultBuf controls channel buffer size (use extractCap for back-pressure). The channel is closed after all workers finish.

func (*Pool) DownloadOne added in v0.4.0

func (p *Pool) DownloadOne(ctx context.Context, task DownloadTask) (DownloadResult, error)

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL