net

package
v0.0.103 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 18, 2026 License: MIT Imports: 21 Imported by: 7

Documentation

Overview

Package net provides the core download functions

Package net provides net related functions

Index

Constants

View Source
const (
	// ContentType is for the http header of content type
	ContentType = "Content-Type"
	// ApplicationForm is for the form submit
	ApplicationForm = "application/x-www-form-urlencoded"
)

Variables

This section is empty.

Functions

func DetectSizeWithRoundTripper added in v0.0.68

func DetectSizeWithRoundTripper(targetURL, output string, showProgress, noProxy, insecureSkipVerify bool,
	roundTripper http.RoundTripper, timeout time.Duration) (total int64, rangeSupport bool, err error)

DetectSizeWithRoundTripper returns the size of target resource Deprecated, use DetectSizeWithRoundTripperAndAuth instead

func DetectSizeWithRoundTripperAndAuth added in v0.0.98

func DetectSizeWithRoundTripperAndAuth(targetURL, output string, showProgress, noProxy, insecureSkipVerify bool,
	roundTripper http.RoundTripper, username, password string, timeout time.Duration) (total int64, rangeSupport bool, err error)

DetectSizeWithRoundTripperAndAuth returns the size of target resource

func DetectSizeWithRoundTripperAndAuthStream added in v0.0.99

func DetectSizeWithRoundTripperAndAuthStream(targetURL string, output io.Writer, showProgress, noProxy, insecureSkipVerify bool,
	roundTripper http.RoundTripper, username, password string, timeout time.Duration) (total int64, rangeSupport bool, err error)

DetectSizeWithRoundTripperAndAuthStream returns the size of target resource

func DownloadFileWithMultipleThreadKeepParts

func DownloadFileWithMultipleThreadKeepParts(targetURL, targetFilePath string, thread int, keepParts, showProgress bool) (err error)

DownloadFileWithMultipleThreadKeepParts downloads the files with multiple threads

func ParseSuggestedFilename added in v0.0.88

func ParseSuggestedFilename(header http.Header, filepath string) (filename string)

ParseSuggestedFilename parse the filename from resp header,More details from https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Disposition

func SetProxy

func SetProxy(proxy, proxyAuth string, tr *http.Transport) (err error)

SetProxy set the proxy for a http

Types

type ContinueDownloader added in v0.0.32

type ContinueDownloader struct {
	UserName, Password string
	Timeout            time.Duration
	Context            context.Context
	// contains filtered or unexported fields
}

ContinueDownloader is a downloader which support continuously download

func (*ContinueDownloader) DownloadWithContinue added in v0.0.32

func (c *ContinueDownloader) DownloadWithContinue(targetURL, output string, index, continueAt, end int64, showProgress bool) (err error)

DownloadWithContinue downloads the files continuously. When the transfer breaks in the middle (e.g. an unexpected EOF) it automatically retries and resumes from the bytes that are already on the local disk via range requests, until the attempts are exhausted or the error is not retryable.

func (*ContinueDownloader) DownloadWithContinueAsStream added in v0.0.99

func (c *ContinueDownloader) DownloadWithContinueAsStream(targetURL string, output io.Writer, index, continueAt, end int64, showProgress bool) (err error)

DownloadWithContinueAsStream downloads the files continuously

func (*ContinueDownloader) GetSuggestedFilename added in v0.0.81

func (c *ContinueDownloader) GetSuggestedFilename() string

GetSuggestedFilename returns the suggested filename

func (*ContinueDownloader) WithBasicAuth added in v0.0.98

func (c *ContinueDownloader) WithBasicAuth(username, password string) *ContinueDownloader

WithBasicAuth sets the basic auth

func (*ContinueDownloader) WithContext added in v0.0.83

WithContext sets the context

func (*ContinueDownloader) WithInsecureSkipVerify added in v0.0.81

func (c *ContinueDownloader) WithInsecureSkipVerify(insecureSkipVerify bool) *ContinueDownloader

WithInsecureSkipVerify set if skip the insecure verify

func (*ContinueDownloader) WithMaxAttempts added in v0.0.102

func (c *ContinueDownloader) WithMaxAttempts(maxAttempts int) *ContinueDownloader

WithMaxAttempts sets the max times a download is attempted when the transfer breaks in the middle, zero or a negative value means no retry

func (*ContinueDownloader) WithProgressIndicator added in v0.0.102

func (c *ContinueDownloader) WithProgressIndicator(progress *ProgressIndicator) *ContinueDownloader

WithProgressIndicator sets a shared progress indicator which aggregates the progress of all the downloads that share it. Passing an indicator that was not initialized (Init) disables the progress of the downloads

func (*ContinueDownloader) WithRoundTripper added in v0.0.69

func (c *ContinueDownloader) WithRoundTripper(roundTripper http.RoundTripper) *ContinueDownloader

WithRoundTripper set WithRoundTripper

func (*ContinueDownloader) WithTimeout added in v0.0.83

func (c *ContinueDownloader) WithTimeout(timeout time.Duration) *ContinueDownloader

WithTimeout sets the timeout

func (*ContinueDownloader) WithoutProxy added in v0.0.69

func (c *ContinueDownloader) WithoutProxy(noProxy bool) *ContinueDownloader

WithoutProxy indicate no HTTP proxy use

type DownloadError

type DownloadError struct {
	StatusCode int
	Message    string
}

DownloadError represents the error of HTTP download

func (*DownloadError) Error

func (e *DownloadError) Error() string

Error print the error message

type FakeReader added in v0.0.77

type FakeReader struct {
	ExpectErr error
}

FakeReader is a fake reader for the test purpose

func (*FakeReader) Read added in v0.0.77

func (e *FakeReader) Read(p []byte) (n int, err error)

Read is a fake method

type HTTPDownloader

type HTTPDownloader struct {
	TargetFilePath     string
	URL                string
	ShowProgress       bool
	InsecureSkipVerify bool
	Context            context.Context

	UserName string
	Password string

	NoProxy   bool
	Proxy     string
	ProxyAuth string

	Header map[string]string

	// PreStart returns false will don't continue
	PreStart func(*http.Response) bool

	Thread  int
	Title   string
	Timeout time.Duration

	Debug        bool
	RoundTripper http.RoundTripper

	// Append indicates to append the downloaded data to the target file
	// instead of truncating it, used by the resumed downloads
	Append bool
	// ProgressBase is the number of bytes that were already downloaded
	// before this download started, used by the progress indicator
	ProgressBase int64
	// contains filtered or unexported fields
}

HTTPDownloader is the downloader for http request

func (*HTTPDownloader) DownloadAsStream added in v0.0.99

func (h *HTTPDownloader) DownloadAsStream(writer io.Writer) (err error)

DownloadAsStream downloads the file as stream

func (*HTTPDownloader) DownloadFile

func (h *HTTPDownloader) DownloadFile() (err error)

DownloadFile download a file with the progress

func (*HTTPDownloader) GetSuggestedFilename added in v0.0.81

func (h *HTTPDownloader) GetSuggestedFilename() string

GetSuggestedFilename returns the suggested filename which comes from the HTTP response header. Returns empty string if the filename is same with the given name.

type MultiProgressIndicator added in v0.0.102

type MultiProgressIndicator struct {
	// contains filtered or unexported fields
}

MultiProgressIndicator renders one progress bar per download part, every bar stays on its own terminal line. All the bars and the relative cursor movements go through a single locked renderer, so the lines never garble each other - unlike bars rendered independently. It is only suitable for an interactive terminal, use a single shared ProgressIndicator instead when the output is piped or redirected.

func NewMultiProgressIndicator added in v0.0.102

func NewMultiProgressIndicator() *MultiProgressIndicator

NewMultiProgressIndicator creates an empty multi-bar group. The cursor is expected to stay at the beginning of a fresh line.

func (*MultiProgressIndicator) Close added in v0.0.102

func (m *MultiProgressIndicator) Close()

Close finishes all the bars and leaves the cursor below them.

func (*MultiProgressIndicator) Register added in v0.0.102

func (m *MultiProgressIndicator) Register(title string, total float64) *ProgressIndicator

Register allocates the next terminal line for a new bar and returns the indicator of it.

type MultiThreadDownloader added in v0.0.69

type MultiThreadDownloader struct {
	// contains filtered or unexported fields
}

MultiThreadDownloader is a download with multi-thread

func (*MultiThreadDownloader) Download added in v0.0.69

func (d *MultiThreadDownloader) Download(targetURL, targetFilePath string, thread int) (err error)

Download starts to download the target URL

func (*MultiThreadDownloader) DownloadWithContext added in v0.0.99

func (d *MultiThreadDownloader) DownloadWithContext(ctx context.Context, targetURL string, outputWriter io.Writer, thread int) (err error)

DownloadWithContext starts to download the target URL with context

func (*MultiThreadDownloader) GetSuggestedFilename added in v0.0.81

func (d *MultiThreadDownloader) GetSuggestedFilename() string

GetSuggestedFilename returns the suggested filename

func (*MultiThreadDownloader) WithBasicAuth added in v0.0.98

func (d *MultiThreadDownloader) WithBasicAuth(username, password string) *MultiThreadDownloader

WithBasicAuth sets the basic auth

func (*MultiThreadDownloader) WithBearerToken added in v0.0.99

func (d *MultiThreadDownloader) WithBearerToken(bearerToken string) *MultiThreadDownloader

WithBearerToken sets the bearer token

func (*MultiThreadDownloader) WithInsecureSkipVerify added in v0.0.81

func (d *MultiThreadDownloader) WithInsecureSkipVerify(insecureSkipVerify bool) *MultiThreadDownloader

WithInsecureSkipVerify set if skip the insecure verify

func (*MultiThreadDownloader) WithKeepParts added in v0.0.69

func (d *MultiThreadDownloader) WithKeepParts(keepParts bool) *MultiThreadDownloader

WithKeepParts indicates if keeping the part files

func (*MultiThreadDownloader) WithMaxAttempts added in v0.0.102

func (d *MultiThreadDownloader) WithMaxAttempts(maxAttempts int) *MultiThreadDownloader

WithMaxAttempts sets the max times a download part is attempted when the transfer breaks in the middle, zero or a negative value means no retry

func (*MultiThreadDownloader) WithRoundTripper added in v0.0.69

func (d *MultiThreadDownloader) WithRoundTripper(roundTripper http.RoundTripper) *MultiThreadDownloader

WithRoundTripper sets RoundTripper

func (*MultiThreadDownloader) WithShowProgress added in v0.0.69

func (d *MultiThreadDownloader) WithShowProgress(showProgress bool) *MultiThreadDownloader

WithShowProgress indicate if show the download progress

func (*MultiThreadDownloader) WithTimeout added in v0.0.83

func (d *MultiThreadDownloader) WithTimeout(timeout time.Duration) *MultiThreadDownloader

WithTimeout sets the timeout

func (*MultiThreadDownloader) WithoutProxy added in v0.0.69

func (d *MultiThreadDownloader) WithoutProxy(noProxy bool) *MultiThreadDownloader

WithoutProxy indicates not use HTTP proxy

type ProgressIndicator

type ProgressIndicator struct {
	Writer io.Writer
	Reader io.Reader
	Title  string

	// Total is the expected total size in bytes. Zero or a negative value
	// means the size is unknown, in that case the bar works in the spinner
	// mode which still reports the transferred bytes.
	Total float64
	// contains filtered or unexported fields
}

ProgressIndicator hold the progress of io operation.

Multiple indicators are allowed to share one underlying progress bar, for example the parts of a multi-thread download can feed a single aggregated bar (see Share) when the output is not an interactive terminal. On an interactive terminal use MultiProgressIndicator instead, it renders one bar per part on dedicated terminal lines.

func (*ProgressIndicator) Close added in v0.0.32

func (i *ProgressIndicator) Close()

Close shutdowns the progress rendering. Only the owner of the underlying bar finishes it, shared indicators are not affected.

func (*ProgressIndicator) Init

func (i *ProgressIndicator) Init()

Init set the default value for progress indicator

func (*ProgressIndicator) Read

func (i *ProgressIndicator) Read(p []byte) (n int, err error)

Read reads the progress

func (*ProgressIndicator) Share added in v0.0.102

Share returns a new indicator which feeds the same progress bar with this indicator. The returned indicator does not own the bar, which means closing it does not finish the bar. Typically used by the multi-thread downloads: all the parts write into one aggregated bar. It returns nil if the current indicator was not initialized (Init), so that the progress can be disabled globally.

func (*ProgressIndicator) Write

func (i *ProgressIndicator) Write(p []byte) (n int, err error)

Write writes the progress

type RetryClient

type RetryClient struct {
	http.Client
	MaxAttempts int
	// contains filtered or unexported fields
}

RetryClient is the wrap of http.Client

func NewRetryClient added in v0.0.83

func NewRetryClient(client http.Client) *RetryClient

NewRetryClient creates the instance of RetryClient

func (*RetryClient) Do

func (c *RetryClient) Do(req *http.Request) (rsp *http.Response, err error)

Do is the wrap of http.Client.Do

type SuggestedFilenameAware added in v0.0.81

type SuggestedFilenameAware interface {
	GetSuggestedFilename() string
}

SuggestedFilenameAware is the interface for getting suggested filename

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL