http

package
v1.2.17 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 27, 2026 License: Apache-2.0 Imports: 13 Imported by: 0

Documentation

Overview

Package http implements a Gatherer for HTTP and HTTPS sources.

Index

Constants

View Source
const DefaultMaxResponseBytes int64 = 80 << 20

DefaultMaxResponseBytes bounds the size of a downloaded response body, preventing a malicious or misbehaving server from exhausting disk. It matches the Conforma CLI's MaxRequestBodySize (80 MiB).

Variables

This section is empty.

Functions

This section is empty.

Types

type HTTPGatherer

type HTTPGatherer struct {
	Client http.Client
	// MaxResponseBytes bounds the response body size. A value <= 0 disables the
	// limit. NewHTTPGatherer sets it to DefaultMaxResponseBytes.
	MaxResponseBytes int64
}

HTTPGatherer gathers resources over HTTP/HTTPS.

func NewHTTPGatherer

func NewHTTPGatherer(opts ...Option) *HTTPGatherer

NewHTTPGatherer returns an HTTPGatherer with a default 30-second timeout.

func (*HTTPGatherer) Gather

func (h *HTTPGatherer) Gather(ctx context.Context, rawSource, dst string) (meta metadata.Metadata, err error)

Gather downloads a file from rawSource via HTTP and writes it to dst.

func (*HTTPGatherer) Matcher

func (h *HTTPGatherer) Matcher(uri string) bool

Matcher returns true if the URI uses an HTTP or HTTPS scheme and is not a known git host.

type HTTPMetadata

type HTTPMetadata struct {
	URI          string
	Path         string
	ResponseCode int
	Size         int64
	Timestamp    string
}

HTTPMetadata holds metadata about a gathered HTTP resource.

func (HTTPMetadata) Get

func (h HTTPMetadata) Get() interface{}

Get returns the HTTPMetadata value.

func (HTTPMetadata) GetPinnedURL

func (h HTTPMetadata) GetPinnedURL(u string) (string, error)

GetPinnedURL returns an http:: prefixed URL for the given address.

type Option added in v1.1.6

type Option func(*HTTPGatherer)

Option configures an HTTPGatherer.

func WithMaxResponseBytes added in v1.2.1

func WithMaxResponseBytes(n int64) Option

WithMaxResponseBytes sets the maximum number of bytes read from a response body. A value <= 0 disables the limit.

func WithTransport added in v1.1.6

func WithTransport(t http.RoundTripper) Option

WithTransport sets the http.RoundTripper used by the HTTPGatherer's client.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL