Documentation
¶
Overview ¶
Package s3http is the node-side HTTP transfer primitive for the direct-S3 data plane described in ADR-001 §2.2 and §5.
Presigned URLs carry their own authentication (SigV4 query parameters), so no AWS SDK or S3 credentials are needed on the node. This package wraps net/http only.
Ranged reads (#18) ¶
GetRange implements the ranged-read primitive for ADR-001 §5 ("#18 readahead / ranged reads — node-side parallel presigned ranged GETs replacing whole- slice sequential reads (chunkstore.go:94-99)"). Callers should issue ranged GETs in parallel goroutines over a shared Client to saturate S3 throughput.
URL confidentiality ¶
Presigned URLs embed a signature in the query string. The Client never logs full URLs at any level; if debug logging is added in the future, redact the query string (url.URL.Path only) before writing to any log sink.
Retries ¶
GET and HEAD are idempotent and are retried on transient 5xx or network errors up to Client.maxRetries times (default 3). PUT is NOT auto-retried (non-idempotent: an S3 PUT that returns a network error after the bytes were received would duplicate the object). Callers that know their PUT is safe to retry (e.g. content-addressed packs whose key is deterministic) may wrap Put in their own retry loop.
Index ¶
- type Client
- func (c *Client) Get(ctx context.Context, url string) ([]byte, error)
- func (c *Client) GetRange(ctx context.Context, url string, offset, length int64) ([]byte, error)
- func (c *Client) Head(ctx context.Context, url string) (size int64, err error)
- func (c *Client) Put(ctx context.Context, url string, body io.Reader, size int64) error
- type Option
- type StatusError
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client performs presigned-URL S3 HTTP transfers. It is safe for concurrent use by multiple goroutines. The zero value is not usable; create one with NewClient.
func NewClient ¶
NewClient constructs a Client with the given options. Without any options, the Client uses a transport tuned for parallel S3 fanout and retries GET/HEAD up to 3 times on transient errors.
func (*Client) Get ¶
Get fetches the full object at the presigned URL and returns its bytes. Non-2xx responses are returned as *StatusError.
func (*Client) GetRange ¶
GetRange fetches bytes [offset, offset+length) of the object at the presigned URL using the HTTP Range header.
Servers that honour the Range header return 206 Partial Content with exactly the requested bytes. Servers that ignore the Range header return 200 OK with the full object; GetRange handles this transparently by slicing the full body to [offset, offset+length) as a fallback (the full body is still downloaded, so callers should prefer servers that support ranged responses for large objects).
This is the #18 ranged-read primitive (ADR-001 §5).
func (*Client) Head ¶
Head fetches the Content-Length of the object at the presigned URL. Non-2xx responses are returned as *StatusError.
func (*Client) Put ¶
Put uploads body to the presigned URL using HTTP PUT. size must be the exact byte count of body; S3 presigned PUTs require Content-Length.
PUT is NOT auto-retried. If the caller's body is content-addressed and therefore safe to re-send, wrap Put in an external retry loop. See package-level doc for rationale.
Non-2xx responses are returned as *StatusError with a snippet of the S3 XML error body for diagnostics.
type Option ¶
type Option func(*Client)
Option configures a Client.
func WithHTTPClient ¶
WithHTTPClient replaces the default http.Client. Use this to inject a custom Transport (e.g. for testing with httptest or custom TLS config).
func WithMaxRetries ¶
WithMaxRetries sets the number of additional attempts for idempotent requests (GET, HEAD) on transient errors. 0 means no retries (one attempt total). Default is 3.
type StatusError ¶
type StatusError struct {
// Code is the HTTP status code.
Code int
// URLPath is the request path with the query string redacted to avoid
// logging presigned URL signatures.
URLPath string
// Snippet is up to snippetMax bytes of the response body, trimmed of
// leading/trailing whitespace. S3 error bodies are XML; the snippet is
// for human diagnostics only.
Snippet string
}
StatusError is returned when the server responds with a non-success HTTP status. It carries the status code, the URL path (query string redacted), and a snippet of the response body for diagnostics.
Use errors.As to unwrap from a wrapping error:
var se *s3http.StatusError
if errors.As(err, &se) && se.Code == 403 { /* expired presign */ }
func (*StatusError) Error ¶
func (e *StatusError) Error() string
Source Files