Documentation
¶
Overview ¶
Package objstore is a small stdlib S3 client: SigV4, path style, ListObjectsV2 with pagination, GetObject, PutObject, HeadBucket and CreateBucket (EPIC D19, P7-ADOPT-23).
It exists because the loader, the exporter and the nhost source must move objects through the loopback S3 port of the project's object store (MinIO compatible or SeaweedFS) without a host rclone/mc dependency. Requests go through internal/httptimeout (NoProxy: a signed request reaches only the host the caller named, never follows a redirect).
Secrets: errors are redacted; the secret key and any signature never appear in error text.
Layering: L1; imports internal/httptimeout (L0) and the standard library.
Index ¶
- Constants
- type Client
- func (c *Client) EnsureBucket(ctx context.Context, bucket string) error
- func (c *Client) Get(ctx context.Context, bucket, key string) (io.ReadCloser, error)
- func (c Client) GoString() string
- func (c *Client) List(ctx context.Context, bucket, prefix string) ([]Object, error)
- func (c *Client) ListBuckets(ctx context.Context) ([]string, error)
- func (c *Client) ListPage(ctx context.Context, bucket, prefix, token string) (Page, error)
- func (c *Client) Put(ctx context.Context, bucket, key string, body io.Reader, size int64) error
- func (c Client) String() string
- type Error
- type Object
- type Page
- type Signer
Constants ¶
const ( DefaultMaxListPages = 100_000 DefaultMaxListObjects = 10_000_000 )
Limits of List when the Client sets none.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Client ¶
type Client struct {
Endpoint string
Region string
AccessKey string
SecretKey string
PageSize int
MaxListPages int
MaxListObjects int
HTTP *http.Client
Now func() time.Time
}
Client talks to one S3-compatible endpoint in path style (http://host:port/<bucket>/<key>).
Inputs: Endpoint is scheme://host[:port] with no userinfo, path or query; Region defaults to "us-east-1" when empty; AccessKey and SecretKey are the credentials (the secret never appears in an error, in String or in %#v). PageSize caps keys per List page (0 = server default); MaxListPages and MaxListObjects bound List (0 = DefaultMaxListPages and DefaultMaxListObjects). HTTP defaults to a shared no-proxy, no-redirect client with dial, TLS and response-header timeouts and no whole-request timeout, so a large object can take as long as it needs: bound a call with its context. Now defaults to time.Now (tests pin it). Constraints: a PUT streams with UNSIGNED-PAYLOAD only over plain http to a loopback endpoint; every other PUT is signed over the payload hash.
func (*Client) EnsureBucket ¶
EnsureBucket creates the bucket when it does not exist (HEAD, then PUT). Success when it already exists or another caller created it first.
func (*Client) List ¶
List returns every object under prefix, sorted by key, following continuation tokens. A repeated token (a server that never advances) is an error, not an endless loop, and so is a listing past MaxListPages pages or MaxListObjects objects (a hostile endpoint cannot grow memory without bound).
func (*Client) ListBuckets ¶ added in v1.4.15
ListBuckets returns all buckets visible to the configured credentials.
func (*Client) ListPage ¶
ListPage returns one ListObjectsV2 page for prefix, starting at token ("" for the first page). The continuation token is URL-encoded on the wire. The request asks for encoding-type=url, so a key with a character XML cannot carry (U+0001 and the like) still lists; keys are decoded here when the server says it encoded them.
func (*Client) Put ¶
Put uploads size bytes from body as bucket/key.
Over plain http to a loopback endpoint the body streams with UNSIGNED-PAYLOAD. Otherwise the payload is hashed first and signed: a seekable body is hashed and rewound, any other body is spooled to a temporary file (removed before return).
type Error ¶
type Error struct {
// Op is the operation ("list", "get", "put", "head-bucket", "create-bucket").
Op string
// Status is the HTTP status code.
Status int
// Code is the S3 error code ("NoSuchBucket", "SignatureDoesNotMatch", ...), if the body had one.
Code string
// Message is the server's message, redacted.
Message string
}
Error is a non-success S3 response.
func (*Error) IsNotFound ¶
IsNotFound reports whether the response was 404.
type Page ¶
type Page struct {
Objects []Object
// NextToken is the continuation token of the next page; empty when done.
NextToken string
}
Page is one ListObjectsV2 page.
type Signer ¶
Signer signs requests with AWS Signature Version 4.
Purpose: stdlib-only SigV4 (crypto/hmac, crypto/sha256). Inputs: Service is "s3" for object stores (the path is encoded once and x-amz-content-sha256 is set); any other service is double-path-encoded and gets no content header, matching the AWS SigV4 test suite. Constraints: never log a Signer; SecretKey is the secret.
Source Files