Documentation
¶
Overview ¶
Package client provides an S3 client wrapper with configuration management.
Index ¶
- Constants
- Variables
- func SanitizeAWSEnvVars()
- type BucketInfo
- type Client
- func (c *Client) Close() error
- func (c *Client) Config() *Config
- func (c *Client) ConnectionName() string
- func (c *Client) CopyObject(ctx context.Context, input *CopyObjectInput) (*CopyObjectOutput, error)
- func (c *Client) DeleteObject(ctx context.Context, bucket, key string) error
- func (c *Client) GetObject(ctx context.Context, bucket, key string) (*ObjectContent, error)
- func (c *Client) GetObjectMetadata(ctx context.Context, bucket, key string) (*ObjectMetadata, error)
- func (c *Client) GetObjectRange(ctx context.Context, bucket, key string, offset, length int64) (*ObjectContent, error)
- func (c *Client) ListBuckets(ctx context.Context) ([]BucketInfo, error)
- func (c *Client) ListObjects(ctx context.Context, bucket, prefix, delimiter string, maxKeys int32, ...) (*ListObjectsOutput, error)
- func (c *Client) PresignGetURL(ctx context.Context, bucket, key string, expires time.Duration) (*PresignedURL, error)
- func (c *Client) PresignPutURL(ctx context.Context, bucket, key string, expires time.Duration) (*PresignedURL, error)
- func (c *Client) PutObject(ctx context.Context, input *PutObjectInput) (*PutObjectOutput, error)
- func (c *Client) PutObjectStream(ctx context.Context, input *PutObjectStreamInput) (*PutObjectOutput, error)
- type Config
- type CopyObjectInput
- type CopyObjectOutput
- type ListObjectsOutput
- type ObjectContent
- type ObjectInfo
- type ObjectMetadata
- type ObjectUploader
- type PresignAPI
- type PresignedURL
- type PutObjectInput
- type PutObjectOutput
- type PutObjectStreamInput
- type S3API
Constants ¶
const ( DefaultRegion = "us-east-1" DefaultTimeout = 30 * time.Second DefaultReadIdleTimeout = 30 * time.Second )
Default values for configuration.
Variables ¶
var ErrInvalidRange = errors.New("invalid byte range")
ErrInvalidRange is returned by GetObjectRange for an offset or length no byte range can be built from. No request is made.
var ErrRangeNotSatisfiable = errors.New("byte range not satisfiable")
ErrRangeNotSatisfiable is returned by GetObjectRange when the store answers 416: the offset is at or past the end of the object, which includes every offset of a zero-byte object.
var ErrReadIdleTimeout = errors.New("object body read stalled")
ErrReadIdleTimeout is returned by GetObject and GetObjectRange when an object body stops arriving for longer than Config.ReadIdleTimeout.
var ErrStreamTooLarge = errors.New("stream exceeds maximum allowed size")
ErrStreamTooLarge indicates that a streaming upload was aborted because the body exceeded the caller-supplied size limit. Callers can test for it with errors.Is.
Functions ¶
func SanitizeAWSEnvVars ¶ added in v0.1.3
func SanitizeAWSEnvVars()
SanitizeAWSEnvVars clears AWS environment variables that contain unresolved template variables. This prevents the AWS SDK from using invalid values when mcpb passes through unresolved templates. This function must be called before config.LoadDefaultConfig() as the AWS SDK reads these environment variables directly.
Types ¶
type BucketInfo ¶
BucketInfo contains information about an S3 bucket.
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client wraps the AWS S3 SDK client with convenience methods.
func (*Client) Close ¶
Close closes the S3 client and releases resources. Currently a no-op as the AWS SDK manages its own connection pool.
func (*Client) ConnectionName ¶
ConnectionName returns the configured connection name.
func (*Client) CopyObject ¶
func (c *Client) CopyObject(ctx context.Context, input *CopyObjectInput) (*CopyObjectOutput, error)
CopyObject copies an object within or between buckets.
func (*Client) DeleteObject ¶
DeleteObject deletes an object from S3.
func (*Client) GetObjectMetadata ¶
func (c *Client) GetObjectMetadata(ctx context.Context, bucket, key string) (*ObjectMetadata, error)
GetObjectMetadata retrieves an object's metadata without downloading the content.
func (*Client) GetObjectRange ¶ added in v1.5.0
func (c *Client) GetObjectRange(ctx context.Context, bucket, key string, offset, length int64) (*ObjectContent, error)
GetObjectRange reads length bytes of an object starting at offset.
The returned Size is the object's TOTAL size, read from the response's Content-Range, not the length of Body: a caller reading the tail of a file learns how large the file is from the same call. Size is -1 when the store reports no total ("bytes 0-99/*"). Body may be shorter than length when the range runs past the end of the object.
A store that ignores Range answers with the whole object. Body is still the bytes at offset, at most length of them: the bytes before offset are read and discarded, never held.
A zero-byte object has no byte to start a range at, and S3 refuses any ranged GET of one with 416 InvalidRange. That is returned as ErrRangeNotSatisfiable, the same error as an offset past the end of a non-empty object, and not as an empty Body: an empty Body with Size 0 would be indistinguishable from a read that failed. A caller that needs to tell the two apart asks GetObjectMetadata for the size.
func (*Client) ListBuckets ¶
func (c *Client) ListBuckets(ctx context.Context) ([]BucketInfo, error)
ListBuckets returns a list of all buckets accessible to the client.
func (*Client) ListObjects ¶
func (c *Client) ListObjects( ctx context.Context, bucket, prefix, delimiter string, maxKeys int32, continueToken string, ) (*ListObjectsOutput, error)
ListObjects lists objects in a bucket with optional prefix, delimiter, and pagination.
func (*Client) PresignGetURL ¶
func (c *Client) PresignGetURL(ctx context.Context, bucket, key string, expires time.Duration) (*PresignedURL, error)
PresignGetURL generates a presigned URL for downloading an object.
func (*Client) PresignPutURL ¶
func (c *Client) PresignPutURL(ctx context.Context, bucket, key string, expires time.Duration) (*PresignedURL, error)
PresignPutURL generates a presigned URL for uploading an object.
func (*Client) PutObject ¶
func (c *Client) PutObject(ctx context.Context, input *PutObjectInput) (*PutObjectOutput, error)
PutObject uploads an object to S3.
func (*Client) PutObjectStream ¶ added in v1.2.0
func (c *Client) PutObjectStream(ctx context.Context, input *PutObjectStreamInput) (*PutObjectOutput, error)
PutObjectStream uploads an object from an io.Reader using the AWS SDK transfer manager, which splits the body into parts and uploads them without buffering the full payload in memory.
Unlike the buffered operations on Client, the per-operation timeout (S3_TIMEOUT) is intentionally NOT applied here: a streaming upload of a large object can legitimately run far longer than an ordinary request. Callers control the deadline through ctx.
Like the other write methods on Client, PutObjectStream performs the upload directly; the read-only and size-limit MCP extensions guard the tool layer, not direct library calls. Use MaxBytes to bound a stream at the library level.
type Config ¶
type Config struct {
// Region is the AWS region for the S3 service.
Region string
// Endpoint is an optional custom endpoint URL for S3-compatible services (SeaweedFS, LocalStack, etc.).
Endpoint string
// PresignEndpoint is an optional public-facing endpoint used only when
// signing presigned URLs. When set, presigned URLs are signed against it so
// they are reachable from outside the cluster, while data operations keep
// using Endpoint (typically an internal/cluster address). Empty falls back
// to Endpoint, preserving prior behavior.
PresignEndpoint string
// AccessKeyID is the AWS access key ID. If empty, the SDK credential chain is used.
AccessKeyID string
// SecretAccessKey is the AWS secret access key.
SecretAccessKey string
// SessionToken is an optional session token for temporary credentials.
SessionToken string //#nosec G117 -- field name is an AWS credential, not a secret exposure
// Profile is an optional AWS profile name to use from shared credentials/config.
Profile string
// UsePathStyle enables path-style addressing instead of virtual-hosted style.
// Required for some S3-compatible services like SeaweedFS.
UsePathStyle bool
// Timeout is the timeout for S3 operations. For GetObject and
// GetObjectRange it bounds the request up to the response headers, not the
// body read that follows; ReadIdleTimeout governs that.
Timeout time.Duration
// ReadIdleTimeout bounds how long a GetObject or GetObjectRange body read
// may go without receiving a byte. It is a stall detector, not a total:
// a transfer that keeps making progress is never cut off, whatever the
// object's size.
ReadIdleTimeout time.Duration
// Name is an optional identifier for this connection (used in multi-connection setups).
Name string
// DisableSSL disables SSL/TLS for the connection (useful for local development).
DisableSSL bool
}
Config holds the configuration for connecting to an S3-compatible storage service.
func FromEnv ¶
func FromEnv() Config
FromEnv creates a Config populated from environment variables.
Environment variables:
- AWS_REGION: AWS region (default: us-east-1)
- AWS_ACCESS_KEY_ID: Access key ID
- AWS_SECRET_ACCESS_KEY: Secret access key
- AWS_SESSION_TOKEN: Session token (optional)
- AWS_PROFILE: Profile name (optional)
- S3_ENDPOINT: Custom endpoint URL (optional)
- S3_PRESIGN_ENDPOINT: Public endpoint for presigned URLs (optional)
- S3_USE_PATH_STYLE: Use path-style URLs (default: false)
- S3_TIMEOUT: Operation timeout (default: 30s)
- S3_READ_IDLE_TIMEOUT: Longest stall allowed while reading an object body (default: 30s)
- S3_CONNECTION_NAME: Connection name (optional)
- S3_DISABLE_SSL: Disable SSL (default: false)
func (*Config) HasCredentials ¶
HasCredentials returns true if explicit credentials are configured.
func (*Config) HasEndpoint ¶
HasEndpoint returns true if a custom endpoint is configured.
type CopyObjectInput ¶
type CopyObjectInput struct {
SourceBucket string
SourceKey string
DestBucket string
DestKey string
Metadata map[string]string
}
CopyObjectInput contains the parameters for copying an object.
type CopyObjectOutput ¶
CopyObjectOutput contains the result of copying an object.
type ListObjectsOutput ¶
type ListObjectsOutput struct {
Objects []ObjectInfo
CommonPrefixes []string
IsTruncated bool
NextContinueToken string
KeyCount int32
}
ListObjectsOutput contains the result of listing objects.
type ObjectContent ¶
type ObjectContent struct {
Key string
Body []byte
ContentType string
// Size is the whole object's length. From GetObjectRange it can exceed
// len(Body).
Size int64
LastModified time.Time
ETag string
Metadata map[string]string
}
ObjectContent contains the content and metadata of an S3 object.
type ObjectInfo ¶
type ObjectInfo struct {
Key string
Size int64
LastModified time.Time
ETag string
StorageClass string
}
ObjectInfo contains information about an S3 object.
type ObjectMetadata ¶
type ObjectMetadata struct {
Key string
Size int64
LastModified time.Time
ETag string
ContentType string
ContentLength int64
Metadata map[string]string
}
ObjectMetadata contains metadata about an S3 object (from HEAD request).
type ObjectUploader ¶ added in v1.2.0
type ObjectUploader interface {
UploadObject(
ctx context.Context, input *transfermanager.UploadObjectInput, opts ...func(*transfermanager.Options),
) (*transfermanager.UploadObjectOutput, error)
}
ObjectUploader abstracts a streaming/multipart upload. It is satisfied by *transfermanager.Client from the AWS SDK and is defined here, at the consumer, so the streaming path can be mocked in unit tests.
type PresignAPI ¶
type PresignAPI interface {
PresignGetObject(ctx context.Context, params *s3.GetObjectInput, optFns ...func(*s3.PresignOptions)) (*v4.PresignedHTTPRequest, error)
PresignPutObject(ctx context.Context, params *s3.PutObjectInput, optFns ...func(*s3.PresignOptions)) (*v4.PresignedHTTPRequest, error)
}
PresignAPI defines the interface for presigning operations.
type PresignedURL ¶
PresignedURL contains information about a presigned URL.
type PutObjectInput ¶
type PutObjectInput struct {
Bucket string
Key string
Body []byte
ContentType string
Metadata map[string]string
}
PutObjectInput contains the parameters for uploading an object.
type PutObjectOutput ¶
PutObjectOutput contains the result of uploading an object.
type PutObjectStreamInput ¶ added in v1.2.0
type PutObjectStreamInput struct {
Bucket string
Key string
Body io.Reader
ContentType string
Metadata map[string]string
// MaxBytes, when greater than zero, aborts the upload once more than
// MaxBytes have been read from Body, returning an error that wraps
// ErrStreamTooLarge. A value of zero means no limit is enforced here.
MaxBytes int64
}
PutObjectStreamInput contains the parameters for a streaming/multipart upload.
Unlike PutObjectInput, the body is an io.Reader rather than a []byte, so the payload is never fully buffered in memory. This makes it suitable for large or unbounded sources such as query exports.
type S3API ¶
type S3API interface {
ListBuckets(ctx context.Context, params *s3.ListBucketsInput, optFns ...func(*s3.Options)) (*s3.ListBucketsOutput, error)
ListObjectsV2(ctx context.Context, params *s3.ListObjectsV2Input, optFns ...func(*s3.Options)) (*s3.ListObjectsV2Output, error)
GetObject(ctx context.Context, params *s3.GetObjectInput, optFns ...func(*s3.Options)) (*s3.GetObjectOutput, error)
HeadObject(ctx context.Context, params *s3.HeadObjectInput, optFns ...func(*s3.Options)) (*s3.HeadObjectOutput, error)
PutObject(ctx context.Context, params *s3.PutObjectInput, optFns ...func(*s3.Options)) (*s3.PutObjectOutput, error)
DeleteObject(ctx context.Context, params *s3.DeleteObjectInput, optFns ...func(*s3.Options)) (*s3.DeleteObjectOutput, error)
CopyObject(ctx context.Context, params *s3.CopyObjectInput, optFns ...func(*s3.Options)) (*s3.CopyObjectOutput, error)
}
S3API defines the interface for S3 operations used by Client. This interface enables mocking for unit tests.