Documentation
¶
Index ¶
- Variables
- func FetchFileFromGCSWithZstd(ctx context.Context, client ObjectStorage, gsURL string, localFile *os.File) error
- func FetchFromGCS(ctx context.Context, client ObjectStorage, gsURL string) ([]byte, error)
- func FetchLocalFileFromGCSWithZstd(ctx context.Context, client ObjectStorage, gsURL string, localFilePath string) (err error)
- func Open(ctx context.Context, client ObjectStorage, gsURL string) (io.ReadCloser, error)
- func SendBytesToGCS(ctx context.Context, client ObjectStorage, gsURL string, content []byte) error
- func SendFileToGCSWithZstd(ctx context.Context, client ObjectStorage, gsURL string, localFile *os.File) error
- func SendLocalFileToGCSWithZstd(ctx context.Context, client ObjectStorage, gsURL string, localFilePath string) (err error)
- type ObjectStorage
Constants ¶
This section is empty.
Variables ¶
var ErrObjectNotFound = errors.New("object not found in storage backend")
ErrObjectNotFound reports the object or its bucket is absent from the storage backend, as opposed to a transient fetch failure.
Functions ¶
func FetchFileFromGCSWithZstd ¶
func FetchFileFromGCSWithZstd(ctx context.Context, client ObjectStorage, gsURL string, localFile *os.File) error
FetchFileFromGCSWithZstd downloads and decompresses into an already-open file. The caller retains ownership of localFile.
func FetchFromGCS ¶
func Open ¶
func Open(ctx context.Context, client ObjectStorage, gsURL string) (io.ReadCloser, error)
Open streams the object at gsURL; the caller must Close the returned reader. Unlike FetchFromGCS it does not buffer the whole object in memory.
func SendBytesToGCS ¶
SendBytesToGCS uploads the given bytes (uncompressed) to gsURL. Intended for small objects such as the snapshot manifest.
func SendFileToGCSWithZstd ¶
func SendFileToGCSWithZstd(ctx context.Context, client ObjectStorage, gsURL string, localFile *os.File) error
SendFileToGCSWithZstd compresses and uploads an already-open file. The caller retains ownership of localFile.
Types ¶
type ObjectStorage ¶
type ObjectStorage interface {
GetObject(ctx context.Context, bucket, object string) (io.ReadCloser, error)
PutObject(ctx context.Context, bucket, object string, reader io.Reader) error
}
func NewGCSClient ¶
func NewGCSClient(ctx context.Context, opts ...option.ClientOption) (ObjectStorage, error)
NewGCSClient returns a GCS-backed ObjectStorage. It builds its own storage.Client from opts (and more from the same opts for the part-upload pool, see uploadClient) rather than accepting one, because it installs a RetryAlways policy (see setRetry) that is only safe for this package's own operations and must not leak onto a client shared with other code.
func NewS3Client ¶
func NewS3Client(client *s3.Client) ObjectStorage
Source Files