cloud_storage

package
v1.106.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 8, 2026 License: MIT Imports: 34 Imported by: 0

Documentation

Index

Constants

View Source
const Domain = "storage.googleapis.com"
View Source
const MaxSignedUrlExpires = 7 * 24 * time.Hour

MaxSignedUrlExpires is the upper bound on V4 signed URL lifetime enforced by GCS.

Variables

View Source
var ErrUploadStalled = errors.New("upload stalled")

ErrUploadStalled is an upload that stopped getting any further: sessions that kept ending, or chunks that were taken and not kept.

Functions

func Crc32cOf added in v1.105.0

func Crc32cOf(r io.Reader) (string, error)

Crc32cOf reads r to its end and returns the CRC32C of what it read, encoded as Cloud Storage writes it.

func DecodeCrc32c added in v1.105.0

func DecodeCrc32c(encoded string) (uint32, error)

DecodeCrc32c reads a CRC32C written as Cloud Storage writes it.

func EncodeCrc32c added in v1.105.0

func EncodeCrc32c(checksum uint32) string

EncodeCrc32c renders a CRC32C as Cloud Storage writes it in an object's metadata: the four bytes big-endian, in standard base64.

func NewCrc32c added in v1.105.0

func NewCrc32c() hash.Hash32

NewCrc32c returns a hash computing the CRC32C Cloud Storage keeps for an object.

Types

type Client

type Client struct {
	// contains filtered or unexported fields
}

func NewClient

func NewClient(options ...cloud_storage_config.Option) *Client

func (*Client) CopyObject added in v1.76.0

func (c *Client) CopyObject(
	ctx context.Context,
	sourceBucketName string,
	sourceObjectName string,
	destinationBucketName string,
	destinationObjectName string,
	options ...fetch_config.Option,
) (*object.Object, error)

CopyObject copies an object to another name, in the same bucket or another one, and returns the copy's metadata.

The copy happens inside Cloud Storage: the bytes never travel to the caller and back, which is what makes it the right way to move an object between buckets rather than a download followed by an insert. It is a copy and not a move -- the source is left alone, so a caller that wants a move deletes the source once the copy has returned, and a failure between the two leaves two copies rather than none.

An existing destination is replaced, which is what makes a retry safe: the same source copied again writes the same bytes.

func (*Client) DeleteObject

func (c *Client) DeleteObject(ctx context.Context, bucketName string, objectName string, options ...fetch_config.Option) error

DeleteObject deletes an object from a bucket.

func (*Client) DeleteObjectGeneration added in v1.105.0

func (c *Client) DeleteObjectGeneration(
	ctx context.Context,
	bucketName string,
	objectName string,
	generation int64,
	options ...fetch_config.Option,
) error

DeleteObjectGeneration deletes an object only while generation is its live generation. A name written again since its generation was read is left alone, and the failure is a Non2xxStatusCodeError with status 412; a name already gone fails with 404.

It is the delete for a caller that decided from a listing: between the listing and the delete, the name may have been written anew, and what was decided about the old object says nothing about the new one.

func (*Client) DownloadObject

func (c *Client) DownloadObject(ctx context.Context, bucketName string, objectName string, options ...fetch_config.Option) ([]byte, error)

DownloadObject downloads an object's content.

func (*Client) GetObject

func (c *Client) GetObject(ctx context.Context, bucketName string, objectName string, options ...fetch_config.Option) (*object.Object, error)

GetObject retrieves an object's metadata.

func (*Client) InsertBucket

func (c *Client) InsertBucket(ctx context.Context, project string, bucketConfig *bucket.Bucket, options ...fetch_config.Option) (*bucket.Bucket, error)

InsertBucket creates a new bucket in the specified project.

func (*Client) InsertObject

func (c *Client) InsertObject(ctx context.Context, bucketName string, metadata *object.Object, data []byte, contentType string, options ...fetch_config.Option) (*object.Object, error)

InsertObject uploads an object to a bucket using a multipart upload. The metadata should have at least its Name field set. The data parameter contains the object content, and contentType specifies its MIME type.

func (*Client) InsertObjectIfAbsent

func (c *Client) InsertObjectIfAbsent(ctx context.Context, bucketName string, metadata *object.Object, data []byte, contentType string, options ...fetch_config.Option) (*object.Object, bool, error)

InsertObjectIfAbsent uploads an object only if the name is unused, reporting whether it did. It is for objects whose name determines their content — the contents of a commit, say — where a replacement is not an update but the same thing written again, and where writing it again is not harmless: replacing an object deletes the generation readers of it may hold.

func (*Client) ListObjects

func (c *Client) ListObjects(ctx context.Context, bucketName string, query url.Values, options ...fetch_config.Option) (*object_list.ObjectList, error)

ListObjects lists objects in a bucket. Use the query parameter to specify prefix, delimiter, maxResults, pageToken, and other query parameters.

func (*Client) NewReader added in v1.105.0

func (c *Client) NewReader(
	ctx context.Context,
	bucketName string,
	objectName string,
	options ...reader_config.Option,
) (*Reader, error)

NewReader opens an object for reading, from the live generation unless reader_config.WithGeneration pins one, and over the whole object unless reader_config.WithRange narrows it.

func (*Client) PatchBucket

func (c *Client) PatchBucket(ctx context.Context, bucketName string, bucketConfig *bucket.Bucket, options ...fetch_config.Option) (*bucket.Bucket, error)

PatchBucket updates an existing bucket using patch semantics.

func (*Client) PatchObject

func (c *Client) PatchObject(ctx context.Context, bucketName string, objectName string, objectConfig *object.Object, options ...fetch_config.Option) (*object.Object, error)

PatchObject updates an object's metadata using patch semantics.

func (*Client) SignedUrl

func (c *Client) SignedUrl(
	ctx context.Context,
	s signer.Signer,
	method string,
	bucketName string,
	objectName string,
	expires time.Duration,
) (string, error)

SignedUrl returns a V4 signed URL for the given object. The URL grants temporary access to perform the given HTTP method against the object, valid for the given duration (capped at 7 days by GCS). The signer's identity is embedded in the URL — the runtime must be able to sign on behalf of that identity (e.g. an in-process RSA key, or via IAM Credentials' signBlob).

Only the host header is signed; payloads are sent as UNSIGNED-PAYLOAD, so the resulting URL works with any request body content.

func (*Client) UploadObject added in v1.105.0

func (c *Client) UploadObject(
	ctx context.Context,
	bucketName string,
	metadata *object.Object,
	body io.ReaderAt,
	size int64,
	options ...upload_config.Option,
) (*object.Object, bool, error)

UploadObject uploads size bytes read from body as an object, through a resumable upload session, and returns the object's metadata and whether it was created.

It is InsertObject for what should not be held in memory. The body is read a chunk at a time, straight from body, and never passes through the fetch logging, which records only a body it is handed as bytes. A chunk broken off is not sent again from its start: the session is asked how much it kept, and the upload goes on from there.

The object's CRC32C is declared when the session is opened -- metadata's own if it has one, otherwise computed here with a first pass over body -- so Cloud Storage refuses to create an object whose bytes are not the ones meant, and the object that comes back is checked against the declared size and CRC32C all the same.

With upload_config.WithIfAbsent, a name already taken, when the session is opened or when it completes, is reported as nothing created rather than as an error, as InsertObjectIfAbsent does.

type Reader added in v1.105.0

type Reader struct {
	// contains filtered or unexported fields
}

Reader reads an object's bytes as they arrive, rather than all at once as DownloadObject does.

A read broken off midway is resumed where it stopped, from the same object generation: what it yields is one generation's bytes or an error, never the start of one object followed by the rest of another written in its place.

The bytes are the object's as stored. An object stored gzip-compressed comes back compressed, as its size and CRC32C describe it, rather than decompressed on the way by Cloud Storage or by the HTTP transport.

func (*Reader) Close added in v1.105.0

func (r *Reader) Close() error

func (*Reader) Crc32c added in v1.105.0

func (r *Reader) Crc32c() string

Crc32c is the whole object's CRC32C, as Cloud Storage reported it.

func (*Reader) Generation added in v1.105.0

func (r *Reader) Generation() int64

Generation is the generation being read.

func (*Reader) Read added in v1.105.0

func (r *Reader) Read(p []byte) (int, error)

func (*Reader) Size added in v1.105.0

func (r *Reader) Size() int64

Size is the size of the whole object, whatever range is being read.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL