Help us shape the future of Go! Take the 2026 Go Developer Survey

gcpstorage

package
v1.8.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 20, 2026 License: MIT Imports: 31 Imported by: 0

README

Google Cloud Storage

Parity grade: B · SDK cloud.google.com/go/storage@v1.65.1 · last audited 2026-10-18 (3a537de68)

Coverage

Metric Value
PARITY entries audited 23 (22 ok, 1 partial)
Feature families 7 (5 ok, 2 partial)
Known gaps 8
Deferred items 0
Resource leaks clean
Known gaps
  • No bucket or object ACL endpoints (/b/{b}/acl, /b/{b}/defaultObjectAcl, /b/{b}/o/{o}/acl); acl fields on bucket bodies are stored but predefinedAcl is ignored.
  • No gzip transcoding: an object stored with contentEncoding gzip is always served compressed.
  • No notification configs, HMAC keys, bucket relocation, soft-delete restore, object restore, or the project serviceAccount endpoint.
  • Lifecycle, retention policies, holds and CORS are stored but not enforced.
  • Signed URLs (V2/V4) are not verified and no request is authenticated.
  • Virtual-hosted-style URLs (bucket.storage.googleapis.com) and XML bucket listing, multipart XML uploads and x-goog-copy-source are not supported.
  • Customer-supplied and KMS encryption keys are accepted but not applied.
  • ListObjects ignores matchGlob; page tokens are index-based, so they shift if objects change between pages.

More

Documentation

Index

Constants

View Source 
const DefaultPort = 4443

DefaultPort is the dedicated listener's default TCP port.

Variables

View Source 
var ErrNilAppContext = errors.New("gcpstorage: nil app context")

ErrNilAppContext is returned when Init is called with a nil AppContext.

View Source 
var ErrSnapshotBucketNull = errors.New("gcpstorage: restore snapshot: bucket is null")

ErrSnapshotBucketNull is returned by Restore when a snapshot holds a null bucket.

Functions

func LocationType 

func LocationType(loc string) string

LocationType returns the GCS locationType of a bucket location.

Types

type APIError 

type APIError struct {
	Reason  string
	Message string
	Status  int
}

APIError is an error carrying the HTTP status and Google error reason.

func (*APIError) Error 

func (e *APIError) Error() string

Error implements error.

type Bucket 

type Bucket struct {
	Created               time.Time                  `json:"created"`
	Updated               time.Time                  `json:"updated"`
	Objects               map[string][]*Object       `json:"objects"`
	Labels                map[string]string          `json:"labels,omitempty"`
	Config                map[string]json.RawMessage `json:"config,omitempty"`
	Name                  string                     `json:"name"`
	Project               string                     `json:"project"`
	Location              string                     `json:"location"`
	StorageClass          string                     `json:"storageClass"`
	IAM                   IAMPolicy                  `json:"iam"`
	Metageneration        int64                      `json:"metageneration"`
	Versioning            bool                       `json:"versioning,omitempty"`
	DefaultEventBasedHold bool                       `json:"defaultEventBasedHold,omitempty"`
}

Bucket is a bucket with all of its object versions.

type ComposeSource 

type ComposeSource struct {
	Pre        Preconds
	Name       string
	Generation int64
}

ComposeSource is one source object of a compose request.

type ConfigProvider 

type ConfigProvider interface {
	GetGCPStorageSettings() Settings
}

ConfigProvider extracts GCS configuration from the AppContext Config.

type CopyRequest 

type CopyRequest struct {
	DstPre        Preconds
	SrcPre        Preconds
	Meta          Patch
	SrcBucket     string
	SrcName       string
	DstBucket     string
	DstName       string
	SrcGeneration int64
}

CopyRequest describes a copy or rewrite.

type Handler 

type Handler struct {
	Backend *InMemoryBackend

	Port int
	// contains filtered or unexported fields
}

Handler serves the Cloud Storage JSON and XML APIs.

func NewHandler 

func NewHandler(backend *InMemoryBackend) *Handler

NewHandler creates a Handler; callers override Port from Settings.

func (*Handler) ExtractOperation 

func (h *Handler) ExtractOperation(c *echo.Context) string

ExtractOperation returns the operation name for metrics.

func (*Handler) ExtractResource 

func (h *Handler) ExtractResource(c *echo.Context) string

ExtractResource returns bucket[/object] for metrics.

func (*Handler) GetSupportedOperations 

func (h *Handler) GetSupportedOperations() []string

GetSupportedOperations lists the supported operations.

func (*Handler) Handler 

func (h *Handler) Handler() echo.HandlerFunc

Handler returns the Echo handler for all GCS requests.

func (*Handler) MatchPriority 

func (h *Handler) MatchPriority() int

MatchPriority is irrelevant because RouteMatcher never matches.

func (*Handler) Name 

func (h *Handler) Name() string

Name returns the service name.

func (*Handler) Reset 

func (h *Handler) Reset()

Reset clears all in-memory state.

func (*Handler) Restore 

func (h *Handler) Restore(ctx context.Context, data []byte) error

Restore implements persistence.Persistable by delegating to the backend.

func (*Handler) RouteMatcher 

func (h *Handler) RouteMatcher() service.Matcher

RouteMatcher never matches: GCS runs on its own listener.

func (*Handler) Shutdown 

func (h *Handler) Shutdown(ctx context.Context)

Shutdown stops the dedicated listener, forcing a close if graceful shutdown fails.

func (*Handler) Snapshot 

func (h *Handler) Snapshot(ctx context.Context) []byte

Snapshot implements persistence.Persistable by delegating to the backend.

func (*Handler) StartWorker 

func (h *Handler) StartWorker(ctx context.Context) error

StartWorker binds the dedicated listener synchronously so a port conflict is returned to the caller.

type IAMBinding 

type IAMBinding struct {
	Condition json.RawMessage `json:"condition,omitempty"`
	Role      string          `json:"role"`
	Members   []string        `json:"members"`
}

IAMBinding is one role binding of a bucket IAM policy.

type IAMPolicy 

type IAMPolicy struct {
	Etag     string       `json:"etag"`
	Bindings []IAMBinding `json:"bindings"`
	Version  int          `json:"version"`
}

IAMPolicy is a bucket IAM policy.

type InMemoryBackend 

type InMemoryBackend struct {
	// contains filtered or unexported fields
}

InMemoryBackend holds all buckets, objects and resumable upload sessions under one lock.

func NewInMemoryBackend 

func NewInMemoryBackend() *InMemoryBackend

NewInMemoryBackend creates an empty backend.

func (*InMemoryBackend) AppendUpload 

func (b *InMemoryBackend) AppendUpload(id string, offset int64, chunk []byte, total int64) (UploadProgress, error)

AppendUpload appends chunk at offset. total is the declared object size or -1 when unknown; once the received bytes reach it the object is committed.

func (*InMemoryBackend) CancelUpload 

func (b *InMemoryBackend) CancelUpload(id string) error

CancelUpload discards a resumable session.

func (*InMemoryBackend) ComposeObject 

func (b *InMemoryBackend) ComposeObject(
	bucket, dst string, sources []ComposeSource, meta Patch, pre Preconds,
) (*Object, error)

ComposeObject concatenates sources (same bucket) into dst.

func (*InMemoryBackend) CopyObject 

func (b *InMemoryBackend) CopyObject(req CopyRequest) (*Object, error)

CopyObject copies an object. A nil Meta inherits the source metadata; otherwise Meta supplies the destination metadata.

func (*InMemoryBackend) CreateBucket 

func (b *InMemoryBackend) CreateBucket(project string, p Patch) (*Bucket, error)

CreateBucket creates a bucket in project from the request body p.

func (*InMemoryBackend) DeleteBucket 

func (b *InMemoryBackend) DeleteBucket(name string, pre Preconds) error

DeleteBucket deletes an empty bucket.

func (*InMemoryBackend) DeleteObject 

func (b *InMemoryBackend) DeleteObject(bucket, name string, gen int64, pre Preconds) error

DeleteObject deletes the live version (archiving it in a versioned bucket) or, with gen set, permanently removes that generation.

func (*InMemoryBackend) GetBucket 

func (b *InMemoryBackend) GetBucket(name string, pre Preconds) (*Bucket, error)

GetBucket returns the bucket metadata.

func (*InMemoryBackend) GetIAMPolicy 

func (b *InMemoryBackend) GetIAMPolicy(name string) (IAMPolicy, error)

GetIAMPolicy returns the bucket IAM policy.

func (*InMemoryBackend) GetObject 

func (b *InMemoryBackend) GetObject(bucket, name string, gen int64, pre Preconds) (*Object, error)

GetObject returns generation gen of an object, or the live one when gen is 0.

func (*InMemoryBackend) ListBuckets 

func (b *InMemoryBackend) ListBuckets(project, prefix, token string, limit int) page.Page[*Bucket]

ListBuckets lists the project's buckets whose names start with prefix.

func (*InMemoryBackend) ListObjects 

func (b *InMemoryBackend) ListObjects(bucket string, opts ListOptions) (ListResult, error)

ListObjects lists objects in name order, collapsing delimited prefixes.

func (*InMemoryBackend) PutObject 

func (b *InMemoryBackend) PutObject(bucket, name string, data []byte, p Patch, pre Preconds) (*Object, error)

PutObject creates or overwrites an object.

func (*InMemoryBackend) Reset 

func (b *InMemoryBackend) Reset()

Reset clears all state.

func (*InMemoryBackend) Restore 

func (b *InMemoryBackend) Restore(ctx context.Context, data []byte) error

Restore loads backend state from a snapshot.

func (*InMemoryBackend) SetIAMPolicy 

func (b *InMemoryBackend) SetIAMPolicy(name string, pol IAMPolicy) (IAMPolicy, error)

SetIAMPolicy replaces the bucket IAM policy; a non-empty etag must match.

func (*InMemoryBackend) Snapshot 

func (b *InMemoryBackend) Snapshot(ctx context.Context) []byte

Snapshot serialises the backend state. In-flight resumable sessions are not persisted.

func (*InMemoryBackend) StartUpload 

func (b *InMemoryBackend) StartUpload(bucket, name string, meta Patch, pre Preconds) (string, error)

StartUpload opens a resumable upload session and returns its id.

func (*InMemoryBackend) UpdateBucket 

func (b *InMemoryBackend) UpdateBucket(name string, p Patch, replace bool, pre Preconds) (*Bucket, error)

UpdateBucket applies a PATCH (replace=false) or PUT (replace=true) body.

func (*InMemoryBackend) UpdateObject 

func (b *InMemoryBackend) UpdateObject(
	bucket, name string, gen int64, p Patch, replace bool, pre Preconds,
) (*Object, error)

UpdateObject applies a PATCH (replace=false) or PUT (replace=true) body.

type ListOptions 

type ListOptions struct {
	Prefix                   string
	Delimiter                string
	StartOffset              string
	EndOffset                string
	PageToken                string
	MaxResults               int
	Versions                 bool
	IncludeTrailingDelimiter bool
}

ListOptions are the object list parameters.

type ListResult 

type ListResult struct {
	Next     string
	Items    []*Object
	Prefixes []string
}

ListResult is one page of objects and collapsed prefixes.

type Object 

type Object struct {
	Created            time.Time         `json:"created"`
	Updated            time.Time         `json:"updated"`
	Deleted            time.Time         `json:"deleted"`
	CustomTime         time.Time         `json:"customTime"`
	Metadata           map[string]string `json:"metadata,omitempty"`
	Bucket             string            `json:"bucket"`
	Name               string            `json:"name"`
	ContentType        string            `json:"contentType"`
	ContentEncoding    string            `json:"contentEncoding,omitempty"`
	ContentDisposition string            `json:"contentDisposition,omitempty"`
	ContentLanguage    string            `json:"contentLanguage,omitempty"`
	CacheControl       string            `json:"cacheControl,omitempty"`
	StorageClass       string            `json:"storageClass"`
	MD5                string            `json:"md5,omitempty"`
	CRC32C             string            `json:"crc32c"`
	Data               []byte            `json:"data"`
	Generation         int64             `json:"generation"`
	Metageneration     int64             `json:"metageneration"`
	ComponentCount     int64             `json:"componentCount,omitempty"`
	TemporaryHold      bool              `json:"temporaryHold,omitempty"`
	EventBasedHold     bool              `json:"eventBasedHold,omitempty"`
}

Object is one stored object version. A version whose Deleted time is set is noncurrent; the live version, if any, is always last in its slice.

func (*Object) Size 

func (o *Object) Size() int64

Size returns the object size in bytes.

type Patch 

type Patch map[string]json.RawMessage

Patch is a decoded JSON request body, keyed by field name. A null value clears the field, as in a JSON merge patch.

type Preconds 

type Preconds struct {
	GenMatch     *int64
	GenNotMatch  *int64
	MetaMatch    *int64
	MetaNotMatch *int64
}

Preconds are the ifGeneration*/ifMetageneration* request conditions.

type Provider 

type Provider struct{}

Provider implements service.Provider for Google Cloud Storage. Like the Azure services it runs on its own listener (see Handler.StartWorker) and never matches on the shared AWS router.

func (*Provider) Init 

func (p *Provider) Init(ctx *service.AppContext) (service.Registerable, error)

Init creates the backend and handler; the port is bound in StartWorker.

func (*Provider) Name 

func (p *Provider) Name() string

Name returns the service provider name.

type Settings 

type Settings struct {
	Port int `` //nolint:lll // config struct tags are intentionally verbose
	/* 168-byte string literal not displayed */
}

Settings holds service-level configuration, embedded in the root CLI command as CLI.GCPStorage.

func DefaultSettings 

func DefaultSettings() Settings

DefaultSettings returns the settings used when no ConfigProvider is available.

type UploadProgress 

type UploadProgress struct {
	Object   *Object
	Received int64
}

UploadProgress reports a session's state after an append.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.