gcpstorage

package
v1.8.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 20, 2026 License: MIT Imports: 31 Imported by: 0

README

Google Cloud Storage

Parity grade: B · SDK cloud.google.com/go/storage@v1.65.1 · last audited 2026-10-18 (3a537de68)

Coverage

Metric Value
PARITY entries audited 23 (22 ok, 1 partial)
Feature families 7 (5 ok, 2 partial)
Known gaps 8
Deferred items 0
Resource leaks clean
Known gaps
  • No bucket or object ACL endpoints (/b/{b}/acl, /b/{b}/defaultObjectAcl, /b/{b}/o/{o}/acl); acl fields on bucket bodies are stored but predefinedAcl is ignored.
  • No gzip transcoding: an object stored with contentEncoding gzip is always served compressed.
  • No notification configs, HMAC keys, bucket relocation, soft-delete restore, object restore, or the project serviceAccount endpoint.
  • Lifecycle, retention policies, holds and CORS are stored but not enforced.
  • Signed URLs (V2/V4) are not verified and no request is authenticated.
  • Virtual-hosted-style URLs (bucket.storage.googleapis.com) and XML bucket listing, multipart XML uploads and x-goog-copy-source are not supported.
  • Customer-supplied and KMS encryption keys are accepted but not applied.
  • ListObjects ignores matchGlob; page tokens are index-based, so they shift if objects change between pages.

More

Documentation

Index

Constants

View Source 
const DefaultPort = 4443

DefaultPort is the dedicated listener's default TCP port.

Variables

View Source 
var ErrNilAppContext = errors.New("gcpstorage: nil app context")

ErrNilAppContext is returned when Init is called with a nil AppContext.

View Source 
var ErrSnapshotBucketNull = errors.New("gcpstorage: restore snapshot: bucket is null")

ErrSnapshotBucketNull is returned by Restore when a snapshot holds a null bucket.

Functions

func LocationType 

func LocationType(loc string) string

LocationType returns the GCS locationType of a bucket location.

Types

type APIError 

type APIError struct {
	Reason  string
	Message string
	Status  int
}

APIError is an error carrying the HTTP status and Google error reason.

func (*APIError) Error 

func (e *APIError) Error() string

Error implements error.

type Bucket 

type Bucket struct {
	Created               time.Time                  `json:"created"`
	Updated               time.Time                  `json:"updated"`
	Objects               map[string][]*Object       `json:"objects"`
	Labels                map[string]string          `json:"labels,omitempty"`
	Config                map[string]json.RawMessage `json:"config,omitempty"`
	Name                  string                     `json:"name"`
	Project               string                     `json:"project"`
	Location              string                     `json:"location"`
	StorageClass          string                     `json:"storageClass"`
	IAM                   IAMPolicy                  `json:"iam"`
	Metageneration        int64                      `json:"metageneration"`
	Versioning            bool                       `json:"versioning,omitempty"`
	DefaultEventBasedHold bool                       `json:"defaultEventBasedHold,omitempty"`
}

Bucket is a bucket with all of its object versions.

type ComposeSource 

type ComposeSource struct {
	Pre        Preconds
	Name       string
	Generation int64
}

ComposeSource is one source object of a compose request.

type ConfigProvider 

type ConfigProvider interface {
	GetGCPStorageSettings() Settings
}

ConfigProvider extracts GCS configuration from the AppContext Config.

type CopyRequest 

type CopyRequest struct {
	DstPre        Preconds
	SrcPre        Preconds
	Meta          Patch
	SrcBucket     string
	SrcName       string
	DstBucket     string
	DstName       string
	SrcGeneration int64
}

CopyRequest describes a copy or rewrite.

type Handler 

type Handler struct {
	Backend *InMemoryBackend

	Port int
	// contains filtered or unexported fields
}

Handler serves the Cloud Storage JSON and XML APIs.

func NewHandler 

func NewHandler(backend *InMemoryBackend) *Handler

NewHandler creates a Handler; callers override Port from Settings.

func (*Handler) ExtractOperation 

func (h *Handler) ExtractOperation(c *echo.Context) string

ExtractOperation returns the operation name for metrics.

func (*Handler) ExtractResource 

func (h *Handler) ExtractResource(c *echo.Context) string

ExtractResource returns bucket[/object] for metrics.

func (*Handler) GetSupportedOperations 

func (h *Handler) GetSupportedOperations() []string

GetSupportedOperations lists the supported operations.

func (*Handler) Handler 

func (h *Handler) Handler() echo.HandlerFunc

Handler returns the Echo handler for all GCS requests.

func (*Handler) MatchPriority 

func (h *Handler) MatchPriority() int

MatchPriority is irrelevant because RouteMatcher never matches.

func (*Handler) Name 

func (h *Handler) Name() string

Name returns the service name.

func (*Handler) Reset 

func (h *Handler) Reset()

Reset clears all in-memory state.

func (*Handler) Restore 

func (h *Handler) Restore(ctx context.Context, data []byte) error

Restore implements persistence.Persistable by delegating to the backend.

func (*Handler) RouteMatcher 

func (h *Handler) RouteMatcher() service.Matcher

RouteMatcher never matches: GCS runs on its own listener.

func (*Handler) Shutdown 

func (h *Handler) Shutdown(ctx context.Context)

Shutdown stops the dedicated listener, forcing a close if graceful shutdown fails.

func (*Handler) Snapshot 

func (h *Handler) Snapshot(ctx context.Context) []byte

Snapshot implements persistence.Persistable by delegating to the backend.

func (*Handler) StartWorker 

func (h *Handler) StartWorker(ctx context.Context) error

StartWorker binds the dedicated listener synchronously so a port conflict is returned to the caller.

type IAMBinding 

type IAMBinding struct {
	Condition json.RawMessage `json:"condition,omitempty"`
	Role      string          `json:"role"`
	Members   []string        `json:"members"`
}

IAMBinding is one role binding of a bucket IAM policy.

type IAMPolicy 

type IAMPolicy struct {
	Etag     string       `json:"etag"`
	Bindings []IAMBinding `json:"bindings"`
	Version  int          `json:"version"`
}

IAMPolicy is a bucket IAM policy.

type InMemoryBackend 

type InMemoryBackend struct {
	// contains filtered or unexported fields
}

InMemoryBackend holds all buckets, objects and resumable upload sessions under one lock.

func NewInMemoryBackend 

func NewInMemoryBackend() *InMemoryBackend

NewInMemoryBackend creates an empty backend.

func (*InMemoryBackend) AppendUpload 

func (b *InMemoryBackend) AppendUpload(id string, offset int64, chunk []byte, total int64) (UploadProgress, error)

AppendUpload appends chunk at offset. total is the declared object size or -1 when unknown; once the received bytes reach it the object is committed.

func (*InMemoryBackend) CancelUpload 

func (b *InMemoryBackend) CancelUpload(id string) error

CancelUpload discards a resumable session.

func (*InMemoryBackend) ComposeObject 

func (b *InMemoryBackend) ComposeObject(
	bucket, dst string, sources []ComposeSource, meta Patch, pre Preconds,
) (*Object, error)

ComposeObject concatenates sources (same bucket) into dst.

func (*InMemoryBackend) CopyObject 

func (b *InMemoryBackend) CopyObject(req CopyRequest) (*Object, error)

CopyObject copies an object. A nil Meta inherits the source metadata; otherwise Meta supplies the destination metadata.

func (*InMemoryBackend) CreateBucket 

func (b *InMemoryBackend) CreateBucket(project string, p Patch) (*Bucket, error)

CreateBucket creates a bucket in project from the request body p.

func (*InMemoryBackend) DeleteBucket 

func (b *InMemoryBackend) DeleteBucket(name string, pre Preconds) error

DeleteBucket deletes an empty bucket.

func (*InMemoryBackend) DeleteObject 

func (b *InMemoryBackend) DeleteObject(bucket, name string, gen int64, pre Preconds) error

DeleteObject deletes the live version (archiving it in a versioned bucket) or, with gen set, permanently removes that generation.

func (*InMemoryBackend) GetBucket 

func (b *InMemoryBackend) GetBucket(name string, pre Preconds) (*Bucket, error)

GetBucket returns the bucket metadata.

func (*InMemoryBackend) GetIAMPolicy 

func (b *InMemoryBackend) GetIAMPolicy(name string) (IAMPolicy, error)

GetIAMPolicy returns the bucket IAM policy.

func (*InMemoryBackend) GetObject 

func (b *InMemoryBackend) GetObject(bucket, name string, gen int64, pre Preconds) (*Object, error)

GetObject returns generation gen of an object, or the live one when gen is 0.

func (*InMemoryBackend) ListBuckets 

func (b *InMemoryBackend) ListBuckets(project, prefix, token string, limit int) page.Page[*Bucket]

ListBuckets lists the project's buckets whose names start with prefix.

func (*InMemoryBackend) ListObjects 

func (b *InMemoryBackend) ListObjects(bucket string, opts ListOptions) (ListResult, error)

ListObjects lists objects in name order, collapsing delimited prefixes.

func (*InMemoryBackend) PutObject 

func (b *InMemoryBackend) PutObject(bucket, name string, data []byte, p Patch, pre Preconds) (*Object, error)

PutObject creates or overwrites an object.

func (*InMemoryBackend) Reset 

func (b *InMemoryBackend) Reset()

Reset clears all state.

func (*InMemoryBackend) Restore 

func (b *InMemoryBackend) Restore(ctx context.Context, data []byte) error

Restore loads backend state from a snapshot.

func (*InMemoryBackend) SetIAMPolicy 

func (b *InMemoryBackend) SetIAMPolicy(name string, pol IAMPolicy) (IAMPolicy, error)

SetIAMPolicy replaces the bucket IAM policy; a non-empty etag must match.

func (*InMemoryBackend) Snapshot 

func (b *InMemoryBackend) Snapshot(ctx context.Context) []byte

Snapshot serialises the backend state. In-flight resumable sessions are not persisted.

func (*InMemoryBackend) StartUpload 

func (b *InMemoryBackend) StartUpload(bucket, name string, meta Patch, pre Preconds) (string, error)

StartUpload opens a resumable upload session and returns its id.

func (*InMemoryBackend) UpdateBucket 

func (b *InMemoryBackend) UpdateBucket(name string, p Patch, replace bool, pre Preconds) (*Bucket, error)

UpdateBucket applies a PATCH (replace=false) or PUT (replace=true) body.

func (*InMemoryBackend) UpdateObject 

func (b *InMemoryBackend) UpdateObject(
	bucket, name string, gen int64, p Patch, replace bool, pre Preconds,
) (*Object, error)

UpdateObject applies a PATCH (replace=false) or PUT (replace=true) body.

type ListOptions 

type ListOptions struct {
	Prefix                   string
	Delimiter                string
	StartOffset              string
	EndOffset                string
	PageToken                string
	MaxResults               int
	Versions                 bool
	IncludeTrailingDelimiter bool
}

ListOptions are the object list parameters.

type ListResult 

type ListResult struct {
	Next     string
	Items    []*Object
	Prefixes []string
}

ListResult is one page of objects and collapsed prefixes.

type Object 

type Object struct {
	Created            time.Time         `json:"created"`
	Updated            time.Time         `json:"updated"`
	Deleted            time.Time         `json:"deleted"`
	CustomTime         time.Time         `json:"customTime"`
	Metadata           map[string]string `json:"metadata,omitempty"`
	Bucket             string            `json:"bucket"`
	Name               string            `json:"name"`
	ContentType        string            `json:"contentType"`
	ContentEncoding    string            `json:"contentEncoding,omitempty"`
	ContentDisposition string            `json:"contentDisposition,omitempty"`
	ContentLanguage    string            `json:"contentLanguage,omitempty"`
	CacheControl       string            `json:"cacheControl,omitempty"`
	StorageClass       string            `json:"storageClass"`
	MD5                string            `json:"md5,omitempty"`
	CRC32C             string            `json:"crc32c"`
	Data               []byte            `json:"data"`
	Generation         int64             `json:"generation"`
	Metageneration     int64             `json:"metageneration"`
	ComponentCount     int64             `json:"componentCount,omitempty"`
	TemporaryHold      bool              `json:"temporaryHold,omitempty"`
	EventBasedHold     bool              `json:"eventBasedHold,omitempty"`
}

Object is one stored object version. A version whose Deleted time is set is noncurrent; the live version, if any, is always last in its slice.

func (*Object) Size 

func (o *Object) Size() int64

Size returns the object size in bytes.

type Patch 

type Patch map[string]json.RawMessage

Patch is a decoded JSON request body, keyed by field name. A null value clears the field, as in a JSON merge patch.

type Preconds 

type Preconds struct {
	GenMatch     *int64
	GenNotMatch  *int64
	MetaMatch    *int64
	MetaNotMatch *int64
}

Preconds are the ifGeneration*/ifMetageneration* request conditions.

type Provider 

type Provider struct{}

Provider implements service.Provider for Google Cloud Storage. Like the Azure services it runs on its own listener (see Handler.StartWorker) and never matches on the shared AWS router.

func (*Provider) Init 

func (p *Provider) Init(ctx *service.AppContext) (service.Registerable, error)

Init creates the backend and handler; the port is bound in StartWorker.

func (*Provider) Name 

func (p *Provider) Name() string

Name returns the service provider name.

type Settings 

type Settings struct {
	Port int `` //nolint:lll // config struct tags are intentionally verbose
	/* 168-byte string literal not displayed */
}

Settings holds service-level configuration, embedded in the root CLI command as CLI.GCPStorage.

func DefaultSettings 

func DefaultSettings() Settings

DefaultSettings returns the settings used when no ConfigProvider is available.

type UploadProgress 

type UploadProgress struct {
	Object   *Object
	Received int64
}

UploadProgress reports a session's state after an append.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.