Help us shape the future of Go! Take the 2026 Go Developer Survey

gcpbigquery

package
v1.9.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 27, 2026 License: MIT Imports: 33 Imported by: 0

README

BigQuery

Parity grade: B · SDK cloud.google.com/go/bigquery@v1.85.0 · last audited 2026-10-19 (15ea1e35b)

Coverage

Metric Value
PARITY entries audited 23 (22 ok, 1 partial)
Feature families 10 (7 ok, 3 partial)
Known gaps 8
Deferred items 0
Resource leaks clean
Known gaps
  • Not served: routines, models, row access policies, table and dataset IAM policies, datasets undelete, snapshots/clones as storage objects, materialized views, external and BigLake tables, reservations, connections, data transfers, analytics hub, and the Storage Read/Write gRPC APIs.
  • Unsupported GoogleSQL: WITH RECURSIVE, GROUP BY ROLLUP/CUBE/GROUPING SETS, PIVOT/UNPIVOT, TABLESAMPLE, FOR SYSTEM_TIME AS OF, named WINDOW clauses, ARRAY_AGG ... LIMIT, INTERSECT/EXCEPT ALL, CAST ... FORMAT, AT TIME ZONE, scripting (DECLARE, SET, IF, LOOP, several statements, transactions), CREATE FUNCTION/PROCEDURE and other UDFs, wildcard tables (_TABLE_SUFFIX), partition pseudo columns, selecting a whole row as a STRUCT, legacy SQL.
  • Unsupported functions: GEOGRAPHY (ST_), INTERVAL and RANGE values, HLL and approximate aggregates other than APPROX_COUNT_DISTINCT, APPROX_QUANTILES/TOP_COUNT, FARM_FINGERPRINT, NET., ML.*, SEARCH, JSON constructors and mutators beyond JSON_VALUE/JSON_QUERY/PARSE_JSON/TO_JSON_STRING, FORMAT patterns beyond %s %d %i %f %g %e %t %T %x %o, collation.
  • Engine fidelity: integer overflow in +, - and * is not detected, an out-of-range array OFFSET returns NULL instead of an error, NUMERIC and BIGNUMERIC are computed in float64, and result types of untyped expressions (for example an all-NULL column) default to STRING.
  • Jobs run synchronously: there are no PENDING/RUNNING states, no slot or priority scheduling, no query cache, no timeoutMs waiting, no cancellation of running work and no requestId idempotency; anonymous result tables are not created.
  • Table settings are stored but not enforced: partitioning and clustering do not prune, requirePartitionFilter and expirationTime do nothing, and streaming-buffer metadata is not reported.
  • Load: AVRO, ORC, Datastore and Firestore backups, hive partitioning, nested CSV, custom quote characters, schemaUpdateOptions and column-name flexibility options are not supported. Extract: AVRO is not supported, output is a single object (no size-based sharding, only the first destination URI is written), and the NDJSON TIMESTAMP and CSV float rendering have not been checked against the live service.
  • insertAll does not honour templateSuffix and de-duplicates by insertId only within one minute; getQueryResults ignores timeoutMs.

More

Documentation

Index

Constants

View Source 
const DefaultPort = 9050

DefaultPort is the dedicated listener's default TCP port.

Variables

View Source 
var ErrNilAppContext = errors.New("gcpbigquery: nil app context")

ErrNilAppContext is returned when Init is called with a nil AppContext.

View Source 
var ErrObjectNotFound = errors.New("object not found")

ErrObjectNotFound is returned by an ObjectStore when a bucket or object does not exist.

View Source 
var ErrSnapshotNull = errors.New("gcpbigquery: snapshot contains a null dataset")

ErrSnapshotNull is returned when a snapshot holds a null dataset.

Functions

This section is empty.

Types

type APIError 

type APIError struct {
	Reason  string
	Message string
	Status  int
}

APIError is a REST error with the HTTP status and BigQuery reason it surfaces as.

func (*APIError) Error 

func (e *APIError) Error() string

Error implements error.

type ConfigProvider 

type ConfigProvider interface {
	GetGCPBigQuerySettings() Settings
}

ConfigProvider extracts BigQuery configuration from the AppContext Config.

type DataOptions 

type DataOptions struct {
	PageToken      string
	SelectedFields string
	StartIndex     int
	Max            int
	Int64Timestamp bool
}

DataOptions are the tabledata.list parameters.

type Dataset 

type Dataset struct {
	Meta     map[string]any    `json:"meta"`
	Tables   map[string]*Table `json:"tables"`
	Location string            `json:"location"`
	Created  int64             `json:"created"`
	Modified int64             `json:"modified"`
	Hidden   bool              `json:"hidden,omitempty"`
}

Dataset is a BigQuery dataset. Meta holds the writable passthrough fields (labels, access, description, ...).

type Handler 

type Handler struct {
	Backend *InMemoryBackend

	Port int
	// contains filtered or unexported fields
}

Handler serves the BigQuery v2 REST API.

func NewHandler 

func NewHandler(backend *InMemoryBackend) *Handler

NewHandler creates a Handler; callers override Port from Settings.

func (*Handler) ExtractOperation 

func (h *Handler) ExtractOperation(c *echo.Context) string

ExtractOperation returns the operation name for metrics.

func (*Handler) ExtractResource 

func (h *Handler) ExtractResource(c *echo.Context) string

ExtractResource returns project[/dataset[.table]] for metrics.

func (*Handler) GetSupportedOperations 

func (h *Handler) GetSupportedOperations() []string

GetSupportedOperations lists the supported operations.

func (*Handler) Handler 

func (h *Handler) Handler() echo.HandlerFunc

Handler returns the Echo handler for all BigQuery requests.

func (*Handler) MatchPriority 

func (h *Handler) MatchPriority() int

MatchPriority is irrelevant because RouteMatcher never matches.

func (*Handler) Name 

func (h *Handler) Name() string

Name returns the service name.

func (*Handler) Reset 

func (h *Handler) Reset()

Reset clears all in-memory state.

func (*Handler) Restore 

func (h *Handler) Restore(ctx context.Context, data []byte) error

Restore implements persistence.Persistable by delegating to the backend.

func (*Handler) RouteMatcher 

func (h *Handler) RouteMatcher() service.Matcher

RouteMatcher never matches: BigQuery runs on its own listener.

func (*Handler) Shutdown 

func (h *Handler) Shutdown(ctx context.Context)

Shutdown stops the dedicated listener, forcing a close if graceful shutdown fails.

func (*Handler) Snapshot 

func (h *Handler) Snapshot(ctx context.Context) []byte

Snapshot implements persistence.Persistable by delegating to the backend.

func (*Handler) StartWorker 

func (h *Handler) StartWorker(ctx context.Context) error

StartWorker binds the dedicated listener synchronously so a port conflict is returned to the caller.

type InMemoryBackend 

type InMemoryBackend struct {
	// contains filtered or unexported fields
}

InMemoryBackend holds every dataset, table, row and job under one lock.

func NewInMemoryBackend 

func NewInMemoryBackend() *InMemoryBackend

NewInMemoryBackend creates an empty backend.

func (*InMemoryBackend) CancelJob 

func (b *InMemoryBackend) CancelJob(project, id string) (map[string]any, error)

CancelJob implements jobs.cancel; every job finishes inside the insert call so this reports the final job.

func (*InMemoryBackend) CreateDataset 

func (b *InMemoryBackend) CreateDataset(project string, body map[string]any) (map[string]any, error)

CreateDataset implements datasets.insert.

func (*InMemoryBackend) CreateTable 

func (b *InMemoryBackend) CreateTable(project, ds string, body map[string]any) (map[string]any, error)

CreateTable implements tables.insert.

func (*InMemoryBackend) DeleteDataset 

func (b *InMemoryBackend) DeleteDataset(project, id string, deleteContents bool) error

DeleteDataset implements datasets.delete.

func (*InMemoryBackend) DeleteJob 

func (b *InMemoryBackend) DeleteJob(project, id string) error

DeleteJob implements jobs.delete.

func (*InMemoryBackend) DeleteTable 

func (b *InMemoryBackend) DeleteTable(project, ds, id string) error

DeleteTable implements tables.delete.

func (*InMemoryBackend) GetDataset 

func (b *InMemoryBackend) GetDataset(project, id string) (map[string]any, error)

GetDataset implements datasets.get.

func (*InMemoryBackend) GetJob 

func (b *InMemoryBackend) GetJob(project, id string) (map[string]any, error)

GetJob implements jobs.get.

func (*InMemoryBackend) GetQueryResults 

func (b *InMemoryBackend) GetQueryResults(project, id string, o QueryResultOptions) (map[string]any, error)

GetQueryResults implements jobs.getQueryResults.

func (*InMemoryBackend) GetTable 

func (b *InMemoryBackend) GetTable(project, ds, id string) (map[string]any, error)

GetTable implements tables.get.

func (*InMemoryBackend) InsertAll 

func (b *InMemoryBackend) InsertAll(project, ds, id string, req InsertRequest) (map[string]any, error)

InsertAll implements tabledata.insertAll.

func (*InMemoryBackend) InsertJob 

func (b *InMemoryBackend) InsertJob(
	ctx context.Context,
	project string,
	body map[string]any,
	media []byte,
) (map[string]any, error)

InsertJob implements jobs.insert; media carries the body of a multipart or resumable upload for load jobs.

func (*InMemoryBackend) ListData 

func (b *InMemoryBackend) ListData(project, ds, id string, o DataOptions) (map[string]any, error)

ListData implements tabledata.list.

func (*InMemoryBackend) ListDatasets 

func (b *InMemoryBackend) ListDatasets(project string, o ListOptions) page.Page[map[string]any]

ListDatasets implements datasets.list.

func (*InMemoryBackend) ListJobs 

func (b *InMemoryBackend) ListJobs(project string, o JobListOptions) page.Page[map[string]any]

ListJobs implements jobs.list, newest first.

func (*InMemoryBackend) ListProjects 

func (b *InMemoryBackend) ListProjects() []map[string]any

ListProjects implements projects.list over every project that owns a dataset or a job.

func (*InMemoryBackend) ListTables 

func (b *InMemoryBackend) ListTables(project, ds string, o ListOptions) (page.Page[map[string]any], error)

ListTables implements tables.list.

func (*InMemoryBackend) Reset 

func (b *InMemoryBackend) Reset()

Reset clears all state.

func (*InMemoryBackend) Restore 

func (b *InMemoryBackend) Restore(ctx context.Context, data []byte) error

Restore loads backend state from a snapshot.

func (*InMemoryBackend) RunQuery 

func (b *InMemoryBackend) RunQuery(ctx context.Context, project string, req map[string]any) (map[string]any, error)

RunQuery implements jobs.query: it creates a query job, runs it and returns the first page of results.

func (*InMemoryBackend) SetObjectStore 

func (b *InMemoryBackend) SetObjectStore(s ObjectStore)

SetObjectStore wires the object store load and extract jobs read from and write to.

func (*InMemoryBackend) Snapshot 

func (b *InMemoryBackend) Snapshot(ctx context.Context) []byte

Snapshot serialises the backend state.

func (*InMemoryBackend) UpdateDataset 

func (b *InMemoryBackend) UpdateDataset(project, id string, body map[string]any, replace bool) (map[string]any, error)

UpdateDataset implements datasets.patch (merge) and datasets.update (replace).

func (*InMemoryBackend) UpdateTable 

func (b *InMemoryBackend) UpdateTable(
	project, ds, id string,
	body map[string]any,
	replace bool,
) (map[string]any, error)

UpdateTable implements tables.patch (merge) and tables.update (replace).

type InsertRequest 

type InsertRequest struct {
	Rows                []InsertRow `json:"rows"`
	SkipInvalidRows     bool        `json:"skipInvalidRows"`
	IgnoreUnknownValues bool        `json:"ignoreUnknownValues"`
}

InsertRequest is the tabledata.insertAll request body.

type InsertRow 

type InsertRow struct {
	JSON     map[string]any `json:"json"`
	InsertID string         `json:"insertId"`
}

InsertRow is one row of a streaming insert.

type Job 

type Job struct {
	Config      map[string]any `json:"config"`
	Stats       map[string]any `json:"stats"`
	ErrorResult *jobError      `json:"errorResult,omitempty"`
	Dest        *tableRef      `json:"dest,omitempty"`
	Project     string         `json:"project"`
	ID          string         `json:"id"`
	Location    string         `json:"location"`
	Type        string         `json:"type"`
	State       string         `json:"state"`
	User        string         `json:"user"`
	Schema      []bqsql.Field  `json:"schema,omitempty"`
	Rows        [][]any        `json:"rows,omitempty"`
	Created     int64          `json:"created"`
	Started     int64          `json:"started"`
	Ended       int64          `json:"ended"`
	NumDML      int64          `json:"numDml"`
	DryRun      bool           `json:"dryRun,omitempty"`
	HasResult   bool           `json:"hasResult,omitempty"`
}

Job is a BigQuery job. Query jobs keep their result rows so getQueryResults can page them.

type JobListOptions 

type JobListOptions struct {
	PageToken   string
	StateFilter []string
	MinCreated  int64
	MaxCreated  int64
	Max         int
}

JobListOptions are the jobs.list parameters.

type ListOptions 

type ListOptions struct {
	Filter    string
	PageToken string
	Max       int
	All       bool
}

ListOptions carries the paging parameters shared by list calls.

type ObjectStore 

type ObjectStore interface {
	// ListObjects returns the names of every object in bucket that starts with prefix.
	ListObjects(bucket, prefix string) ([]string, error)
	// ReadObject returns the content of one object.
	ReadObject(bucket, name string) ([]byte, error)
	// WriteObject creates or replaces one object.
	WriteObject(bucket, name string, data []byte, contentType string) error
}

ObjectStore is the slice of Cloud Storage that load and extract jobs need.

type Provider 

type Provider struct{}

Provider implements service.Provider for Google BigQuery. It serves the REST API on its own listener (see Handler.StartWorker) and never matches on the shared AWS router.

func (*Provider) Init 

func (p *Provider) Init(ctx *service.AppContext) (service.Registerable, error)

Init creates the backend and handler; the port is bound in StartWorker.

func (*Provider) Name 

func (p *Provider) Name() string

Name returns the service provider name.

type QueryResultOptions 

type QueryResultOptions struct {
	PageToken      string
	StartIndex     int
	Max            int
	Int64Timestamp bool
}

QueryResultOptions are the getQueryResults parameters.

type Settings 

type Settings struct {
	Port int `` //nolint:lll // config struct tags are intentionally verbose
	/* 164-byte string literal not displayed */
}

Settings holds service-level configuration, embedded in the root CLI command as CLI.GCPBigQuery.

func DefaultSettings 

func DefaultSettings() Settings

DefaultSettings returns the settings used when no ConfigProvider is available.

type Table 

type Table struct {
	Meta      map[string]any   `json:"meta"`
	Seen      map[string]int64 `json:"seen,omitempty"`
	ViewQuery string           `json:"viewQuery,omitempty"`
	Schema    []bqsql.Field    `json:"schema"`
	Rows      [][]any          `json:"rows"`
	Created   int64            `json:"created"`
	Modified  int64            `json:"modified"`
	Seq       int64            `json:"seq"`
}

Table is a table or view with its rows in the canonical stored encoding.

Source Files

View all Source files

Directories

Path Synopsis
Package bqsql runs a GoogleSQL subset on SQLite and converts BigQuery values between their REST encodings.
 Package bqsql runs a GoogleSQL subset on SQLite and converts BigQuery values between their REST encodings.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.