bigquery

package
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 10, 2026 License: Apache-2.0 Imports: 23 Imported by: 0

Documentation

Index

Constants

View Source
const (
	// No write operations are allowed.
	WriteModeBlocked string = "blocked"
	// Only protected write operations are allowed in a BigQuery session.
	WriteModeProtected string = "protected"
	// All write operations are allowed.
	WriteModeAllowed string = "allowed"
)
View Source
const CloudPlatformScope = "https://www.googleapis.com/auth/cloud-platform"

CloudPlatformScope is a broad scope for Google Cloud Platform services.

View Source
const SourceType string = "bigquery"

Variables

This section is empty.

Functions

func NormalizeValue

func NormalizeValue(v any) any

NormalizeValue converts BigQuery specific types to standard JSON-compatible types. Specifically, it handles *big.Rat (used for NUMERIC/BIGNUMERIC) by converting them to decimal strings with up to 38 digits of precision, trimming trailing zeros. It recursively handles slices (arrays) and maps (structs) using reflection.

Types

type BigQuerySessionProvider

type BigQuerySessionProvider func(ctx context.Context) (*Session, error)

type BigqueryClientCreator

type BigqueryClientCreator func(tokenString string, wantRestService bool) (*bigqueryapi.Client, *bigqueryrestapi.Service, error)

type Config

type Config struct {
	// BigQuery configs
	Name                      string              `yaml:"name" validate:"required"`
	Type                      string              `yaml:"type" validate:"required"`
	Project                   string              `yaml:"project" validate:"required"`
	Location                  string              `yaml:"location"`
	WriteMode                 string              `yaml:"writeMode"`
	AllowedDatasets           StringOrStringSlice `yaml:"allowedDatasets"`
	UseClientOAuth            string              `yaml:"useClientOAuth"`
	ImpersonateServiceAccount string              `yaml:"impersonateServiceAccount"`
	Scopes                    StringOrStringSlice `yaml:"scopes"`
	MaxQueryResultRows        int                 `yaml:"maxQueryResultRows"`
}

func (Config) Initialize

func (r Config) Initialize(ctx context.Context, tracer trace.Tracer) (sources.Source, error)

func (Config) SourceConfigType

func (r Config) SourceConfigType() string

type DataplexClientCreator

type DataplexClientCreator func(tokenString string) (*dataplexapi.CatalogClient, error)

type Session

type Session struct {
	ID           string
	ProjectID    string
	DatasetID    string
	CreationTime time.Time
	LastUsed     time.Time
}

type Source

type Source struct {
	Config
	Client              *bigqueryapi.Client
	RestService         *bigqueryrestapi.Service
	TokenSource         oauth2.TokenSource
	AuthTokenHeaderName string
	MaxQueryResultRows  int
	ClientCreator       BigqueryClientCreator
	AllowedDatasets     map[string]struct{}

	SessionProvider BigQuerySessionProvider
	Session         *Session
	// contains filtered or unexported fields
}

func (*Source) BigQueryAllowedDatasets

func (s *Source) BigQueryAllowedDatasets() []string

func (*Source) BigQueryClient

func (s *Source) BigQueryClient() *bigqueryapi.Client

func (*Source) BigQueryClientCreator

func (s *Source) BigQueryClientCreator() BigqueryClientCreator

func (*Source) BigQueryLocation

func (s *Source) BigQueryLocation() string

func (*Source) BigQueryProject

func (s *Source) BigQueryProject() string

func (*Source) BigQueryRestService

func (s *Source) BigQueryRestService() *bigqueryrestapi.Service

func (*Source) BigQuerySession

func (s *Source) BigQuerySession() BigQuerySessionProvider

func (*Source) BigQueryTokenSource

func (s *Source) BigQueryTokenSource() oauth2.TokenSource

func (*Source) BigQueryTokenSourceWithScope

func (s *Source) BigQueryTokenSourceWithScope(ctx context.Context, scopes []string) (oauth2.TokenSource, error)

func (*Source) BigQueryWriteMode

func (s *Source) BigQueryWriteMode() string

func (*Source) GetAuthTokenHeaderName

func (s *Source) GetAuthTokenHeaderName() string

func (*Source) GetMaxQueryResultRows

func (s *Source) GetMaxQueryResultRows() int

func (*Source) IsDatasetAllowed

func (s *Source) IsDatasetAllowed(projectID, datasetID string) bool

IsDatasetAllowed checks if a given dataset is accessible based on the source's configuration.

func (*Source) MakeDataplexCatalogClient

func (s *Source) MakeDataplexCatalogClient() func() (*dataplexapi.CatalogClient, DataplexClientCreator, error)

func (*Source) RetrieveClientAndService

func (s *Source) RetrieveClientAndService(accessToken tools.AccessToken) (*bigqueryapi.Client, *bigqueryrestapi.Service, error)

func (*Source) RunSQL

func (s *Source) RunSQL(ctx context.Context, bqClient *bigqueryapi.Client, statement, statementType string, params []bigqueryapi.QueryParameter, connProps []*bigqueryapi.ConnectionProperty) (any, error)

func (*Source) SourceType

func (s *Source) SourceType() string

func (*Source) ToConfig

func (s *Source) ToConfig() sources.SourceConfig

func (*Source) UseClientAuthorization

func (s *Source) UseClientAuthorization() bool

type StringOrStringSlice

type StringOrStringSlice []string

StringOrStringSlice is a custom type that can unmarshal both a single string (which it splits by comma) and a sequence of strings into a string slice.

func (*StringOrStringSlice) UnmarshalYAML

func (s *StringOrStringSlice) UnmarshalYAML(unmarshal func(any) error) error

UnmarshalYAML implements the yaml.Unmarshaler interface.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL