Documentation
¶
Index ¶
- func CollectBigQueryStats(ctx context.Context, job *bigquery.Job)
- func IsPermissionError(err error) bool
- func NewHTTPClient(ctx context.Context, conf *BigQueryConf, scopes ...string) (*http.Client, error)
- type BigQueryConf
- type BigQueryExecutor
- func (e *BigQueryExecutor) Close() error
- func (e *BigQueryExecutor) Exec(ctx context.Context, sql string) error
- func (e *BigQueryExecutor) GetBigQueryClient() *bigquery.Client
- func (e *BigQueryExecutor) NewQuery(ctx context.Context, sql string, args ...any) *bigquery.Query
- func (e *BigQueryExecutor) QueryRowsIterator(ctx context.Context, sql string, args ...interface{}) (*bigquery.RowIterator, error)
- type Executor
- type Querier
- func (q *Querier[T]) Close() error
- func (q *Querier[T]) Exec(ctx context.Context, sql string) error
- func (q *Querier[T]) QueryAndProcessMany(ctx context.Context, sql string, ...) error
- func (q *Querier[T]) QueryMany(ctx context.Context, sql string, opts ...exec.QueryManyOpt[T]) ([]*T, error)
- func (q *Querier[T]) QueryMaps(ctx context.Context, sql string) ([]exec.QueryMapResult, error)
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func CollectBigQueryStats ¶ added in v0.9.0
CollectBigQueryStats populates the DriverStats with BigQuery job statistics if a stats callback is registered in the context.
func IsPermissionError ¶ added in v0.15.3
IsPermissionError reports whether err indicates the DWH credentials lack the privileges required for the attempted operation.
func NewHTTPClient ¶ added in v0.21.0
NewHTTPClient returns an HTTP client authenticated the way conf says (AccessToken > CredentialsJson > CredentialsFile) for the given OAuth scopes, with conf's per-request timeout. The BigQuery client and the scrapper's Resource Manager and IAM calls all go through it, so they act as the same identity.
Types ¶
type BigQueryConf ¶
type BigQueryConf struct {
// ProjectId is the GCP project containing BigQuery datasets.
ProjectId string
// Region restricts queries to a specific BQ region (e.g. "EU", "us-central1").
Region string
// CredentialsJson is a GCP service account key in JSON format.
CredentialsJson string
// CredentialsFile is a path to a GCP service account key file.
CredentialsFile string
// AccessToken is an OAuth access token for user-level authentication.
// When set, it takes precedence over CredentialsJson and CredentialsFile.
AccessToken string
// HTTPTimeout caps each BigQuery REST round-trip (table/dataset metadata,
// table listing, query-result pages). Guards against a stalled connection
// hanging a scrape until the caller's job deadline. Zero uses
// defaultHTTPTimeout.
HTTPTimeout time.Duration
}
BigQueryConf configures the BigQuery executor connection. Authentication precedence: AccessToken > CredentialsJson > CredentialsFile.
type BigQueryExecutor ¶
type BigQueryExecutor struct {
// contains filtered or unexported fields
}
func NewBigqueryExecutor ¶
func NewBigqueryExecutor(ctx context.Context, conf *BigQueryConf) (*BigQueryExecutor, error)
func NewBigqueryExecutorFromClient ¶ added in v0.21.0
func NewBigqueryExecutorFromClient(client *bigquery.Client, conf *BigQueryConf) *BigQueryExecutor
NewBigqueryExecutorFromClient wraps a client the caller has already built, without the connection check NewBigqueryExecutor makes.
func (*BigQueryExecutor) Close ¶
func (e *BigQueryExecutor) Close() error
func (*BigQueryExecutor) Exec ¶
func (e *BigQueryExecutor) Exec(ctx context.Context, sql string) error
func (*BigQueryExecutor) GetBigQueryClient ¶
func (e *BigQueryExecutor) GetBigQueryClient() *bigquery.Client
func (*BigQueryExecutor) NewQuery ¶ added in v0.21.0
NewQuery builds a query job for sql that carries the query context of ctx: the SQL comment, which INFORMATION_SCHEMA.JOBS keeps in the query text, and the job labels. args become positional parameters. Every job we start is built here, so query-history readers can tell our own traffic apart.
func (*BigQueryExecutor) QueryRowsIterator ¶
func (e *BigQueryExecutor) QueryRowsIterator( ctx context.Context, sql string, args ...interface{}, ) (*bigquery.RowIterator, error)
type Querier ¶
type Querier[T any] struct { // contains filtered or unexported fields }
func NewQuerier ¶
func NewQuerier[T any](conn *BigQueryExecutor) *Querier[T]
Source Files