bigqueryread

package
v0.60.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 5, 2026 License: Apache-2.0 Imports: 24 Imported by: 0

README

DataTug BigQuery CLI composition

This package connects the released github.com/dal-go/dalgo2bigquery v0.1.1 protocol to the actual datatug query bigquery Cobra commands. Fixture tests exercise protected preview, approval, one submission and two pages through the real command tree. Live Google authentication and paid execution have not been executed for this change. Source rights/admission and both runtime live gates remain open.

--file accepts typed JSON containing profile and scalar DTQL query. Alternatively, --source-profile accepts the reviewed profile JSON and --file accepts scalar DTQL JSON separately. Each source/query input is bounded to 256 KiB, with unknown fields, duplicate keys, invalid Unicode, unsafe query shapes and excessive depth rejected before policy substitution or authentication. Only explicit fields, bounded scalar conditions/order and a positive limit are supported. Whole-AST guards precede generic policy/substitution paths, and the released compiler validates the policy-effective result. The copied driver AST normalizes DALgo policy In arrays to constant slices; empty lists are supported, 1000 elements are the limit, and nested, null, mixed-type or expression elements refuse before authentication. The generic policy AST is never mutated.

The commands are:

  • connect --auth google --ledger <absolute-private-directory> deliberately opens Google consent using the existing credential storage owner. Its callback binds IPv4 loopback before browser launch, uses random state and PKCE, admits one matching GET callback, launches an owned cancellable OS opener, and closes its listener and accepted HTTP connections under a bounded context. Read-only plus openid is the baseline; --enable-cancellation separately requests BigQuery scope. It separately attests the fresh consent token and stored credential and refuses success if their account/grant bindings differ. Other operations never launch consent or a listener.
  • preview requires explicit --auth adc|google, --execution-project and --maximum-bytes-billed. --session-budget-bytes defaults to that exact cap; --page-size is bound by the preview. Stdout includes effective SQL/typed parameters, observed source, independently verified Google principal, estimate, cap, bounds and approval digest. --preview-out optionally writes a private preview-*.json artifact within the ledger directory.
  • run accepts --preview and exact --approve-digest, with the same protected source/query/context. It reauthorizes and rechecks before one submission and emits the first typed page. --receipt-out writes only receipt/cursor to a private receipt-*.json artifact within the ledger directory, without rows. Artifacts are committed before stdout, and operation, persistence and output failures remain distinct.
  • page --receipt <receipt-or-page-json> reads the next page of that job. --cursor can supply the original opaque cursor separately. --reconnect is deliberate same-subject reauthorization, preserving original approval, counters, deadline and job; it cannot replay a query. status and separately enabled cancel use the same receipt and truthful authoritative control outcome. Control and the driver's local authoritative Snapshot share one bounded context. The latest receipt/counters/billing and original issued cursor are exported with meaningful control outcomes, including partial failures. A failed snapshot preserves the previous artifact and both errors. Control outputs do not read or replay rows, even after execution expiry.

Every operation reuses the same private --ledger directory. A new directory is a new authorization and budget session, not continuation of an existing job. No cap/row/page/wall bound can be renewed through a continuation flag. Ctrl-C, local close and expiry keep known job receipts or submission_unknown; cancellation is never inferred from stopping local waiting. Error output preserves existing receipt authority when available and never includes raw service errors or tokens. Policy stderr contains policy/rule/field metadata, never predicate literals, bindings or explanations; explicit preview JSON retains the requested parameters. Output is --format json only. Preview recovery is bounded to 2 MiB and page recovery to 5 MiB; compact receipt artifacts retain the 256 KiB input bound.

--as/--role/--group/--var/--policy/--policies-dir/--no-policies retain DataTug's policy conventions. Those labels do not attest Google execution identity. --execution-project remains distinct from saved-query --project.

Google-user ADC is an explicit opt-in and does not choose an execution project. Workload/service-account ADC is refused here; its authoritative identity belongs to an operator-injected provider in the driver's server composition. --auth google accurately names the existing user-controlled stored OAuth grant. Scope options and stored requested scopes are not capability evidence. Only current Google token-response scope, explicit expiry, and same-token UserInfo sub attest the current grant. Missing scope, openid, BigQuery grant or stable subject refuses with setup guidance; cloud-platform never substitutes for the BigQuery grant. No token enters this package's ledger, artifacts or diagnostics.

For this Go CLI, generation binds a persistent nonsecret private authorization session nonce, verified Google-user subject and exact current granted-scope set. Refreshing an access token retains that generation only after identity/grants are verified again. Subject/grant changes invalidate pending approvals; deliberate same-subject job rebind is required for grant changes. The returned transport uses an oauth2.StaticTokenSource snapshot of the exact verified token over the driver request guard. This narrow Go session interpretation does not change browser GIS rules, and remains subject to independent review.

Documentation

Overview

Package bigqueryread composes the released BigQuery driver with DataTug policies.

Index

Constants

View Source 
const MaxInputBytes = 256 << 10

Variables

View Source 
var ErrIdentity = errors.New("google execution identity or granted scopes unavailable; explicitly sign in with Google BigQuery read-only and openid scopes, then preview again")
View Source 
var ErrInput = errors.New("invalid BigQuery input: expected bounded typed JSON")

Functions

func Decode 

func Decode(r io.Reader, target any) error

func DecodeLimit 

func DecodeLimit(r io.Reader, target any, limit int) error

DecodeLimit is used only for bounded preview/result recovery envelopes.

func Guard 

func Guard(query dal.Query) error

Guard permits bounded RHS parameters before shared substitution. All other shape checks use the released driver's whole-AST guard, without a paid leaf.

Types

type Column 

type Column struct {
	Field string `json:"field"`
}

type Condition 

type Condition struct {
	Op        string      `json:"op,omitempty"`
	Left      *Expression `json:"left,omitempty"`
	Right     *Expression `json:"right,omitempty"`
	And       []Condition `json:"and,omitempty"`
	Or        []Condition `json:"or,omitempty"`
	IsNull    *Column     `json:"isNull,omitempty"`
	IsNotNull *Column     `json:"isNotNull,omitempty"`
}

type Expression 

type Expression struct {
	Field string          `json:"field,omitempty"`
	Param string          `json:"param,omitempty"`
	Value json.RawMessage `json:"value,omitempty"`
}

type From 

type From struct {
	Name string `json:"name"`
}

type GoogleProvider 

type GoogleProvider struct {
	// contains filtered or unexported fields
}

GoogleProvider uses the existing explicit Google login's refresh-token source. It never opens a browser or uses ADC. Granted scope comes only from the Google token response, and the SAME token verifies sub at fixed HTTPS UserInfo.

func NewADCProvider 

func NewADCProvider(dir string, enableCancel bool) (*GoogleProvider, error)

NewADCProvider reads user-controlled ADC only after explicit --auth adc. Scope options request grants but cannot attest them. Workload ADC is refused; its authoritative subject requires a separate trusted operator provider.

func NewGoogleProvider 

func NewGoogleProvider(dir string, enableCancel bool) (*GoogleProvider, error)

NewGoogleProvider must follow NewFileLedger admission of dir's private path. The nonce is nonsecret and contains no token, email, policy label or credential.

func (*GoogleProvider) Authorize 

func (p *GoogleProvider) Authorize(ctx context.Context, guarded http.RoundTripper) (bigquery.Identity, http.RoundTripper, error)

func (*GoogleProvider) AuthorizeToken 

func (p *GoogleProvider) AuthorizeToken(ctx context.Context, token *oauth2.Token, guarded http.RoundTripper) (bigquery.Identity, http.RoundTripper, error)

AuthorizeToken attests the exact token returned by an explicit Google consent exchange. It is used to compare that consent with the credential store before connect reports success; it never stores the access token or opens consent.

type Input 

type Input struct {
	Profile bigquery.SourceProfile `json:"profile"`
	Query   QueryShape             `json:"query"`
}

Input is an explicit scalar DTQL query and reviewed native source profile. QueryShape intentionally excludes SQL, joins, subqueries, aliases and offsets.

func (Input) DALQuery 

func (in Input) DALQuery() (dal.Query, error)

type Order 

type Order struct {
	Field string `json:"field"`
	Desc  bool   `json:"desc,omitempty"`
}

type Preparer 

type Preparer struct {
	Input   Input
	Options func() (accesspolicies.Options, error)
}

Preparer reloads the exact policy documents on every operation. User-supplied policy principal labels never attest the independent Google execution subject.

func (Preparer) Prepare 

func (p Preparer) Prepare(ctx context.Context) (bigquery.ReadPlan, string, error)

func (Preparer) PrepareWithReport 

func (p Preparer) PrepareWithReport(ctx context.Context) (bigquery.ReadPlan, string, []accesspolicies.Line, error)

type QueryShape 

type QueryShape struct {
	From    From       `json:"from"`
	Columns []Column   `json:"columns"`
	Where   *Condition `json:"where,omitempty"`
	OrderBy []Order    `json:"orderBy,omitempty"`
	Limit   int        `json:"limit"`
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.