Documentation
¶
Overview ¶
Package warehouse provides read-only analytics queries (backyard.warehouse) against BigQuery, Postgres-compatible databases (incl. Redshift) and Athena. Sources live in sub-packages; service.Service enforces policy (raw SQL gating, statement guard, named queries, row and size caps, timeouts).
Index ¶
Constants ¶
const DefaultMaxResultBytes = 32 << 20
DefaultMaxResultBytes is the default result-size cap (Limits.MaxBytes): 32 MiB of estimated row data.
Variables ¶
var ( // ErrUpstream: the engine failed to run the query (connection failure, // internal engine error, quota exhaustion, …). ErrUpstream = errors.New("warehouse: upstream error") // ErrTimeout: the query exceeded warehouse.timeout (or the engine's // statement timeout). ErrTimeout = errors.New("warehouse: query timed out") )
Additional sentinel errors. Together with ErrDisabled, ErrForbidden, ErrNotFound and ErrBadRequest they let the HTTP layer map every failure to a SPEC §5 error code without inspecting engine-specific error types:
ErrDisabled → 501 unsupported (or 404 not_found) ErrBadRequest → 400 bad_request (invalid params, SQL syntax / semantic errors) ErrForbidden → 403 forbidden (raw SQL disabled, statement guard, read-only violation, permission denied) ErrNotFound → 404 not_found (unknown source or named query) ErrTimeout → 504 upstream_error (warehouse.timeout exceeded) ErrUpstream → 502 upstream_error (engine or connectivity failure)
Errors returned by Service.Query wrap exactly one of these sentinels, except when the caller's own context was canceled (then they wrap context.Canceled).
var ( // ErrDisabled: no warehouse configured. ErrDisabled = errors.New("warehouse: not configured") // ErrForbidden: raw SQL disabled, or statement not allowed. ErrForbidden = errors.New("warehouse: query not allowed") // ErrNotFound: unknown source or named query. ErrNotFound = errors.New("warehouse: not found") // ErrBadRequest: invalid parameters. ErrBadRequest = errors.New("warehouse: bad request") )
Functions ¶
This section is empty.
Types ¶
type Column ¶
type Column struct {
Name string `json:"name"`
Type string `json:"type"` // engine type name, e.g. STRING, INT64, DATE, int4, varchar
}
Column describes a result column.
type Limits ¶
type Limits struct {
// MaxRows is the maximum number of rows returned (values <= 0 mean 1).
MaxRows int
// MaxBytes caps the estimated size of the returned rows: their JSON
// encoding plus a fixed per-value overhead for holding them in memory
// (values <= 0 mean DefaultMaxResultBytes). Sources stop at the first
// row that does not fit and never buffer a row much larger than what
// is left, so one query cannot make the server hold an arbitrarily
// large result.
MaxBytes int
}
Limits bound the result of Source.Query.
func (Limits) WithDefaults ¶
WithDefaults returns l with its zero or negative fields replaced by their defaults.
type QueryInfo ¶
type QueryInfo struct {
Name string `json:"name"`
Description string `json:"description,omitempty"`
Params []string `json:"params"`
Source string `json:"source"`
}
QueryInfo describes a named query.
type QueryRequest ¶
type QueryRequest struct {
SQL string `json:"sql,omitempty"`
Query string `json:"query,omitempty"` // named query
Params any `json:"params,omitempty"` // []any for sql, map[string]any for named
Source string `json:"source,omitempty"`
Extra map[string]any `json:"-"`
}
QueryRequest is the client request (SPEC §5.5).
type Result ¶
type Result struct {
Columns []Column `json:"columns"`
Rows []map[string]any `json:"rows"`
RowCount int `json:"row_count"`
Truncated bool `json:"truncated"`
}
Result is a query result. Values are JSON-friendly: strings, float64/int64, bool, nil, RFC3339 strings for timestamps, ISO dates for dates. Truncated reports that the query had more rows than the result holds (row or size cap).
type Service ¶
type Service interface {
Query(ctx context.Context, req QueryRequest) (*Result, error)
Describe() (sources []SourceInfo, queries []QueryInfo)
Enabled() bool
Close() error
}
Service is what the HTTP layer uses. Implemented by service.Service.
type Source ¶
type Source interface {
// Query runs sql with positional args and returns the rows that fit
// within lim (Truncated=true if more were available).
Query(ctx context.Context, sql string, args []any, lim Limits) (*Result, error)
// Placeholder returns the engine's positional placeholder for argument i
// (0-based): "$1" for postgres, "?" for bigquery/athena.
Placeholder(i int) string
// Close releases resources.
Close() error
}
Source executes SQL against one engine. sql uses the engine's native placeholder syntax already (the service translates "?" / "@name").
type SourceInfo ¶
type SourceInfo struct {
Name string `json:"name"`
Type string `json:"type"`
Description string `json:"description,omitempty"`
}
SourceInfo / QueryInfo are returned by GET /warehouse.
Directories
¶
| Path | Synopsis |
|---|---|
|
Package athena implements warehouse.Source for Amazon Athena (Trino SQL) with the AWS SDK for Go v2.
|
Package athena implements warehouse.Source for Amazon Athena (Trino SQL) with the AWS SDK for Go v2. |
|
Package bigquery implements warehouse.Source for Google BigQuery (GoogleSQL) with cloud.google.com/go/bigquery.
|
Package bigquery implements warehouse.Source for Google BigQuery (GoogleSQL) with cloud.google.com/go/bigquery. |
|
internal
|
|
|
conv
Package conv holds JSON-friendly value conversions shared by the warehouse sources (sqlsource, bigquery, athena), so every engine renders numbers, decimals, bytes and timestamps the same way (SPEC §5.5).
|
Package conv holds JSON-friendly value conversions shared by the warehouse sources (sqlsource, bigquery, athena), so every engine renders numbers, decimals, bytes and timestamps the same way (SPEC §5.5). |
|
Package service implements warehouse.Service on top of the configured sources (SPEC §5.5).
|
Package service implements warehouse.Service on top of the configured sources (SPEC §5.5). |
|
Package sqlsource implements warehouse.Source for PostgreSQL and Postgres-compatible engines (Amazon Redshift, AlloyDB, Aurora, Neon, CockroachDB, …) on a pgx/v5 connection pool.
|
Package sqlsource implements warehouse.Source for PostgreSQL and Postgres-compatible engines (Amazon Redshift, AlloyDB, Aurora, Neon, CockroachDB, …) on a pgx/v5 connection pool. |