warehouse

package
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 15, 2026 License: Apache-2.0 Imports: 2 Imported by: 0

Documentation

Overview

Package warehouse provides read-only analytics queries (backyard.warehouse) against BigQuery, Postgres-compatible databases (incl. Redshift) and Athena. Sources live in sub-packages; service.Service enforces policy (raw SQL gating, statement guard, named queries, row and size caps, timeouts).

Index

Constants

View Source
const DefaultMaxResultBytes = 32 << 20

DefaultMaxResultBytes is the default result-size cap (Limits.MaxBytes): 32 MiB of estimated row data.

Variables

View Source
var (
	// ErrUpstream: the engine failed to run the query (connection failure,
	// internal engine error, quota exhaustion, …).
	ErrUpstream = errors.New("warehouse: upstream error")
	// ErrTimeout: the query exceeded warehouse.timeout (or the engine's
	// statement timeout).
	ErrTimeout = errors.New("warehouse: query timed out")
)

Additional sentinel errors. Together with ErrDisabled, ErrForbidden, ErrNotFound and ErrBadRequest they let the HTTP layer map every failure to a SPEC §5 error code without inspecting engine-specific error types:

ErrDisabled   → 501 unsupported (or 404 not_found)
ErrBadRequest → 400 bad_request   (invalid params, SQL syntax / semantic errors)
ErrForbidden  → 403 forbidden     (raw SQL disabled, statement guard, read-only violation, permission denied)
ErrNotFound   → 404 not_found     (unknown source or named query)
ErrTimeout    → 504 upstream_error (warehouse.timeout exceeded)
ErrUpstream   → 502 upstream_error (engine or connectivity failure)

Errors returned by Service.Query wrap exactly one of these sentinels, except when the caller's own context was canceled (then they wrap context.Canceled).

View Source
var (
	// ErrDisabled: no warehouse configured.
	ErrDisabled = errors.New("warehouse: not configured")
	// ErrForbidden: raw SQL disabled, or statement not allowed.
	ErrForbidden = errors.New("warehouse: query not allowed")
	// ErrNotFound: unknown source or named query.
	ErrNotFound = errors.New("warehouse: not found")
	// ErrBadRequest: invalid parameters.
	ErrBadRequest = errors.New("warehouse: bad request")
)

Functions

This section is empty.

Types

type Column

type Column struct {
	Name string `json:"name"`
	Type string `json:"type"` // engine type name, e.g. STRING, INT64, DATE, int4, varchar
}

Column describes a result column.

type Limits

type Limits struct {
	// MaxRows is the maximum number of rows returned (values <= 0 mean 1).
	MaxRows int
	// MaxBytes caps the estimated size of the returned rows: their JSON
	// encoding plus a fixed per-value overhead for holding them in memory
	// (values <= 0 mean DefaultMaxResultBytes). Sources stop at the first
	// row that does not fit and never buffer a row much larger than what
	// is left, so one query cannot make the server hold an arbitrarily
	// large result.
	MaxBytes int
}

Limits bound the result of Source.Query.

func (Limits) WithDefaults

func (l Limits) WithDefaults() Limits

WithDefaults returns l with its zero or negative fields replaced by their defaults.

type QueryInfo

type QueryInfo struct {
	Name        string   `json:"name"`
	Description string   `json:"description,omitempty"`
	Params      []string `json:"params"`
	Source      string   `json:"source"`
}

QueryInfo describes a named query.

type QueryRequest

type QueryRequest struct {
	SQL    string         `json:"sql,omitempty"`
	Query  string         `json:"query,omitempty"`  // named query
	Params any            `json:"params,omitempty"` // []any for sql, map[string]any for named
	Source string         `json:"source,omitempty"`
	Extra  map[string]any `json:"-"`
}

QueryRequest is the client request (SPEC §5.5).

type Result

type Result struct {
	Columns   []Column         `json:"columns"`
	Rows      []map[string]any `json:"rows"`
	RowCount  int              `json:"row_count"`
	Truncated bool             `json:"truncated"`
}

Result is a query result. Values are JSON-friendly: strings, float64/int64, bool, nil, RFC3339 strings for timestamps, ISO dates for dates. Truncated reports that the query had more rows than the result holds (row or size cap).

type Service

type Service interface {
	Query(ctx context.Context, req QueryRequest) (*Result, error)
	Describe() (sources []SourceInfo, queries []QueryInfo)
	Enabled() bool
	Close() error
}

Service is what the HTTP layer uses. Implemented by service.Service.

type Source

type Source interface {
	// Query runs sql with positional args and returns the rows that fit
	// within lim (Truncated=true if more were available).
	Query(ctx context.Context, sql string, args []any, lim Limits) (*Result, error)
	// Placeholder returns the engine's positional placeholder for argument i
	// (0-based): "$1" for postgres, "?" for bigquery/athena.
	Placeholder(i int) string
	// Close releases resources.
	Close() error
}

Source executes SQL against one engine. sql uses the engine's native placeholder syntax already (the service translates "?" / "@name").

type SourceInfo

type SourceInfo struct {
	Name        string `json:"name"`
	Type        string `json:"type"`
	Description string `json:"description,omitempty"`
}

SourceInfo / QueryInfo are returned by GET /warehouse.

Directories

Path Synopsis
Package athena implements warehouse.Source for Amazon Athena (Trino SQL) with the AWS SDK for Go v2.
Package athena implements warehouse.Source for Amazon Athena (Trino SQL) with the AWS SDK for Go v2.
Package bigquery implements warehouse.Source for Google BigQuery (GoogleSQL) with cloud.google.com/go/bigquery.
Package bigquery implements warehouse.Source for Google BigQuery (GoogleSQL) with cloud.google.com/go/bigquery.
internal
conv
Package conv holds JSON-friendly value conversions shared by the warehouse sources (sqlsource, bigquery, athena), so every engine renders numbers, decimals, bytes and timestamps the same way (SPEC §5.5).
Package conv holds JSON-friendly value conversions shared by the warehouse sources (sqlsource, bigquery, athena), so every engine renders numbers, decimals, bytes and timestamps the same way (SPEC §5.5).
Package service implements warehouse.Service on top of the configured sources (SPEC §5.5).
Package service implements warehouse.Service on top of the configured sources (SPEC §5.5).
Package sqlsource implements warehouse.Source for PostgreSQL and Postgres-compatible engines (Amazon Redshift, AlloyDB, Aurora, Neon, CockroachDB, …) on a pgx/v5 connection pool.
Package sqlsource implements warehouse.Source for PostgreSQL and Postgres-compatible engines (Amazon Redshift, AlloyDB, Aurora, Neon, CockroachDB, …) on a pgx/v5 connection pool.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL