Documentation
¶
Overview ¶
Package query is the query use-case layer: it checks a (dataset, AQL) request, turns it into SQL and runs it, by loading the repo's data sources and driving the sidecars via their clients. What an invalid query is, and the code it fails with, are its own. It depends on the clients (not the process managers), so it works equally against host-spawned or external sidecars. The CLI command and the future HTTP/MCP handler are thin shells over these functions.
Index ¶
- Constants
- Variables
- func CheckUngoverned(perms dataperm.Set) error
- func Compile(ctx context.Context, node *anfranode.Client, src Source, dataset, aql string, ...) (anfranode.CompileToSQLResult, error)
- func CompileRequest(ctx context.Context, src Source, dataset, aql string) (anfranode.CompileToSQLRequest, error)
- func ExtractLimit(aql string) (string, int, error)
- func NewPageTooLarge(maxBytes int) error
- func Stream(ctx context.Context, canal *canalquery.Client, sources datasource.DataSources, ...) error
- type Limits
- type PageLimit
- type QueryValidation
- type ResultBounds
- type ResultLimit
- type Run
- type Source
- type UngovernedSQL
Constants ¶
const DefaultMaxPageBytes = 16 << 20
DefaultMaxPageBytes is the most a page of a query's result may hold, in its rows' bytes, when the host sets no maximum (Limits.MaxPageBytes).
const DefaultMaxResultBytes = 256 << 20
DefaultMaxResultBytes is the most a query's result may hold, in bytes, when the host sets no maximum (Limits.MaxResultBytes).
const DefaultMaxResultRows = 1_000_000
DefaultMaxResultRows is the most rows a query's result may hold when the host sets no maximum (Limits.MaxResultRows).
const NoLimit = -1
NoLimit is the row limit meaning "no truncation" (canal's truncate_rows uses a negative value for this).
Variables ¶
var PageTooLarge = apperr.DefinePublicCodeWith[PageLimit](errcode.NS, "page_too_large", apperr.User,
"The page is larger than this server answers at once.")
PageTooLarge is a page of a query's result larger, in bytes, than the host answers at once: it fails rather than being cut short, so that page numbers stay exact. The maximum is in its details.
var QueryInvalid = apperr.DefinePublicCodeWith[QueryValidation](errcode.NS, "query_invalid", apperr.User, "The query is invalid.")
QueryInvalid is a query that cannot run or compile, with its diagnostics: the QueryValidation that says why. A command that judges the query answers the QueryValidation instead.
var ResultTooLarge = apperr.DefinePublicCodeWith[ResultLimit](errcode.NS, "result_too_large", apperr.User,
"The query's result is larger than this server allows.")
ResultTooLarge is a query whose result has more rows than the host allows: it fails rather than being cut short, since a chart drawn from part of a result says something the whole would not. The maximum is in its details.
var Unbounded = ResultBounds{Truncate: NoLimit}
Unbounded is every row, however many: for a result streamed to a file, which holds none of it.
Functions ¶
func CheckUngoverned ¶ added in v0.11.1
CheckUngoverned refuses SQL as written for a governed caller: nothing can govern it.
func Compile ¶
func Compile(ctx context.Context, node *anfranode.Client, src Source, dataset, aql string, run Run) (anfranode.CompileToSQLResult, error)
Compile compiles an AQL query against a dataset into SQL plus the data source it targets (dialect + execution routing), without executing. Shared by --generate and the run path so both fail identically on a bad query: with QueryInvalid and its diagnostics when the query does not compile. Any other refusal (a dataset that cannot be used, the run's shaping) is returned as it is, for the command to place on its arg (anfranode.Violation). The result's AQL is the query with run's Query Input applied.
func CompileRequest ¶
func CompileRequest(ctx context.Context, src Source, dataset, aql string) (anfranode.CompileToSQLRequest, error)
CompileRequest builds the sidecar compile request for a (dataset, aql) against src, with its data sources' dialects and no connection. Shared by SQL generation and AQL validation (both feed the same {repo, datasetFqn, aql, dataSources} to the sidecar).
func ExtractLimit ¶
ExtractLimit pulls an anfra-level `limit:` directive out of an AQL query and returns the query with the directive removed plus the row limit. The AQL engine does not support `limit`, so anfra strips it here and applies it at execution time via canal's truncate_rows. Returns NoLimit when no directive is present. TODO: remove this when AQL supports limit
func NewPageTooLarge ¶ added in v0.11.1
NewPageTooLarge is the error of a page past maxBytes.
func Stream ¶
func Stream(ctx context.Context, canal *canalquery.Client, sources datasource.DataSources, compiled anfranode.CompileToSQLResult, bounds ResultBounds, h canalquery.Handler) error
Stream runs already-compiled SQL via canal-query against the data source the dataset targets, handing its rows within bounds to h as they arrive, its header first, holding none of it. Split from Compile so callers can distinguish a compile failure (a query problem) from an execution failure (a data-source/DB problem).
Types ¶
type Limits ¶ added in v0.11.1
type Limits struct {
// MaxResultRows is the most rows a query's result may hold: a query that
// produces more fails with ResultTooLarge rather than being cut short. Zero:
// DefaultMaxResultRows.
MaxResultRows int
// MaxResultBytes is the most a query's result may hold, in bytes, as
// canal-query measures it while streaming it (each batch's Arrow data, about
// what its JSON takes; neither it nor the engine holds the result whole): a
// query that produces more fails with ResultTooLarge. It bounds how much one
// query moves, not memory. Zero: DefaultMaxResultBytes.
MaxResultBytes int64
// MaxPageBytes is the most a page answered whole may hold, in its rows'
// bytes: a page over it fails with PageTooLarge rather than being cut short,
// so that page numbers stay exact. Zero: DefaultMaxPageBytes.
MaxPageBytes int
}
Limits bound what a query may cost the host. The zero value is the defaults.
func (Limits) ResultBytes ¶ added in v0.11.1
ResultBytes is MaxResultBytes, or its default.
func (Limits) ResultRows ¶ added in v0.11.1
ResultRows is MaxResultRows, or its default.
type PageLimit ¶ added in v0.11.1
type PageLimit struct {
MaxBytes int `json:"max_bytes"`
}
PageLimit is the maximum a page went past.
type QueryValidation ¶
type QueryValidation struct {
// Valid: no diagnostic has "error" severity.
Valid bool `json:"valid"`
Diagnostics []anfranode.AQLDiagnostic `json:"diagnostics"`
}
QueryValidation is the outcome of validating a query: its verdict, and the type-check diagnostics for a single AQL query.
type ResultBounds ¶ added in v0.11.1
type ResultBounds struct {
Truncate int // NoLimit: every row the query produces
MaxRows int // zero: no maximum
MaxBytes int64 // zero: no maximum
}
ResultBounds bound how much of a query's result is fetched: up to Truncate rows, where the result is cut as the query asked (its limit: directive, or a page); and no more than MaxRows rows or MaxBytes bytes, past which the query fails with ResultTooLarge — the host's maximums (Limits), bounding what it holds.
func Bounded ¶ added in v0.11.1
func Bounded(truncate int, limits Limits) ResultBounds
Bounded are the bounds of the host's maximums (limits), the result truncated at truncate rows.
type ResultLimit ¶ added in v0.11.1
type ResultLimit struct {
MaxRows int `json:"max_rows,omitempty"`
MaxBytes int64 `json:"max_bytes,omitempty"`
}
ResultLimit is the maximum a result went past: its rows, or its bytes.
type Run ¶
type Run struct {
Input *anfranode.QueryTransforms
Pagination *anfranode.Pagination
Timezone string
}
Run is what shapes one run of a query beyond its AQL: its Query Input, applied to the AQL before it compiles; a page of rows (nil: every row); and the time zone relative dates and date truncation use ("": anfra-node's default).
type Source ¶ added in v0.9.0
type Source struct {
Repo repo.Repo
DataSources datasource.DataSources
}
Source is what a query compiles against and runs on: the repo's AML, and its data sources, whose dialects compiling asks for and one of whose connections a run asks for.
type UngovernedSQL ¶ added in v0.11.1
type UngovernedSQL struct {
// contains filtered or unexported fields
}
UngovernedSQL is SQL as written, with the data source it runs on, connection included. Only ConnectUngoverned makes one.
func ConnectUngoverned ¶ added in v0.11.1
func ConnectUngoverned(ctx context.Context, perms dataperm.Set, sources datasource.DataSources, name, sql string) (UngovernedSQL, error)
ConnectUngoverned is sql, as written, on the data source named name: refused for a governed caller (CheckUngoverned), and connected in the ungoverned mode. A name the repo has no data source by is datasource.ErrUnknown.
func (UngovernedSQL) Stream ¶ added in v0.11.1
func (s UngovernedSQL) Stream(ctx context.Context, canal *canalquery.Client, bounds ResultBounds, h canalquery.Handler) error
Stream runs s as it is: written in its data source's dialect, with its own LIMIT. bounds are as for the package's Stream.