query

package
v0.17.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 5, 2026 License: Apache-2.0 Imports: 22 Imported by: 0

Documentation

Overview

Package query is the query commands: query runs a query, query.compile compiles one to SQL, and query.validate checks one. What they share, and only they, stays in here.

Index

Constants

View Source
const MaxPageSize = 10_000

MaxPageSize is the most rows a page holds: page_size's maximum, and the page a query without one is answered with when it is not streamed.

Variables

View Source
var Compile = command.Define(command.Def[QueryRunInput, CompiledQuery]{
	Name:     "query.compile",
	Short:    "Compile a query to SQL, without running it",
	ReadOnly: true,
	Timeout:  2 * time.Minute,
	Needs:    func(in QueryRunInput) command.Sidecars { return command.Sidecars{Node: in.Lang != "sql"} },
	Check:    QueryRunInput.check,
	Errors:   []apperr.AnyCode{query.QueryInvalid, errcode.DataPermsUnenforceable},
	Run: func(ctx context.Context, cc command.CommandContext, in QueryRunInput) (CompiledQuery, error) {
		if in.Lang == "sql" {

			if err := in.checkDataSource(ctx, cc); err != nil {
				return CompiledQuery{}, err
			}
			return CompiledQuery{SQL: in.Query}, nil
		}
		aql, _, err := in.aql()
		if err != nil {
			return CompiledQuery{}, err
		}
		if err := command.RequireSidecars(cc, command.Sidecars{Node: true}); err != nil {
			return CompiledQuery{}, err
		}
		src, err := sourceOf(cc)
		if err != nil {
			return CompiledQuery{}, err
		}
		compiled, err := compileAQL(ctx, cc, src, in.Dataset, aql, in.run())
		if err != nil {
			return CompiledQuery{}, err
		}
		return CompiledQuery{SQL: compiled.SQL, AQL: compiled.AQL, Columns: compiledColumns(compiled.Columns)}, nil
	},
})

Compile compiles a query to SQL, without running it.

View Source
var Export = command.Define(command.Def[ExportInput, command.ExportLink]{
	Name:  "query.export",
	Short: "Export a query's whole result as a file: a link to download it",
	Long: "Export a query's whole result as a file: a link to download it.\n\n" +
		"The query runs to completion before the answer, so a query that fails fails here. " +
		"The CLI downloads the file, to stdout or --output; --link prints the answer instead.",

	NonDestructive: true,
	Timeout:        30 * time.Minute,
	Needs: func(in ExportInput) command.Sidecars {
		return command.Sidecars{Node: in.Lang != "sql", CanalQuery: true}
	},
	Check:  ExportInput.check,
	Errors: []apperr.AnyCode{query.QueryInvalid, errcode.QueryFailed, errcode.DataPermsUnenforceable, errcode.ExportsUnavailable},
	Run:    runExport,
})

Export runs a query and answers its whole result as a file: a link to download it. See docs/designs/exports.md.

View Source
var Query = command.Define(command.Def[QueryRunInput, QueryRun]{
	Name:  "query",
	Short: "Run a query: its rows, and the SQL that produced them",
	Long: `Answers one page of rows: page_size's, or without it the first 10000, with has_more saying whether there may be more.
Ask the query for what you need (aggregate, filter, a limit: directive) rather than paging through rows; for every row of a large result, export it.
Over HTTP, a request that accepts application/x-ndjson is answered as the query runs: its phase, its SQL, its columns, then its rows in batches — every row without page_size, up to the server's maximum.`,
	ReadOnly: true,
	Timeout:  10 * time.Minute,
	Needs: func(in QueryRunInput) command.Sidecars {
		return command.Sidecars{Node: in.Lang != "sql", CanalQuery: true}
	},
	Check:  QueryRunInput.check,
	Errors: []apperr.AnyCode{query.QueryInvalid, errcode.QueryFailed, errcode.DataPermsUnenforceable, query.ResultTooLarge, query.PageTooLarge},
	Events: queryEvents,
	Run:    newQueryRun,
})

Query runs a query: its rows, and the SQL that produced them — as one page, or streamed as it runs (QueryRun).

View Source
var Validate = command.Define(command.Def[QueryInput, query.QueryValidation]{
	Name:     "query.validate",
	Short:    "Validate a query: its diagnostics, without running it",
	ReadOnly: true,
	Timeout:  2 * time.Minute,
	Needs:    func(QueryInput) command.Sidecars { return command.Sidecars{Node: true} },
	Check: func(in QueryInput) error {
		if in.Lang == "sql" {

			return command.InvalidArg("lang", "unsupported", "validating a SQL query is not supported yet; you can still run it directly")
		}
		return in.target()
	},
	Run: func(ctx context.Context, cc command.CommandContext, in QueryInput) (query.QueryValidation, error) {
		aql, _, err := in.aql()
		if err != nil {
			return query.QueryValidation{}, err
		}
		if err := command.RequireSidecars(cc, command.Sidecars{Node: true}); err != nil {
			return query.QueryValidation{}, err
		}
		src, err := sourceOf(cc)
		if err != nil {
			return query.QueryValidation{}, err
		}
		return checkAQL(ctx, cc, src, in.Dataset, aql)
	},
	Valid: func(r query.QueryValidation) bool { return r.Valid },
})

Validate checks a query: its diagnostics, without running it.

Functions

This section is empty.

Types

type Column

type Column struct {
	Name        string `json:"name" doc:"the column's key, as in fields"`
	FieldName   string `json:"fieldName" doc:"the field it draws; for an adhoc column, its name"`
	ModelID     string `` /* 126-byte string literal not displayed */
	Label       string `json:"label"`
	Adhoc       bool   `json:"adhoc" doc:"a query-local expression, not a field the dataset defines"`
	IsMeasure   bool   `json:"isMeasure"`
	Aggregation string `json:"aggregation,omitempty" doc:"an aggregated field's aggregation, such as sum or count distinct"`

	SemanticType SemanticType         `` /* 389-byte string literal not displayed */
	ArrowType    canalquery.ArrowType `` /* 135-byte string literal not displayed */
	DBType       string               `` /* 180-byte string literal not displayed */
}

Column is one column of a query's answer: what it draws from the dataset, as anfra-node describes it, in amql's names (camelCase); and its types, which the core adds, in the API's (snake_case). See docs/designs/result-values.md.

type CompiledQuery

type CompiledQuery struct {
	SQL     string   `json:"sql"`
	AQL     string   `json:"aql,omitempty" doc:"the AQL compiled: the query with its Query Input applied; absent for a SQL query"`
	Columns []Column `` /* 133-byte string literal not displayed */
}

CompiledQuery is the `query compile` result: the SQL the query compiles to.

type ExportFormatOptions

type ExportFormatOptions struct {
	Header string `` /* 203-byte string literal not displayed */
	BOM    bool   `` /* 155-byte string literal not displayed */
}

ExportFormatOptions are a format's options. Each applies to the formats it names; given to another, it is refused (check).

type ExportInput

type ExportInput struct {
	QueryInput
	Input         anfranode.QueryTransforms `` /* 126-byte string literal not displayed */
	Timezone      string                    `json:"timezone,omitempty" doc:"the IANA time zone relative dates and date truncation use, such as Asia/Ho_Chi_Minh"`
	Format        string                    `json:"format,omitempty" enum:"csv" default:"csv" doc:"the file's format"`
	FormatOptions ExportFormatOptions       `json:"format_options,omitempty" doc:"the format's options"`
	Filename      string                    `` /* 181-byte string literal not displayed */
}

ExportInput is what query.export takes: a query and what shapes its run, as query takes them, but no page, since an export is the whole result; and the file's format.

func (ExportInput) TransformSchema

func (ExportInput) TransformSchema(_ huma.Registry, s *huma.Schema) *huma.Schema

type QueryCompiled added in v0.11.1

type QueryCompiled struct {
	Type string `json:"type" enum:"compiled"`
	SQL  string `json:"sql"`
	AQL  string `json:"aql,omitempty" doc:"the AQL that runs: the query with its Query Input applied; absent for a SQL query"`
}

QueryCompiled is the SQL the query runs, as soon as it is known.

type QueryEvent added in v0.11.1

type QueryEvent interface {
	// contains filtered or unexported methods
}

QueryEvent is one of query's own events, in a stream: compiled, then warehouse_query once the warehouse has the query, then header, then rows in batches, then summary. Which phase the query is in — compiling_query, then canal-query's connecting_to_dw, executing_query, fetching_query_result, and retrying_query — is the stream's status.

type QueryHeader added in v0.11.1

type QueryHeader struct {
	Type    string   `json:"type" enum:"header"`
	Columns []Column `json:"columns" doc:"one per field, in order"`
	Fields  []string `json:"fields"`
}

QueryHeader is what each column of the result is, once the result's shape is known: before its rows.

type QueryInput

type QueryInput struct {
	Query      string `json:"query" cli:"positional,stdin" doc:"the query"`
	Lang       string `json:"lang,omitempty" short:"l" enum:"aql,sql" default:"aql" doc:"the language the query is written in"`
	Dataset    string `json:"dataset,omitempty" short:"d" doc:"the dataset an AQL query runs against"`
	DataSource string `json:"data_source,omitempty" short:"s" doc:"the data source a SQL query runs against"`
}

QueryInput is a query: its text, the language it is written in, and the target it runs against. Which target a language runs against is target's to say: AQL against a dataset, and SQL against a data source.

func (QueryInput) TransformSchema

func (QueryInput) TransformSchema(_ huma.Registry, s *huma.Schema) *huma.Schema

TransformSchema publishes the required query (command.ArgsSchema). The target is not a group in the schema: which one is required depends on lang, which a group cannot say, so target checks it.

type QueryRecords added in v0.11.1

type QueryRecords struct {
	Type    string           `json:"type" enum:"rows"`
	Records []canalquery.Row `json:"records" doc:"one array per row, its values in fields' order, written as a page's records are"`
}

QueryRecords is a batch of the result's rows, as a page's records are.

type QueryResult

type QueryResult struct {
	SQL     string    `json:"sql"`
	AQL     string    `json:"aql,omitempty" doc:"the AQL that ran: the query with its Query Input applied; absent for a SQL query"`
	Columns []Column  `json:"columns" doc:"one per field of result, in order"`
	Result  QueryRows `json:"result"`
	HasMore bool      `` /* 133-byte string literal not displayed */
	Timings Timings   `` /* 253-byte string literal not displayed */
}

QueryResult is the `query` result answered whole: the SQL that ran, the AQL it was compiled from, what each column is, and one page of rows — page_size's, or the first MaxPageSize without one.

type QueryRows

type QueryRows struct {
	Fields  []string         `json:"fields"`
	Records []canalquery.Row `` /* 309-byte string literal not displayed */
}

type QueryRun added in v0.11.1

type QueryRun = apikit.Run[QueryResult, QueryEvent]

QueryRun is how query answers: as a page (QueryResult), or as a stream of its events (QueryEvent).

type QueryRunInput

type QueryRunInput struct {
	QueryInput
	Input    anfranode.QueryTransforms `` /* 126-byte string literal not displayed */
	Page     int                       `json:"page,omitempty" minimum:"1" doc:"the 1-based page of rows to answer; needs a page size"`
	PageSize int                       `json:"page_size,omitempty" minimum:"1" maximum:"10000" doc:"rows per page; alone, the first page"`
	Timezone string                    `json:"timezone,omitempty" doc:"the IANA time zone relative dates and date truncation use, such as Asia/Ho_Chi_Minh"`
}

QueryRunInput is what query and query.compile take: a query, and what shapes its run. query.validate takes the query alone (QueryInput), since none of it applies to checking one.

func (QueryRunInput) TransformSchema

func (QueryRunInput) TransformSchema(_ huma.Registry, s *huma.Schema) *huma.Schema

type QuerySummary added in v0.11.1

type QuerySummary struct {
	Type     string `json:"type" enum:"summary"`
	RowCount int    `json:"row_count"`
	HasMore  bool   `` /* 139-byte string literal not displayed */
	// BytesScanned is what the warehouse read to run the query, as it reports
	// it: BigQuery and Athena.
	BytesScanned *int64  `json:"bytes_scanned,omitempty" doc:"the bytes the warehouse read to run the query, where it reports them (BigQuery, Athena)"`
	Timings      Timings `` /* 253-byte string literal not displayed */
}

QuerySummary ends the result: how many rows it held, and, for a page, whether there are more.

type QueryWarehouse added in v0.11.1

type QueryWarehouse struct {
	Type string `json:"type" enum:"warehouse_query"`
	ID   string `json:"id" doc:"the warehouse's own id of the query; for Postgres, the process id of the backend running it"`
}

QueryWarehouse is the query's id on the warehouse, as soon as it has one: to find it there — in its query history, its billing — or to cancel it there.

type SemanticType

type SemanticType string

SemanticType is what a column's values mean: AML's types, plus bit.

const (
	SemanticText      SemanticType = "text"
	SemanticNumber    SemanticType = "number"
	SemanticDate      SemanticType = "date"
	SemanticDatetime  SemanticType = "datetime"
	SemanticTruefalse SemanticType = "truefalse"
	SemanticJSON      SemanticType = "json"
	SemanticBit       SemanticType = "bit"
	SemanticUnknown   SemanticType = "unknown"
)

func (SemanticType) Schema

Schema publishes the semantic types semanticTypes gives, and no other, so the enum can neither miss one the core answers nor list one it never does.

type TimedStep added in v0.11.1

type TimedStep = command.TimedStep

Timings and TimedStep are a run's timeline, command's (CommandContext's observers report them too), named here as the query's answer has always named them. A query's steps are documented on its answer's fields.

type Timings added in v0.11.1

type Timings = command.Timings

Timings and TimedStep are a run's timeline, command's (CommandContext's observers report them too), named here as the query's answer has always named them. A query's steps are documented on its answer's fields.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL