statementexecution

package
v0.0.1-dev.8 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 10, 2026 License: Apache-2.0 Imports: 19 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type CancelStatementRequest

type CancelStatementRequest struct {
	// The statement ID is returned upon successfully submitting a SQL statement,
	// and is a required reference for all subsequent calls.
	StatementId *string
}

type CancelStatementResponse

type CancelStatementResponse struct {
}

type ChunkInfo

type ChunkInfo struct {
	// The position within the sequence of result set chunks.
	ChunkIndex *int
	// The starting row offset within the result set.
	RowOffset *int64
	// The number of rows within the result chunk.
	RowCount *int64
	// The number of bytes in the result chunk. This field is not available when
	// using `INLINE` disposition.
	ByteCount *int64
	// When fetching, provides the `chunk_index` for the _next_ chunk. If absent,
	// indicates there are no more chunks. The next chunk can be fetched with a
	// :method:statementexecution/getstatementresultchunkn request.
	NextChunkIndex *int
	// When fetching, provides a link to fetch the _next_ chunk. If absent,
	// indicates there are no more chunks. This link is an absolute `path` to be
	// joined with your `$DATABRICKS_HOST`, and should be treated as an opaque link.
	// This is an alternative to using `next_chunk_index`.
	NextChunkInternalLink *string
}

type Client

type Client struct {
	// contains filtered or unexported fields
}

func NewClient

func NewClient(ctx context.Context, opts ...client.Option) (*Client, error)

func (*Client) CancelStatement

func (c *Client) CancelStatement(ctx context.Context, req CancelStatementRequest, opts ...call.Option) (*CancelStatementResponse, error)

Requests that an executing statement be canceled. Callers must poll for status to see the terminal state. Cancel response is empty; receiving response indicates successful receipt.

func (*Client) ExecuteStatement

func (c *Client) ExecuteStatement(ctx context.Context, req ExecuteStatementRequest, opts ...call.Option) (*StatementResponse, error)

Execute a SQL statement and optionally await its results for a specified time.

**Use case: small result sets with INLINE + JSON_ARRAY**

For flows that generate small and predictable result sets (<= 25 MiB), `INLINE` responses of `JSON_ARRAY` result data are typically the simplest way to execute and fetch result data.

**Use case: large result sets with EXTERNAL_LINKS**

Using `EXTERNAL_LINKS` to fetch result data allows you to fetch large result sets efficiently. The main differences from using `INLINE` disposition are that the result data is accessed with URLs, and that there are 3 supported formats: `JSON_ARRAY`, `ARROW_STREAM` and `CSV` compared to only `JSON_ARRAY` with `INLINE`.

** URLs**

External links point to data stored within your workspace's internal storage, in the form of a URL. The URLs are valid for only a short period, <= 15 minutes. Alongside each `external_link` is an expiration field indicating the time at which the URL is no longer valid. In `EXTERNAL_LINKS` mode, chunks can be resolved and fetched multiple times and in parallel.

----

### **Warning: Databricks strongly recommends that you protect the URLs that are returned by the `EXTERNAL_LINKS` disposition.**

When you use the `EXTERNAL_LINKS` disposition, a short-lived, URL is generated, which can be used to download the results directly from . As a short-lived is embedded in this URL, you should protect the URL.

Because URLs are already generated with embedded temporary s, you must not set an `Authorization` header in the download requests.

The `EXTERNAL_LINKS` disposition can be disabled upon request by creating a support case.

See also [Security best practices](/sql/admin/sql-execution-tutorial.html#security-best-practices).

----

StatementResponse contains `statement_id` and `status`; other fields might be absent or present depending on context. If the SQL warehouse fails to execute the provided statement, a 200 response is returned with `status.state` set to `FAILED` (in contrast to a failure when accepting the request, which results in a non-200 response). Details of the error can be found at `status.error` in case of execution failures.

func (*Client) GetResultData

func (c *Client) GetResultData(ctx context.Context, req GetResultDataRequest, opts ...call.Option) (*ResultData, error)

After the statement execution has `SUCCEEDED`, this request can be used to fetch any chunk by index. Whereas the first chunk with `chunk_index=0` is typically fetched with :method:statementexecution/executeStatement or :method:statementexecution/getStatement, this request can be used to fetch subsequent chunks. The response structure is identical to the nested `result` element described in the :method:statementexecution/getStatement request, and similarly includes the `next_chunk_index` and `next_chunk_internal_link` fields for simple iteration through the result set. Depending on `disposition`, the response returns chunks of data either inline, or as links.

func (*Client) GetStatementResult

func (c *Client) GetStatementResult(ctx context.Context, req GetStatementResultRequest, opts ...call.Option) (*StatementResponse, error)

This request can be used to poll for the statement's status. StatementResponse contains `statement_id` and `status`; other fields might be absent or present depending on context. When the `status.state` field is `SUCCEEDED` it will also return the result manifest and the first chunk of the result data. When the statement is in the terminal states `CANCELED`, `CLOSED` or `FAILED`, it returns HTTP 200 with the state set. After at least 12 hours in terminal state, the statement is removed from the warehouse and further calls will receive an HTTP 404 response.

**NOTE** This call currently might take up to 5 seconds to get the latest status and result.

type ColumnInfo

type ColumnInfo struct {
	// The name of the column.
	Name *string
	// The full SQL type specification.
	TypeText *string
	// The name of the base data type. This doesn't include details for complex
	// types such as STRUCT, MAP or ARRAY.
	TypeName ColumnTypeName
	// The ordinal position of the column (starting at position 0).
	Position *int
	// Specifies the number of digits in a number. This applies to the DECIMAL type.
	TypePrecision *int
	// Specifies the number of digits to the right of the decimal point in a number.
	// This applies to the DECIMAL type.
	TypeScale *int
	// The format of the interval type.
	TypeIntervalType *string
}

type ColumnTypeName

type ColumnTypeName string

The name of the base data type. This doesn't include details for complex types such as STRUCT, MAP or ARRAY.

const (
	ColumnTypeName_Unspecified     ColumnTypeName = ""
	ColumnTypeName_Boolean         ColumnTypeName = "BOOLEAN"
	ColumnTypeName_Byte            ColumnTypeName = "BYTE"
	ColumnTypeName_Short           ColumnTypeName = "SHORT"
	ColumnTypeName_Int             ColumnTypeName = "INT"
	ColumnTypeName_Long            ColumnTypeName = "LONG"
	ColumnTypeName_Float           ColumnTypeName = "FLOAT"
	ColumnTypeName_Double          ColumnTypeName = "DOUBLE"
	ColumnTypeName_Date            ColumnTypeName = "DATE"
	ColumnTypeName_Timestamp       ColumnTypeName = "TIMESTAMP"
	ColumnTypeName_String          ColumnTypeName = "STRING"
	ColumnTypeName_Binary          ColumnTypeName = "BINARY"
	ColumnTypeName_Decimal         ColumnTypeName = "DECIMAL"
	ColumnTypeName_Interval        ColumnTypeName = "INTERVAL"
	ColumnTypeName_Array           ColumnTypeName = "ARRAY"
	ColumnTypeName_Struct          ColumnTypeName = "STRUCT"
	ColumnTypeName_Map             ColumnTypeName = "MAP"
	ColumnTypeName_Char            ColumnTypeName = "CHAR"
	ColumnTypeName_Null            ColumnTypeName = "NULL"
	ColumnTypeName_UserDefinedType ColumnTypeName = "USER_DEFINED_TYPE"
)

type Disposition

type Disposition string
const (
	Disposition_Unspecified   Disposition = ""
	Disposition_Inline        Disposition = "INLINE"
	Disposition_ExternalLinks Disposition = "EXTERNAL_LINKS"
)

type ExecuteStatementRequest

type ExecuteStatementRequest struct {
	// The SQL statement to execute. The statement can optionally be parameterized,
	// see `parameters`. The maximum query text size is 16 MiB.
	Statement *string
	// Warehouse upon which to execute a statement. See also [What are SQL
	// warehouses?]
	//
	// [What are SQL warehouses?]: https://docs.databricks.com/sql/admin/warehouse-type.html
	WarehouseId *string
	// Sets default catalog for statement execution, similar to [`USE CATALOG`] in
	// SQL.
	//
	// [`USE CATALOG`]: https://docs.databricks.com/sql/language-manual/sql-ref-syntax-ddl-use-catalog.html
	Catalog *string
	// Sets default schema for statement execution, similar to [`USE SCHEMA`] in
	// SQL.
	//
	// [`USE SCHEMA`]: https://docs.databricks.com/sql/language-manual/sql-ref-syntax-ddl-use-schema.html
	Schema *string
	// Applies the given row limit to the statement's result set, but unlike the
	// `LIMIT` clause in SQL, it also sets the `truncated` field in the response to
	// indicate whether the result was trimmed due to the limit or not.
	RowLimit *int64
	// Applies the given byte limit to the statement's result size. Byte counts are
	// based on internal data representations and might not match the final size in
	// the requested `format`. If the result was truncated due to the byte limit,
	// then `truncated` in the response is set to `true`. When using
	// `EXTERNAL_LINKS` disposition, a default `byte_limit` of 100 GiB is applied if
	// `byte_limit` is not explicitly set.
	ByteLimit *int64
	// Statement execution supports three result formats: `JSON_ARRAY` (default),
	// `ARROW_STREAM`, and `CSV`.
	//
	// Important: The formats `ARROW_STREAM` and `CSV` are supported only with
	// `EXTERNAL_LINKS` disposition. `JSON_ARRAY` is supported in `INLINE` and
	// `EXTERNAL_LINKS` disposition.
	//
	// When specifying `format=JSON_ARRAY`, result data will be formatted as an
	// array of arrays of values, where each value is either the *string
	// representation* of a value, or `null`. For example, the output of `SELECT
	// concat('id-', id) AS strCol, id AS intCol, null AS nullCol FROM range(3)`
	// would look like this:
	//
	// “` [ [ "id-1", "1", null ], [ "id-2", "2", null ], [ "id-3", "3", null ], ]
	// “`
	//
	// When specifying `format=JSON_ARRAY` and `disposition=EXTERNAL_LINKS`, each
	// chunk in the result contains compact JSON with no indentation or extra
	// whitespace.
	//
	// When specifying `format=ARROW_STREAM` and `disposition=EXTERNAL_LINKS`, each
	// chunk in the result will be formatted as Apache Arrow Stream. See the [Apache
	// Arrow streaming format].
	//
	// When specifying `format=CSV` and `disposition=EXTERNAL_LINKS`, each chunk in
	// the result will be a CSV according to [RFC 4180] standard. All the columns
	// values will have *string representation* similar to the `JSON_ARRAY` format,
	// and `null` values will be encoded as “null”. Only the first chunk in the
	// result would contain a header row with column names. For example, the output
	// of `SELECT concat('id-', id) AS strCol, id AS intCol, null as nullCol FROM
	// range(3)` would look like this:
	//
	// “` strCol,intCol,nullCol id-1,1,null id-2,2,null id-3,3,null “`
	//
	// [Apache Arrow streaming format]: https://arrow.apache.org/docs/format/Columnar.html#ipc-streaming-format
	// [RFC 4180]: https://www.rfc-editor.org/rfc/rfc4180
	Format Format
	// The fetch disposition provides two modes of fetching results: `INLINE` and
	// `EXTERNAL_LINKS`.
	//
	// Statements executed with `INLINE` disposition will return result data inline,
	// in `JSON_ARRAY` format, in a series of chunks. If a given statement produces
	// a result set with a size larger than 25 MiB, that statement execution is
	// aborted, and no result set will be available.
	//
	// **NOTE** Byte limits are computed based upon internal representations of the
	// result set data, and might not match the sizes visible in JSON responses.
	//
	// Statements executed with `EXTERNAL_LINKS` disposition will return result data
	// as external links: URLs that point to cloud storage internal to the
	// workspace. Using `EXTERNAL_LINKS` disposition allows statements to generate
	// arbitrarily sized result sets for fetching up to 100 GiB. The resulting links
	// have two important properties:
	//
	// 1. They point to resources _external_ to the <Databricks> compute; therefore
	// any associated authentication information (typically a personal access token,
	// OAuth token, or similar) _must be removed_ when fetching from these links.
	//
	// 2. These are URLs with a specific expiration, indicated in the response. The
	// behavior when attempting to use an expired link is cloud specific.
	Disposition Disposition
	// The time in seconds the call will wait for the statement's result set as
	// `Ns`, where `N` can be set to 0 or to a value between 5 and 50.
	//
	// When set to `0s`, the statement will execute in asynchronous mode and the
	// call will not wait for the execution to finish. In this case, the call
	// returns directly with `PENDING` state and a statement ID which can be used
	// for polling with :method:statementexecution/getStatement.
	//
	// When set between 5 and 50 seconds, the call will behave synchronously up to
	// this timeout and wait for the statement execution to finish. If the execution
	// finishes within this time, the call returns immediately with a manifest and
	// result data (or a `FAILED` state in case of an execution error). If the
	// statement takes longer to execute, `on_wait_timeout` determines what should
	// happen after the timeout is reached.
	WaitTimeout *string
	// When `wait_timeout > 0s`, the call will block up to the specified time. If
	// the statement execution doesn't finish within this time, `on_wait_timeout`
	// determines whether the execution should continue or be canceled. When set to
	// `CONTINUE`, the statement execution continues asynchronously and the call
	// returns a statement ID which can be used for polling with
	// :method:statementexecution/getStatement. When set to `CANCEL`, the statement
	// execution is canceled and the call returns with a `CANCELED` state.
	OnWaitTimeout TimeoutAction
	// A list of parameters to pass into a SQL statement containing parameter
	// markers. A parameter consists of a name, a value, and optionally a type. To
	// represent a NULL value, the `value` field may be omitted or set to `null`
	// explicitly. If the `type` field is omitted, the value is interpreted as a
	// string.
	//
	// If the type is given, parameters will be checked for type correctness
	// according to the given type. A value is correct if the provided string can be
	// converted to the requested type using the `cast` function. The exact
	// semantics are described in the section [`cast` function] of the SQL language
	// reference.
	//
	// For example, the following statement contains two parameters, `my_name` and
	// `my_date`:
	//
	// “` SELECT * FROM my_table WHERE name = :my_name AND date = :my_date “`
	//
	// The parameters can be passed in the request body as follows:
	//
	// ` { ..., "statement": "SELECT * FROM my_table WHERE name = :my_name AND date
	// = :my_date", "parameters": [ { "name": "my_name", "value": "the name" }, {
	// "name": "my_date", "value": "2020-01-01", "type": "DATE" } ] } `
	//
	// Currently, positional parameters denoted by a `?` marker are not supported by
	// the Databricks SQL Statement Execution API.
	//
	// Also see the section [Parameter markers] of the SQL language reference.
	//
	// [Parameter markers]: https://docs.databricks.com/sql/language-manual/sql-ref-parameter-marker.html
	// [`cast` function]: https://docs.databricks.com/sql/language-manual/functions/cast.html
	Parameters []StatementParameter
	// An array of query tags to annotate a SQL statement. A query tag consists of a
	// non-empty key and, optionally, a value. To represent a NULL value, either
	// omit the `value` field or manually set it to `null` or white space. Refer to
	// the SQL language reference for the format specification of query tags.
	// There's no significance to the order of tags. Only one value per key will be
	// recorded. A sequence in excess of 20 query tags will be coerced to 20.
	// Example:
	//
	// { ..., "query_tags": [ { "key": "team", "value": "eng" }, { "key": "some key
	// only tag" } ] }
	QueryTags []QueryTag
}
type ExternalLink struct {
	// A URL pointing to a chunk of result data, hosted by an external service, with
	// a short expiration time (<= 15 minutes). As this URL contains a temporary
	// credential, it should be considered sensitive and the client should not
	// expose this URL in a log.
	ExternalLink *string
	// Indicates the date-time that the given external link will expire and becomes
	// invalid, after which point a new `external_link` must be requested.
	Expiration *string
	// HTTP headers that must be included with a GET request to the `external_link`.
	// Each header is provided as a key-value pair. Headers are typically used to
	// pass a decryption key to the external service. The values of these headers
	// should be considered sensitive and the client should not expose these values
	// in a log.
	HttpHeaders map[string]string
	// The position within the sequence of result set chunks.
	ChunkIndex *int
	// The starting row offset within the result set.
	RowOffset *int64
	// The number of rows within the result chunk.
	RowCount *int64
	// The number of bytes in the result chunk. This field is not available when
	// using `INLINE` disposition.
	ByteCount *int64
	// When fetching, provides the `chunk_index` for the _next_ chunk. If absent,
	// indicates there are no more chunks. The next chunk can be fetched with a
	// :method:statementexecution/getstatementresultchunkn request.
	NextChunkIndex *int
	// When fetching, provides a link to fetch the _next_ chunk. If absent,
	// indicates there are no more chunks. This link is an absolute `path` to be
	// joined with your `$DATABRICKS_HOST`, and should be treated as an opaque link.
	// This is an alternative to using `next_chunk_index`.
	NextChunkInternalLink *string
}

type Format

type Format string
const (
	Format_Unspecified Format = ""
	Format_JsonArray   Format = "JSON_ARRAY"
	Format_ArrowStream Format = "ARROW_STREAM"
	Format_Csv         Format = "CSV"
)

type GetResultDataRequest

type GetResultDataRequest struct {
	// The statement ID is returned upon successfully submitting a SQL statement,
	// and is a required reference for all subsequent calls.
	StatementId *string
	ChunkIndex  *int
}

type GetStatementResultRequest

type GetStatementResultRequest struct {
	// The statement ID is returned upon successfully submitting a SQL statement,
	// and is a required reference for all subsequent calls.
	StatementId *string
}

type QueryTag

type QueryTag struct {
	Key   *string
	Value *string
}

* A query execution can be annotated with an optional key-value pair to allow users to attribute the executions by key and optional value to filter by. QueryTag is the user-facing representation..

type ResultData

type ResultData struct {
	ExternalLinks []ExternalLink
	// The `JSON_ARRAY` format is an array of arrays of values, where each non-null
	// value is formatted as a string. Null values are encoded as JSON `null`.
	DataArray [][]json.RawMessage
	// The position within the sequence of result set chunks.
	ChunkIndex *int
	// The starting row offset within the result set.
	RowOffset *int64
	// The number of rows within the result chunk.
	RowCount *int64
	// The number of bytes in the result chunk. This field is not available when
	// using `INLINE` disposition.
	ByteCount *int64
	// When fetching, provides the `chunk_index` for the _next_ chunk. If absent,
	// indicates there are no more chunks. The next chunk can be fetched with a
	// :method:statementexecution/getstatementresultchunkn request.
	NextChunkIndex *int
	// When fetching, provides a link to fetch the _next_ chunk. If absent,
	// indicates there are no more chunks. This link is an absolute `path` to be
	// joined with your `$DATABRICKS_HOST`, and should be treated as an opaque link.
	// This is an alternative to using `next_chunk_index`.
	NextChunkInternalLink *string
}

Contains the result data of a single chunk when using `INLINE` disposition. When using `EXTERNAL_LINKS` disposition, the array `external_links` is used instead to provide URLs to the result data in cloud storage. Exactly one of these alternatives is used. (While the `external_links` array prepares the API to return multiple links in a single response. Currently only a single link is returned.).

type ResultManifest

type ResultManifest struct {
	Format Format
	Schema *Schema
	// The total number of chunks that the result set has been divided into.
	TotalChunkCount *int
	// Array of result set chunk metadata.
	Chunks []ChunkInfo
	// The total number of rows in the result set.
	TotalRowCount *int64
	// The total number of bytes in the result set. This field is not available when
	// using `INLINE` disposition.
	TotalByteCount *int64
	// Indicates whether the result is truncated due to `row_limit` or `byte_limit`.
	Truncated *bool
}

The result manifest provides schema and metadata for the result set..

type Schema

type Schema struct {
	ColumnCount *int
	Columns     []ColumnInfo
}

The schema is an ordered list of column descriptions..

type ServiceError

type ServiceError struct {
	ErrorCode ServiceErrorCode
	// A brief summary of the error condition.
	Message *string
}

type ServiceErrorCode

type ServiceErrorCode string
const (
	ServiceErrorCode_Unspecified                     ServiceErrorCode = ""
	ServiceErrorCode_Unknown                         ServiceErrorCode = "UNKNOWN"
	ServiceErrorCode_InternalError                   ServiceErrorCode = "INTERNAL_ERROR"
	ServiceErrorCode_TemporarilyUnavailable          ServiceErrorCode = "TEMPORARILY_UNAVAILABLE"
	ServiceErrorCode_IoError                         ServiceErrorCode = "IO_ERROR"
	ServiceErrorCode_BadRequest                      ServiceErrorCode = "BAD_REQUEST"
	ServiceErrorCode_ServiceUnderMaintenance         ServiceErrorCode = "SERVICE_UNDER_MAINTENANCE"
	ServiceErrorCode_WorkspaceTemporarilyUnavailable ServiceErrorCode = "WORKSPACE_TEMPORARILY_UNAVAILABLE"
	ServiceErrorCode_DeadlineExceeded                ServiceErrorCode = "DEADLINE_EXCEEDED"
	ServiceErrorCode_Cancelled                       ServiceErrorCode = "CANCELLED"
	ServiceErrorCode_ResourceExhausted               ServiceErrorCode = "RESOURCE_EXHAUSTED"
	ServiceErrorCode_Aborted                         ServiceErrorCode = "ABORTED"
	ServiceErrorCode_NotFound                        ServiceErrorCode = "NOT_FOUND"
	ServiceErrorCode_AlreadyExists                   ServiceErrorCode = "ALREADY_EXISTS"
	ServiceErrorCode_Unauthenticated                 ServiceErrorCode = "UNAUTHENTICATED"
)

type StatementParameter

type StatementParameter struct {
	// The name of a parameter marker to be substituted in the statement.
	Name *string
	// The value to substitute, represented as a string. If omitted, the value is
	// interpreted as NULL.
	Value *string
	// The data type, given as a string. For example: `INT`, `STRING`,
	// `DECIMAL(10,2)`. If no type is given the type is assumed to be `STRING`.
	// Complex types, such as `ARRAY`, `MAP`, and `STRUCT` are not supported. For
	// valid types, refer to the section [Data types] of the SQL language reference.
	//
	// [Data types]: https://docs.databricks.com/sql/language-manual/functions/cast.html
	Type *string
}

type StatementResponse

type StatementResponse struct {
	// The statement ID is returned upon successfully submitting a SQL statement,
	// and is a required reference for all subsequent calls.
	StatementId *string
	Status      *StatementStatus
	Manifest    *ResultManifest
	Result      *ResultData
}

type StatementStatus

type StatementStatus struct {
	// Statement execution state: - `PENDING`: waiting for warehouse - `RUNNING`:
	// running - `SUCCEEDED`: execution was successful, result data available for
	// fetch - `FAILED`: execution failed; reason for failure described in
	// accompanying error message - `CANCELED`: user canceled; can come from
	// explicit cancel call, or timeout with `on_wait_timeout=CANCEL` - `CLOSED`:
	// execution successful, and statement closed; result no longer available for
	// fetch
	State StatementStatus_State
	Error *ServiceError
	// SQLSTATE error code returned when the statement execution fails. Only
	// populated when the statement status is `FAILED`.
	SqlState *string
}

The status response includes execution state and if relevant, error information..

type StatementStatus_State

type StatementStatus_State string
const (
	StatementStatus_State_Unspecified StatementStatus_State = ""
	StatementStatus_State_Pending     StatementStatus_State = "PENDING"
	StatementStatus_State_Running     StatementStatus_State = "RUNNING"
	StatementStatus_State_Succeeded   StatementStatus_State = "SUCCEEDED"
	StatementStatus_State_Failed      StatementStatus_State = "FAILED"
	StatementStatus_State_Canceled    StatementStatus_State = "CANCELED"
	StatementStatus_State_Closed      StatementStatus_State = "CLOSED"
)

type TimeoutAction

type TimeoutAction string

When `wait_timeout > 0s`, the call will block up to the specified time. If the statement execution doesn't finish within this time, `on_wait_timeout` determines whether the execution should continue or be canceled. When set to `CONTINUE`, the statement execution continues asynchronously and the call returns a statement ID which can be used for polling with :method:statementexecution/getStatement. When set to `CANCEL`, the statement execution is canceled and the call returns with a `CANCELED` state.

const (
	TimeoutAction_Unspecified TimeoutAction = ""
	TimeoutAction_Continue    TimeoutAction = "CONTINUE"
	TimeoutAction_Cancel      TimeoutAction = "CANCEL"
)

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL