Documentation
¶
Overview ¶
Package driver is Databricks' implementation of the driver contract in github.com/bytebase/omni/driver: it reaches a Databricks workspace through the Databricks SDK, syncs Unity Catalog's catalogs, schemas and tables into omni/metadata, runs statements through a SQL warehouse, and dumps a workspace's DDL as the server states it.
Everything that reaches a workspace is behind one environment gate: DATABRICKS_DSN names the workspace, the personal access token, the SQL warehouse and the catalog a live test uses. Unset, a live test skips naming the gate; set and naming no catalog, it fails, so a recording cannot write no tape and read as a pass.
The tree is driver-only. Bytebase dispatches Databricks through parser/standard's lexical splitter and its EXPLAIN FORMATTED rewrite, and omni has no databricks tree and no standard tree to call, so the driver carries that splitter and rewrite itself, as clickhouse/driver and hive/driver carry theirs until their engine trees exist. When a databricks tree lands, those two pieces belong there and this package should call them.
What it implements ¶
The package was verified against a scratch workspace on 2026-11-17: the live tests passed, the recordings and a sync golden are under testdata, and the parity harness in VERIFY.md agreed with Bytebase's driver on the sync, the instance, queries and Execute. Dump agrees except where the workspace showed Bytebase wrong: a materialized view is the server's own SHOW CREATE TABLE.
It implements Driver, Syncer, Dumper, Querier and Executor. SyncDatabase reads a catalog's schemas and tables through Unity Catalog. SyncInstance reads the SQL version and the catalog names. Dump asks the server to state each object with SHOW CREATE TABLE, because omni has no databricks catalog to render a synced schema as Databricks DDL; the metadata argument is not read, as in hive/driver, so a caller that edits or narrows it changes nothing. A view's server statement names its schema only, so Dump qualifies the target with the catalog. Bytebase's Dump wrote the same server statements, except that it rendered a materialized view from DESC FORMATTED, which is not a statement the workspace accepts, and left a view's target unqualified.
The connection ¶
Open does not reach the workspace. The SDK's configuration resolution would otherwise fetch /.well-known/databricks-config on a best-effort basis, so Open installs a resolver that reads no host metadata; Ping is the first request.
Config.Host is the workspace URL, such as https://dbc-00000000-0000.cloud.databricks.com. Config.Password is the personal access token; a token is a resolved credential, as the contract says, and this driver supports token authentication only, which is what Bytebase's driver supported. Config.Database is the catalog unqualified names resolve in and SyncDatabase reads; empty leaves the workspace's default catalog. Config.Params[ParamWarehouseID] is the SQL warehouse every statement runs on; "warehouse_id" is accepted as an alias. Config.User does nothing: a personal access token has no login name beside it. OAuth credentials are the caller's to exchange; what arrives here is the token.
Config.TLS supplies the TLS settings for the workspace's host and makes TLS required, so a caller can pin a certificate. Config.Dial opens the connections the SDK's HTTP client makes, so an SSH tunnel or a cloud connector reaches the workspace unchanged. Both are installed on the SDK's HTTP transport; nil leaves the SDK's own transport and its system-root verification. Config.ApplicationName does nothing: the service carries no application name. Config.ReadOnly does nothing: a SQL warehouse has no read-only session setting, and Bytebase ignored the flag. Config.EngineVersion does nothing: SyncInstance asks SELECT VERSION(), and nothing version- dependent follows it.
Queries and change scripts ¶
A Session is the driver: the SQL Statement Execution API has no session state, so a statement's Catalog and Schema are sent with the statement and what it sets does not hold for the next. QueryOptions.Schema names the schema sent with each statement of the call. QueryOptions.Limit is the request's RowLimit, as Bytebase sent it, rather than a rewrite of the statement. QueryOptions.MaxResultBytes stops reading once the rows kept pass the cap and sets Result.Truncated. QueryOptions.Explain prefixes EXPLAIN FORMATTED, or leaves a statement that already asks for a plan as the caller wrote it; Result.Plan is TEXT, and Executed is true only for EXPLAIN ANALYZE. QueryOptions.MaskingEnabled does nothing: a Databricks result is the columns its statement traces.
A row's cells are the strings the JSON_ARRAY format states, decoded by the column's type: BOOLEAN a bool, BINARY bytes, SHORT and INT an int32, LONG an int64, FLOAT a float32 widened to float64, DOUBLE a float64, and every other type its text, which is how Bytebase read a DECIMAL, DATE, TIMESTAMP, ARRAY, MAP and STRUCT. A JSON null arrives as the empty string, so an empty value in any non-STRING column is NULL, exactly as Bytebase read it; an empty STRING is the empty string. A result the service splits into chunks is read to the end, or until MaxResultBytes stops it. Result.AffectedRows stays nil because the API states a write's count as a num_affected_rows result column rather than as a field; Execute sums those columns.
Execute splits the script and sends its statements in order; a failing statement ends the script and leaves the statements before it committed. Databricks' Statement Execution API exposes no transaction control, so ExecuteOptions.Transaction, .Isolation, .CreateDatabase and .MaxRetries change nothing, which is also what Bytebase's driver did. ExecuteOptions.Log receives each command and its response, with the rows its num_affected_rows states, or 0 when it states none.
A failure the workspace's REST API reports is a *apierr.APIError, which this package wraps without replacing, and driver.ErrNotFound is joined when the API answers 404. A statement the engine refuses is the SDK helper's error text, "FAILED: <ERROR_CODE> <message>"; the helper discards the typed ServiceError, and this package keeps its text as Bytebase received it.
Index ¶
- Constants
- type Driver
- func (*Driver) Close() error
- func (d *Driver) Dump(ctx context.Context, writer io.Writer, _ *metadatapb.DatabaseSchemaMetadata) error
- func (d *Driver) Execute(ctx context.Context, script string, opts driver.ExecuteOptions) (int64, error)
- func (d *Driver) Ping(ctx context.Context) error
- func (d *Driver) Session(context.Context) (driver.Session, error)
- func (d *Driver) SyncDatabase(ctx context.Context) (*metadatapb.DatabaseSchemaMetadata, error)
- func (d *Driver) SyncInstance(ctx context.Context) (*driver.Instance, error)
Constants ¶
const (
// ParamWarehouseID is the SQL warehouse every statement runs on.
ParamWarehouseID = "warehouseId"
)
The names this driver reads out of driver.Config.Params.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Driver ¶
type Driver struct {
// contains filtered or unexported fields
}
Driver is an open Databricks driver: one workspace client, the SQL warehouse statements run on, and the catalog its syncs and unqualified names use.
func Open ¶
Open returns a driver for the workspace and catalog cfg names. It builds the SDK client but does not reach the workspace; Ping does.
func (*Driver) Dump ¶
func (d *Driver) Dump(ctx context.Context, writer io.Writer, _ *metadatapb.DatabaseSchemaMetadata) error
Dump writes every catalog's, schema's and object's DDL as the server states it. The metadata is not rendered and does not narrow the dump: omni has no databricks catalog that turns a synced schema back into Databricks DDL, so SHOW CREATE TABLE states a managed or external table, a view and a materialized view. The system catalog and each catalog's information_schema are left out, as Bytebase left them out.
func (*Driver) Execute ¶
func (d *Driver) Execute(ctx context.Context, script string, opts driver.ExecuteOptions) (int64, error)
Execute runs every statement of script in order and returns the rows they affected in total, reporting each step to opts.Log as it goes.
Databricks' Statement Execution API exposes no transaction control: a statement commits itself. ExecuteOptions.Transaction, .Isolation, .CreateDatabase and .MaxRetries therefore change nothing, which is what Bytebase's driver did. A failing statement ends the script and leaves the statements before it committed. A write states its count in a num_affected_rows result column, which Execute sums; a statement that states no count contributes 0.
func (*Driver) Ping ¶
Ping asks the workspace for its root folder, which is what Bytebase's driver used as a connectivity check: Databricks has no SQL-level ping.
func (*Driver) SyncDatabase ¶
func (d *Driver) SyncDatabase(ctx context.Context) (*metadatapb.DatabaseSchemaMetadata, error)
SyncDatabase reads the catalog the driver was opened on. An empty catalog answers nothing, as Bytebase's sync did.
func (*Driver) SyncInstance ¶
SyncInstance reads the SQL version and the workspace's catalog names. A catalog is the unit a caller syncs, so the databases are names only: SyncDatabase reads one catalog's schemas and tables. Roles stay empty because Unity Catalog exposes no role a sync reads, which is where Bytebase's TODO left it.