telemetry

package
v1.11.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 21, 2026 License: Apache-2.0 Imports: 18 Imported by: 0

Documentation

Index

Constants

View Source
const (
	OperationTypeUnspecified      = "TYPE_UNSPECIFIED"
	OperationTypeCreateSession    = "CREATE_SESSION"
	OperationTypeDeleteSession    = "DELETE_SESSION"
	OperationTypeExecuteStatement = "EXECUTE_STATEMENT"
	OperationTypeCloseStatement   = "CLOSE_STATEMENT"
)
View Source
const (
	TagWorkspaceID   = "workspace.id"
	TagSessionID     = "session.id"
	TagDriverVersion = "driver.version"
	TagDriverOS      = "driver.os"
	TagDriverRuntime = "driver.runtime"
	TagServerAddress = "server.address" // Not exported to Databricks
)

Tag names for connection metrics

View Source
const (
	TagStatementID        = "statement.id"
	TagResultFormat       = "result.format"
	TagChunkCount         = "chunk_count"
	TagBytesDownloaded    = "bytes_downloaded"
	TagCompressionEnabled = "result.compression_enabled"
	TagOperationType      = "operation_type"
	TagPollCount          = "poll_count"
	TagPollLatency        = "poll.latency_ms"
)

Tag names for statement metrics. Values must match the keys used in metricContext.tags and read by createTelemetryRequest / aggregator — keep them in sync.

View Source
const (
	TagChunkInitialLatencyMs = "chunk_initial_latency_ms"
	TagChunkSlowestLatencyMs = "chunk_slowest_latency_ms"
	TagChunkSumLatencyMs     = "chunk_sum_latency_ms"
	TagChunkTotalPresent     = "chunk_total_present"
)

Tag names for chunk timing metrics

View Source
const (
	TagErrorType = "error.type"
	TagErrorCode = "error.code"
)

Tag names for error metrics

View Source
const (
	TagFeatureCloudFetch    = "feature.cloudfetch"
	TagFeatureLZ4           = "feature.lz4"
	TagFeatureDirectResults = "feature.direct_results"
)

Feature flag tags

Variables

View Source
var ErrCircuitOpen = errors.New("circuit breaker is open")

ErrCircuitOpen is returned when circuit is open.

Functions

func ReleaseForConnection added in v1.11.0

func ReleaseForConnection(host string)

ReleaseForConnection releases telemetry resources for a connection. Should be called when the connection is closed.

Parameters:

  • host: Databricks host

Types

type ChunkDetails added in v1.11.0

type ChunkDetails struct {
	InitialChunkLatencyMs   int64 `json:"initial_chunk_latency_millis,omitempty"`
	SlowestChunkLatencyMs   int64 `json:"slowest_chunk_latency_millis,omitempty"`
	TotalChunksPresent      int32 `json:"total_chunks_present,omitempty"`
	TotalChunksIterated     int32 `json:"total_chunks_iterated,omitempty"`
	SumChunksDownloadTimeMs int64 `json:"sum_chunks_download_time_millis,omitempty"`
}

ChunkDetails maps to ChunkDetails in the proto schema.

type Config

type Config struct {
	// Enabled controls whether telemetry is active
	Enabled bool

	// EnableTelemetry is a tristate for the client DSN setting:
	//   nil   — not set by the client; server feature flag controls enablement
	//   &true — client explicitly opted in (overrides server flag)
	//   &false— client explicitly opted out (overrides server flag)
	EnableTelemetry *bool

	// BatchSize is the number of metrics to batch before flushing
	BatchSize int

	// FlushInterval is how often to flush metrics
	FlushInterval time.Duration

	// MaxRetries is the maximum number of retry attempts
	MaxRetries int

	// RetryDelay is the base delay between retries
	RetryDelay time.Duration

	// CircuitBreakerEnabled enables circuit breaker protection
	CircuitBreakerEnabled bool

	// CircuitBreakerThreshold is failures before opening circuit
	CircuitBreakerThreshold int

	// CircuitBreakerTimeout is time before retrying after open
	CircuitBreakerTimeout time.Duration
}

Config holds telemetry configuration.

func DefaultConfig

func DefaultConfig() *Config

DefaultConfig returns default telemetry configuration.

BEHAVIORAL NOTE (SDR-approved): When EnableTelemetry is nil (the default), telemetry enablement is controlled by the server-side feature flag (databricks.partnerplatform.clientConfigsFeatureFlags.enableTelemetryForGoDriver). This means telemetry may be active without the user explicitly opting in. The user can always override by setting enableTelemetry=true or enableTelemetry=false in the DSN or via WithEnableTelemetry(). No PII is collected; only aggregate driver performance metrics are sent to the Databricks telemetry endpoint.

func ParseTelemetryConfig

func ParseTelemetryConfig(params map[string]string) *Config

ParseTelemetryConfig extracts telemetry config from connection parameters.

type DriverConnectionParameters added in v1.11.0

type DriverConnectionParameters struct {
	HTTPPath             string       `json:"http_path,omitempty"`
	Mode                 string       `json:"mode,omitempty"`
	HostInfo             *HostDetails `json:"host_info,omitempty"`
	UseProxy             bool         `json:"use_proxy,omitempty"`
	AuthMech             string       `json:"auth_mech,omitempty"`
	AuthFlow             string       `json:"auth_flow,omitempty"`
	AuthScope            string       `json:"auth_scope,omitempty"`
	UseSystemProxy       bool         `json:"use_system_proxy,omitempty"`
	UseCFProxy           bool         `json:"use_cf_proxy,omitempty"`
	EnableArrow          bool         `json:"enable_arrow,omitempty"`
	EnableDirectResults  bool         `json:"enable_direct_results,omitempty"`
	QueryTags            string       `json:"query_tags,omitempty"`
	EnableMetricViewMeta bool         `json:"enable_metric_view_metadata,omitempty"`
	SocketTimeout        int64        `json:"socket_timeout,omitempty"`
}

DriverConnectionParameters maps to DriverConnectionParameters in the proto schema. Only fields populated by the Go driver are included; others are omitted.

type DriverErrorInfo added in v1.11.0

type DriverErrorInfo struct {
	ErrorName  string `json:"error_name,omitempty"`
	StackTrace string `json:"stack_trace,omitempty"`
}

DriverErrorInfo maps to DriverErrorInfo in the proto schema.

type DriverSystemConfiguration added in v1.11.0

type DriverSystemConfiguration struct {
	DriverVersion   string `json:"driver_version,omitempty"`
	RuntimeName     string `json:"runtime_name,omitempty"`
	RuntimeVersion  string `json:"runtime_version,omitempty"`
	RuntimeVendor   string `json:"runtime_vendor,omitempty"`
	OSName          string `json:"os_name,omitempty"`
	OSVersion       string `json:"os_version,omitempty"`
	OSArch          string `json:"os_arch,omitempty"`
	DriverName      string `json:"driver_name,omitempty"`
	ClientAppName   string `json:"client_app_name,omitempty"`
	LocaleName      string `json:"locale_name,omitempty"`
	CharSetEncoding string `json:"char_set_encoding,omitempty"`
	ProcessName     string `json:"process_name,omitempty"`
}

DriverSystemConfiguration maps to DriverSystemConfiguration in the proto schema.

type FrontendLogContext added in v1.11.0

type FrontendLogContext struct {
	ClientContext *TelemetryClientContext `json:"client_context,omitempty"`
}

FrontendLogContext contains the client context.

type FrontendLogEntry added in v1.11.0

type FrontendLogEntry struct {
	SQLDriverLog *TelemetryEvent `json:"sql_driver_log,omitempty"`
}

FrontendLogEntry contains the actual telemetry event.

type HostDetails added in v1.11.0

type HostDetails struct {
	HostURL       string `json:"host_url,omitempty"`
	Port          int32  `json:"port,omitempty"`
	ProxyAuthType string `json:"proxy_auth_type,omitempty"`
}

HostDetails maps to HostDetails in the proto schema.

type Interceptor added in v1.11.0

type Interceptor struct {
	// contains filtered or unexported fields
}

Interceptor wraps driver operations to collect metrics. Exported for use by the driver package.

func InitializeForConnection added in v1.11.0

func InitializeForConnection(ctx context.Context, opts TelemetryInitOptions) *Interceptor

InitializeForConnection initializes telemetry for a database connection. Returns an Interceptor if telemetry is enabled, nil otherwise. This function handles all the logic for checking feature flags and creating the interceptor.

func (*Interceptor) AddTag added in v1.11.0

func (i *Interceptor) AddTag(ctx context.Context, key string, value interface{})

AddTag adds a tag to the current metric context. Exported for use by the driver package.

func (*Interceptor) AfterExecute added in v1.11.0

func (i *Interceptor) AfterExecute(ctx context.Context, err error)

AfterExecute is called after statement execution. Records the metric with timing and error information. Exported for use by the driver package.

func (*Interceptor) BeforeExecute added in v1.11.0

func (i *Interceptor) BeforeExecute(ctx context.Context, sessionID string, statementID string) context.Context

BeforeExecute is called before statement execution. Returns a new context with metric tracking attached. Exported for use by the driver package.

func (*Interceptor) BeforeExecuteWithTime added in v1.11.0

func (i *Interceptor) BeforeExecuteWithTime(ctx context.Context, sessionID string, statementID string, startTime time.Time) context.Context

BeforeExecuteWithTime is called before statement execution with a custom start time. This is useful when the statement ID is not known until after execution starts. Exported for use by the driver package.

func (*Interceptor) Close added in v1.11.0

func (i *Interceptor) Close(ctx context.Context) error

Close flushes any pending per-connection metrics. Does NOT close the shared aggregator — its lifecycle is managed via ReleaseForConnection, which uses reference counting across all connections to the same host. Exported for use by the driver package.

func (*Interceptor) CompleteStatement added in v1.11.0

func (i *Interceptor) CompleteStatement(ctx context.Context, statementID string, failed bool)

CompleteStatement marks a statement as complete and flushes aggregated metrics. Exported for use by the driver package.

func (*Interceptor) FinalizeLatency added in v1.11.0

func (i *Interceptor) FinalizeLatency(ctx context.Context)

FinalizeLatency freezes the elapsed time as the statement's execution latency. Call this when the execute phase is complete (i.e. when QueryContext returns) so that AfterExecute, even if called later from rows.Close(), still reports execute-only latency rather than total latency that would include row iteration. Exported for use by the driver package.

func (*Interceptor) RecordOperation added in v1.11.0

func (i *Interceptor) RecordOperation(ctx context.Context, sessionID string, statementID string, operationType string, latencyMs int64, err error)

RecordOperation records an operation with type, latency, and optional error. statementID is included when the operation is scoped to a specific statement (e.g. CLOSE_STATEMENT). Pass "" for session-level operations (CREATE_SESSION, DELETE_SESSION). Exported for use by the driver package.

type OperationDetail added in v1.11.0

type OperationDetail struct {
	NOperationStatusCalls    int32  `json:"n_operation_status_calls,omitempty"`
	OperationStatusLatencyMs int64  `json:"operation_status_latency_millis,omitempty"`
	OperationType            string `json:"operation_type,omitempty"`
	IsInternalCall           bool   `json:"is_internal_call,omitempty"`
}

OperationDetail maps to OperationDetail in the proto schema.

type ResultLatency added in v1.11.0

type ResultLatency struct {
	ResultSetReadyLatencyMs       int64 `json:"result_set_ready_latency_millis,omitempty"`
	ResultSetConsumptionLatencyMs int64 `json:"result_set_consumption_latency_millis,omitempty"`
}

ResultLatency maps to ResultLatency in the proto schema.

type SQLExecutionEvent added in v1.11.0

type SQLExecutionEvent struct {
	StatementType        string           `json:"statement_type,omitempty"`
	IsCompressed         bool             `json:"is_compressed,omitempty"`
	ExecutionResult      string           `json:"execution_result,omitempty"`
	ChunkID              int64            `json:"chunk_id,omitempty"`
	RetryCount           int64            `json:"retry_count,omitempty"`
	ChunkDetails         *ChunkDetails    `json:"chunk_details,omitempty"`
	ResultLatency        *ResultLatency   `json:"result_latency,omitempty"`
	OperationDetail      *OperationDetail `json:"operation_detail,omitempty"`
	JavaUsesPatchedArrow bool             `json:"java_uses_patched_arrow,omitempty"`
}

SQLExecutionEvent maps to SqlExecutionEvent in the proto schema.

type TelemetryClientContext added in v1.11.0

type TelemetryClientContext struct {
	ClientType    string `json:"client_type,omitempty"`
	ClientVersion string `json:"client_version,omitempty"`
}

TelemetryClientContext contains client-level information.

type TelemetryEvent added in v1.11.0

type TelemetryEvent struct {
	SessionID                  string                      `json:"session_id,omitempty"`
	SQLStatementID             string                      `json:"sql_statement_id,omitempty"`
	SystemConfiguration        *DriverSystemConfiguration  `json:"system_configuration,omitempty"`
	DriverConnectionParameters *DriverConnectionParameters `json:"driver_connection_params,omitempty"`
	AuthType                   string                      `json:"auth_type,omitempty"`
	VolumeOperation            *VolumeOperationEvent       `json:"vol_operation,omitempty"`
	SQLOperation               *SQLExecutionEvent          `json:"sql_operation,omitempty"`
	ErrorInfo                  *DriverErrorInfo            `json:"error_info,omitempty"`
	OperationLatencyMs         int64                       `json:"operation_latency_ms"`
}

TelemetryEvent maps to OssSqlDriverTelemetryLog in the proto schema.

type TelemetryFrontendLog added in v1.11.0

type TelemetryFrontendLog struct {
	WorkspaceID        int64               `json:"workspace_id,omitempty"`
	FrontendLogEventID string              `json:"frontend_log_event_id,omitempty"`
	Context            *FrontendLogContext `json:"context,omitempty"`
	Entry              *FrontendLogEntry   `json:"entry,omitempty"`
}

TelemetryFrontendLog represents a single telemetry log entry.

type TelemetryInitOptions added in v1.11.0

type TelemetryInitOptions struct {
	// Host is the Databricks host.
	Host string

	// DriverVersion is the driver version string.
	DriverVersion string

	// HTTPClient is the HTTP client used for both feature-flag checks and
	// telemetry export. The /telemetry-ext endpoint requires authentication,
	// so this should be the authenticated driver client.
	HTTPClient *http.Client

	// EnableTelemetry is a tristate from the client DSN:
	//   unset  — server feature flag controls enablement
	//   true   — client explicitly opted in
	//   false  — client explicitly opted out
	EnableTelemetry config.ConfigValue[bool]

	// BatchSize is the number of metrics per batch (0 = use default 100).
	BatchSize int

	// FlushInterval is the flush interval (0 = use default 5s).
	FlushInterval time.Duration

	// RetryCount is max retry attempts (-1 = use default 3; 0 = disable retries).
	// IMPORTANT: Go's zero-value for int is 0, which disables retries. Callers
	// constructing TelemetryInitOptions must set RetryCount = -1 explicitly to
	// get the default retry behavior.
	RetryCount int

	// RetryDelay is the base delay between retries (0 = use default 100ms).
	RetryDelay time.Duration
}

TelemetryInitOptions bundles the parameters for InitializeForConnection.

type TelemetryRequest added in v1.11.0

type TelemetryRequest struct {
	UploadTime int64    `json:"uploadTime"`
	Items      []string `json:"items"`
	ProtoLogs  []string `json:"protoLogs"`
}

TelemetryRequest is the top-level request sent to the telemetry endpoint.

type TelemetryResponse added in v1.11.0

type TelemetryResponse struct {
	Errors             []string `json:"errors"`
	NumSuccess         int      `json:"numSuccess"`
	NumProtoSuccess    int      `json:"numProtoSuccess"`
	NumRealtimeSuccess int      `json:"numRealtimeSuccess"`
}

TelemetryResponse is the response from the telemetry endpoint.

type VolumeOperationEvent added in v1.11.0

type VolumeOperationEvent struct {
	VolumeOperationType string `json:"volume_operation_type,omitempty"`
	VolumePath          string `json:"volume_path,omitempty"`
	LocalFile           string `json:"local_file,omitempty"`
}

VolumeOperationEvent maps to VolumeOperationEvent in the proto schema.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL