clusters

package
v0.0.1-dev.7 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 8, 2026 License: Apache-2.0 Imports: 22 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Adlsgen2Info

type Adlsgen2Info struct {
	// abfss destination, e.g.
	// `abfss://<container-name>@<storage-account-name>.dfs.core.windows.net/<directory-name>`.
	Destination *string
}

A storage location in Adls Gen2.

type AutoScale

type AutoScale struct {
	// The minimum number of workers to which the cluster can scale down when
	// underutilized. It is also the initial number of workers the cluster will have
	// after creation.
	MinWorkers *int `fieldmask:"min_workers"`
	// The maximum number of workers to which the cluster can scale up when
	// overloaded. Note that `max_workers` must be strictly greater than
	// `min_workers`.
	MaxWorkers *int `fieldmask:"max_workers"`
}

type AwsAttributes

type AwsAttributes struct {
	// The first `first_on_demand` nodes of the cluster will be placed on on-demand
	// instances. If this value is greater than 0, the cluster driver node in
	// particular will be placed on an on-demand instance. If this value is greater
	// than or equal to the current cluster size, all nodes will be placed on
	// on-demand instances. If this value is less than the current cluster size,
	// `first_on_demand` nodes will be placed on on-demand instances and the
	// remainder will be placed on `availability` instances. Note that this value
	// does not affect cluster size and cannot currently be mutated over the
	// lifetime of a cluster.
	FirstOnDemand *int            `fieldmask:"first_on_demand"`
	Availability  AwsAvailability `fieldmask:"availability"`
	// Identifier for the availability zone/datacenter in which the cluster resides.
	// This string will be of a form like "us-west-2a". The provided availability
	// zone must be in the same region as the <Databricks> deployment. For example,
	// "us-west-2a" is not a valid zone id if the <Databricks> deployment resides in
	// the "us-east-1" region. This is an optional field at cluster creation, and if
	// not specified, the zone "auto" will be used. If the zone specified is "auto",
	// will try to place cluster in a zone with high availability, and will retry
	// placement in a different AZ if there is not enough capacity. The list of
	// available zones as well as the default value can be found by using the `List
	// Zones` method.
	ZoneId *string `fieldmask:"zone_id"`
	// Nodes for this cluster will only be placed on AWS instances with this
	// instance profile. If ommitted, nodes will be placed on instances without an
	// IAM instance profile. The instance profile must have previously been added to
	// the <Databricks> environment by an account administrator.
	//
	// This feature may only be available to certain customer plans.
	InstanceProfileArn *string `fieldmask:"instance_profile_arn"`
	// The bid price for AWS spot instances, as a percentage of the corresponding
	// instance type's on-demand price. For example, if this field is set to 50, and
	// the cluster needs a new `r3.xlarge` spot instance, then the bid price is half
	// of the price of on-demand `r3.xlarge` instances. Similarly, if this field is
	// set to 200, the bid price is twice the price of on-demand `r3.xlarge`
	// instances. If not specified, the default value is 100. When spot instances
	// are requested for this cluster, only spot instances whose bid price
	// percentage matches this field will be considered. Note that, for safety, we
	// enforce this field to be no more than 10000.
	SpotBidPricePercent *int `fieldmask:"spot_bid_price_percent"`
	// The type of EBS volumes that will be launched with this cluster.
	EbsVolumeType EbsVolumeType `fieldmask:"ebs_volume_type"`
	// The number of volumes launched for each instance. Users can choose up to 10
	// volumes. This feature is only enabled for supported node types. Legacy node
	// types cannot specify custom EBS volumes. For node types with no instance
	// store, at least one EBS volume needs to be specified; otherwise, cluster
	// creation will fail.
	//
	// These EBS volumes will be mounted at `/ebs0`, `/ebs1`, and etc. Instance
	// store volumes will be mounted at `/local_disk0`, `/local_disk1`, and etc.
	//
	// If EBS volumes are attached, <Databricks> will configure Spark to use only
	// the EBS volumes for scratch storage because heterogenously sized scratch
	// devices can lead to inefficient disk utilization. If no EBS volumes are
	// attached, <Databricks> will configure Spark to use instance store volumes.
	//
	// Please note that if EBS volumes are specified, then the Spark configuration
	// `spark.local.dir` will be overridden.
	EbsVolumeCount *int `fieldmask:"ebs_volume_count"`
	// The size of each EBS volume (in GiB) launched for each instance. For general
	// purpose SSD, this value must be within the range 100 - 4096. For throughput
	// optimized HDD, this value must be within the range 500 - 4096.
	EbsVolumeSize *int `fieldmask:"ebs_volume_size"`
	// If using gp3 volumes, what IOPS to use for the disk. If this is not set, the
	// maximum performance of a gp2 volume with the same volume size will be used.
	EbsVolumeIops *int `fieldmask:"ebs_volume_iops"`
	// If using gp3 volumes, what throughput to use for the disk. If this is not
	// set, the maximum performance of a gp2 volume with the same volume size will
	// be used.
	EbsVolumeThroughput *int `fieldmask:"ebs_volume_throughput"`
}

Attributes set during cluster creation which are related to Amazon Web Services..

type AwsAvailability

type AwsAvailability string

Availability type used for all subsequent nodes past the `first_on_demand` ones.

Note: If `first_on_demand` is zero, this availability type will be used for the entire cluster.

const (
	AwsAvailability_Unspecified AwsAvailability = ""
	// Use spot instances.
	AwsAvailability_Spot AwsAvailability = "SPOT"
	// Use on-demand instances.
	AwsAvailability_OnDemand AwsAvailability = "ON_DEMAND"
	// Preferably use spot instances, but fall back to on-demand instances if spot
	// instances cannot be acquired (e.g., if AWS spot prices are too high).
	AwsAvailability_SpotWithFallback AwsAvailability = "SPOT_WITH_FALLBACK"
)

type AzureAttributes

type AzureAttributes struct {
	// Defines values necessary to configure and run Azure Log Analytics agent
	LogAnalyticsInfo *LogAnalyticsInfo `fieldmask:"log_analytics_info"`
	// The first `first_on_demand` nodes of the cluster will be placed on on-demand
	// instances. This value should be greater than 0, to make sure the cluster
	// driver node is placed on an on-demand instance. If this value is greater than
	// or equal to the current cluster size, all nodes will be placed on on-demand
	// instances. If this value is less than the current cluster size,
	// `first_on_demand` nodes will be placed on on-demand instances and the
	// remainder will be placed on `availability` instances. Note that this value
	// does not affect cluster size and cannot currently be mutated over the
	// lifetime of a cluster.
	FirstOnDemand *int `fieldmask:"first_on_demand"`
	// Availability type used for all subsequent nodes past the `first_on_demand`
	// ones. Note: If `first_on_demand` is zero, this availability type will be used
	// for the entire cluster.
	Availability AzureAvailability `fieldmask:"availability"`
	// The max bid price to be used for Azure spot instances. The Max price for the
	// bid cannot be higher than the on-demand price of the instance. If not
	// specified, the default value is -1, which specifies that the instance cannot
	// be evicted on the basis of price, and only on the basis of availability.
	// Further, the value should > 0 or -1.
	SpotBidMaxPrice *float64 `fieldmask:"spot_bid_max_price"`
	// The Azure capacity reservation group resource ID to use for launching VMs.
	// When specified, VMs will be launched using the provided capacity reservation.
	//
	// Capacity reservations can only be specified when the workspace uses injected
	// vnet (i.e. customer defined vnet not managed by databricks). Ensure the
	// databricks-login-prod Enterprise Application is granted the following four
	// permissions: 1. Microsoft.Compute/capacityReservationGroups/read 2.
	// Microsoft.Compute/capacityReservationGroups/deploy/action 3.
	// Microsoft.Compute/capacityReservationGroups/capacityReservations/read 4.
	// Microsoft.Compute/capacityReservationGroups/capacityReservations/deploy/action
	//
	// Format:
	// `/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Compute/capacityReservationGroups/{capacityReservationGroupName}`
	CapacityReservationGroup *string `fieldmask:"capacity_reservation_group"`
}

Attributes set during cluster creation which are related to Microsoft Azure..

type AzureAvailability

type AzureAvailability string

Availability type used for all subsequent nodes past the `first_on_demand` ones. Note: If `first_on_demand` is zero, this availability type will be used for the entire cluster.

const (
	AzureAvailability_Unspecified AzureAvailability = ""
	// Use spot instances.
	AzureAvailability_SpotAzure AzureAvailability = "SPOT_AZURE"
	// Use on-demand instances.
	AzureAvailability_OnDemandAzure AzureAvailability = "ON_DEMAND_AZURE"
	// Preferably use spot instances, but fall back to on-demand instances if spot
	// instances cannot be acquired (e.g., if Azure is out of Quota).
	AzureAvailability_SpotWithFallbackAzure AzureAvailability = "SPOT_WITH_FALLBACK_AZURE"
)

type CancelPendingClusterEnforcementRequest

type CancelPendingClusterEnforcementRequest struct {
	// The ID of the cluster to cancel the pending enforcement for.
	ClusterId *string
	// If true and no pending enforcement exists, the request will succeed but no
	// action will be taken.
	AllowMissing *bool
}

Request to cancel the pending enforcement for a cluster..

type CancelPendingClusterEnforcementResponse

type CancelPendingClusterEnforcementResponse struct {
}

Response for canceling the pending enforcement for a cluster. If the cancel request succeeds, an empty response object is returned. Otherwise, an error response is returned..

type ChangeClusterOwnerRequest

type ChangeClusterOwnerRequest struct {
	ClusterId *string
	// New owner of the cluster_id after this RPC.
	OwnerUsername *string
}

type ChangeClusterOwnerResponse

type ChangeClusterOwnerResponse struct {
}

type Client

type Client struct {
	// contains filtered or unexported fields
}

func NewClient

func NewClient(ctx context.Context, opts ...client.Option) (*Client, error)

func (*Client) CancelPendingClusterEnforcement

func (c *Client) CancelPendingClusterEnforcement(ctx context.Context, req CancelPendingClusterEnforcementRequest, opts ...call.Option) (*CancelPendingClusterEnforcementResponse, error)

Cancels a pending enforcement on a cluster. After canceling the pending enforcement, the cluster will no longer update on the next termination or restart. Pending enforcements cannot be canceled when a cluster is in `TERMINATING` state. Only workspace admins can cancel pending enforcements.

func (*Client) ChangeClusterOwner

func (c *Client) ChangeClusterOwner(ctx context.Context, req ChangeClusterOwnerRequest, opts ...call.Option) (*ChangeClusterOwnerResponse, error)

Change the owner of the cluster. You must be an admin and the cluster must be terminated to perform this operation. The service principal application ID can be supplied as an argument to `owner_username`.

func (*Client) CreateCluster

func (c *Client) CreateCluster(ctx context.Context, req CreateClusterRequest, opts ...call.Option) (*CreateClusterWaiter, error)

Creates a new Spark cluster. This method will acquire new instances from the cloud provider if necessary. This method is asynchronous; the returned “cluster_id“ can be used to poll the cluster status. When this method returns, the cluster will be in a “PENDING“ state. The cluster will be usable once it enters a “RUNNING“ state. Note: <Databricks> may not be able to acquire some of the requested nodes, due to cloud provider limitations (account limits, spot price, etc.) or transient network issues.

If <Databricks> acquires at least 85% of the requested on-demand nodes, cluster creation will succeed. Otherwise the cluster will terminate with an informative error message.

Rather than authoring the cluster's JSON definition from scratch, Databricks recommends filling out the [create compute UI](/compute/configure.html) and then copying the generated JSON definition from the UI.

func (*Client) DeleteCluster

func (c *Client) DeleteCluster(ctx context.Context, req DeleteClusterRequest, opts ...call.Option) (*DeleteClusterWaiter, error)

Terminates the Spark cluster with the specified ID. The cluster is removed asynchronously. Once the termination has completed, the cluster will be in a `TERMINATED` state. If the cluster is already in a `TERMINATING` or `TERMINATED` state, nothing will happen.

func (*Client) EditCluster

func (c *Client) EditCluster(ctx context.Context, req EditClusterRequest, opts ...call.Option) (*EditClusterWaiter, error)

Updates the configuration of a cluster to match the provided attributes and size. A cluster can be updated if it is in a `RUNNING` or `TERMINATED` state.

If a cluster is updated while in a `RUNNING` state, it will be restarted so that the new attributes can take effect.

If a cluster is updated while in a `TERMINATED` state, it will remain `TERMINATED`. The next time it is started using the `clusters/start` API, the new attributes will take effect. Any attempt to update a cluster in any other state will be rejected with an `INVALID_STATE` error code.

Clusters created by the Databricks Jobs service cannot be edited.

func (*Client) EnforcePolicyComplianceForCluster

func (c *Client) EnforcePolicyComplianceForCluster(ctx context.Context, req EnforcePolicyComplianceForClusterRequest, opts ...call.Option) (*EnforcePolicyComplianceForClusterResponse, error)

Updates a cluster to be compliant with the current version of its policy.

If a cluster is updated while in a `TERMINATED` state, it will remain `TERMINATED`. The next time the cluster is started, the new attributes will take effect.

For clusters in other states, the behavior depends on the `enforce_mode` used.

Clusters created by the Databricks Jobs, SDP, or Models services cannot be enforced by this API. Instead, use the "Enforce job policy compliance" API to enforce policy compliance on jobs.

func (*Client) GetCluster

func (c *Client) GetCluster(ctx context.Context, req GetClusterRequest, opts ...call.Option) (*ClusterInfo, error)

Retrieves the information for a cluster given its identifier. Clusters can be described while they are running, or up to 60 days after they are terminated.

func (*Client) GetPolicyComplianceForCluster

func (c *Client) GetPolicyComplianceForCluster(ctx context.Context, req GetPolicyComplianceForClusterRequest, opts ...call.Option) (*GetPolicyComplianceForClusterResponse, error)

Returns the policy compliance status of a cluster. Clusters could be out of compliance if their policy was updated after the cluster was last edited.

func (*Client) ListAvailableZones

func (c *Client) ListAvailableZones(ctx context.Context, req ListAvailableZonesRequest, opts ...call.Option) (*ListAvailableZonesResponse, error)

Returns a list of availability zones where clusters can be created in (For example, us-west-2a). These zones can be used to launch a cluster.

func (*Client) ListClusterComplianceForPolicy

func (c *Client) ListClusterComplianceForPolicy(ctx context.Context, req ListClusterComplianceForPolicyRequest, opts ...call.Option) (*ListClusterComplianceForPolicyResponse, error)

Returns the policy compliance status of all clusters that use a given policy. Clusters could be out of compliance if their policy was updated after the cluster was last edited.

func (*Client) ListClusterComplianceForPolicyIter

func (c *Client) ListClusterComplianceForPolicyIter(ctx context.Context, req ListClusterComplianceForPolicyRequest, opts ...call.Option) iter.Seq2[*ClusterCompliance, error]

ListClusterComplianceForPolicyIter returns an iterator that iterates over the results of ListClusterComplianceForPolicy.

For example:

for item, err := range c.ListClusterComplianceForPolicyIter(ctx, ListClusterComplianceForPolicyRequest{}) {
  if err != nil {
    return err
  }
  fmt.Println(item)
}

Options opts are passed to each ListClusterComplianceForPolicy call made by the iterator under the hood.

Callers who need custom pagination logic should use ListClusterComplianceForPolicy directly.

func (*Client) ListClusters

func (c *Client) ListClusters(ctx context.Context, req ListClustersRequest, opts ...call.Option) (*ListClustersResponse, error)

Return information about all pinned and active clusters, and all clusters terminated within the last 30 days. Clusters terminated prior to this period are not included.

func (*Client) ListClustersIter

func (c *Client) ListClustersIter(ctx context.Context, req ListClustersRequest, opts ...call.Option) iter.Seq2[*ClusterInfo, error]

ListClustersIter returns an iterator that iterates over the results of ListClusters.

For example:

for item, err := range c.ListClustersIter(ctx, ListClustersRequest{}) {
  if err != nil {
    return err
  }
  fmt.Println(item)
}

Options opts are passed to each ListClusters call made by the iterator under the hood.

Callers who need custom pagination logic should use ListClusters directly.

func (*Client) ListEvents

func (c *Client) ListEvents(ctx context.Context, req ListEventsRequest, opts ...call.Option) (*GetEventsResponse, error)

Retrieves a list of events about the activity of a cluster. This API is paginated. If there are more events to read, the response includes all the parameters necessary to request the next page of events.

func (*Client) ListEventsIter

func (c *Client) ListEventsIter(ctx context.Context, req ListEventsRequest, opts ...call.Option) iter.Seq2[*ClusterEvent, error]

ListEventsIter returns an iterator that iterates over the results of ListEvents.

For example:

for item, err := range c.ListEventsIter(ctx, ListEventsRequest{}) {
  if err != nil {
    return err
  }
  fmt.Println(item)
}

Options opts are passed to each ListEvents call made by the iterator under the hood.

Callers who need custom pagination logic should use ListEvents directly.

func (*Client) ListNodeTypes

func (c *Client) ListNodeTypes(ctx context.Context, req ListNodeTypesRequest, opts ...call.Option) (*ListNodeTypesResponse, error)

Returns a list of supported Spark node types. These node types can be used to launch a cluster.

func (*Client) ListSparkVersions

func (c *Client) ListSparkVersions(ctx context.Context, req GetSparkVersionsRequest, opts ...call.Option) (*GetSparkVersionsResponse, error)

Returns the list of available Spark versions. These versions can be used to launch a cluster.

func (*Client) PermanentDeleteCluster

func (c *Client) PermanentDeleteCluster(ctx context.Context, req PermanentDeleteClusterRequest, opts ...call.Option) (*PermanentDeleteClusterResponse, error)

Permanently deletes a Spark cluster. This cluster is terminated and resources are asynchronously removed.

In addition, users will no longer see permanently deleted clusters in the cluster list, and API users can no longer perform any action on permanently deleted clusters.

func (*Client) PinCluster

func (c *Client) PinCluster(ctx context.Context, req PinClusterRequest, opts ...call.Option) (*PinClusterResponse, error)

Pinning a cluster ensures that the cluster will always be returned by the ListClusters API. Pinning a cluster that is already pinned will have no effect. This API can only be called by workspace admins.

func (*Client) ResizeCluster

func (c *Client) ResizeCluster(ctx context.Context, req ResizeClusterRequest, opts ...call.Option) (*ResizeClusterWaiter, error)

Resizes a cluster to have a desired number of workers. This will fail unless the cluster is in a `RUNNING` state.

func (*Client) RestartCluster

func (c *Client) RestartCluster(ctx context.Context, req RestartClusterRequest, opts ...call.Option) (*RestartClusterWaiter, error)

Restarts a Spark cluster with the supplied ID. If the cluster is not currently in a `RUNNING` state, nothing will happen.

func (*Client) StartCluster

func (c *Client) StartCluster(ctx context.Context, req StartClusterRequest, opts ...call.Option) (*StartClusterWaiter, error)

Starts a terminated Spark cluster with the supplied ID. This works similar to `createCluster` except: - The previous cluster id and attributes are preserved. - The cluster starts with the last specified cluster size. - If the previous cluster was an autoscaling cluster, the current cluster starts with the minimum number of nodes. - If the cluster is not currently in a “TERMINATED“ state, nothing will happen. - Clusters launched to run a job cannot be started.

func (*Client) UnpinCluster

func (c *Client) UnpinCluster(ctx context.Context, req UnpinClusterRequest, opts ...call.Option) (*UnpinClusterResponse, error)

Unpinning a cluster will allow the cluster to eventually be removed from the ListClusters API. Unpinning a cluster that is not pinned will have no effect. This API can only be called by workspace admins.

func (*Client) UpdateCluster

func (c *Client) UpdateCluster(ctx context.Context, req UpdateClusterRequest, opts ...call.Option) (*UpdateClusterWaiter, error)

Updates the configuration of a cluster to match the partial set of attributes and size. Denote which fields to update using the `update_mask` field in the request body. A cluster can be updated if it is in a `RUNNING` or `TERMINATED` state. If a cluster is updated while in a `RUNNING` state, it will be restarted so that the new attributes can take effect. If a cluster is updated while in a `TERMINATED` state, it will remain `TERMINATED`. The updated attributes will take effect the next time the cluster is started using the `clusters/start` API. Attempts to update a cluster in any other state will be rejected with an `INVALID_STATE` error code. Clusters created by the Databricks Jobs service cannot be updated.

type CloneCluster

type CloneCluster struct {
	// The cluster that is being cloned.
	SourceClusterId *string
}

type CloudProviderNodeInfo

type CloudProviderNodeInfo struct {
	// Status as reported by the cloud provider
	Status []CloudProviderNodeStatus
}

type CloudProviderNodeStatus

type CloudProviderNodeStatus string
const (
	CloudProviderNodeStatus_Unspecified              CloudProviderNodeStatus = ""
	CloudProviderNodeStatus_NotEnabledOnSubscription CloudProviderNodeStatus = "NotEnabledOnSubscription"
	CloudProviderNodeStatus_NotAvailableInRegion     CloudProviderNodeStatus = "NotAvailableInRegion"
)

type ClusterAttributes

type ClusterAttributes struct {
	// Cluster name requested by the user. This doesn't have to be unique. If not
	// specified at creation, the cluster name will be an empty string. For job
	// clusters, the cluster name is automatically set based on the job and job run
	// IDs.
	ClusterName *string
	// The Spark version of the cluster, e.g. `3.3.x-scala2.11`. A list of available
	// Spark versions can be retrieved by using the [clusters/sparkVersions] API
	// call.
	//
	// [clusters/sparkVersions]: https://docs.databricks.com/api/workspace/clusters/sparkversions
	SparkVersion *string
	// An object containing a set of optional, user-specified Spark configuration
	// key-value pairs. Users can also pass in a string of extra JVM options to the
	// driver and the executors via `spark.driver.extraJavaOptions` and
	// `spark.executor.extraJavaOptions` respectively.
	SparkConf map[string]string
	// Attributes related to clusters running on Amazon Web Services. If not
	// specified at cluster creation, a set of default values will be used.
	AwsAttributes *AwsAttributes
	// Attributes related to clusters running on Microsoft Azure. If not specified
	// at cluster creation, a set of default values will be used.
	AzureAttributes *AzureAttributes
	// Attributes related to clusters running on Google Cloud Platform. If not
	// specified at cluster creation, a set of default values will be used.
	GcpAttributes *GcpAttributes
	// This field encodes, through a single value, the resources available to each
	// of the Spark nodes in this cluster. For example, the Spark nodes can be
	// provisioned and optimized for memory or compute intensive workloads. A list
	// of available node types can be retrieved by using the
	// [clusters/listNodeTypes] API call.
	//
	// [clusters/listNodeTypes]: https://docs.databricks.com/api/workspace/clusters/listnodetypes
	NodeTypeId *string
	// The node type of the Spark driver. Note that this field is optional; if
	// unset, the driver node type will be set as the same value as `node_type_id`
	// defined above.
	//
	// This field, along with node_type_id, should not be set if
	// virtual_cluster_size is set. If both driver_node_type_id, node_type_id, and
	// virtual_cluster_size are specified, driver_node_type_id and node_type_id take
	// precedence.
	DriverNodeTypeId *string
	// Flexible node type configuration for worker nodes.
	WorkerNodeTypeFlexibility *NodeTypeFlexibility
	// Flexible node type configuration for the driver node.
	DriverNodeTypeFlexibility *NodeTypeFlexibility
	// SSH public key contents that will be added to each Spark node in this
	// cluster. The corresponding private keys can be used to login with the user
	// name `ubuntu` on port `2200`. Up to 10 keys can be specified.
	SshPublicKeys []string
	// Additional tags for cluster resources. <Databricks> will tag all cluster
	// resources (e.g., AWS instances and EBS volumes) with these tags in addition
	// to `default_tags`. Notes:
	//
	// - Currently, <Databricks> allows at most 45 custom tags
	//
	// - Clusters can only reuse cloud resources if the resources' tags are a subset
	// of the cluster tags
	CustomTags map[string]string
	// The configuration for delivering spark logs to a long-term storage
	// destination. Three kinds of destinations (DBFS, S3 and Unity Catalog volumes)
	// are supported. Only one destination can be specified for one cluster. If the
	// conf is given, the logs will be delivered to the destination every `5 mins`.
	// The destination of driver logs is `$destination/$clusterId/driver`, while the
	// destination of executor logs is `$destination/$clusterId/executor`.
	ClusterLogConf *ClusterLogConf
	// An object containing a set of optional, user-specified environment variable
	// key-value pairs. Please note that key-value pair of the form (X,Y) will be
	// exported as is (i.e., `export X='Y'`) while launching the driver and workers.
	//
	// In order to specify an additional set of `SPARK_DAEMON_JAVA_OPTS`, we
	// recommend appending them to `$SPARK_DAEMON_JAVA_OPTS` as shown in the example
	// below. This ensures that all default databricks managed environmental
	// variables are included as well.
	//
	// Example Spark environment variables: `{"SPARK_WORKER_MEMORY": "28000m",
	// "SPARK_LOCAL_DIRS": "/local_disk0"}` or `{"SPARK_DAEMON_JAVA_OPTS":
	// "$SPARK_DAEMON_JAVA_OPTS -Dspark.shuffle.service.enabled=true"}`
	SparkEnvVars map[string]string
	// Automatically terminates the cluster after it is inactive for this time in
	// minutes. If not set, this cluster will not be automatically terminated. If
	// specified, the threshold must be between 10 and 10000 minutes. Users can also
	// set this value to 0 to explicitly disable automatic termination.
	AutoterminationMinutes *int
	// Autoscaling Local Storage: when enabled, this cluster will dynamically
	// acquire additional disk space when its Spark workers are running low on disk
	// space.
	EnableElasticDisk *bool
	// The configuration for storing init scripts. Any number of destinations can be
	// specified. The scripts are executed sequentially in the order provided. If
	// `cluster_log_conf` is specified, init script logs are sent to
	// `<destination>/<cluster-ID>/init_scripts`.
	InitScripts []InitScriptInfo
	// Custom docker image BYOC
	DockerImage *DockerImage
	// The optional ID of the instance pool to which the cluster belongs.
	InstancePoolId *string
	// Single user name if data_security_mode is `SINGLE_USER`
	SingleUserName *string
	// The ID of the cluster policy used to create the cluster if applicable.
	PolicyId *string
	// Whether to enable LUKS on cluster VMs' local disks
	EnableLocalDiskEncryption *bool
	// The optional ID of the instance pool for the driver of the cluster belongs.
	// The pool cluster uses the instance pool with id (instance_pool_id) if the
	// driver pool is not assigned.
	DriverInstancePoolId *string
	WorkloadType         *WorkloadType
	DataSecurityMode     DataSecurityMode
	// Determines the cluster's runtime engine, either standard or Photon.
	//
	// This field is not compatible with legacy `spark_version` values that contain
	// `-photon-`. Remove `-photon-` from the `spark_version` and set
	// `runtime_engine` to `PHOTON`.
	//
	// If left unspecified, the runtime engine defaults to standard unless the
	// spark_version contains -photon-, in which case Photon will be used.
	RuntimeEngine RuntimeEngine
	Kind          ComputeKind
	// This field can only be used when `kind = CLASSIC_PREVIEW`.
	//
	// `effective_spark_version` is determined by `spark_version` (DBR release),
	// this field `use_ml_runtime`, and whether `node_type_id` is gpu node or not.
	UseMlRuntime *bool
	// This field can only be used when `kind = CLASSIC_PREVIEW`.
	//
	// When set to true, <Databricks> will automatically set single node related
	// `custom_tags`, `spark_conf`, and `num_workers`
	IsSingleNode *bool
	// If set, what the configurable throughput (in Mb/s) for the remote disk is.
	// Currently only supported for GCP HYPERDISK_BALANCED disks.
	RemoteDiskThroughput *int
	// If set, what the total initial volume size (in GB) of the remote disks should
	// be. Supported for GCP.
	TotalInitialRemoteDiskSize *int
	// Controls dependency configuration for the cluster.
	DependencyMode DependencyMode
}

Common set of attributes set during cluster creation. These attributes cannot be changed over the lifetime of a cluster..

type ClusterCompliance

type ClusterCompliance struct {
	// Canonical unique identifier for a cluster.
	ClusterId *string
	// Whether this cluster is in compliance with the latest version of its policy.
	IsCompliant *bool
	// An object containing key-value mappings representing the first 200 policy
	// validation errors. The keys indicate the path where the policy validation
	// error is occurring. The values indicate an error message describing the
	// policy validation error.
	Violations map[string]string
	// Information about the pending enforcement for the cluster. Only present if a
	// pending enforcement is scheduled for the cluster.
	PendingEnforcement *PendingEnforcement
}

type ClusterEvent

type ClusterEvent struct {
	ClusterId *string
	// The timestamp when the event occurred, stored as the number of milliseconds
	// since the Unix epoch. If not provided, this will be assigned by the Timeline
	// service.
	Timestamp             *int64
	Type                  ClusterEventType_ClusterEventType
	Details               *EventDetails
	DataPlaneEventDetails *DataPlaneEventDetails
}

type ClusterEventType

type ClusterEventType struct {
}

type ClusterEventType_ClusterEventType

type ClusterEventType_ClusterEventType string
const (
	ClusterEventType_ClusterEventType_Unspecified ClusterEventType_ClusterEventType = ""
	// Indicates that the cluster is being created by someone.
	ClusterEventType_ClusterEventType_Creating ClusterEventType_ClusterEventType = "CREATING"
	// Indicates that the cluster is being started by someone.
	ClusterEventType_ClusterEventType_Starting ClusterEventType_ClusterEventType = "STARTING"
	// Indicates that the cluster is being started by someone.
	ClusterEventType_ClusterEventType_Restarting ClusterEventType_ClusterEventType = "RESTARTING"
	// Indicates that the cluster is being terminating.
	ClusterEventType_ClusterEventType_Terminating ClusterEventType_ClusterEventType = "TERMINATING"
	// Indicates that the cluster has been edited by someone.
	ClusterEventType_ClusterEventType_Edited ClusterEventType_ClusterEventType = "EDITED"
	// Indicates the cluster finished creating, starting, or restarting. Includes
	// the number of nodes in the cluster, and a failure reason if some nodes could
	// not be acquired.
	ClusterEventType_ClusterEventType_Running ClusterEventType_ClusterEventType = "RUNNING"
	// Indicates a change in the target size of the cluster (upsize or downsize).
	ClusterEventType_ClusterEventType_Resizing ClusterEventType_ClusterEventType = "RESIZING"
	// Indicates that some nodes were lost from the cluster.
	ClusterEventType_ClusterEventType_NodesLost ClusterEventType_ClusterEventType = "NODES_LOST"
	// Indicates that nodes finished to be added to the cluster. Includes the number
	// of nodes in the cluster, and a failure reason if some nodes could not be
	// acquired.
	ClusterEventType_ClusterEventType_UpsizeCompleted ClusterEventType_ClusterEventType = "UPSIZE_COMPLETED"
	// Init Scripts V2 have started executing. Includes the list of Global and
	// Cluster scoped init scripts that are about to be fetched & executed.
	ClusterEventType_ClusterEventType_InitScriptsStarted ClusterEventType_ClusterEventType = "INIT_SCRIPTS_STARTED"
	// Init Scripts V2 have finished executing.
	ClusterEventType_ClusterEventType_InitScriptsFinished ClusterEventType_ClusterEventType = "INIT_SCRIPTS_FINISHED"
	// Indicates that a disk is low on space, but adding disks would put it over the
	// max capacity
	ClusterEventType_ClusterEventType_DidNotExpandDisk ClusterEventType_ClusterEventType = "DID_NOT_EXPAND_DISK"
	// Indicates that a disk is low on space, and we did expand its disks.
	ClusterEventType_ClusterEventType_ExpandedDisk ClusterEventType_ClusterEventType = "EXPANDED_DISK"
	// Indicates we failed to expand the disk space
	ClusterEventType_ClusterEventType_FailedToExpandDisk ClusterEventType_ClusterEventType = "FAILED_TO_EXPAND_DISK"
	// Indicates that driver is up and running
	ClusterEventType_ClusterEventType_DriverHealthy ClusterEventType_ClusterEventType = "DRIVER_HEALTHY"
	// Indicates that driver is overloaded(one case is when it is GCing)
	ClusterEventType_ClusterEventType_DriverNotResponding ClusterEventType_ClusterEventType = "DRIVER_NOT_RESPONDING"
	// Indicates that the container that hosts driver and chauffeur is unavailable
	ClusterEventType_ClusterEventType_DriverUnavailable ClusterEventType_ClusterEventType = "DRIVER_UNAVAILABLE"
	// Indicates that spark context is null or there was a spark exception thrown
	// from driver
	ClusterEventType_ClusterEventType_SparkException ClusterEventType_ClusterEventType = "SPARK_EXCEPTION"
	// Indicates that driver is up but metastore is down
	ClusterEventType_ClusterEventType_MetastoreDown ClusterEventType_ClusterEventType = "METASTORE_DOWN"
	// Indicates that driver is up but dbfs is down
	ClusterEventType_ClusterEventType_DbfsDown ClusterEventType_ClusterEventType = "DBFS_DOWN"
	// Autoscaling stat, including wasted instance minutes, reported
	ClusterEventType_ClusterEventType_AutoscalingStatsReport ClusterEventType_ClusterEventType = "AUTOSCALING_STATS_REPORT"
	// Indicates that a node has been blacklisted.
	ClusterEventType_ClusterEventType_NodeBlacklisted ClusterEventType_ClusterEventType = "NODE_BLACKLISTED"
	// Indicates the cluster was pinned.
	ClusterEventType_ClusterEventType_Pinned ClusterEventType_ClusterEventType = "PINNED"
	// Indicates the cluster was unpinned.
	ClusterEventType_ClusterEventType_Unpinned ClusterEventType_ClusterEventType = "UNPINNED"
	// Indicates that a node has been decommissioned because of exclusion
	ClusterEventType_ClusterEventType_NodeExcludedDecommissioned ClusterEventType_ClusterEventType = "NODE_EXCLUDED_DECOMMISSIONED"
	// Indicates add node failure
	ClusterEventType_ClusterEventType_AddNodesFailed ClusterEventType_ClusterEventType = "ADD_NODES_FAILED"
	// Indicates the cluster autoscaling has been retried several times. The waiting
	// time has reached the max waiting time.
	ClusterEventType_ClusterEventType_AutoscalingBackoff ClusterEventType_ClusterEventType = "AUTOSCALING_BACKOFF"
	// Indicates that the cluster is going to be restarted because of the automatic
	// worker image update
	ClusterEventType_ClusterEventType_AutomaticClusterUpdate ClusterEventType_ClusterEventType = "AUTOMATIC_CLUSTER_UPDATE"
	// Indicates there was a failure during autoscaling of a cluster. These are
	// failures that we may want to surface to the customer such as: -
	// DatabricksServiceException(REQUEST_LIMIT_EXCEEDED)
	ClusterEventType_ClusterEventType_AutoscalingFailed ClusterEventType_ClusterEventType = "AUTOSCALING_FAILED"
	// Indicates that the cluster was migrated for the GCP CMv1 migration. The
	// cluster may be migrated from GKE architecture to GCE or rolled back from GCE
	// to GKE.
	ClusterEventType_ClusterEventType_ClusterMigrated ClusterEventType_ClusterEventType = "CLUSTER_MIGRATED"
	// Indicates that decommission started.
	ClusterEventType_ClusterEventType_DecommissionStarted ClusterEventType_ClusterEventType = "DECOMMISSION_STARTED"
	// Indicates that decommission ended.
	ClusterEventType_ClusterEventType_DecommissionEnded ClusterEventType_ClusterEventType = "DECOMMISSION_ENDED"
	// Indicates that a deferred policy enforcement was scheduled.
	ClusterEventType_ClusterEventType_DeferredPolicyEnforcementScheduled ClusterEventType_ClusterEventType = "DEFERRED_POLICY_ENFORCEMENT_SCHEDULED"
	// Indicates that a deferred policy enforcement failed.
	ClusterEventType_ClusterEventType_DeferredPolicyEnforcementFailed ClusterEventType_ClusterEventType = "DEFERRED_POLICY_ENFORCEMENT_FAILED"
	// Indicates that the configured UC volume for log delivery is misconfigured
	// (permission does not exist or volume is invalid)
	ClusterEventType_ClusterEventType_UcVolumeMisconfigured ClusterEventType_ClusterEventType = "UC_VOLUME_MISCONFIGURED"
)

type ClusterInfo

type ClusterInfo struct {
	// Canonical identifier for the cluster. This id is retained during cluster
	// restarts and resizes, while each new cluster has a globally unique id.
	ClusterId *string
	// Creator user name. The field won't be included in the response if the user
	// has already been deleted.
	CreatorUserName *string
	// Current state of the cluster.
	State ClusterState_ClusterState
	// A message associated with the most recent state transition (e.g., the reason
	// why the cluster entered a `TERMINATED` state).
	StateMessage *string
	// Total amount of cluster memory, in megabytes
	ClusterMemoryMb *int64
	// Number of CPU cores available for this cluster. Note that this can be
	// fractional, e.g. 7.5 cores, since certain node types are configured to share
	// cores between Spark nodes on the same instance.
	ClusterCores *float32
	// Tags that are added by <Databricks> regardless of any `custom_tags`,
	// including:
	//
	// - Vendor: <Databricks>
	//
	// - Creator: <username_of_creator>
	//
	// - ClusterName: <name_of_cluster>
	//
	// - ClusterId: <id_of_cluster>
	//
	// - Name: <<Databricks> internal use>
	DefaultTags map[string]string
	// Cluster log delivery status.
	ClusterLogStatus *LogSyncStatus
	// Information about why the cluster was terminated. This field only appears
	// when the cluster is in a `TERMINATING` or `TERMINATED` state.
	TerminationReason *TerminationReason
	// The spec contains a snapshot of the latest user specified settings that were
	// used to create/edit the cluster. Note: not included in the response of the
	// ListClusters API.
	Spec *ClusterInfo_ComputeSpec
	// Node on which the Spark driver resides. The driver node contains the Spark
	// master and the <Databricks> application that manages the per-notebook Spark
	// REPLs.
	Driver *SparkInfo_SparkNode
	// Nodes on which the Spark executors reside.
	Executors []SparkInfo_SparkNode
	// A canonical SparkContext identifier. This value *does* change when the Spark
	// driver restarts. The pair `(cluster_id, spark_context_id)` is a globally
	// unique identifier over all Spark contexts.
	SparkContextId *int64
	// Port on which Spark JDBC server is listening, in the driver nod. No service
	// will be listeningon on this port in executor nodes.
	JdbcPort *int
	// Cluster name requested by the user. This doesn't have to be unique. If not
	// specified at creation, the cluster name will be an empty string. For job
	// clusters, the cluster name is automatically set based on the job and job run
	// IDs.
	ClusterName *string
	// The Spark version of the cluster, e.g. `3.3.x-scala2.11`. A list of available
	// Spark versions can be retrieved by using the [clusters/sparkVersions] API
	// call.
	//
	// [clusters/sparkVersions]: https://docs.databricks.com/api/workspace/clusters/sparkversions
	SparkVersion *string
	// An object containing a set of optional, user-specified Spark configuration
	// key-value pairs. Users can also pass in a string of extra JVM options to the
	// driver and the executors via `spark.driver.extraJavaOptions` and
	// `spark.executor.extraJavaOptions` respectively.
	SparkConf map[string]string
	// Attributes related to clusters running on Amazon Web Services. If not
	// specified at cluster creation, a set of default values will be used.
	AwsAttributes *AwsAttributes
	// Attributes related to clusters running on Microsoft Azure. If not specified
	// at cluster creation, a set of default values will be used.
	AzureAttributes *AzureAttributes
	// Attributes related to clusters running on Google Cloud Platform. If not
	// specified at cluster creation, a set of default values will be used.
	GcpAttributes *GcpAttributes
	// This field encodes, through a single value, the resources available to each
	// of the Spark nodes in this cluster. For example, the Spark nodes can be
	// provisioned and optimized for memory or compute intensive workloads. A list
	// of available node types can be retrieved by using the
	// [clusters/listNodeTypes] API call.
	//
	// [clusters/listNodeTypes]: https://docs.databricks.com/api/workspace/clusters/listnodetypes
	NodeTypeId *string
	// The node type of the Spark driver. Note that this field is optional; if
	// unset, the driver node type will be set as the same value as `node_type_id`
	// defined above.
	//
	// This field, along with node_type_id, should not be set if
	// virtual_cluster_size is set. If both driver_node_type_id, node_type_id, and
	// virtual_cluster_size are specified, driver_node_type_id and node_type_id take
	// precedence.
	DriverNodeTypeId *string
	// Flexible node type configuration for worker nodes.
	WorkerNodeTypeFlexibility *NodeTypeFlexibility
	// Flexible node type configuration for the driver node.
	DriverNodeTypeFlexibility *NodeTypeFlexibility
	// SSH public key contents that will be added to each Spark node in this
	// cluster. The corresponding private keys can be used to login with the user
	// name `ubuntu` on port `2200`. Up to 10 keys can be specified.
	SshPublicKeys []string
	// Additional tags for cluster resources. <Databricks> will tag all cluster
	// resources (e.g., AWS instances and EBS volumes) with these tags in addition
	// to `default_tags`. Notes:
	//
	// - Currently, <Databricks> allows at most 45 custom tags
	//
	// - Clusters can only reuse cloud resources if the resources' tags are a subset
	// of the cluster tags
	CustomTags map[string]string
	// The configuration for delivering spark logs to a long-term storage
	// destination. Three kinds of destinations (DBFS, S3 and Unity Catalog volumes)
	// are supported. Only one destination can be specified for one cluster. If the
	// conf is given, the logs will be delivered to the destination every `5 mins`.
	// The destination of driver logs is `$destination/$clusterId/driver`, while the
	// destination of executor logs is `$destination/$clusterId/executor`.
	ClusterLogConf *ClusterLogConf
	// An object containing a set of optional, user-specified environment variable
	// key-value pairs. Please note that key-value pair of the form (X,Y) will be
	// exported as is (i.e., `export X='Y'`) while launching the driver and workers.
	//
	// In order to specify an additional set of `SPARK_DAEMON_JAVA_OPTS`, we
	// recommend appending them to `$SPARK_DAEMON_JAVA_OPTS` as shown in the example
	// below. This ensures that all default databricks managed environmental
	// variables are included as well.
	//
	// Example Spark environment variables: `{"SPARK_WORKER_MEMORY": "28000m",
	// "SPARK_LOCAL_DIRS": "/local_disk0"}` or `{"SPARK_DAEMON_JAVA_OPTS":
	// "$SPARK_DAEMON_JAVA_OPTS -Dspark.shuffle.service.enabled=true"}`
	SparkEnvVars map[string]string
	// Automatically terminates the cluster after it is inactive for this time in
	// minutes. If not set, this cluster will not be automatically terminated. If
	// specified, the threshold must be between 10 and 10000 minutes. Users can also
	// set this value to 0 to explicitly disable automatic termination.
	AutoterminationMinutes *int
	// Autoscaling Local Storage: when enabled, this cluster will dynamically
	// acquire additional disk space when its Spark workers are running low on disk
	// space.
	EnableElasticDisk *bool
	// The configuration for storing init scripts. Any number of destinations can be
	// specified. The scripts are executed sequentially in the order provided. If
	// `cluster_log_conf` is specified, init script logs are sent to
	// `<destination>/<cluster-ID>/init_scripts`.
	InitScripts []InitScriptInfo
	// Custom docker image BYOC
	DockerImage *DockerImage
	// The optional ID of the instance pool to which the cluster belongs.
	InstancePoolId *string
	// Single user name if data_security_mode is `SINGLE_USER`
	SingleUserName *string
	// The ID of the cluster policy used to create the cluster if applicable.
	PolicyId *string
	// Whether to enable LUKS on cluster VMs' local disks
	EnableLocalDiskEncryption *bool
	// The optional ID of the instance pool for the driver of the cluster belongs.
	// The pool cluster uses the instance pool with id (instance_pool_id) if the
	// driver pool is not assigned.
	DriverInstancePoolId *string
	WorkloadType         *WorkloadType
	DataSecurityMode     DataSecurityMode
	// Determines the cluster's runtime engine, either standard or Photon.
	//
	// This field is not compatible with legacy `spark_version` values that contain
	// `-photon-`. Remove `-photon-` from the `spark_version` and set
	// `runtime_engine` to `PHOTON`.
	//
	// If left unspecified, the runtime engine defaults to standard unless the
	// spark_version contains -photon-, in which case Photon will be used.
	RuntimeEngine RuntimeEngine
	Kind          ComputeKind
	// This field can only be used when `kind = CLASSIC_PREVIEW`.
	//
	// `effective_spark_version` is determined by `spark_version` (DBR release),
	// this field `use_ml_runtime`, and whether `node_type_id` is gpu node or not.
	UseMlRuntime *bool
	// This field can only be used when `kind = CLASSIC_PREVIEW`.
	//
	// When set to true, <Databricks> will automatically set single node related
	// `custom_tags`, `spark_conf`, and `num_workers`
	IsSingleNode *bool
	// If set, what the configurable throughput (in Mb/s) for the remote disk is.
	// Currently only supported for GCP HYPERDISK_BALANCED disks.
	RemoteDiskThroughput *int
	// If set, what the total initial volume size (in GB) of the remote disks should
	// be. Supported for GCP.
	TotalInitialRemoteDiskSize *int
	// Controls dependency configuration for the cluster.
	DependencyMode DependencyMode
	// Time (in epoch milliseconds) when the cluster creation request was received
	// (when the cluster entered a `PENDING` state).
	StartTime *int64
	// Time (in epoch milliseconds) when the cluster was terminated, if applicable.
	TerminatedTime *int64
	// Time when the cluster driver last lost its state (due to a restart or driver
	// failure).
	LastStateLossTime *int64
	// the timestamp that the cluster was started/restarted
	LastRestartedTime *int64
	Size              isClusterInfo_Size
}

Describes all of the metadata about a single Spark cluster in <Databricks>..

type ClusterInfo_ComputeSpec

type ClusterInfo_ComputeSpec struct {
	// When set to true, fixed and default values from the policy will be used for
	// fields that are omitted. When set to false, only fixed values from the policy
	// will be applied.
	ApplyPolicyDefaultValues *bool
	// Cluster name requested by the user. This doesn't have to be unique. If not
	// specified at creation, the cluster name will be an empty string. For job
	// clusters, the cluster name is automatically set based on the job and job run
	// IDs.
	ClusterName *string
	// The Spark version of the cluster, e.g. `3.3.x-scala2.11`. A list of available
	// Spark versions can be retrieved by using the [clusters/sparkVersions] API
	// call.
	//
	// [clusters/sparkVersions]: https://docs.databricks.com/api/workspace/clusters/sparkversions
	SparkVersion *string
	// An object containing a set of optional, user-specified Spark configuration
	// key-value pairs. Users can also pass in a string of extra JVM options to the
	// driver and the executors via `spark.driver.extraJavaOptions` and
	// `spark.executor.extraJavaOptions` respectively.
	SparkConf map[string]string
	// Attributes related to clusters running on Amazon Web Services. If not
	// specified at cluster creation, a set of default values will be used.
	AwsAttributes *AwsAttributes
	// Attributes related to clusters running on Microsoft Azure. If not specified
	// at cluster creation, a set of default values will be used.
	AzureAttributes *AzureAttributes
	// Attributes related to clusters running on Google Cloud Platform. If not
	// specified at cluster creation, a set of default values will be used.
	GcpAttributes *GcpAttributes
	// This field encodes, through a single value, the resources available to each
	// of the Spark nodes in this cluster. For example, the Spark nodes can be
	// provisioned and optimized for memory or compute intensive workloads. A list
	// of available node types can be retrieved by using the
	// [clusters/listNodeTypes] API call.
	//
	// [clusters/listNodeTypes]: https://docs.databricks.com/api/workspace/clusters/listnodetypes
	NodeTypeId *string
	// The node type of the Spark driver. Note that this field is optional; if
	// unset, the driver node type will be set as the same value as `node_type_id`
	// defined above.
	//
	// This field, along with node_type_id, should not be set if
	// virtual_cluster_size is set. If both driver_node_type_id, node_type_id, and
	// virtual_cluster_size are specified, driver_node_type_id and node_type_id take
	// precedence.
	DriverNodeTypeId *string
	// Flexible node type configuration for worker nodes.
	WorkerNodeTypeFlexibility *NodeTypeFlexibility
	// Flexible node type configuration for the driver node.
	DriverNodeTypeFlexibility *NodeTypeFlexibility
	// SSH public key contents that will be added to each Spark node in this
	// cluster. The corresponding private keys can be used to login with the user
	// name `ubuntu` on port `2200`. Up to 10 keys can be specified.
	SshPublicKeys []string
	// Additional tags for cluster resources. <Databricks> will tag all cluster
	// resources (e.g., AWS instances and EBS volumes) with these tags in addition
	// to `default_tags`. Notes:
	//
	// - Currently, <Databricks> allows at most 45 custom tags
	//
	// - Clusters can only reuse cloud resources if the resources' tags are a subset
	// of the cluster tags
	CustomTags map[string]string
	// The configuration for delivering spark logs to a long-term storage
	// destination. Three kinds of destinations (DBFS, S3 and Unity Catalog volumes)
	// are supported. Only one destination can be specified for one cluster. If the
	// conf is given, the logs will be delivered to the destination every `5 mins`.
	// The destination of driver logs is `$destination/$clusterId/driver`, while the
	// destination of executor logs is `$destination/$clusterId/executor`.
	ClusterLogConf *ClusterLogConf
	// An object containing a set of optional, user-specified environment variable
	// key-value pairs. Please note that key-value pair of the form (X,Y) will be
	// exported as is (i.e., `export X='Y'`) while launching the driver and workers.
	//
	// In order to specify an additional set of `SPARK_DAEMON_JAVA_OPTS`, we
	// recommend appending them to `$SPARK_DAEMON_JAVA_OPTS` as shown in the example
	// below. This ensures that all default databricks managed environmental
	// variables are included as well.
	//
	// Example Spark environment variables: `{"SPARK_WORKER_MEMORY": "28000m",
	// "SPARK_LOCAL_DIRS": "/local_disk0"}` or `{"SPARK_DAEMON_JAVA_OPTS":
	// "$SPARK_DAEMON_JAVA_OPTS -Dspark.shuffle.service.enabled=true"}`
	SparkEnvVars map[string]string
	// Automatically terminates the cluster after it is inactive for this time in
	// minutes. If not set, this cluster will not be automatically terminated. If
	// specified, the threshold must be between 10 and 10000 minutes. Users can also
	// set this value to 0 to explicitly disable automatic termination.
	AutoterminationMinutes *int
	// Autoscaling Local Storage: when enabled, this cluster will dynamically
	// acquire additional disk space when its Spark workers are running low on disk
	// space.
	EnableElasticDisk *bool
	// The configuration for storing init scripts. Any number of destinations can be
	// specified. The scripts are executed sequentially in the order provided. If
	// `cluster_log_conf` is specified, init script logs are sent to
	// `<destination>/<cluster-ID>/init_scripts`.
	InitScripts []InitScriptInfo
	// Custom docker image BYOC
	DockerImage *DockerImage
	// The optional ID of the instance pool to which the cluster belongs.
	InstancePoolId *string
	// Single user name if data_security_mode is `SINGLE_USER`
	SingleUserName *string
	// The ID of the cluster policy used to create the cluster if applicable.
	PolicyId *string
	// Whether to enable LUKS on cluster VMs' local disks
	EnableLocalDiskEncryption *bool
	// The optional ID of the instance pool for the driver of the cluster belongs.
	// The pool cluster uses the instance pool with id (instance_pool_id) if the
	// driver pool is not assigned.
	DriverInstancePoolId *string
	WorkloadType         *WorkloadType
	DataSecurityMode     DataSecurityMode
	// Determines the cluster's runtime engine, either standard or Photon.
	//
	// This field is not compatible with legacy `spark_version` values that contain
	// `-photon-`. Remove `-photon-` from the `spark_version` and set
	// `runtime_engine` to `PHOTON`.
	//
	// If left unspecified, the runtime engine defaults to standard unless the
	// spark_version contains -photon-, in which case Photon will be used.
	RuntimeEngine RuntimeEngine
	Kind          ComputeKind
	// This field can only be used when `kind = CLASSIC_PREVIEW`.
	//
	// `effective_spark_version` is determined by `spark_version` (DBR release),
	// this field `use_ml_runtime`, and whether `node_type_id` is gpu node or not.
	UseMlRuntime *bool
	// This field can only be used when `kind = CLASSIC_PREVIEW`.
	//
	// When set to true, <Databricks> will automatically set single node related
	// `custom_tags`, `spark_conf`, and `num_workers`
	IsSingleNode *bool
	// If set, what the configurable throughput (in Mb/s) for the remote disk is.
	// Currently only supported for GCP HYPERDISK_BALANCED disks.
	RemoteDiskThroughput *int
	// If set, what the total initial volume size (in GB) of the remote disks should
	// be. Supported for GCP.
	TotalInitialRemoteDiskSize *int
	// Controls dependency configuration for the cluster.
	DependencyMode DependencyMode
	Size           isClusterInfo_ComputeSpec_Size
}

Contains a snapshot of the latest user specified settings that were used to create/edit the cluster..

type ClusterInfo_ComputeSpec_Size_Autoscale

type ClusterInfo_ComputeSpec_Size_Autoscale struct {
	Autoscale AutoScale
}

ClusterInfo_ComputeSpec_Size_Autoscale selects Autoscale for ClusterInfo_ComputeSpec.Size. Parameters needed in order to automatically scale clusters up and down based on load. Note: autoscaling works best with DB runtime versions 3.0 or later.

type ClusterInfo_ComputeSpec_Size_NumWorkers

type ClusterInfo_ComputeSpec_Size_NumWorkers struct {
	NumWorkers int
}

ClusterInfo_ComputeSpec_Size_NumWorkers selects NumWorkers for ClusterInfo_ComputeSpec.Size. Number of worker nodes that this cluster should have. A cluster has one Spark Driver and `num_workers` Executors for a total of `num_workers` + 1 Spark nodes.

Note: When reading the properties of a cluster, this field reflects the desired number of workers rather than the actual current number of workers. For instance, if a cluster is resized from 5 to 10 workers, this field will immediately be updated to reflect the target size of 10 workers, whereas the workers listed in `spark_info` will gradually increase from 5 to 10 as the new nodes are provisioned.

type ClusterInfo_Size_Autoscale

type ClusterInfo_Size_Autoscale struct {
	Autoscale AutoScale
}

ClusterInfo_Size_Autoscale selects Autoscale for ClusterInfo.Size. Parameters needed in order to automatically scale clusters up and down based on load. Note: autoscaling works best with DB runtime versions 3.0 or later.

type ClusterInfo_Size_NumWorkers

type ClusterInfo_Size_NumWorkers struct {
	NumWorkers int
}

ClusterInfo_Size_NumWorkers selects NumWorkers for ClusterInfo.Size. Number of worker nodes that this cluster should have. A cluster has one Spark Driver and `num_workers` Executors for a total of `num_workers` + 1 Spark nodes.

Note: When reading the properties of a cluster, this field reflects the desired number of workers rather than the actual current number of workers. For instance, if a cluster is resized from 5 to 10 workers, this field will immediately be updated to reflect the target size of 10 workers, whereas the workers listed in `spark_info` will gradually increase from 5 to 10 as the new nodes are provisioned.

type ClusterLogConf

type ClusterLogConf struct {
	StorageInfo isClusterLogConf_StorageInfo
	// contains filtered or unexported fields
}

Cluster log delivery config.

type ClusterLogConf_StorageInfo_Dbfs

type ClusterLogConf_StorageInfo_Dbfs struct {
	Dbfs DbfsStorageInfo `fieldmask:"dbfs"`
}

ClusterLogConf_StorageInfo_Dbfs selects Dbfs for ClusterLogConf.StorageInfo. destination needs to be provided. e.g. `{ "dbfs" : { "destination" : "dbfs:/home/cluster_log" } }`

type ClusterLogConf_StorageInfo_S3

type ClusterLogConf_StorageInfo_S3 struct {
	S3 S3StorageInfo `fieldmask:"s3"`
}

ClusterLogConf_StorageInfo_S3 selects S3 for ClusterLogConf.StorageInfo. destination and either the region or endpoint need to be provided. e.g. `{ "s3": { "destination" : "s3://cluster_log_bucket/prefix", "region" : "us-west-2" } }` Cluster iam role is used to access s3, please make sure the cluster iam role in `instance_profile_arn` has permission to write data to the s3 destination.

type ClusterLogConf_StorageInfo_Volumes

type ClusterLogConf_StorageInfo_Volumes struct {
	Volumes VolumesStorageInfo `fieldmask:"volumes"`
}

ClusterLogConf_StorageInfo_Volumes selects Volumes for ClusterLogConf.StorageInfo. destination needs to be provided, e.g. `{ "volumes": { "destination": "/Volumes/catalog/schema/volume/cluster_log" } }`

type ClusterSize

type ClusterSize struct {
	Size isClusterSize_Size
}

type ClusterSize_Size_Autoscale

type ClusterSize_Size_Autoscale struct {
	Autoscale AutoScale
}

ClusterSize_Size_Autoscale selects Autoscale for ClusterSize.Size. Parameters needed in order to automatically scale clusters up and down based on load. Note: autoscaling works best with DB runtime versions 3.0 or later.

type ClusterSize_Size_NumWorkers

type ClusterSize_Size_NumWorkers struct {
	NumWorkers int
}

ClusterSize_Size_NumWorkers selects NumWorkers for ClusterSize.Size. Number of worker nodes that this cluster should have. A cluster has one Spark Driver and `num_workers` Executors for a total of `num_workers` + 1 Spark nodes.

Note: When reading the properties of a cluster, this field reflects the desired number of workers rather than the actual current number of workers. For instance, if a cluster is resized from 5 to 10 workers, this field will immediately be updated to reflect the target size of 10 workers, whereas the workers listed in `spark_info` will gradually increase from 5 to 10 as the new nodes are provisioned.

type ClusterState

type ClusterState struct {
}

type ClusterState_ClusterState

type ClusterState_ClusterState string

The state of a Cluster. The current allowable state transitions are as follows:

- `PENDING` -> `RUNNING` - `PENDING` -> `TERMINATING` - `RUNNING` -> `RESIZING` - `RUNNING` -> `RESTARTING` - `RUNNING` -> `TERMINATING` - `RESTARTING` -> `RUNNING` - `RESTARTING` -> `TERMINATING` - `RESIZING` -> `RUNNING` - `RESIZING` -> `TERMINATING` - `TERMINATING` -> `TERMINATED`

const (
	ClusterState_ClusterState_Unspecified ClusterState_ClusterState = ""
	// Indicates a cluster that is in progress of being created.
	ClusterState_ClusterState_Pending ClusterState_ClusterState = "PENDING"
	// Indicates a cluster that has been started and is ready for use.
	ClusterState_ClusterState_Running ClusterState_ClusterState = "RUNNING"
	// Indicates that a cluster is in the process of restarting.
	ClusterState_ClusterState_Restarting ClusterState_ClusterState = "RESTARTING"
	// Indicates that a cluster is in the process of adding or removing nodes.
	ClusterState_ClusterState_Resizing ClusterState_ClusterState = "RESIZING"
	// Indicates that a cluster is in the process of being destroyed.
	ClusterState_ClusterState_Terminating ClusterState_ClusterState = "TERMINATING"
	// Indicates a cluster which has been successfully destroyed.
	ClusterState_ClusterState_Terminated ClusterState_ClusterState = "TERMINATED"
	// This state is not used anymore. It was used to indicate a cluster which
	// failed to be created. Terminating and Terminated are used instead.
	ClusterState_ClusterState_Error ClusterState_ClusterState = "ERROR"
	// Indicates a cluster which is an unknown state. A cluster should never be in
	// this state.
	ClusterState_ClusterState_Unknown ClusterState_ClusterState = "UNKNOWN"
)

type ComputeKind

type ComputeKind string

The kind of compute described by this compute specification.

Depending on `kind`, different validations and default values will be applied.

Clusters with `kind = CLASSIC_PREVIEW` support the following fields, whereas clusters with no specified `kind` do not. * [is_single_node](/api/workspace/clusters/create#is_single_node) * [use_ml_runtime](/api/workspace/clusters/create#use_ml_runtime)

By using the simple form, your clusters are automatically using `kind = CLASSIC_PREVIEW`.

const (
	ComputeKind_Unspecified    ComputeKind = ""
	ComputeKind_ClassicPreview ComputeKind = "CLASSIC_PREVIEW"
)

type ConfidentialComputeType

type ConfidentialComputeType string

Confidential computing technology for GCP instances. Aligns with gcloud's --confidential-compute-type flag and the REST API's confidentialInstanceConfig.confidentialInstanceType field. See: https://cloud.google.com/confidential-computing/confidential-vm/docs/create-a-confidential-vm-instance

const (
	ConfidentialComputeType_Unspecified                 ConfidentialComputeType = ""
	ConfidentialComputeType_ConfidentialComputeTypeNone ConfidentialComputeType = "CONFIDENTIAL_COMPUTE_TYPE_NONE"
	ConfidentialComputeType_SevSnp                      ConfidentialComputeType = "SEV_SNP"
)

type CreateClusterRequest

type CreateClusterRequest struct {
	// When set to true, fixed and default values from the policy will be used for
	// fields that are omitted. When set to false, only fixed values from the policy
	// will be applied.
	ApplyPolicyDefaultValues *bool
	// When specified, this clones libraries from a source cluster during the
	// creation of a new cluster.
	CloneFrom *CloneCluster
	Size      isCreateClusterRequest_Size
	// Cluster name requested by the user. This doesn't have to be unique. If not
	// specified at creation, the cluster name will be an empty string. For job
	// clusters, the cluster name is automatically set based on the job and job run
	// IDs.
	ClusterName *string
	// The Spark version of the cluster, e.g. `3.3.x-scala2.11`. A list of available
	// Spark versions can be retrieved by using the [clusters/sparkVersions] API
	// call.
	//
	// [clusters/sparkVersions]: https://docs.databricks.com/api/workspace/clusters/sparkversions
	SparkVersion *string
	// An object containing a set of optional, user-specified Spark configuration
	// key-value pairs. Users can also pass in a string of extra JVM options to the
	// driver and the executors via `spark.driver.extraJavaOptions` and
	// `spark.executor.extraJavaOptions` respectively.
	SparkConf map[string]string
	// Attributes related to clusters running on Amazon Web Services. If not
	// specified at cluster creation, a set of default values will be used.
	AwsAttributes *AwsAttributes
	// Attributes related to clusters running on Microsoft Azure. If not specified
	// at cluster creation, a set of default values will be used.
	AzureAttributes *AzureAttributes
	// Attributes related to clusters running on Google Cloud Platform. If not
	// specified at cluster creation, a set of default values will be used.
	GcpAttributes *GcpAttributes
	// This field encodes, through a single value, the resources available to each
	// of the Spark nodes in this cluster. For example, the Spark nodes can be
	// provisioned and optimized for memory or compute intensive workloads. A list
	// of available node types can be retrieved by using the
	// [clusters/listNodeTypes] API call.
	//
	// [clusters/listNodeTypes]: https://docs.databricks.com/api/workspace/clusters/listnodetypes
	NodeTypeId *string
	// The node type of the Spark driver. Note that this field is optional; if
	// unset, the driver node type will be set as the same value as `node_type_id`
	// defined above.
	//
	// This field, along with node_type_id, should not be set if
	// virtual_cluster_size is set. If both driver_node_type_id, node_type_id, and
	// virtual_cluster_size are specified, driver_node_type_id and node_type_id take
	// precedence.
	DriverNodeTypeId *string
	// Flexible node type configuration for worker nodes.
	WorkerNodeTypeFlexibility *NodeTypeFlexibility
	// Flexible node type configuration for the driver node.
	DriverNodeTypeFlexibility *NodeTypeFlexibility
	// SSH public key contents that will be added to each Spark node in this
	// cluster. The corresponding private keys can be used to login with the user
	// name `ubuntu` on port `2200`. Up to 10 keys can be specified.
	SshPublicKeys []string
	// Additional tags for cluster resources. <Databricks> will tag all cluster
	// resources (e.g., AWS instances and EBS volumes) with these tags in addition
	// to `default_tags`. Notes:
	//
	// - Currently, <Databricks> allows at most 45 custom tags
	//
	// - Clusters can only reuse cloud resources if the resources' tags are a subset
	// of the cluster tags
	CustomTags map[string]string
	// The configuration for delivering spark logs to a long-term storage
	// destination. Three kinds of destinations (DBFS, S3 and Unity Catalog volumes)
	// are supported. Only one destination can be specified for one cluster. If the
	// conf is given, the logs will be delivered to the destination every `5 mins`.
	// The destination of driver logs is `$destination/$clusterId/driver`, while the
	// destination of executor logs is `$destination/$clusterId/executor`.
	ClusterLogConf *ClusterLogConf
	// An object containing a set of optional, user-specified environment variable
	// key-value pairs. Please note that key-value pair of the form (X,Y) will be
	// exported as is (i.e., `export X='Y'`) while launching the driver and workers.
	//
	// In order to specify an additional set of `SPARK_DAEMON_JAVA_OPTS`, we
	// recommend appending them to `$SPARK_DAEMON_JAVA_OPTS` as shown in the example
	// below. This ensures that all default databricks managed environmental
	// variables are included as well.
	//
	// Example Spark environment variables: `{"SPARK_WORKER_MEMORY": "28000m",
	// "SPARK_LOCAL_DIRS": "/local_disk0"}` or `{"SPARK_DAEMON_JAVA_OPTS":
	// "$SPARK_DAEMON_JAVA_OPTS -Dspark.shuffle.service.enabled=true"}`
	SparkEnvVars map[string]string
	// Automatically terminates the cluster after it is inactive for this time in
	// minutes. If not set, this cluster will not be automatically terminated. If
	// specified, the threshold must be between 10 and 10000 minutes. Users can also
	// set this value to 0 to explicitly disable automatic termination.
	AutoterminationMinutes *int
	// Autoscaling Local Storage: when enabled, this cluster will dynamically
	// acquire additional disk space when its Spark workers are running low on disk
	// space.
	EnableElasticDisk *bool
	// The configuration for storing init scripts. Any number of destinations can be
	// specified. The scripts are executed sequentially in the order provided. If
	// `cluster_log_conf` is specified, init script logs are sent to
	// `<destination>/<cluster-ID>/init_scripts`.
	InitScripts []InitScriptInfo
	// Custom docker image BYOC
	DockerImage *DockerImage
	// The optional ID of the instance pool to which the cluster belongs.
	InstancePoolId *string
	// Single user name if data_security_mode is `SINGLE_USER`
	SingleUserName *string
	// The ID of the cluster policy used to create the cluster if applicable.
	PolicyId *string
	// Whether to enable LUKS on cluster VMs' local disks
	EnableLocalDiskEncryption *bool
	// The optional ID of the instance pool for the driver of the cluster belongs.
	// The pool cluster uses the instance pool with id (instance_pool_id) if the
	// driver pool is not assigned.
	DriverInstancePoolId *string
	WorkloadType         *WorkloadType
	DataSecurityMode     DataSecurityMode
	// Determines the cluster's runtime engine, either standard or Photon.
	//
	// This field is not compatible with legacy `spark_version` values that contain
	// `-photon-`. Remove `-photon-` from the `spark_version` and set
	// `runtime_engine` to `PHOTON`.
	//
	// If left unspecified, the runtime engine defaults to standard unless the
	// spark_version contains -photon-, in which case Photon will be used.
	RuntimeEngine RuntimeEngine
	Kind          ComputeKind
	// This field can only be used when `kind = CLASSIC_PREVIEW`.
	//
	// `effective_spark_version` is determined by `spark_version` (DBR release),
	// this field `use_ml_runtime`, and whether `node_type_id` is gpu node or not.
	UseMlRuntime *bool
	// This field can only be used when `kind = CLASSIC_PREVIEW`.
	//
	// When set to true, <Databricks> will automatically set single node related
	// `custom_tags`, `spark_conf`, and `num_workers`
	IsSingleNode *bool
	// If set, what the configurable throughput (in Mb/s) for the remote disk is.
	// Currently only supported for GCP HYPERDISK_BALANCED disks.
	RemoteDiskThroughput *int
	// If set, what the total initial volume size (in GB) of the remote disks should
	// be. Supported for GCP.
	TotalInitialRemoteDiskSize *int
	// Controls dependency configuration for the cluster.
	DependencyMode DependencyMode
}

type CreateClusterRequest_Size_Autoscale

type CreateClusterRequest_Size_Autoscale struct {
	Autoscale AutoScale
}

CreateClusterRequest_Size_Autoscale selects Autoscale for CreateClusterRequest.Size. Parameters needed in order to automatically scale clusters up and down based on load. Note: autoscaling works best with DB runtime versions 3.0 or later.

type CreateClusterRequest_Size_NumWorkers

type CreateClusterRequest_Size_NumWorkers struct {
	NumWorkers int
}

CreateClusterRequest_Size_NumWorkers selects NumWorkers for CreateClusterRequest.Size. Number of worker nodes that this cluster should have. A cluster has one Spark Driver and `num_workers` Executors for a total of `num_workers` + 1 Spark nodes.

Note: When reading the properties of a cluster, this field reflects the desired number of workers rather than the actual current number of workers. For instance, if a cluster is resized from 5 to 10 workers, this field will immediately be updated to reflect the target size of 10 workers, whereas the workers listed in `spark_info` will gradually increase from 5 to 10 as the new nodes are provisioned.

type CreateClusterResponse

type CreateClusterResponse struct {
	ClusterId *string
}

type CreateClusterWaiter

type CreateClusterWaiter struct {
	// contains filtered or unexported fields
}

CreateClusterWaiter tracks the state of the operation started by CreateCluster.

func (*CreateClusterWaiter) Done

func (w *CreateClusterWaiter) Done(ctx context.Context, opts ...call.Option) (bool, error)

Done polls once and reports whether the operation has reached a terminal state.

func (*CreateClusterWaiter) GetClusterId

func (w *CreateClusterWaiter) GetClusterId() string

GetClusterId returns the ClusterId value used to identify the operation.

func (*CreateClusterWaiter) Wait

func (w *CreateClusterWaiter) Wait(ctx context.Context, opts ...lro.Option) (*ClusterInfo, error)

Wait polls until the operation reaches a terminal state.

type DataPlaneClusterEventType

type DataPlaneClusterEventType string
const (
	DataPlaneClusterEventType_Unspecified                DataPlaneClusterEventType = ""
	DataPlaneClusterEventType_NodeBlacklisted            DataPlaneClusterEventType = "NODE_BLACKLISTED"
	DataPlaneClusterEventType_NodeExcludedDecommissioned DataPlaneClusterEventType = "NODE_EXCLUDED_DECOMMISSIONED"
)

type DataPlaneEventDetails

type DataPlaneEventDetails struct {
	EventType        DataPlaneClusterEventType
	Timestamp        *int64
	HostId           *string
	ExecutorFailures *int
}

type DataSecurityMode

type DataSecurityMode string

Data security mode decides what data governance model to use when accessing data from a cluster.

* `DATA_SECURITY_MODE_AUTO`: <Databricks> will choose the most appropriate access mode depending on your compute configuration. * `DATA_SECURITY_MODE_STANDARD`: A secure cluster that can be shared by multiple users. Cluster users are fully isolated so that they cannot see each other’s data and credentials. Most data governance features are supported in this mode. But programming languages and cluster features might be limited. * `DATA_SECURITY_MODE_DEDICATED`: A secure cluster that can only be exclusively used by a single user specified in `single_user_name`. Most programming languages, cluster features and data governance features are available in this mode.

The following modes are legacy aliases for the above modes:

* `USER_ISOLATION`: Legacy alias for `DATA_SECURITY_MODE_STANDARD`. * `SINGLE_USER`: Legacy alias for `DATA_SECURITY_MODE_DEDICATED`.

The following modes are deprecated starting with Databricks Runtime 15.0 and will be removed for future Databricks Runtime versions:

* `LEGACY_TABLE_ACL`: This mode is for users migrating from legacy Table ACL clusters. * `LEGACY_PASSTHROUGH`: This mode is for users migrating from legacy Passthrough on high concurrency clusters. * `LEGACY_SINGLE_USER`: This mode is for users migrating from legacy Passthrough on standard clusters. * `LEGACY_SINGLE_USER_STANDARD`: This mode provides a way that doesn’t have UC nor passthrough enabled.

const (
	DataSecurityMode_Unspecified DataSecurityMode = ""
	// No security isolation for multiple users sharing the cluster. Data governance
	// features are not available in this mode.
	DataSecurityMode_None DataSecurityMode = "NONE"
	// Legacy alias for `DATA_SECURITY_MODE_DEDICATED`.
	DataSecurityMode_SingleUser DataSecurityMode = "SINGLE_USER"
	// Legacy alias for `DATA_SECURITY_MODE_STANDARD`.
	DataSecurityMode_UserIsolation DataSecurityMode = "USER_ISOLATION"
	// This mode is for users migrating from legacy Table ACL clusters.
	DataSecurityMode_LegacyTableAcl DataSecurityMode = "LEGACY_TABLE_ACL"
	// This mode is for users migrating from legacy Passthrough on high concurrency
	// clusters.
	DataSecurityMode_LegacyPassthrough DataSecurityMode = "LEGACY_PASSTHROUGH"
	// This mode is for users migrating from legacy Passthrough on standard
	// clusters.
	DataSecurityMode_LegacySingleUser DataSecurityMode = "LEGACY_SINGLE_USER"
	// This is mode where single user is enforced but no actual security feature
	// enabled.
	DataSecurityMode_LegacySingleUserStandard DataSecurityMode = "LEGACY_SINGLE_USER_STANDARD"
	// A secure cluster that can be shared by multiple users. Cluster users are
	// fully isolated so that they cannot see each other's data and credentials.
	// Most data governance features are supported in this mode. But programming
	// languages and cluster features might be limited.
	DataSecurityMode_DataSecurityModeStandard DataSecurityMode = "DATA_SECURITY_MODE_STANDARD"
	// A secure cluster that can only be exclusively used by a single user specified
	// in `single_user_name`. Most programming languages, cluster features and data
	// governance features are available in this mode.
	DataSecurityMode_DataSecurityModeDedicated DataSecurityMode = "DATA_SECURITY_MODE_DEDICATED"
	// Databricks will choose `DATA_SECURITY_MODE_STANDARD` or
	// `DATA_SECURITY_MODE_DEDICATED` depending on the compute configuration.
	DataSecurityMode_DataSecurityModeAuto DataSecurityMode = "DATA_SECURITY_MODE_AUTO"
)

type DbfsStorageInfo

type DbfsStorageInfo struct {
	// dbfs destination, e.g. `dbfs:/my/path`
	Destination *string `fieldmask:"destination"`
}

A storage location in DBFS.

type DeleteClusterRequest

type DeleteClusterRequest struct {
	// The cluster to be terminated.
	ClusterId *string
}

type DeleteClusterResponse

type DeleteClusterResponse struct {
}

type DeleteClusterWaiter

type DeleteClusterWaiter struct {
	// contains filtered or unexported fields
}

DeleteClusterWaiter tracks the state of the operation started by DeleteCluster.

func (*DeleteClusterWaiter) Done

func (w *DeleteClusterWaiter) Done(ctx context.Context, opts ...call.Option) (bool, error)

Done polls once and reports whether the operation has reached a terminal state.

func (*DeleteClusterWaiter) GetClusterId

func (w *DeleteClusterWaiter) GetClusterId() string

GetClusterId returns the ClusterId value used to identify the operation.

func (*DeleteClusterWaiter) Wait

func (w *DeleteClusterWaiter) Wait(ctx context.Context, opts ...lro.Option) (*ClusterInfo, error)

Wait polls until the operation reaches a terminal state.

type DependencyMode

type DependencyMode string

Controls dependency configuration for the cluster.

* `DEPENDENCY_MODE_AUTO`: <Databricks> will choose the most appropriate dependency mode based on your compute configuration. * `DEPENDENCY_MODE_ENVIRONMENTS`: Enables a unified dependency management experience across classic and serverless, resulting in increased stability and performance. Supported only on DBR 19+ in Standard access mode. * `DEPENDENCY_MODE_CLUSTER_LIBRARIES`: Legacy mode: dependencies come from cluster libraries and init scripts.

const (
	DependencyMode_Unspecified                    DependencyMode = ""
	DependencyMode_DependencyModeEnvironments     DependencyMode = "DEPENDENCY_MODE_ENVIRONMENTS"
	DependencyMode_DependencyModeClusterLibraries DependencyMode = "DEPENDENCY_MODE_CLUSTER_LIBRARIES"
	DependencyMode_DependencyModeAuto             DependencyMode = "DEPENDENCY_MODE_AUTO"
)

type DockerBasicAuth

type DockerBasicAuth struct {
	// Name of the user
	Username *string `fieldmask:"username"`
	// Password of the user
	Password *string `fieldmask:"password"`
}

type DockerImage

type DockerImage struct {
	// URL of the docker image.
	Url        *string `fieldmask:"url"`
	CredsOneof isDockerImage_CredsOneof
	// contains filtered or unexported fields
}

type DockerImage_CredsOneof_BasicAuth

type DockerImage_CredsOneof_BasicAuth struct {
	BasicAuth DockerBasicAuth `fieldmask:"basic_auth"`
}

DockerImage_CredsOneof_BasicAuth selects BasicAuth for DockerImage.CredsOneof. Basic auth with username and password

type EbsVolumeType

type EbsVolumeType string

All EBS volume types that <Databricks> supports. See https://aws.amazon.com/ebs/details/ for details.

const (
	EbsVolumeType_Unspecified EbsVolumeType = ""
	// Provision extra storage using AWS gp2 EBS volumes.
	EbsVolumeType_GeneralPurposeSsd EbsVolumeType = "GENERAL_PURPOSE_SSD"
	// Provision extra storage using AWS st1 volumes.
	EbsVolumeType_ThroughputOptimizedHdd EbsVolumeType = "THROUGHPUT_OPTIMIZED_HDD"
)

type EditClusterRequest

type EditClusterRequest struct {
	// ID of the cluster
	ClusterId *string
	// When set to true, fixed and default values from the policy will be used for
	// fields that are omitted. When set to false, only fixed values from the policy
	// will be applied.
	ApplyPolicyDefaultValues *bool
	Size                     isEditClusterRequest_Size
	// Cluster name requested by the user. This doesn't have to be unique. If not
	// specified at creation, the cluster name will be an empty string. For job
	// clusters, the cluster name is automatically set based on the job and job run
	// IDs.
	ClusterName *string
	// The Spark version of the cluster, e.g. `3.3.x-scala2.11`. A list of available
	// Spark versions can be retrieved by using the [clusters/sparkVersions] API
	// call.
	//
	// [clusters/sparkVersions]: https://docs.databricks.com/api/workspace/clusters/sparkversions
	SparkVersion *string
	// An object containing a set of optional, user-specified Spark configuration
	// key-value pairs. Users can also pass in a string of extra JVM options to the
	// driver and the executors via `spark.driver.extraJavaOptions` and
	// `spark.executor.extraJavaOptions` respectively.
	SparkConf map[string]string
	// Attributes related to clusters running on Amazon Web Services. If not
	// specified at cluster creation, a set of default values will be used.
	AwsAttributes *AwsAttributes
	// Attributes related to clusters running on Microsoft Azure. If not specified
	// at cluster creation, a set of default values will be used.
	AzureAttributes *AzureAttributes
	// Attributes related to clusters running on Google Cloud Platform. If not
	// specified at cluster creation, a set of default values will be used.
	GcpAttributes *GcpAttributes
	// This field encodes, through a single value, the resources available to each
	// of the Spark nodes in this cluster. For example, the Spark nodes can be
	// provisioned and optimized for memory or compute intensive workloads. A list
	// of available node types can be retrieved by using the
	// [clusters/listNodeTypes] API call.
	//
	// [clusters/listNodeTypes]: https://docs.databricks.com/api/workspace/clusters/listnodetypes
	NodeTypeId *string
	// The node type of the Spark driver. Note that this field is optional; if
	// unset, the driver node type will be set as the same value as `node_type_id`
	// defined above.
	//
	// This field, along with node_type_id, should not be set if
	// virtual_cluster_size is set. If both driver_node_type_id, node_type_id, and
	// virtual_cluster_size are specified, driver_node_type_id and node_type_id take
	// precedence.
	DriverNodeTypeId *string
	// Flexible node type configuration for worker nodes.
	WorkerNodeTypeFlexibility *NodeTypeFlexibility
	// Flexible node type configuration for the driver node.
	DriverNodeTypeFlexibility *NodeTypeFlexibility
	// SSH public key contents that will be added to each Spark node in this
	// cluster. The corresponding private keys can be used to login with the user
	// name `ubuntu` on port `2200`. Up to 10 keys can be specified.
	SshPublicKeys []string
	// Additional tags for cluster resources. <Databricks> will tag all cluster
	// resources (e.g., AWS instances and EBS volumes) with these tags in addition
	// to `default_tags`. Notes:
	//
	// - Currently, <Databricks> allows at most 45 custom tags
	//
	// - Clusters can only reuse cloud resources if the resources' tags are a subset
	// of the cluster tags
	CustomTags map[string]string
	// The configuration for delivering spark logs to a long-term storage
	// destination. Three kinds of destinations (DBFS, S3 and Unity Catalog volumes)
	// are supported. Only one destination can be specified for one cluster. If the
	// conf is given, the logs will be delivered to the destination every `5 mins`.
	// The destination of driver logs is `$destination/$clusterId/driver`, while the
	// destination of executor logs is `$destination/$clusterId/executor`.
	ClusterLogConf *ClusterLogConf
	// An object containing a set of optional, user-specified environment variable
	// key-value pairs. Please note that key-value pair of the form (X,Y) will be
	// exported as is (i.e., `export X='Y'`) while launching the driver and workers.
	//
	// In order to specify an additional set of `SPARK_DAEMON_JAVA_OPTS`, we
	// recommend appending them to `$SPARK_DAEMON_JAVA_OPTS` as shown in the example
	// below. This ensures that all default databricks managed environmental
	// variables are included as well.
	//
	// Example Spark environment variables: `{"SPARK_WORKER_MEMORY": "28000m",
	// "SPARK_LOCAL_DIRS": "/local_disk0"}` or `{"SPARK_DAEMON_JAVA_OPTS":
	// "$SPARK_DAEMON_JAVA_OPTS -Dspark.shuffle.service.enabled=true"}`
	SparkEnvVars map[string]string
	// Automatically terminates the cluster after it is inactive for this time in
	// minutes. If not set, this cluster will not be automatically terminated. If
	// specified, the threshold must be between 10 and 10000 minutes. Users can also
	// set this value to 0 to explicitly disable automatic termination.
	AutoterminationMinutes *int
	// Autoscaling Local Storage: when enabled, this cluster will dynamically
	// acquire additional disk space when its Spark workers are running low on disk
	// space.
	EnableElasticDisk *bool
	// The configuration for storing init scripts. Any number of destinations can be
	// specified. The scripts are executed sequentially in the order provided. If
	// `cluster_log_conf` is specified, init script logs are sent to
	// `<destination>/<cluster-ID>/init_scripts`.
	InitScripts []InitScriptInfo
	// Custom docker image BYOC
	DockerImage *DockerImage
	// The optional ID of the instance pool to which the cluster belongs.
	InstancePoolId *string
	// Single user name if data_security_mode is `SINGLE_USER`
	SingleUserName *string
	// The ID of the cluster policy used to create the cluster if applicable.
	PolicyId *string
	// Whether to enable LUKS on cluster VMs' local disks
	EnableLocalDiskEncryption *bool
	// The optional ID of the instance pool for the driver of the cluster belongs.
	// The pool cluster uses the instance pool with id (instance_pool_id) if the
	// driver pool is not assigned.
	DriverInstancePoolId *string
	WorkloadType         *WorkloadType
	DataSecurityMode     DataSecurityMode
	// Determines the cluster's runtime engine, either standard or Photon.
	//
	// This field is not compatible with legacy `spark_version` values that contain
	// `-photon-`. Remove `-photon-` from the `spark_version` and set
	// `runtime_engine` to `PHOTON`.
	//
	// If left unspecified, the runtime engine defaults to standard unless the
	// spark_version contains -photon-, in which case Photon will be used.
	RuntimeEngine RuntimeEngine
	Kind          ComputeKind
	// This field can only be used when `kind = CLASSIC_PREVIEW`.
	//
	// `effective_spark_version` is determined by `spark_version` (DBR release),
	// this field `use_ml_runtime`, and whether `node_type_id` is gpu node or not.
	UseMlRuntime *bool
	// This field can only be used when `kind = CLASSIC_PREVIEW`.
	//
	// When set to true, <Databricks> will automatically set single node related
	// `custom_tags`, `spark_conf`, and `num_workers`
	IsSingleNode *bool
	// If set, what the configurable throughput (in Mb/s) for the remote disk is.
	// Currently only supported for GCP HYPERDISK_BALANCED disks.
	RemoteDiskThroughput *int
	// If set, what the total initial volume size (in GB) of the remote disks should
	// be. Supported for GCP.
	TotalInitialRemoteDiskSize *int
	// Controls dependency configuration for the cluster.
	DependencyMode DependencyMode
}

type EditClusterRequest_Size_Autoscale

type EditClusterRequest_Size_Autoscale struct {
	Autoscale AutoScale
}

EditClusterRequest_Size_Autoscale selects Autoscale for EditClusterRequest.Size. Parameters needed in order to automatically scale clusters up and down based on load. Note: autoscaling works best with DB runtime versions 3.0 or later.

type EditClusterRequest_Size_NumWorkers

type EditClusterRequest_Size_NumWorkers struct {
	NumWorkers int
}

EditClusterRequest_Size_NumWorkers selects NumWorkers for EditClusterRequest.Size. Number of worker nodes that this cluster should have. A cluster has one Spark Driver and `num_workers` Executors for a total of `num_workers` + 1 Spark nodes.

Note: When reading the properties of a cluster, this field reflects the desired number of workers rather than the actual current number of workers. For instance, if a cluster is resized from 5 to 10 workers, this field will immediately be updated to reflect the target size of 10 workers, whereas the workers listed in `spark_info` will gradually increase from 5 to 10 as the new nodes are provisioned.

type EditClusterResponse

type EditClusterResponse struct {
}

type EditClusterWaiter

type EditClusterWaiter struct {
	// contains filtered or unexported fields
}

EditClusterWaiter tracks the state of the operation started by EditCluster.

func (*EditClusterWaiter) Done

func (w *EditClusterWaiter) Done(ctx context.Context, opts ...call.Option) (bool, error)

Done polls once and reports whether the operation has reached a terminal state.

func (*EditClusterWaiter) GetClusterId

func (w *EditClusterWaiter) GetClusterId() string

GetClusterId returns the ClusterId value used to identify the operation.

func (*EditClusterWaiter) Wait

func (w *EditClusterWaiter) Wait(ctx context.Context, opts ...lro.Option) (*ClusterInfo, error)

Wait polls until the operation reaches a terminal state.

type EnforcePolicyComplianceForClusterRequest

type EnforcePolicyComplianceForClusterRequest struct {
	// The ID of the cluster you want to enforce policy compliance on.
	ClusterId *string
	// If set, previews the changes that would be made to a cluster to enforce
	// compliance but does not update the cluster.
	ValidateOnly *bool
	// Determines how changes should be made to clusters that are not in
	// `TERMINATED` state.
	//
	// - `ENFORCE_IMMEDIATELY`: If the cluster is in a `RUNNING` state, it will be
	// restarted so that the new attributes can take effect. For other states aside
	// from `TERMINATED` state, the request will be rejected. -
	// `WAIT_FOR_TERMINATION`: The cluster is not immediately edited. Instead, a
	// pending enforcement is scheduled to update the cluster when it terminates or
	// restarts. When this occurs, `enforce_result` will contain `DEFERRED`. Only
	// workspace admins can use this mode.
	//
	// Regardless of the enforce mode, clusters in `TERMINATED` state are
	// immediately edited.
	EnforceMode EnforcePolicyComplianceForClusterRequest_EnforceMode
}

type EnforcePolicyComplianceForClusterRequest_EnforceMode

type EnforcePolicyComplianceForClusterRequest_EnforceMode string
const (
	EnforcePolicyComplianceForClusterRequest_EnforceMode_Unspecified EnforcePolicyComplianceForClusterRequest_EnforceMode = ""
	// If the cluster is in the TERMINATED state, edit the cluster immediately. If
	// the cluster is in the RUNNING state, edit and restart the cluster. Else, the
	// operation fails.
	EnforcePolicyComplianceForClusterRequest_EnforceMode_EnforceImmediately EnforcePolicyComplianceForClusterRequest_EnforceMode = "ENFORCE_IMMEDIATELY"
	// If the cluster is in the TERMINATED state, edit the cluster immediately.
	// Else, the cluster is not edited. Instead, a pending enforcement is scheduled
	// to update the cluster when it terminates or restarts. Only workspace admins
	// can use this mode.
	EnforcePolicyComplianceForClusterRequest_EnforceMode_WaitForTermination EnforcePolicyComplianceForClusterRequest_EnforceMode = "WAIT_FOR_TERMINATION"
)

type EnforcePolicyComplianceForClusterResponse

type EnforcePolicyComplianceForClusterResponse struct {
	// Whether any changes have been made to the cluster settings for the cluster to
	// become compliant with its policy.
	HasChanges *bool
	// A list of changes that have been made to the cluster settings for the cluster
	// to become compliant with its policy.
	Changes []EnforcePolicyComplianceForClusterResponse_ClusterSettingsChange
	// Describes whether changes have been applied to the cluster.
	EnforceResult EnforcePolicyComplianceForClusterResponse_EnforceResult
}

type EnforcePolicyComplianceForClusterResponse_ClusterSettings

type EnforcePolicyComplianceForClusterResponse_ClusterSettings struct {
	// Cluster name requested by the user. This doesn't have to be unique. If not
	// specified at creation, the cluster name will be an empty string. For job
	// clusters, the cluster name is automatically set based on the job and job run
	// IDs.
	ClusterName *string
	// The Spark version of the cluster, e.g. `3.3.x-scala2.11`. A list of available
	// Spark versions can be retrieved by using the [clusters/sparkVersions] API
	// call.
	//
	// [clusters/sparkVersions]: https://docs.databricks.com/api/workspace/clusters/sparkversions
	SparkVersion *string
	// An object containing a set of optional, user-specified Spark configuration
	// key-value pairs. Users can also pass in a string of extra JVM options to the
	// driver and the executors via `spark.driver.extraJavaOptions` and
	// `spark.executor.extraJavaOptions` respectively.
	SparkConf map[string]string
	// Attributes related to clusters running on Amazon Web Services. If not
	// specified at cluster creation, a set of default values will be used.
	AwsAttributes *AwsAttributes
	// Attributes related to clusters running on Microsoft Azure. If not specified
	// at cluster creation, a set of default values will be used.
	AzureAttributes *AzureAttributes
	// Attributes related to clusters running on Google Cloud Platform. If not
	// specified at cluster creation, a set of default values will be used.
	GcpAttributes *GcpAttributes
	// This field encodes, through a single value, the resources available to each
	// of the Spark nodes in this cluster. For example, the Spark nodes can be
	// provisioned and optimized for memory or compute intensive workloads. A list
	// of available node types can be retrieved by using the
	// [clusters/listNodeTypes] API call.
	//
	// [clusters/listNodeTypes]: https://docs.databricks.com/api/workspace/clusters/listnodetypes
	NodeTypeId *string
	// The node type of the Spark driver. Note that this field is optional; if
	// unset, the driver node type will be set as the same value as `node_type_id`
	// defined above.
	//
	// This field, along with node_type_id, should not be set if
	// virtual_cluster_size is set. If both driver_node_type_id, node_type_id, and
	// virtual_cluster_size are specified, driver_node_type_id and node_type_id take
	// precedence.
	DriverNodeTypeId *string
	// Flexible node type configuration for worker nodes.
	WorkerNodeTypeFlexibility *NodeTypeFlexibility
	// Flexible node type configuration for the driver node.
	DriverNodeTypeFlexibility *NodeTypeFlexibility
	// SSH public key contents that will be added to each Spark node in this
	// cluster. The corresponding private keys can be used to login with the user
	// name `ubuntu` on port `2200`. Up to 10 keys can be specified.
	SshPublicKeys []string
	// Additional tags for cluster resources. <Databricks> will tag all cluster
	// resources (e.g., AWS instances and EBS volumes) with these tags in addition
	// to `default_tags`. Notes:
	//
	// - Currently, <Databricks> allows at most 45 custom tags
	//
	// - Clusters can only reuse cloud resources if the resources' tags are a subset
	// of the cluster tags
	CustomTags map[string]string
	// The configuration for delivering spark logs to a long-term storage
	// destination. Three kinds of destinations (DBFS, S3 and Unity Catalog volumes)
	// are supported. Only one destination can be specified for one cluster. If the
	// conf is given, the logs will be delivered to the destination every `5 mins`.
	// The destination of driver logs is `$destination/$clusterId/driver`, while the
	// destination of executor logs is `$destination/$clusterId/executor`.
	ClusterLogConf *ClusterLogConf
	// An object containing a set of optional, user-specified environment variable
	// key-value pairs. Please note that key-value pair of the form (X,Y) will be
	// exported as is (i.e., `export X='Y'`) while launching the driver and workers.
	//
	// In order to specify an additional set of `SPARK_DAEMON_JAVA_OPTS`, we
	// recommend appending them to `$SPARK_DAEMON_JAVA_OPTS` as shown in the example
	// below. This ensures that all default databricks managed environmental
	// variables are included as well.
	//
	// Example Spark environment variables: `{"SPARK_WORKER_MEMORY": "28000m",
	// "SPARK_LOCAL_DIRS": "/local_disk0"}` or `{"SPARK_DAEMON_JAVA_OPTS":
	// "$SPARK_DAEMON_JAVA_OPTS -Dspark.shuffle.service.enabled=true"}`
	SparkEnvVars map[string]string
	// Automatically terminates the cluster after it is inactive for this time in
	// minutes. If not set, this cluster will not be automatically terminated. If
	// specified, the threshold must be between 10 and 10000 minutes. Users can also
	// set this value to 0 to explicitly disable automatic termination.
	AutoterminationMinutes *int
	// Autoscaling Local Storage: when enabled, this cluster will dynamically
	// acquire additional disk space when its Spark workers are running low on disk
	// space.
	EnableElasticDisk *bool
	// The configuration for storing init scripts. Any number of destinations can be
	// specified. The scripts are executed sequentially in the order provided. If
	// `cluster_log_conf` is specified, init script logs are sent to
	// `<destination>/<cluster-ID>/init_scripts`.
	InitScripts []InitScriptInfo
	// Custom docker image BYOC
	DockerImage *DockerImage
	// The optional ID of the instance pool to which the cluster belongs.
	InstancePoolId *string
	// Single user name if data_security_mode is `SINGLE_USER`
	SingleUserName *string
	// The ID of the cluster policy used to create the cluster if applicable.
	PolicyId *string
	// Whether to enable LUKS on cluster VMs' local disks
	EnableLocalDiskEncryption *bool
	// The optional ID of the instance pool for the driver of the cluster belongs.
	// The pool cluster uses the instance pool with id (instance_pool_id) if the
	// driver pool is not assigned.
	DriverInstancePoolId *string
	WorkloadType         *WorkloadType
	DataSecurityMode     DataSecurityMode
	// Determines the cluster's runtime engine, either standard or Photon.
	//
	// This field is not compatible with legacy `spark_version` values that contain
	// `-photon-`. Remove `-photon-` from the `spark_version` and set
	// `runtime_engine` to `PHOTON`.
	//
	// If left unspecified, the runtime engine defaults to standard unless the
	// spark_version contains -photon-, in which case Photon will be used.
	RuntimeEngine RuntimeEngine
	Kind          ComputeKind
	// This field can only be used when `kind = CLASSIC_PREVIEW`.
	//
	// `effective_spark_version` is determined by `spark_version` (DBR release),
	// this field `use_ml_runtime`, and whether `node_type_id` is gpu node or not.
	UseMlRuntime *bool
	// This field can only be used when `kind = CLASSIC_PREVIEW`.
	//
	// When set to true, <Databricks> will automatically set single node related
	// `custom_tags`, `spark_conf`, and `num_workers`
	IsSingleNode *bool
	// If set, what the configurable throughput (in Mb/s) for the remote disk is.
	// Currently only supported for GCP HYPERDISK_BALANCED disks.
	RemoteDiskThroughput *int
	// If set, what the total initial volume size (in GB) of the remote disks should
	// be. Supported for GCP.
	TotalInitialRemoteDiskSize *int
	// Controls dependency configuration for the cluster.
	DependencyMode DependencyMode
	Size           isEnforcePolicyComplianceForClusterResponse_ClusterSettings_Size
}

type EnforcePolicyComplianceForClusterResponse_ClusterSettingsChange

type EnforcePolicyComplianceForClusterResponse_ClusterSettingsChange struct {
	// The field where this change would be made.
	Field *string
	// The previous value of this field before enforcing policy compliance (either a
	// number, a boolean, or a string) converted to a string. This is intended to be
	// read by a human. The type of the field can be retrieved by reading the
	// settings field in the API response.
	PreviousValue *string
	// The new value of this field after enforcing policy compliance (either a
	// number, a boolean, or a string) converted to a string. This is intended to be
	// read by a human. The typed new value of this field can be retrieved by
	// reading the settings field in the API response.
	NewValue *string
}

Represents a change to the cluster settings required for the cluster to become compliant with its policy..

type EnforcePolicyComplianceForClusterResponse_ClusterSettings_Size_Autoscale

type EnforcePolicyComplianceForClusterResponse_ClusterSettings_Size_Autoscale struct {
	Autoscale AutoScale
}

EnforcePolicyComplianceForClusterResponse_ClusterSettings_Size_Autoscale selects Autoscale for EnforcePolicyComplianceForClusterResponse_ClusterSettings.Size. Parameters needed in order to automatically scale clusters up and down based on load. Note: autoscaling works best with DB runtime versions 3.0 or later.

type EnforcePolicyComplianceForClusterResponse_ClusterSettings_Size_NumWorkers

type EnforcePolicyComplianceForClusterResponse_ClusterSettings_Size_NumWorkers struct {
	NumWorkers int
}

EnforcePolicyComplianceForClusterResponse_ClusterSettings_Size_NumWorkers selects NumWorkers for EnforcePolicyComplianceForClusterResponse_ClusterSettings.Size. Number of worker nodes that this cluster should have. A cluster has one Spark Driver and `num_workers` Executors for a total of `num_workers` + 1 Spark nodes.

Note: When reading the properties of a cluster, this field reflects the desired number of workers rather than the actual current number of workers. For instance, if a cluster is resized from 5 to 10 workers, this field will immediately be updated to reflect the target size of 10 workers, whereas the workers listed in `spark_info` will gradually increase from 5 to 10 as the new nodes are provisioned.

type EnforcePolicyComplianceForClusterResponse_EnforceResult

type EnforcePolicyComplianceForClusterResponse_EnforceResult string
const (
	EnforcePolicyComplianceForClusterResponse_EnforceResult_Unspecified EnforcePolicyComplianceForClusterResponse_EnforceResult = ""
	// No changes were made to the cluster.
	EnforcePolicyComplianceForClusterResponse_EnforceResult_NoChanges EnforcePolicyComplianceForClusterResponse_EnforceResult = "NO_CHANGES"
	// Changes were applied to the cluster.
	EnforcePolicyComplianceForClusterResponse_EnforceResult_Applied EnforcePolicyComplianceForClusterResponse_EnforceResult = "APPLIED"
	// Changes were not applied to the cluster yet. Instead, changes will be applied
	// when the cluster terminates or restarts. This is not returned when
	// validate_only is true, even if the enforcement would have normally been
	// deferred. Instead APPLIED will be returned.
	EnforcePolicyComplianceForClusterResponse_EnforceResult_Deferred EnforcePolicyComplianceForClusterResponse_EnforceResult = "DEFERRED"
)

type EventDetails

type EventDetails struct {
	// The current number of nodes in the cluster.
	CurrentNumWorkers *int
	// The targeted number of nodes in the cluster.
	TargetNumWorkers *int
	// The cluster attributes before a cluster was edited.
	PreviousAttributes *ClusterAttributes
	// * For created clusters, the attributes of the cluster. * For edited clusters,
	// the new attributes of the cluster.
	Attributes *ClusterAttributes
	// The size of the cluster before an edit or resize.
	PreviousClusterSize *ClusterSize
	// The actual cluster size that was set in the cluster creation or edit.
	ClusterSize *ClusterSize
	// The cause of a change in target size.
	Cause ResizeCause_ResizeCause
	// A termination reason: * On a TERMINATED event, this is the reason of the
	// termination. * On a RESIZE_COMPLETE event, this indicates the reason that we
	// failed to acquire some nodes.
	Reason *TerminationReason
	// The user that caused the event to occur. (Empty if it was done by the control
	// plane.)
	User *string
	// Previous disk size in bytes
	PreviousDiskSize *int64
	// Current disk size in bytes
	DiskSize  *int64
	FreeSpace *int64
	// Instance Id where the event originated from
	InstanceId         *string
	DidNotExpandReason *string
	// More details about the change in driver's state
	DriverStateMessage *string
	// Unique identifier of the specific job run associated with this cluster event
	// * For clusters created for jobs, this will be the same as the cluster name
	JobRunName *string
	// List of global and cluster init scripts associated with this cluster event.
	InitScripts *InitScriptEventDetails
	// Whether or not a blocklisted node should be terminated. For ClusterEventType
	// NODE_BLACKLISTED.
	EnableTerminationForNodeBlocklisted *bool
	// The current number of vCPUs in the cluster.
	CurrentNumVcpus *int
	// The targeted number of vCPUs in the cluster.
	TargetNumVcpus *int
}

type GcpAttributes

type GcpAttributes struct {
	// This field determines whether the spark executors will be scheduled to run on
	// preemptible VMs (when set to true) versus standard compute engine VMs (when
	// set to false; default). Note: Soon to be deprecated, use the 'availability'
	// field instead.
	UsePreemptibleExecutors *bool `fieldmask:"use_preemptible_executors"`
	// If provided, the cluster will impersonate the google service account when
	// accessing gcloud services (like GCS). The google service account must have
	// previously been added to the <Databricks> environment by an account
	// administrator.
	GoogleServiceAccount *string `fieldmask:"google_service_account"`
	// Boot disk size in GB
	BootDiskSize *int `fieldmask:"boot_disk_size"`
	// This field determines whether the spark executors will be scheduled to run on
	// preemptible VMs, on-demand VMs, or preemptible VMs with a fallback to
	// on-demand VMs if the former is unavailable.
	Availability GcpAvailability `fieldmask:"availability"`
	// Identifier for the availability zone in which the cluster resides. This can
	// be one of the following: - "HA" => High availability, spread nodes across
	// availability zones for a <Databricks> deployment region [default]. - "AUTO"
	// => <Databricks> picks an availability zone to schedule the cluster on. - A
	// GCP availability zone => Pick One of the available zones for (machine type +
	// region) from https://cloud.google.com/compute/docs/regions-zones.
	ZoneId *string `fieldmask:"zone_id"`
	// If provided, each node (workers and driver) in the cluster will have this
	// number of local SSDs attached. Each local SSD is 375GB in size. Refer to [GCP
	// documentation] for the supported number of local SSDs for each instance type.
	//
	// [GCP documentation]: https://cloud.google.com/compute/docs/disks/local-ssd#choose_number_local_ssds
	LocalSsdCount *int `fieldmask:"local_ssd_count"`
	// The first `first_on_demand` nodes of the cluster will be placed on on-demand
	// instances. This value should be greater than 0, to make sure the cluster
	// driver node is placed on an on-demand instance. If this value is greater than
	// or equal to the current cluster size, all nodes will be placed on on-demand
	// instances. If this value is less than the current cluster size,
	// `first_on_demand` nodes will be placed on on-demand instances and the
	// remainder will be placed on `availability` instances. Note that this value
	// does not affect cluster size and cannot currently be mutated over the
	// lifetime of a cluster.
	FirstOnDemand *int `fieldmask:"first_on_demand"`
	// The confidential computing technology for this cluster's instances. Currently
	// only SEV_SNP is supported, and only on N2D instance types. When not set, no
	// confidential computing is applied.
	ConfidentialComputeType ConfidentialComputeType `fieldmask:"confidential_compute_type"`
}

Attributes set during cluster creation which are related to GCP..

type GcpAvailability

type GcpAvailability string

This field determines whether the instance pool will contain preemptible VMs, on-demand VMs, or preemptible VMs with a fallback to on-demand VMs if the former is unavailable.

const (
	GcpAvailability_Unspecified                GcpAvailability = ""
	GcpAvailability_PreemptibleGcp             GcpAvailability = "PREEMPTIBLE_GCP"
	GcpAvailability_OnDemandGcp                GcpAvailability = "ON_DEMAND_GCP"
	GcpAvailability_PreemptibleWithFallbackGcp GcpAvailability = "PREEMPTIBLE_WITH_FALLBACK_GCP"
)

type GcsStorageInfo

type GcsStorageInfo struct {
	// GCS destination/URI, e.g. `gs://my-bucket/some-prefix`
	Destination *string
}

A storage location in Google Cloud Platform's GCS.

type GetClusterRequest

type GetClusterRequest struct {
	// The cluster about which to retrieve information.
	ClusterId *string
}

type GetEventsOrder

type GetEventsOrder string
const (
	GetEventsOrder_Unspecified GetEventsOrder = ""
	GetEventsOrder_Desc        GetEventsOrder = "DESC"
	GetEventsOrder_Asc         GetEventsOrder = "ASC"
)

type GetEventsResponse

type GetEventsResponse struct {
	Events []ClusterEvent
	// Deprecated: use next_page_token or prev_page_token instead.
	//
	// The parameters required to retrieve the next page of events. Omitted if there
	// are no more events to read.
	NextPage *ListEventsRequest
	// Deprecated: Returns 0 when request uses page_token. Will start returning zero
	// when request uses offset/limit soon.
	//
	// The total number of events filtered by the start_time, end_time, and
	// event_types.
	TotalCount *int64
	// This field represents the pagination token to retrieve the next page of
	// results. If the value is "", it means no further results for the request.
	NextPageToken *string
	// This field represents the pagination token to retrieve the previous page of
	// results. If the value is "", it means no further results for the request.
	PrevPageToken *string
}

type GetPolicyComplianceForClusterRequest

type GetPolicyComplianceForClusterRequest struct {
	// The ID of the cluster to get the compliance status
	ClusterId *string
}

type GetPolicyComplianceForClusterResponse

type GetPolicyComplianceForClusterResponse struct {
	// Whether the cluster is compliant with its policy or not. Clusters could be
	// out of compliance if the policy was updated after the cluster was last
	// edited.
	IsCompliant *bool
	// An object containing key-value mappings representing the first 200 policy
	// validation errors. The keys indicate the path where the policy validation
	// error is occurring. The values indicate an error message describing the
	// policy validation error.
	Violations map[string]string
	// Information about the pending enforcement for the cluster. Only present if a
	// pending enforcement is scheduled for the cluster.
	PendingEnforcement *PendingEnforcement
}

type GetSparkVersionsRequest

type GetSparkVersionsRequest struct {
}

Returns the list of all Spark versions that can be used to create clusters..

type GetSparkVersionsResponse

type GetSparkVersionsResponse struct {
	// All the available Spark versions.
	Versions []SparkVersion
}

type InitScriptEventDetails

type InitScriptEventDetails struct {
	// The private ip of the node we are reporting init script execution details for
	// (we will select the execution details from only one node rather than
	// reporting the execution details from every node to keep these event details
	// small)
	//
	// This should only be defined for the INIT_SCRIPTS_FINISHED event
	ReportedForNode *string
	// The global init scripts associated with this cluster event.
	Global []InitScriptEventDetails_InitScriptInfoAndExecutionDetails
	// The cluster scoped init scripts associated with this cluster event.
	Cluster []InitScriptEventDetails_InitScriptInfoAndExecutionDetails
}

type InitScriptEventDetails_InitScriptInfoAndExecutionDetails

type InitScriptEventDetails_InitScriptInfoAndExecutionDetails struct {
	StorageInfo isInitScriptEventDetails_InitScriptInfoAndExecutionDetails_StorageInfo
	// The current status of the script
	Status InitScriptExecutionDetails_InitScriptExecutionStatus
	// The number duration of the script execution in seconds
	ExecutionDurationSeconds *int
	// Additional details regarding errors (such as a file not found message if the
	// status is FAILED_FETCH). This field should only be used to provide
	// *additional* information to the status field, not duplicate it.
	ErrorMessage *string
	// The stderr output from the init script execution. Only populated when init
	// scripts debug is enabled and script execution fails.
	Stderr *string
}

type InitScriptEventDetails_InitScriptInfoAndExecutionDetails_StorageInfo_Abfss

type InitScriptEventDetails_InitScriptInfoAndExecutionDetails_StorageInfo_Abfss struct {
	Abfss Adlsgen2Info
}

InitScriptEventDetails_InitScriptInfoAndExecutionDetails_StorageInfo_Abfss selects Abfss for InitScriptEventDetails_InitScriptInfoAndExecutionDetails.StorageInfo. destination needs to be provided, e.g. `abfss://<container-name>@<storage-account-name>.dfs.core.windows.net/<directory-name>`

type InitScriptEventDetails_InitScriptInfoAndExecutionDetails_StorageInfo_Dbfs

type InitScriptEventDetails_InitScriptInfoAndExecutionDetails_StorageInfo_Dbfs struct {
	Dbfs DbfsStorageInfo
}

InitScriptEventDetails_InitScriptInfoAndExecutionDetails_StorageInfo_Dbfs selects Dbfs for InitScriptEventDetails_InitScriptInfoAndExecutionDetails.StorageInfo. destination needs to be provided. e.g. `{ "dbfs": { "destination" : "dbfs:/home/cluster_log" } }`

type InitScriptEventDetails_InitScriptInfoAndExecutionDetails_StorageInfo_File

type InitScriptEventDetails_InitScriptInfoAndExecutionDetails_StorageInfo_File struct {
	File LocalFileInfo
}

InitScriptEventDetails_InitScriptInfoAndExecutionDetails_StorageInfo_File selects File for InitScriptEventDetails_InitScriptInfoAndExecutionDetails.StorageInfo. destination needs to be provided, e.g. `{ "file": { "destination": "file:/my/local/file.sh" } }`

type InitScriptEventDetails_InitScriptInfoAndExecutionDetails_StorageInfo_Gcs

type InitScriptEventDetails_InitScriptInfoAndExecutionDetails_StorageInfo_Gcs struct {
	Gcs GcsStorageInfo
}

InitScriptEventDetails_InitScriptInfoAndExecutionDetails_StorageInfo_Gcs selects Gcs for InitScriptEventDetails_InitScriptInfoAndExecutionDetails.StorageInfo. destination needs to be provided, e.g. `{ "gcs": { "destination": "gs://my-bucket/file.sh" } }`

type InitScriptEventDetails_InitScriptInfoAndExecutionDetails_StorageInfo_S3

type InitScriptEventDetails_InitScriptInfoAndExecutionDetails_StorageInfo_S3 struct {
	S3 S3StorageInfo
}

InitScriptEventDetails_InitScriptInfoAndExecutionDetails_StorageInfo_S3 selects S3 for InitScriptEventDetails_InitScriptInfoAndExecutionDetails.StorageInfo. destination and either the region or endpoint need to be provided. e.g. `{ \"s3\": { \"destination\": \"s3://cluster_log_bucket/prefix\", \"region\": \"us-west-2\" } }` Cluster iam role is used to access s3, please make sure the cluster iam role in `instance_profile_arn` has permission to write data to the s3 destination.

type InitScriptEventDetails_InitScriptInfoAndExecutionDetails_StorageInfo_Volumes

type InitScriptEventDetails_InitScriptInfoAndExecutionDetails_StorageInfo_Volumes struct {
	Volumes VolumesStorageInfo
}

InitScriptEventDetails_InitScriptInfoAndExecutionDetails_StorageInfo_Volumes selects Volumes for InitScriptEventDetails_InitScriptInfoAndExecutionDetails.StorageInfo. destination needs to be provided. e.g. `{ \"volumes\" : { \"destination\" : \"/Volumes/my-init.sh\" } }`

type InitScriptEventDetails_InitScriptInfoAndExecutionDetails_StorageInfo_Workspace

type InitScriptEventDetails_InitScriptInfoAndExecutionDetails_StorageInfo_Workspace struct {
	Workspace WorkspaceStorageInfo
}

InitScriptEventDetails_InitScriptInfoAndExecutionDetails_StorageInfo_Workspace selects Workspace for InitScriptEventDetails_InitScriptInfoAndExecutionDetails.StorageInfo. destination needs to be provided, e.g. `{ "workspace": { "destination": "/cluster-init-scripts/setup-datadog.sh" } }`

type InitScriptExecutionDetails

type InitScriptExecutionDetails struct {
}

type InitScriptExecutionDetails_InitScriptExecutionStatus

type InitScriptExecutionDetails_InitScriptExecutionStatus string

Result of attempted script execution

const (
	InitScriptExecutionDetails_InitScriptExecutionStatus_Unspecified InitScriptExecutionDetails_InitScriptExecutionStatus = ""
	// The script's execution status is unknown
	InitScriptExecutionDetails_InitScriptExecutionStatus_Unknown InitScriptExecutionDetails_InitScriptExecutionStatus = "UNKNOWN"
	// The NodeDaemon failed to fetch the script
	InitScriptExecutionDetails_InitScriptExecutionStatus_FailedFetch InitScriptExecutionDetails_InitScriptExecutionStatus = "FAILED_FETCH"
	// The script returned a non-zero exit code after execution
	InitScriptExecutionDetails_InitScriptExecutionStatus_FailedExecution InitScriptExecutionDetails_InitScriptExecutionStatus = "FAILED_EXECUTION"
	// The script was successfully fetched but was not executed
	InitScriptExecutionDetails_InitScriptExecutionStatus_NotExecuted InitScriptExecutionDetails_InitScriptExecutionStatus = "NOT_EXECUTED"
	// The NodeDaemon failed to fetch the script, and the script was skippable (i.e.
	// skip_if_fetch_fails was true) so it was skipped without triggering any
	// errors.
	InitScriptExecutionDetails_InitScriptExecutionStatus_Skipped InitScriptExecutionDetails_InitScriptExecutionStatus = "SKIPPED"
	// The script was successfully executed
	InitScriptExecutionDetails_InitScriptExecutionStatus_Succeeded InitScriptExecutionDetails_InitScriptExecutionStatus = "SUCCEEDED"
	// For FUSE mount init scripts (WSFS & Volumes): the fuse mounting was
	// unsuccessful
	InitScriptExecutionDetails_InitScriptExecutionStatus_FuseMountFailed InitScriptExecutionDetails_InitScriptExecutionStatus = "FUSE_MOUNT_FAILED"
)

type InitScriptInfo

type InitScriptInfo struct {
	StorageInfo isInitScriptInfo_StorageInfo
}

Config for an individual init script.

type InitScriptInfo_StorageInfo_Abfss

type InitScriptInfo_StorageInfo_Abfss struct {
	Abfss Adlsgen2Info
}

InitScriptInfo_StorageInfo_Abfss selects Abfss for InitScriptInfo.StorageInfo. destination needs to be provided, e.g. `abfss://<container-name>@<storage-account-name>.dfs.core.windows.net/<directory-name>`

type InitScriptInfo_StorageInfo_Dbfs

type InitScriptInfo_StorageInfo_Dbfs struct {
	Dbfs DbfsStorageInfo
}

InitScriptInfo_StorageInfo_Dbfs selects Dbfs for InitScriptInfo.StorageInfo. destination needs to be provided. e.g. `{ "dbfs": { "destination" : "dbfs:/home/cluster_log" } }`

type InitScriptInfo_StorageInfo_File

type InitScriptInfo_StorageInfo_File struct {
	File LocalFileInfo
}

InitScriptInfo_StorageInfo_File selects File for InitScriptInfo.StorageInfo. destination needs to be provided, e.g. `{ "file": { "destination": "file:/my/local/file.sh" } }`

type InitScriptInfo_StorageInfo_Gcs

type InitScriptInfo_StorageInfo_Gcs struct {
	Gcs GcsStorageInfo
}

InitScriptInfo_StorageInfo_Gcs selects Gcs for InitScriptInfo.StorageInfo. destination needs to be provided, e.g. `{ "gcs": { "destination": "gs://my-bucket/file.sh" } }`

type InitScriptInfo_StorageInfo_S3

type InitScriptInfo_StorageInfo_S3 struct {
	S3 S3StorageInfo
}

InitScriptInfo_StorageInfo_S3 selects S3 for InitScriptInfo.StorageInfo. destination and either the region or endpoint need to be provided. e.g. `{ \"s3\": { \"destination\": \"s3://cluster_log_bucket/prefix\", \"region\": \"us-west-2\" } }` Cluster iam role is used to access s3, please make sure the cluster iam role in `instance_profile_arn` has permission to write data to the s3 destination.

type InitScriptInfo_StorageInfo_Volumes

type InitScriptInfo_StorageInfo_Volumes struct {
	Volumes VolumesStorageInfo
}

InitScriptInfo_StorageInfo_Volumes selects Volumes for InitScriptInfo.StorageInfo. destination needs to be provided. e.g. `{ \"volumes\" : { \"destination\" : \"/Volumes/my-init.sh\" } }`

type InitScriptInfo_StorageInfo_Workspace

type InitScriptInfo_StorageInfo_Workspace struct {
	Workspace WorkspaceStorageInfo
}

InitScriptInfo_StorageInfo_Workspace selects Workspace for InitScriptInfo.StorageInfo. destination needs to be provided, e.g. `{ "workspace": { "destination": "/cluster-init-scripts/setup-datadog.sh" } }`

type ListAvailableZonesRequest

type ListAvailableZonesRequest struct {
}

type ListAvailableZonesResponse

type ListAvailableZonesResponse struct {
	// The list of available zones (e.g., ['us-west-2c', 'us-east-2']).
	Zones []string
	// The availability zone if no “zone_id“ is provided in the cluster creation
	// request.
	DefaultZone *string
}

type ListClusterComplianceForPolicyRequest

type ListClusterComplianceForPolicyRequest struct {
	// Canonical unique identifier for the cluster policy.
	PolicyId *string
	// A page token that can be used to navigate to the next page or previous page
	// as returned by `next_page_token` or `prev_page_token`.
	PageToken *string
	// Use this field to specify the maximum number of results to be returned by the
	// server. The server may further constrain the maximum number of results
	// returned in a single page.
	PageSize *int
}

type ListClusterComplianceForPolicyResponse

type ListClusterComplianceForPolicyResponse struct {
	// A list of clusters and their policy compliance statuses.
	Clusters []ClusterCompliance
	// This field represents the pagination token to retrieve the next page of
	// results. If the value is "", it means no further results for the request.
	NextPageToken *string
	// This field represents the pagination token to retrieve the previous page of
	// results. If the value is "", it means no further results for the request.
	PrevPageToken *string
}

type ListClustersRequest

type ListClustersRequest struct {
	// Use next_page_token or prev_page_token returned from the previous request to
	// list the next or previous page of clusters respectively.
	PageToken *string
	// Use this field to specify the maximum number of results to be returned by the
	// server. The server may further constrain the maximum number of results
	// returned in a single page.
	PageSize *int
}

type ListClustersResponse

type ListClustersResponse struct {
	Clusters []ClusterInfo
	// This field represents the pagination token to retrieve the next page of
	// results. If the value is "", it means no further results for the request.
	NextPageToken *string
	// This field represents the pagination token to retrieve the previous page of
	// results. If the value is "", it means no further results for the request.
	PrevPageToken *string
}

type ListEventsRequest

type ListEventsRequest struct {
	// The ID of the cluster to retrieve events about.
	ClusterId *string
	// The start time in epoch milliseconds. If empty, returns events starting from
	// the beginning of time.
	StartTime *int64
	// The end time in epoch milliseconds. If empty, returns events up to the
	// current time.
	EndTime *int64
	// The order to list events in; either "ASC" or "DESC". Defaults to "DESC".
	Order GetEventsOrder
	// An optional set of event types to filter on. If empty, all event types are
	// returned.
	EventTypes []ClusterEventType_ClusterEventType
	// Deprecated: use page_token in combination with page_size instead.
	//
	// The offset in the result set. Defaults to 0 (no offset). When an offset is
	// specified and the results are requested in descending order, the end_time
	// field is required.
	Offset *int64
	// Deprecated: use page_token in combination with page_size instead.
	//
	// The maximum number of events to include in a page of events. Defaults to 50,
	// and maximum allowed value is 500.
	Limit *int64
	// Use next_page_token or prev_page_token returned from the previous request to
	// list the next or previous page of events respectively. If page_token is
	// empty, the first page is returned.
	PageToken *string
	// The maximum number of events to include in a page of events. The server may
	// further constrain the maximum number of results returned in a single page. If
	// the page_size is empty or 0, the server will decide the number of results to
	// be returned. The field has to be in the range [0,500]. If the value is
	// outside the range, the server enforces 0 or 500.
	PageSize *int
}

type ListNodeTypesRequest

type ListNodeTypesRequest struct {
}

type ListNodeTypesResponse

type ListNodeTypesResponse struct {
	// The list of available Spark node types.
	NodeTypes []NodeType
}

type LocalFileInfo

type LocalFileInfo struct {
	// local file destination, e.g. `file:/my/local/file.sh`
	Destination *string
}

type LogAnalyticsInfo

type LogAnalyticsInfo struct {
	LogAnalyticsWorkspaceId *string `fieldmask:"log_analytics_workspace_id"`
	LogAnalyticsPrimaryKey  *string `fieldmask:"log_analytics_primary_key"`
}

type LogSyncStatus

type LogSyncStatus struct {
	// The timestamp of last attempt. If the last attempt fails, `last_exception`
	// will contain the exception in the last attempt.
	LastAttempted *int64
	// The exception thrown in the last attempt, it would be null (omitted in the
	// response) if there is no exception in last attempted.
	LastException *string
}

The log delivery status.

type NodeInstanceType

type NodeInstanceType struct {
	// Unique identifier across instance types
	InstanceTypeId *string
	// Number of local disks that are present on this instance.
	LocalDisks *int
	// Size of the individual local disks attached to this instance (i.e. per local
	// disk).
	LocalDiskSizeGb *int
	// Size of the individual local nvme disks attached to this instance (i.e. per
	// local disk).
	LocalNvmeDiskSizeGb *int
	// Number of local nvme disks that are present on this instance.
	LocalNvmeDisks *int
}

This structure embodies the machine type that hosts spark containers Note: this should be an internal data structure for now It is defined in proto in case we want to send it over the wire in the future (which is likely).

type NodeType

type NodeType struct {
	// Unique identifier for this node type.
	NodeTypeId *string
	// Memory (in MB) available for this node type.
	MemoryMb *int
	// Number of CPU cores available for this node type. Note that this can be
	// fractional, e.g., 2.5 cores, if the number of cores on a machine instance is
	// not divisible by the number of Spark nodes on that machine.
	NumCores *float32
	// A string description associated with this node type, e.g., "r3.xlarge".
	Description *string
	// An identifier for the type of hardware that this node runs on, e.g.,
	// "r3.2xlarge" in AWS.
	InstanceTypeId *string
	// Whether the node type is deprecated. Non-deprecated node types offer greater
	// performance.
	IsDeprecated *bool
	// A descriptive category for this node type. Examples include "Memory
	// Optimized" and "Compute Optimized".
	Category *string
	// Whether this node type support EBS volumes. EBS volumes is disabled for node
	// types that we could place multiple corresponding containers on the same
	// hosting instance.
	SupportEbsVolumes *bool
	// Whether this node type support cluster tags.
	SupportClusterTags *bool
	// Number of GPUs available for this node type.
	NumGpus *int
	// The NodeInstanceType object corresponding to instance_type_id
	NodeInstanceType *NodeInstanceType
	// Whether this node is hidden from presentation in the UI.
	IsHidden *bool
	// Whether this node type supports port forwarding.
	SupportPortForwarding *bool
	// An optional hint at the display order of node types in the UI. Within a node
	// type category, lowest numbers come first.
	DisplayOrder *int
	// Whether this node comes with IO cache enabled by default.
	IsIoCacheEnabled *bool
	// A collection of node type info reported by the cloud provider
	NodeInfo            *CloudProviderNodeInfo
	PhotonWorkerCapable *bool
	PhotonDriverCapable *bool
	// AWS specific, whether this instance supports encryption in transit, used for
	// hipaa and pci workloads.
	IsEncryptedInTransit *bool
	// Whether this is an Arm-based instance.
	IsGraviton *bool
}

A description of a Spark node type including both the dimensions of the node and the instance type on which it will be hosted..

type NodeTypeFlexibility

type NodeTypeFlexibility struct {
	// A list of node type IDs to use as fallbacks when the primary node type is
	// unavailable.
	AlternateNodeTypeIds []string `fieldmask:"alternate_node_type_ids"`
	// The AWS Context ID for EC2 Fleet. When set (non-empty), the value is passed
	// to AWS CreateFleet API to create the EC2 Fleet.
	AwsContextId *string `fieldmask:"aws_context_id"`
}

Configuration for flexible node types, allowing fallback to alternate node types during cluster launch and upscale..

type PendingEnforcement

type PendingEnforcement struct {
	// The new configuration to apply upon cluster termination or restart.
	TargetSpec *EnforcePolicyComplianceForClusterResponse_ClusterSettings
	// The time the pending enforcement was initiated.
	InitiateTime *types.Time
	// Whether the pending enforcement will be applied. A pending enforcement begins
	// in `ACTIVE` state. If the enforcement fails to apply too many times, the
	// state transitions to `INACTIVE`. Afterwards, the enforcement must be
	// re-scheduled to become `ACTIVE` again.
	EnforcementStatus PendingEnforcement_EnforcementStatus
	// A list of changes that will be made to the cluster configuration when the
	// pending enforcement is applied.
	TargetChanges []EnforcePolicyComplianceForClusterResponse_ClusterSettingsChange
	// The user who initiated the pending enforcement.
	InitiatorUser *string
}

Represents a pending enforcement on a cluster, which contains the changes to make to the cluster configuration when the cluster is next terminated or restarted..

type PendingEnforcement_EnforcementStatus

type PendingEnforcement_EnforcementStatus string
const (
	PendingEnforcement_EnforcementStatus_Unspecified PendingEnforcement_EnforcementStatus = ""
	// The pending enforcement will be attempted on the next cluster terminate or
	// restart.
	PendingEnforcement_EnforcementStatus_Active PendingEnforcement_EnforcementStatus = "ACTIVE"
	// The pending enforcement will not be attempted again because we have already
	// unsuccessfully attempted to apply the enforce.
	PendingEnforcement_EnforcementStatus_Inactive PendingEnforcement_EnforcementStatus = "INACTIVE"
)

type PermanentDeleteClusterRequest

type PermanentDeleteClusterRequest struct {
	// The cluster to be deleted.
	ClusterId *string
}

type PermanentDeleteClusterResponse

type PermanentDeleteClusterResponse struct {
}

type PinClusterRequest

type PinClusterRequest struct {
	ClusterId *string
}

type PinClusterResponse

type PinClusterResponse struct {
}

type ResizeCause

type ResizeCause struct {
}

type ResizeCause_ResizeCause

type ResizeCause_ResizeCause string

The cause of a change in target size.

const (
	ResizeCause_ResizeCause_Unspecified ResizeCause_ResizeCause = ""
	// Automatically resized based on load.
	ResizeCause_ResizeCause_Autoscale ResizeCause_ResizeCause = "AUTOSCALE"
	// User requested a new size.
	ResizeCause_ResizeCause_UserRequest ResizeCause_ResizeCause = "USER_REQUEST"
	// Autorecovery monitor resized the cluster after it lost a nodes.
	ResizeCause_ResizeCause_Autorecovery ResizeCause_ResizeCause = "AUTORECOVERY"
	// Terminate bad nodes and spawn new ones
	ResizeCause_ResizeCause_ReplaceBadNodes ResizeCause_ResizeCause = "REPLACE_BAD_NODES"
	// V2 autoscaler automatically resized based on load (internal use only, events
	// show as AUTOSCALE).
	ResizeCause_ResizeCause_AutoscaleV2 ResizeCause_ResizeCause = "AUTOSCALE_V2"
	// Automatically resized based on decision from the DBR Autoscaler service.
	ResizeCause_ResizeCause_DbrAutoscale ResizeCause_ResizeCause = "DBR_AUTOSCALE"
)

type ResizeClusterRequest

type ResizeClusterRequest struct {
	// The cluster to be resized.
	ClusterId *string
	Size      isResizeClusterRequest_Size
}

type ResizeClusterRequest_Size_Autoscale

type ResizeClusterRequest_Size_Autoscale struct {
	Autoscale AutoScale
}

ResizeClusterRequest_Size_Autoscale selects Autoscale for ResizeClusterRequest.Size. Parameters needed in order to automatically scale clusters up and down based on load. Note: autoscaling works best with DB runtime versions 3.0 or later.

type ResizeClusterRequest_Size_NumWorkers

type ResizeClusterRequest_Size_NumWorkers struct {
	NumWorkers int
}

ResizeClusterRequest_Size_NumWorkers selects NumWorkers for ResizeClusterRequest.Size. Number of worker nodes that this cluster should have. A cluster has one Spark Driver and `num_workers` Executors for a total of `num_workers` + 1 Spark nodes.

Note: When reading the properties of a cluster, this field reflects the desired number of workers rather than the actual current number of workers. For instance, if a cluster is resized from 5 to 10 workers, this field will immediately be updated to reflect the target size of 10 workers, whereas the workers listed in `spark_info` will gradually increase from 5 to 10 as the new nodes are provisioned.

type ResizeClusterResponse

type ResizeClusterResponse struct {
}

type ResizeClusterWaiter

type ResizeClusterWaiter struct {
	// contains filtered or unexported fields
}

ResizeClusterWaiter tracks the state of the operation started by ResizeCluster.

func (*ResizeClusterWaiter) Done

func (w *ResizeClusterWaiter) Done(ctx context.Context, opts ...call.Option) (bool, error)

Done polls once and reports whether the operation has reached a terminal state.

func (*ResizeClusterWaiter) GetClusterId

func (w *ResizeClusterWaiter) GetClusterId() string

GetClusterId returns the ClusterId value used to identify the operation.

func (*ResizeClusterWaiter) Wait

func (w *ResizeClusterWaiter) Wait(ctx context.Context, opts ...lro.Option) (*ClusterInfo, error)

Wait polls until the operation reaches a terminal state.

type RestartClusterRequest

type RestartClusterRequest struct {
	// The cluster to be started.
	ClusterId   *string
	RestartUser *string
}

type RestartClusterResponse

type RestartClusterResponse struct {
}

type RestartClusterWaiter

type RestartClusterWaiter struct {
	// contains filtered or unexported fields
}

RestartClusterWaiter tracks the state of the operation started by RestartCluster.

func (*RestartClusterWaiter) Done

func (w *RestartClusterWaiter) Done(ctx context.Context, opts ...call.Option) (bool, error)

Done polls once and reports whether the operation has reached a terminal state.

func (*RestartClusterWaiter) GetClusterId

func (w *RestartClusterWaiter) GetClusterId() string

GetClusterId returns the ClusterId value used to identify the operation.

func (*RestartClusterWaiter) Wait

func (w *RestartClusterWaiter) Wait(ctx context.Context, opts ...lro.Option) (*ClusterInfo, error)

Wait polls until the operation reaches a terminal state.

type RuntimeEngine

type RuntimeEngine string
const (
	RuntimeEngine_Unspecified RuntimeEngine = ""
	// Use standard engine
	RuntimeEngine_Standard RuntimeEngine = "STANDARD"
	// Use Photon engine
	RuntimeEngine_Photon RuntimeEngine = "PHOTON"
)

type S3StorageInfo

type S3StorageInfo struct {
	// S3 destination, e.g. `s3://my-bucket/some-prefix` Note that logs will be
	// delivered using cluster iam role, please make sure you set cluster iam role
	// and the role has write access to the destination. Please also note that you
	// cannot use AWS keys to deliver logs.
	Destination *string `fieldmask:"destination"`
	// S3 region, e.g. `us-west-2`. Either region or endpoint needs to be set. If
	// both are set, endpoint will be used.
	Region *string `fieldmask:"region"`
	// S3 endpoint, e.g. `https://s3-us-west-2.amazonaws.com`. Either region or
	// endpoint needs to be set. If both are set, endpoint will be used.
	Endpoint *string `fieldmask:"endpoint"`
	// (Optional) Flag to enable server side encryption, `false` by default.
	EnableEncryption *bool `fieldmask:"enable_encryption"`
	// (Optional) The encryption type, it could be `sse-s3` or `sse-kms`. It will be
	// used only when encryption is enabled and the default type is `sse-s3`.
	EncryptionType *string `fieldmask:"encryption_type"`
	// (Optional) Kms key which will be used if encryption is enabled and encryption
	// type is set to `sse-kms`.
	KmsKey *string `fieldmask:"kms_key"`
	// (Optional) Set canned access control list for the logs, e.g.
	// `bucket-owner-full-control`. If `canned_cal` is set, please make sure the
	// cluster iam role has `s3:PutObjectAcl` permission on the destination bucket
	// and prefix. The full list of possible canned acl can be found at
	// http://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl.
	// Please also note that by default only the object owner gets full controls. If
	// you are using cross account role for writing data, you may want to set
	// `bucket-owner-full-control` to make bucket owner able to read the logs.
	CannedAcl *string `fieldmask:"canned_acl"`
}

A storage location in Amazon S3.

type SparkInfo

type SparkInfo struct {
}

Provides information about Spark running inside a cluster. This is used in both the [ClusterInfo] for Cluster APIs and persisted cluster proto..

type SparkInfo_SparkNode

type SparkInfo_SparkNode struct {
	// Private IP address (typically a 10.x.x.x address) of the Spark node. Note
	// that this is different from the private IP address of the host instance.
	PrivateIp *string
	// Public DNS address of this node. This address can be used to access the Spark
	// JDBC server on the driver node. To communicate with the JDBC server, traffic
	// must be manually authorized by adding security group rules to the
	// "worker-unmanaged" security group via the AWS console.
	PublicDns *string
	// Globally unique identifier for this node.
	NodeId *string
	// Globally unique identifier for the host instance from the cloud provider.
	InstanceId *string
	// The timestamp (in millisecond) when the Spark node is launched.
	StartTimestamp *int64
	// Attributes specific to AWS for a Spark node.
	NodeAwsAttributes *SparkInfo_SparkNode_SparkNodeAwsAttributes
	// The private IP address of the host instance.
	HostPrivateIp *string
}

Describes a specific Spark driver or executor..

type SparkInfo_SparkNode_SparkNodeAwsAttributes

type SparkInfo_SparkNode_SparkNodeAwsAttributes struct {
	// Whether this node is on an Amazon spot instance.
	IsSpot *bool
}

Attributes specific to AWS for a Spark node..

type SparkVersion

type SparkVersion struct {
	// Spark version key, for example "2.1.x-scala2.11". This is the value which
	// should be provided as the "spark_version" when creating a new cluster. Note
	// that the exact Spark version may change over time for a "wildcard" version
	// (i.e., "2.1.x-scala2.11" is a "wildcard" version) with minor bug fixes.
	Key *string
	// A descriptive name for this Spark version, for example "Spark 2.1".
	Name *string
}

type StartClusterRequest

type StartClusterRequest struct {
	// The cluster to be started.
	ClusterId *string
}

type StartClusterResponse

type StartClusterResponse struct {
}

type StartClusterWaiter

type StartClusterWaiter struct {
	// contains filtered or unexported fields
}

StartClusterWaiter tracks the state of the operation started by StartCluster.

func (*StartClusterWaiter) Done

func (w *StartClusterWaiter) Done(ctx context.Context, opts ...call.Option) (bool, error)

Done polls once and reports whether the operation has reached a terminal state.

func (*StartClusterWaiter) GetClusterId

func (w *StartClusterWaiter) GetClusterId() string

GetClusterId returns the ClusterId value used to identify the operation.

func (*StartClusterWaiter) Wait

func (w *StartClusterWaiter) Wait(ctx context.Context, opts ...lro.Option) (*ClusterInfo, error)

Wait polls until the operation reaches a terminal state.

type TerminationCode

type TerminationCode string

The status code indicating why the cluster was terminated

const (
	TerminationCode_Unspecified TerminationCode = ""
	// A user terminated the cluster directly. Parameters should include a
	// “username“ field that indicates the specific user who terminated the
	// cluster.
	TerminationCode_UserRequest TerminationCode = "USER_REQUEST"
	// This cluster was launched by a Job, and terminated when the Job completed.
	TerminationCode_JobFinished TerminationCode = "JOB_FINISHED"
	// This cluster was terminated since it was idle.
	TerminationCode_Inactivity TerminationCode = "INACTIVITY"
	// The instance that hosted the spark driver was terminated by the cloud
	// provider. In AWS, for example, AWS may retire instances and directly shut
	// them down. Parameters should include an “aws_instance_state_reason“ field
	// indicating the AWS-provided reason why the instance was terminated.
	TerminationCode_CloudProviderShutdown TerminationCode = "CLOUD_PROVIDER_SHUTDOWN"
	// Databricks may lose connection to services on the driver instance. One such
	// case is when problems arise in cloud networking infrastructure, or when the
	// instance itself becomes unhealthy.
	TerminationCode_CommunicationLost TerminationCode = "COMMUNICATION_LOST"
	// Databricks may hit cloud provider failures when requesting instances to
	// launch clusters. For example, AWS limits the number of running instances and
	// EBS volumes. If you ask Databricks to launch a cluster that requires
	// instances or EBS volumes that exceed your AWS limit, the cluster will fail
	// with this status code. Parameters should include one of
	// “aws_api_error_code“, “aws_instance_state_reason“, or
	// “aws_spot_request_status“ to indicate the AWS-provided reason why
	// Databricks could not request the required instances for the cluster.
	TerminationCode_CloudProviderLaunchFailure TerminationCode = "CLOUD_PROVIDER_LAUNCH_FAILURE"
	// Databricks cannot load and execute a cluster-scoped init script on one of the
	// cluster's nodes, or the init script terminates with a non-zero exit code or
	// there was a general failure during the loading/executing of init scripts that
	// does not pertain to any specific script.
	TerminationCode_InitScriptFailure TerminationCode = "INIT_SCRIPT_FAILURE"
	// The Spark driver failed to start. Possible reasons may include incompatible
	// libraries and initialization scripts that corrupted the Spark container.
	TerminationCode_SparkStartupFailure TerminationCode = "SPARK_STARTUP_FAILURE"
	// Cannot launch the cluster because the user specified an invalid argument. For
	// example, the use might specify an invalid spark version for the cluster.
	TerminationCode_InvalidArgument TerminationCode = "INVALID_ARGUMENT"
	// While launching this cluster, Databricks failed to complete critical setup
	// steps, terminating the cluster.
	TerminationCode_UnexpectedLaunchFailure TerminationCode = "UNEXPECTED_LAUNCH_FAILURE"
	// Databricks encountered an unexpected error which forced the running cluster
	// to be terminated. Please contact Databricks support for additional details.
	TerminationCode_InternalError TerminationCode = "INTERNAL_ERROR"
	// Databricks was not able to access instances in order to start the cluster.
	// This can be a transient networking issue. If the problem persists, this
	// usually indicates a networking environment misconfiguration.
	TerminationCode_InstanceUnreachable TerminationCode = "INSTANCE_UNREACHABLE"
	// Blocked upsize requests for the workspace according to
	// https://databricks.atlassian.net/wiki/spaces/UN/pages/934088320/Banning+Workspace+Upsize+Runbook
	TerminationCode_RequestRejected TerminationCode = "REQUEST_REJECTED"
	// The cluster was terminated because it was running in a trial workspace that
	// expired.
	TerminationCode_TrialExpired TerminationCode = "TRIAL_EXPIRED"
	// The cluster was terminated because no response from the chauffeur could be
	// received. We name this "DRIVER_" instead of "CHAUFFEUR_" since chauffeur is
	// non-external terminology
	TerminationCode_DriverUnreachable TerminationCode = "DRIVER_UNREACHABLE"
	// Spark error on startup
	TerminationCode_SparkError TerminationCode = "SPARK_ERROR"
	// Driver unresponsive
	TerminationCode_DriverUnresponsive TerminationCode = "DRIVER_UNRESPONSIVE"
	// Metastore component unhealthy
	TerminationCode_MetastoreComponentUnhealthy TerminationCode = "METASTORE_COMPONENT_UNHEALTHY"
	// DBFS component unhealthy
	TerminationCode_DbfsComponentUnhealthy TerminationCode = "DBFS_COMPONENT_UNHEALTHY"
	// Execution component unhealthy
	TerminationCode_ExecutionComponentUnhealthy TerminationCode = "EXECUTION_COMPONENT_UNHEALTHY"
	// Databricks may hit the azure resource manager request limit. Which will keep
	// the Azure SDK from issuing any read or write request to Azure resource
	// manager. The request limit is applied to each subscription every hour, thus
	// retry after an hour or changing to a smaller cluster size might help to
	// resolve the issue. Please check the following link for more information:
	// https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-manager-request-limits
	TerminationCode_AzureResourceManagerThrottling TerminationCode = "AZURE_RESOURCE_MANAGER_THROTTLING"
	// Databricks may hit the azure resource provider request limit. Specifically,
	// the API request rate to the specific resource type (Compute, Network, etc..)
	// can't exceed the limit. Retry might help to resolve the issue. Please check
	// the following link for more information:
	// https://docs.microsoft.com/en-us/azure/virtual-machines/troubleshooting/
	// troubleshooting-throttling-errors
	TerminationCode_AzureResourceProviderThrottling TerminationCode = "AZURE_RESOURCE_PROVIDER_THROTTLING"
	// The cluster was terminated due to an error in the network configuration.
	TerminationCode_NetworkConfigurationFailure TerminationCode = "NETWORK_CONFIGURATION_FAILURE"
	// Databricks encountered an unexpected error while launching containers on
	// worker nodes for the cluster, terminating the cluster.
	TerminationCode_ContainerLaunchFailure TerminationCode = "CONTAINER_LAUNCH_FAILURE"
	// Instance pool backed cluster specific failure
	TerminationCode_InstancePoolClusterFailure TerminationCode = "INSTANCE_POOL_CLUSTER_FAILURE"
	// Cluster start successfully completed but skipped some instances which were
	// slow to launch
	TerminationCode_SkippedSlowNodes TerminationCode = "SKIPPED_SLOW_NODES"
	// Attach projects failure
	TerminationCode_AttachProjectFailure TerminationCode = "ATTACH_PROJECT_FAILURE"
	// Attach projects failure
	TerminationCode_UpdateInstanceProfileFailure TerminationCode = "UPDATE_INSTANCE_PROFILE_FAILURE"
	// Cluster terminated due to database failure
	TerminationCode_DatabaseConnectionFailure TerminationCode = "DATABASE_CONNECTION_FAILURE"
	// Databricks cannot handle the request at this moment. Please try again later
	// and contact Databricks if the problem persists.
	TerminationCode_RequestThrottled TerminationCode = "REQUEST_THROTTLED"
	// SelfBootstrap failure. Either self-bootstrap fast fail or node daemon ping
	// timeout
	TerminationCode_SelfBootstrapFailure TerminationCode = "SELF_BOOTSTRAP_FAILURE"
	// Databricks cannot load and execute a global init script on one of the
	// cluster's nodes, or the init script terminates with a non-zero exit code.
	TerminationCode_GlobalInitScriptFailure TerminationCode = "GLOBAL_INIT_SCRIPT_FAILURE"
	// Container launch timed out downloading the spark image. This can happen if
	// the customer has byo-vpc/vnet and the download of large files is being
	// throttled.
	TerminationCode_SlowImageDownload TerminationCode = "SLOW_IMAGE_DOWNLOAD"
	// Container setup failed due to an invalid Spark image.
	TerminationCode_InvalidSparkImage TerminationCode = "INVALID_SPARK_IMAGE"
	// If the ngrok tunnel token provisioning fails for any reason, for example
	// hitting the max capacity of allowed ngrok tokens. (ES-32083)
	TerminationCode_NpipTunnelTokenFailure TerminationCode = "NPIP_TUNNEL_TOKEN_FAILURE"
	// Hive Metastore provisioning failue in launch container step
	TerminationCode_HiveMetastoreProvisioningFailure TerminationCode = "HIVE_METASTORE_PROVISIONING_FAILURE"
	// Occurs when the deployment template we submit to Azure violates their
	// requirements. Typical scenarios: - Wrong parameter key/value used - Exceed
	// the limit for certain parameter
	TerminationCode_AzureInvalidDeploymentTemplate TerminationCode = "AZURE_INVALID_DEPLOYMENT_TEMPLATE"
	// The set of un-categorized failure responses from Azure when we launch
	// instance resources using deployment template
	TerminationCode_AzureUnexpectedDeploymentTemplateFailure TerminationCode = "AZURE_UNEXPECTED_DEPLOYMENT_TEMPLATE_FAILURE"
	// Subnet (typically Azure vnet injected) has run out of ip addresses
	TerminationCode_SubnetExhaustedFailure TerminationCode = "SUBNET_EXHAUSTED_FAILURE"
	// Timeout to ping the nodeDaemon, possible reason: nodeDaemon didn't start
	// (configuration issue), network connectivity issue
	TerminationCode_BootstrapTimeout TerminationCode = "BOOTSTRAP_TIMEOUT"
	// Bootstrap timeout due to script download failure
	TerminationCode_StorageDownloadFailure TerminationCode = "STORAGE_DOWNLOAD_FAILURE"
	// Bootstrap timeout due to get runbook failure
	TerminationCode_ControlPlaneRequestFailure TerminationCode = "CONTROL_PLANE_REQUEST_FAILURE"
	// Bootstrap timeout due to Azure Extension Service Failure
	TerminationCode_BootstrapTimeoutCloudProviderException TerminationCode = "BOOTSTRAP_TIMEOUT_CLOUD_PROVIDER_EXCEPTION"
	// Could not find enough of the requested instance type in the requested AZ.
	// Often related to Auto AZ.
	TerminationCode_AwsInsufficientInstanceCapacityFailure TerminationCode = "AWS_INSUFFICIENT_INSTANCE_CAPACITY_FAILURE"
	// Container setup failure due to docker image pulling failure
	TerminationCode_DockerImagePullFailure TerminationCode = "DOCKER_IMAGE_PULL_FAILURE"
	// Failures during azure vnet configuration. For example, a workspace with VNet
	// injection had incorrect DNS settings that blocked access to worker artifacts.
	TerminationCode_AzureVnetConfigurationFailure TerminationCode = "AZURE_VNET_CONFIGURATION_FAILURE"
	// Bootstrap failure due to Ngrok tunnel setup timeout or failure. For example,
	// if the worker node is unable to reach the Ngrok tunnel domain.
	TerminationCode_NpipTunnelSetupFailure TerminationCode = "NPIP_TUNNEL_SETUP_FAILURE"
	// Lack authorization for cluster operation. For example, awsApiErrorCode:
	// 'AccessDenied' or 'UnauthorizedOperation'.
	TerminationCode_AwsAuthorizationFailure TerminationCode = "AWS_AUTHORIZATION_FAILURE"
	// request comes form Nephos resource pool auto management
	TerminationCode_NephosResourceManagement TerminationCode = "NEPHOS_RESOURCE_MANAGEMENT"
	// Container setup failed during container registration to security daemon due
	// to STS endpoint connection error.
	TerminationCode_StsClientSetupFailure TerminationCode = "STS_CLIENT_SETUP_FAILURE"
	// Container setup failed during registration to security daemon due to an
	// unspecified error.
	TerminationCode_SecurityDaemonRegistrationException TerminationCode = "SECURITY_DAEMON_REGISTRATION_EXCEPTION"
	// The maximum request rate permitted by the Amazon EC2 APIs has been exceeded
	// for your account.
	TerminationCode_AwsRequestLimitExceeded TerminationCode = "AWS_REQUEST_LIMIT_EXCEEDED"
	// We don't have enough addresses in the subnet for the instances in the
	// request.
	TerminationCode_AwsInsufficientFreeAddressesInSubnetFailure TerminationCode = "AWS_INSUFFICIENT_FREE_ADDRESSES_IN_SUBNET_FAILURE"
	// The request is not supported (This is a vague error code that can be thrown
	// for a lot of reasons.)
	TerminationCode_AwsUnsupportedFailure TerminationCode = "AWS_UNSUPPORTED_FAILURE"
	// Could not find enough azure resources to fulfill the request.
	TerminationCode_AzureQuotaExceededException TerminationCode = "AZURE_QUOTA_EXCEEDED_EXCEPTION"
	// NOTE: This is currently used by exceptions with messages that are classified
	// as user errors.
	TerminationCode_AzureOperationNotAllowedException TerminationCode = "AZURE_OPERATION_NOT_ALLOWED_EXCEPTION"
	// Failure when mounting remote NFS to container
	TerminationCode_NfsMountFailure TerminationCode = "NFS_MOUNT_FAILURE"
	// K8S failed to upscale to acquire new nodes
	TerminationCode_K8sAutoscalingFailure TerminationCode = "K8S_AUTOSCALING_FAILURE"
	// DBR Cluster launched on K8s (i.e. CMv2) has failed to start up in time
	TerminationCode_K8sDbrClusterLaunchTimeout TerminationCode = "K8S_DBR_CLUSTER_LAUNCH_TIMEOUT"
	// Container launch failed while downloading the spark image. Catch all for if
	// anything goes wrong while downloading and extracting the spark tarball.
	TerminationCode_SparkImageDownloadFailure TerminationCode = "SPARK_IMAGE_DOWNLOAD_FAILURE"
	// Azure VM Extension failure during instance bootstrap
	TerminationCode_AzureVmExtensionFailure TerminationCode = "AZURE_VM_EXTENSION_FAILURE"
	// Workspace was cancelled hence deny/terminate the cluster
	TerminationCode_WorkspaceCancelledError TerminationCode = "WORKSPACE_CANCELLED_ERROR"
	// The spot instance count in an account has exceeded the limit
	TerminationCode_AwsMaxSpotInstanceCountExceededFailure TerminationCode = "AWS_MAX_SPOT_INSTANCE_COUNT_EXCEEDED_FAILURE"
	// Cluster is terminated because the services are temporarily unavailable. This
	// normally happens when CM is restarting and draining execution contexts, or
	// IM/Delegate is overloaded, so that it will not be able to retry the instance
	// launch request.
	TerminationCode_TemporarilyUnavailable TerminationCode = "TEMPORARILY_UNAVAILABLE"
	// Bootstrap failure due to error during worker setup, usually due to an issue
	// with disk or gpu setup. See SetupCommandBuilder for other possible causes
	TerminationCode_WorkerSetupFailure TerminationCode = "WORKER_SETUP_FAILURE"
	// Cluster failure due to IP space exhaustion. For example on CMv2, Kubernetes
	// will fail to scale up new nodes if the pod IP CIDR block is exhausted.
	TerminationCode_IpExhaustionFailure TerminationCode = "IP_EXHAUSTION_FAILURE"
	// Could not find enough GCP resources to fulfill the request. TODO: It's very
	// unfortunate that we have per-cloud termination reasons while we should have
	// cloud-agnostic termination reasons. For example, we should consolidate
	// {AZURE_QUOTA_EXCEEDED_EXCEPTION, AWS_REQUEST_LIMIT_EXCEEDED and
	// GCP_QUOTA_EXCEEDED}, {AWS_INSUFFICIENT_FREE_ADDRESSES_IN_SUBNET_FAILURE,
	// IP_EXHAUSTION_FAILURE}, etc.
	TerminationCode_GcpQuotaExceeded TerminationCode = "GCP_QUOTA_EXCEEDED"
	// Cloud provider is undergoing a transient resource throttling. This is
	// retryable.
	TerminationCode_CloudProviderResourceStockout TerminationCode = "CLOUD_PROVIDER_RESOURCE_STOCKOUT"
	// The GCP service account associated with the DBR cluster is deleted.
	TerminationCode_GcpServiceAccountDeleted TerminationCode = "GCP_SERVICE_ACCOUNT_DELETED"
	// Legit cluster termination in Azure caused by customer revoking the key
	// permission used for managed-disks encryption
	TerminationCode_AzureByokKeyPermissionFailure TerminationCode = "AZURE_BYOK_KEY_PERMISSION_FAILURE"
	// Termination because of spot instance terminated by cloud provider
	TerminationCode_SpotInstanceTermination TerminationCode = "SPOT_INSTANCE_TERMINATION"
	// Termination because of unsupported azure ephemeral os disk setup
	TerminationCode_AzureEphemeralDiskFailure TerminationCode = "AZURE_EPHEMERAL_DISK_FAILURE"
	// The cluster was terminated because we detected an abusive runtime behavior
	// that violated Terms of Service or Acceptable Use Policy.
	TerminationCode_AbuseDetected TerminationCode = "ABUSE_DETECTED"
	// Failed to pull DBR images due to permission error.
	TerminationCode_ImagePullPermissionDenied TerminationCode = "IMAGE_PULL_PERMISSION_DENIED"
	// Workspace configuration is in error state due to configuration issue or ACL
	// modification by the customer side
	TerminationCode_WorkspaceConfigurationError TerminationCode = "WORKSPACE_CONFIGURATION_ERROR"
	// Catch all error for all secret resolution issues in cluster launch. This
	// should be alerted on, and is considered a server error. This can be split out
	// into other cases if there are client errors - for e.g. INVALID_ARGUMENT is
	// used for secrets that don't exist and permission issues
	TerminationCode_SecretResolutionError TerminationCode = "SECRET_RESOLUTION_ERROR"
	// Failure due to an instance being of an unsupported type. This is used when an
	// instance in an EC2 fleet is of an unrecognized type, or an invalid type (i.e.
	// graviton when we don't want graviton instances). This should be alerted on.
	TerminationCode_UnsupportedInstanceType TerminationCode = "UNSUPPORTED_INSTANCE_TYPE"
	// Failed during instance bootstrap with error code Cannot convert NVMe-based
	// dev id
	TerminationCode_CloudProviderDiskSetupFailure TerminationCode = "CLOUD_PROVIDER_DISK_SETUP_FAILURE"
	// Exception when setting up instances using ssh bootstrap
	TerminationCode_SshBootstrapFailure TerminationCode = "SSH_BOOTSTRAP_FAILURE"
	// Failed during instance bootstrap with error code Cannot convert NVMe-based
	// dev id
	TerminationCode_AwsInaccessibleKmsKeyFailure TerminationCode = "AWS_INACCESSIBLE_KMS_KEY_FAILURE"
	// The bootstrapping init-containers in Spark failed or timed out, blocking the
	// Spark container from bootstrapping. This is a refinement of
	// `SPARK_STARTUP_FAILURE`. (init-containers are a bootstrapping step owned by
	// Databricks)
	TerminationCode_InitContainerNotFinished TerminationCode = "INIT_CONTAINER_NOT_FINISHED"
	// Container launch failed due to storage servers throttling our download of
	// spark images. Can happen due to transient spikes of downloads overloading
	// storage servers or gradual increase in usage. In the latter case we need to
	// increase the number of storage servers in the region to help spread load.
	TerminationCode_SparkImageDownloadThrottled TerminationCode = "SPARK_IMAGE_DOWNLOAD_THROTTLED"
	// The spark image specified for the cluster was not found when attempting to
	// download. Usually due to the customer custom specifying a bad image.
	TerminationCode_SparkImageNotFound TerminationCode = "SPARK_IMAGE_NOT_FOUND"
	// Indicates that the cloud provider operations performed for the cluster were
	// dropped due to an influx in load in the cloud provider and had to be dropped
	// from our end to alleviate pressure within the DelegateRpcClient. Please see
	// go/cmloadshedding for more.
	TerminationCode_ClusterOperationThrottled TerminationCode = "CLUSTER_OPERATION_THROTTLED"
	// The error code can be used to indicate a request misses its deadline. Can be
	// used for either request timeouts or missed deadlines (i.e. a request is not
	// completed as it was processed after its specified deadline)
	TerminationCode_ClusterOperationTimeout TerminationCode = "CLUSTER_OPERATION_TIMEOUT"
	// This error code is used to terminate long-running Generic compute jobs in
	// Serverless Environment as part of the NephosLongRunning watcher running in
	// Cluster Monitor Service.
	TerminationCode_ServerlessLongRunningTerminated TerminationCode = "SERVERLESS_LONG_RUNNING_TERMINATED"
	// This error code is used when the cluster is terminated due to its instances
	// fail with partial failure from Azure packed deployments. In Azure, we might
	// pack multiple launch requests in one deployment template in order to avoid
	// the 800 templates limit on Azure side. If the packed deployment fails
	// multiple times, the cluster could be terminated by this
	// [[AZURE_PACKED_DEPLOYMENT_PARTIAL_FAILURE]] termination code.
	TerminationCode_AzurePackedDeploymentPartialFailure TerminationCode = "AZURE_PACKED_DEPLOYMENT_PARTIAL_FAILURE"
	// The instances acquired from a pool in IMv2 do not have a valid worker image
	// to be used in the cluster launch. This usually occurs after AMI/VHD upgrades,
	// worker branch updates, etc.
	TerminationCode_InvalidWorkerImageFailure TerminationCode = "INVALID_WORKER_IMAGE_FAILURE"
	// Worker environment version was changed due to workspace network or CMK
	// update.
	TerminationCode_WorkspaceUpdate TerminationCode = "WORKSPACE_UPDATE"
	// The parameter user specified or the user account to create the cluster is
	// invalid according to AWS.
	TerminationCode_InvalidAwsParameter TerminationCode = "INVALID_AWS_PARAMETER"
	// ** Only relevant on k8s dataplanes (i.e. clusters launched with CMv2 - not
	// CMv1).
	//
	// k8s evicted the driver pod due to disk pressure on the driver node. This is
	// likely due to a customer job consuming too much disk and so this is
	// classified as a customer issue.
	TerminationCode_DriverOutOfDisk TerminationCode = "DRIVER_OUT_OF_DISK"
	// ** Only relevant on k8s dataplanes (i.e. clusters launched with CMv2 - not
	// CMv1).
	//
	// k8s evicted the driver pod due to memory pressure on the driver node. A
	// customer job consuming significant amounts of memory should not be able to
	// trigger this as the driver container would OOM first (we set memory limits on
	// our pods). Thus this termination reason will be considered a databricks
	// issue.
	TerminationCode_DriverOutOfMemory TerminationCode = "DRIVER_OUT_OF_MEMORY"
	// ** Only relevant on k8s dataplanes (i.e. clusters launched with CMv2 - not
	// CMv1). Original driver pod took too long to become ready and timed out.
	TerminationCode_DriverLaunchTimeout TerminationCode = "DRIVER_LAUNCH_TIMEOUT"
	// ** Only relevant on k8s dataplanes (i.e. clusters launched with CMv2 - not
	// CMv1). Unexpected failure during driver pod launch.
	TerminationCode_DriverUnexpectedFailure TerminationCode = "DRIVER_UNEXPECTED_FAILURE"
	// ** Only relevant on k8s dataplanes (i.e. clusters launched with CMv2 - not
	// CMv1). Unexpected new driver pod created
	TerminationCode_UnexpectedPodRecreation TerminationCode = "UNEXPECTED_POD_RECREATION"
	// Failure due to disabled or inaccessible CMK.
	TerminationCode_GcpInaccessibleKmsKeyFailure TerminationCode = "GCP_INACCESSIBLE_KMS_KEY_FAILURE"
	// Failure due to missing/incorrect permission setup on CMK.
	TerminationCode_GcpKmsKeyPermissionDenied TerminationCode = "GCP_KMS_KEY_PERMISSION_DENIED"
	// Driver pod evicted in Nephos
	TerminationCode_DriverEviction TerminationCode = "DRIVER_EVICTION"
	// User request for termination directly to cloud
	TerminationCode_UserInitiatedVmTermination TerminationCode = "USER_INITIATED_VM_TERMINATION"
	// GCP Specific IAM API timeout issues during Workload Idenitity (Cluster
	// Identity) binding process
	TerminationCode_GcpIamTimeout TerminationCode = "GCP_IAM_TIMEOUT"
	// Could not find enough AWS resources to fulfill the request
	TerminationCode_AwsResourceQuotaExceeded TerminationCode = "AWS_RESOURCE_QUOTA_EXCEEDED"
	// Cloud account setup has some error (e.g. pending email verification, blocked)
	TerminationCode_CloudAccountSetupFailure TerminationCode = "CLOUD_ACCOUNT_SETUP_FAILURE"
	// The specified key pair name does not exist.
	TerminationCode_AwsInvalidKeyPair TerminationCode = "AWS_INVALID_KEY_PAIR"
	// Driver pod creation failure in nephos
	TerminationCode_DriverPodCreationFailure TerminationCode = "DRIVER_POD_CREATION_FAILURE"
	// Cluster terminated manually by on-call due to emergency maintenance
	TerminationCode_MaintenanceMode TerminationCode = "MAINTENANCE_MODE"
	// Nephos internal error due to insufficient provisioned k8s capacity or
	// insufficient cloud quota
	TerminationCode_InternalCapacityFailure TerminationCode = "INTERNAL_CAPACITY_FAILURE"
	// Nephos: could not acquire executor pods from pod pool
	TerminationCode_ExecutorPodUnscheduled TerminationCode = "EXECUTOR_POD_UNSCHEDULED"
	// Artifact download failed because it was too slow
	TerminationCode_StorageDownloadFailureSlow TerminationCode = "STORAGE_DOWNLOAD_FAILURE_SLOW"
	// Artifact download failed because it was throttled by the download server
	TerminationCode_StorageDownloadFailureThrottled TerminationCode = "STORAGE_DOWNLOAD_FAILURE_THROTTLED"
	// The cluster was terminated because the size of the dynamic spark conf
	// exceeded the limit.
	TerminationCode_DynamicSparkConfSizeExceeded TerminationCode = "DYNAMIC_SPARK_CONF_SIZE_EXCEEDED"
	// Failure to update the instance profile for the cluster.
	TerminationCode_AwsInstanceProfileUpdateFailure TerminationCode = "AWS_INSTANCE_PROFILE_UPDATE_FAILURE"
	// The instance pool did not exist when the cluster was launched.
	TerminationCode_InstancePoolNotFound TerminationCode = "INSTANCE_POOL_NOT_FOUND"
	// Attempting to launch more instances was rejected as it would exceed the
	// pool's max capacity.
	TerminationCode_InstancePoolMaxCapacityReached TerminationCode = "INSTANCE_POOL_MAX_CAPACITY_REACHED"
	// The KMS key provided is in an incorrect state.
	TerminationCode_AwsInvalidKmsKeyState TerminationCode = "AWS_INVALID_KMS_KEY_STATE"
	// Insufficient capacity failure from GCE API.
	TerminationCode_GcpInsufficientCapacity TerminationCode = "GCP_INSUFFICIENT_CAPACITY"
	// Rate quota exceeded for GCP API (e.g. Read requests per minute per region).
	TerminationCode_GcpApiRateQuotaExceeded TerminationCode = "GCP_API_RATE_QUOTA_EXCEEDED"
	// Resource quota exceeded (e.g. # of n1 vCPUs in a region).
	TerminationCode_GcpResourceQuotaExceeded TerminationCode = "GCP_RESOURCE_QUOTA_EXCEEDED"
	// Subnet IP space exhausted.
	TerminationCode_GcpIpSpaceExhausted TerminationCode = "GCP_IP_SPACE_EXHAUSTED"
	// Missing permissions to launch VM with service account.
	TerminationCode_GcpServiceAccountAccessDenied TerminationCode = "GCP_SERVICE_ACCOUNT_ACCESS_DENIED"
	// VM attempting to launch with non-existent service account.
	TerminationCode_GcpServiceAccountNotFound TerminationCode = "GCP_SERVICE_ACCOUNT_NOT_FOUND"
	// Forbidden (403) returned by GCP API.
	TerminationCode_GcpForbidden TerminationCode = "GCP_FORBIDDEN"
	// Not found (404) returned by GCP API.
	TerminationCode_GcpNotFound TerminationCode = "GCP_NOT_FOUND"
	// Gatekeeper indicated the cluster should be shutdown
	TerminationCode_ResourceUsageBlocked TerminationCode = "RESOURCE_USAGE_BLOCKED"
	// The data access config of the workspace has changed, and clusters using
	// outdated config will be terminated.
	TerminationCode_DataAccessConfigChanged TerminationCode = "DATA_ACCESS_CONFIG_CHANGED"
	// Failed to fetch internal PAT token required for init script installation from
	// WSFS/UC volumes
	TerminationCode_AccessTokenFailure TerminationCode = "ACCESS_TOKEN_FAILURE"
	// It indicates there is a placement v2 protocol rollout/rollback event for the
	// corresponding workspace when processing the placement session on the
	// instance-manager side. A retry will fix the issue by switching back to the
	// correct placement protocol.
	TerminationCode_InvalidInstancePlacementProtocol TerminationCode = "INVALID_INSTANCE_PLACEMENT_PROTOCOL"
	// The cluster was terminated as it failed to resolve budget policy.
	TerminationCode_BudgetPolicyResolutionFailure TerminationCode = "BUDGET_POLICY_RESOLUTION_FAILURE"
	// This customer/error combination is a known issue and is intentionally
	// excluded from termination metrics
	TerminationCode_InPenaltyBox TerminationCode = "IN_PENALTY_BOX"
	// The cluster was terminated when the primary workspace failed over to the
	// secondary workspace. This is expected because there is no data plane in the
	// secondary workspace.
	TerminationCode_DisasterRecoveryReplication TerminationCode = "DISASTER_RECOVERY_REPLICATION"
	// A bootstrap timeout that was caused by misconfiguration on the customer's
	// side
	TerminationCode_BootstrapTimeoutDueToMisconfig TerminationCode = "BOOTSTRAP_TIMEOUT_DUE_TO_MISCONFIG"
	// Instance unreachable, but due to misconfiguration on the customer's side
	TerminationCode_InstanceUnreachableDueToMisconfig TerminationCode = "INSTANCE_UNREACHABLE_DUE_TO_MISCONFIG"
	// Bootstrap timeout due to script download failure, but due to misconfiguration
	// on the customer's side
	TerminationCode_StorageDownloadFailureDueToMisconfig TerminationCode = "STORAGE_DOWNLOAD_FAILURE_DUE_TO_MISCONFIG"
	// CPRF, but due to misconfiguration on the customer's side
	TerminationCode_ControlPlaneRequestFailureDueToMisconfig TerminationCode = "CONTROL_PLANE_REQUEST_FAILURE_DUE_TO_MISCONFIG"
	// CPLF, but due to misconfiguration on the customer's side
	TerminationCode_CloudProviderLaunchFailureDueToMisconfig TerminationCode = "CLOUD_PROVIDER_LAUNCH_FAILURE_DUE_TO_MISCONFIG"
	// GCP subnet is in transient "resourceNotReady" state.
	TerminationCode_GcpSubnetNotReady TerminationCode = "GCP_SUBNET_NOT_READY"
	// The operation on the cloud provider was cancelled. Possibly due to a user
	// action.
	TerminationCode_CloudOperationCancelled TerminationCode = "CLOUD_OPERATION_CANCELLED"
	// If cloud provider indicates instance creation was a success, yet the instance
	// is never created. This can happen in certain edge cases like quota exhaustion
	// on GCP. We have an open bug here:
	// https://partnerissuetracker.corp.google.com/issues/339061883
	TerminationCode_CloudProviderInstanceNotLaunched TerminationCode = "CLOUD_PROVIDER_INSTANCE_NOT_LAUNCHED"
	// GCP Databricks VM Machine Image is blocked by customer organization policy.
	TerminationCode_GcpTrustedImageProjectsViolated TerminationCode = "GCP_TRUSTED_IMAGE_PROJECTS_VIOLATED"
	// cluster terminate can happened when a budget policy limit enforcement
	// activated
	TerminationCode_BudgetPolicyLimitEnforcementActivated TerminationCode = "BUDGET_POLICY_LIMIT_ENFORCEMENT_ACTIVATED"
	TerminationCode_EosSparkImage                         TerminationCode = "EOS_SPARK_IMAGE"
	// Serverless only. There are no eligible K8s for the cluster.
	TerminationCode_NoMatchedK8s TerminationCode = "NO_MATCHED_K8S"
	// Lazy allocation timeout. Timeout before any internal DBR clusters were
	// allocated.
	TerminationCode_LazyAllocationTimeout TerminationCode = "LAZY_ALLOCATION_TIMEOUT"
	// CMv2 unable to contact chauffeur or node-daemon on the driver node.
	TerminationCode_DriverNodeUnreachable TerminationCode = "DRIVER_NODE_UNREACHABLE"
	// Dynamic secret generation failed.
	TerminationCode_SecretCreationFailure TerminationCode = "SECRET_CREATION_FAILURE"
	// Driver or executor pod failed to be scheduled.
	TerminationCode_PodSchedulingFailure TerminationCode = "POD_SCHEDULING_FAILURE"
	// Driver or executor pod failed to finish assigning.
	TerminationCode_PodAssignmentFailure TerminationCode = "POD_ASSIGNMENT_FAILURE"
	// Lazy allocation timeout with unknown reason.
	TerminationCode_AllocationTimeout TerminationCode = "ALLOCATION_TIMEOUT"
	// Lazy allocation timeout. Maps to NoUnallocatedDbrCluster.
	TerminationCode_AllocationTimeoutNoUnallocatedClusters TerminationCode = "ALLOCATION_TIMEOUT_NO_UNALLOCATED_CLUSTERS"
	// Lazy allocation timeout. Maps to NoMatchedUnallocatedDbrCluster.
	TerminationCode_AllocationTimeoutNoMatchedClusters TerminationCode = "ALLOCATION_TIMEOUT_NO_MATCHED_CLUSTERS"
	// Lazy allocation timeout. Maps to NoUnallocatedReadyDbrCluster.
	TerminationCode_AllocationTimeoutNoReadyClusters TerminationCode = "ALLOCATION_TIMEOUT_NO_READY_CLUSTERS"
	// Lazy allocation timeout. Maps to NoMatchedUnallocatedWarmedUpDbrCluster.
	TerminationCode_AllocationTimeoutNoWarmedUpClusters TerminationCode = "ALLOCATION_TIMEOUT_NO_WARMED_UP_CLUSTERS"
	// Lazy allocation timeout. Maps to NoCandidatesWithNodeDaemonK8sReady.
	TerminationCode_AllocationTimeoutNodeDaemonNotReady TerminationCode = "ALLOCATION_TIMEOUT_NODE_DAEMON_NOT_READY"
	// Lazy allocation timeout. Maps to NoCandidatesHealthy.
	TerminationCode_AllocationTimeoutNoHealthyClusters TerminationCode = "ALLOCATION_TIMEOUT_NO_HEALTHY_CLUSTERS"
	// When nephos blocking wait for netvisor setup ready signal, terminated by
	// timeout. This error code only applies to clusters with the attribute
	// should_block_for_network_readiness: true
	TerminationCode_NetvisorSetupTimeout TerminationCode = "NETVISOR_SETUP_TIMEOUT"
	// Serverless only. The preselected K8s for the cluster is not eligible.
	TerminationCode_NoMatchedK8sTestingTag TerminationCode = "NO_MATCHED_K8S_TESTING_TAG"
	// The customer's repeatedly attempting to launch clusters with some
	// configuration that the CSP's not able to provide
	TerminationCode_CloudProviderResourceStockoutDueToMisconfig TerminationCode = "CLOUD_PROVIDER_RESOURCE_STOCKOUT_DUE_TO_MISCONFIG"
	// For the GCP CMv1 Migration, we will terminate all CMv2 based clusters with
	// this failure.
	TerminationCode_GkeBasedClusterTermination TerminationCode = "GKE_BASED_CLUSTER_TERMINATION"
	// Lazy allocation timeout. Maps to NoCandidatesHealthyAndWarmedUp.
	TerminationCode_AllocationTimeoutNoHealthyAndWarmedUpClusters TerminationCode = "ALLOCATION_TIMEOUT_NO_HEALTHY_AND_WARMED_UP_CLUSTERS"
	// Docker container's OS was not valid.
	TerminationCode_DockerInvalidOsException TerminationCode = "DOCKER_INVALID_OS_EXCEPTION"
	// Something went wrong during the creation of the docker container.
	TerminationCode_DockerContainerCreationException TerminationCode = "DOCKER_CONTAINER_CREATION_EXCEPTION"
	// Customer passed in a docker image that's too large for the instance.
	TerminationCode_DockerImageTooLargeForInstanceException TerminationCode = "DOCKER_IMAGE_TOO_LARGE_FOR_INSTANCE_EXCEPTION"
	// The cluster was terminated because the DNS resolution failed.
	TerminationCode_DnsResolutionError TerminationCode = "DNS_RESOLUTION_ERROR"
	// Org policy is preventing a GCE API operation from being executed.
	TerminationCode_GcpDeniedByOrgPolicy TerminationCode = "GCP_DENIED_BY_ORG_POLICY"
	// Customer passed in a secret that they do not have permissions to resolve.
	TerminationCode_SecretPermissionDenied TerminationCode = "SECRET_PERMISSION_DENIED"
	// Start of network health check generated failures
	TerminationCode_NetworkCheckNicFailure                TerminationCode = "NETWORK_CHECK_NIC_FAILURE"
	TerminationCode_NetworkCheckDnsServerFailure          TerminationCode = "NETWORK_CHECK_DNS_SERVER_FAILURE"
	TerminationCode_NetworkCheckStorageFailure            TerminationCode = "NETWORK_CHECK_STORAGE_FAILURE"
	TerminationCode_NetworkCheckMetadataEndpointFailure   TerminationCode = "NETWORK_CHECK_METADATA_ENDPOINT_FAILURE"
	TerminationCode_NetworkCheckControlPlaneFailure       TerminationCode = "NETWORK_CHECK_CONTROL_PLANE_FAILURE"
	TerminationCode_NetworkCheckMultipleComponentsFailure TerminationCode = "NETWORK_CHECK_MULTIPLE_COMPONENTS_FAILURE"
	// Driver has been down or unresponsive for an extended period of time
	TerminationCode_DriverUnhealthy TerminationCode = "DRIVER_UNHEALTHY"
	// cluster request is denied due to disallowed usage policy entitlement
	TerminationCode_UsagePolicyEntitlementDenied TerminationCode = "USAGE_POLICY_ENTITLEMENT_DENIED"
	// Request exceeded MAX_ACTIVE_DBR_PODS_PER_K8S_CLUSTER quota - too many active
	// pods on the K8s cluster
	TerminationCode_K8sActivePodQuotaExceeded TerminationCode = "K8S_ACTIVE_POD_QUOTA_EXCEEDED"
	// Request exceeded MAX_PODS_PER_CLOUD_ACCOUNT quota - subscription/cloud
	// account pod limit reached
	TerminationCode_CloudAccountPodQuotaExceeded TerminationCode = "CLOUD_ACCOUNT_POD_QUOTA_EXCEEDED"
	// Start of network health check generated failures due to misconfiguration
	TerminationCode_NetworkCheckNicFailureDueToMisconfig                TerminationCode = "NETWORK_CHECK_NIC_FAILURE_DUE_TO_MISCONFIG"
	TerminationCode_NetworkCheckDnsServerFailureDueToMisconfig          TerminationCode = "NETWORK_CHECK_DNS_SERVER_FAILURE_DUE_TO_MISCONFIG"
	TerminationCode_NetworkCheckStorageFailureDueToMisconfig            TerminationCode = "NETWORK_CHECK_STORAGE_FAILURE_DUE_TO_MISCONFIG"
	TerminationCode_NetworkCheckMetadataEndpointFailureDueToMisconfig   TerminationCode = "NETWORK_CHECK_METADATA_ENDPOINT_FAILURE_DUE_TO_MISCONFIG"
	TerminationCode_NetworkCheckControlPlaneFailureDueToMisconfig       TerminationCode = "NETWORK_CHECK_CONTROL_PLANE_FAILURE_DUE_TO_MISCONFIG"
	TerminationCode_NetworkCheckMultipleComponentsFailureDueToMisconfig TerminationCode = "NETWORK_CHECK_MULTIPLE_COMPONENTS_FAILURE_DUE_TO_MISCONFIG"
	// CMv2 could not resolve the DBR image for versionless workloads (REPL,
	// GENERIC). This typically happens when no spark version is found from the
	// channel mapping and the workload is versionless-enabled.
	TerminationCode_DbrImageResolutionFailure                   TerminationCode = "DBR_IMAGE_RESOLUTION_FAILURE"
	TerminationCode_ControlPlaneConnectionFailure               TerminationCode = "CONTROL_PLANE_CONNECTION_FAILURE"
	TerminationCode_ControlPlaneConnectionFailureDueToMisconfig TerminationCode = "CONTROL_PLANE_CONNECTION_FAILURE_DUE_TO_MISCONFIG"
	TerminationCode_RateLimited                                 TerminationCode = "RATE_LIMITED"
	// The cluster was terminated because mutual TLS port 8443 check failed.
	TerminationCode_MtlsPortConnectivityFailure TerminationCode = "MTLS_PORT_CONNECTIVITY_FAILURE"
	// The cluster was terminated because hivemetastore connectivity check failed.
	TerminationCode_HivemetastoreConnectivityFailure TerminationCode = "HIVEMETASTORE_CONNECTIVITY_FAILURE"
)

type TerminationReason

type TerminationReason struct {
	// status code indicating why the cluster was terminated
	Code TerminationCode
	// type of the termination
	Type TerminationType
	// list of parameters that provide additional information about why the cluster
	// was terminated
	Parameters map[string]string
}

type TerminationType

type TerminationType string

type of the termination

const (
	TerminationType_Unspecified TerminationType = ""
	// Termination succeeded normally
	TerminationType_Success TerminationType = "SUCCESS"
	// Non-retryable. Client must fix parameters before reattempting the cluster
	// creation
	TerminationType_ClientError TerminationType = "CLIENT_ERROR"
	// Databricks service issue. Clients may retry
	TerminationType_ServiceFault TerminationType = "SERVICE_FAULT"
	// AWS or Azure infrastructure issue. Clients may retry after the underlying
	// cloud issue is resolved
	TerminationType_CloudFailure TerminationType = "CLOUD_FAILURE"
)

type UnpinClusterRequest

type UnpinClusterRequest struct {
	ClusterId *string
}

type UnpinClusterResponse

type UnpinClusterResponse struct {
}

type UpdateClusterRequest

type UpdateClusterRequest struct {
	// ID of the cluster.
	ClusterId *string
	// The cluster to be updated.
	Cluster *UpdateClusterRequest_UpdateClusterResource
	// Used to specify which cluster attributes and size fields to update. See
	// https://google.aip.dev/161 for more details.
	UpdateMask *types.FieldMask[UpdateClusterRequest_UpdateClusterResource]
}

type UpdateClusterRequest_UpdateClusterResource

type UpdateClusterRequest_UpdateClusterResource struct {
	Size isUpdateClusterRequest_UpdateClusterResource_Size
	// Cluster name requested by the user. This doesn't have to be unique. If not
	// specified at creation, the cluster name will be an empty string. For job
	// clusters, the cluster name is automatically set based on the job and job run
	// IDs.
	ClusterName *string `fieldmask:"cluster_name"`
	// The Spark version of the cluster, e.g. `3.3.x-scala2.11`. A list of available
	// Spark versions can be retrieved by using the [clusters/sparkVersions] API
	// call.
	//
	// [clusters/sparkVersions]: https://docs.databricks.com/api/workspace/clusters/sparkversions
	SparkVersion *string `fieldmask:"spark_version"`
	// An object containing a set of optional, user-specified Spark configuration
	// key-value pairs. Users can also pass in a string of extra JVM options to the
	// driver and the executors via `spark.driver.extraJavaOptions` and
	// `spark.executor.extraJavaOptions` respectively.
	SparkConf map[string]string `fieldmask:"spark_conf"`
	// Attributes related to clusters running on Amazon Web Services. If not
	// specified at cluster creation, a set of default values will be used.
	AwsAttributes *AwsAttributes `fieldmask:"aws_attributes"`
	// Attributes related to clusters running on Microsoft Azure. If not specified
	// at cluster creation, a set of default values will be used.
	AzureAttributes *AzureAttributes `fieldmask:"azure_attributes"`
	// Attributes related to clusters running on Google Cloud Platform. If not
	// specified at cluster creation, a set of default values will be used.
	GcpAttributes *GcpAttributes `fieldmask:"gcp_attributes"`
	// This field encodes, through a single value, the resources available to each
	// of the Spark nodes in this cluster. For example, the Spark nodes can be
	// provisioned and optimized for memory or compute intensive workloads. A list
	// of available node types can be retrieved by using the
	// [clusters/listNodeTypes] API call.
	//
	// [clusters/listNodeTypes]: https://docs.databricks.com/api/workspace/clusters/listnodetypes
	NodeTypeId *string `fieldmask:"node_type_id"`
	// The node type of the Spark driver. Note that this field is optional; if
	// unset, the driver node type will be set as the same value as `node_type_id`
	// defined above.
	//
	// This field, along with node_type_id, should not be set if
	// virtual_cluster_size is set. If both driver_node_type_id, node_type_id, and
	// virtual_cluster_size are specified, driver_node_type_id and node_type_id take
	// precedence.
	DriverNodeTypeId *string `fieldmask:"driver_node_type_id"`
	// Flexible node type configuration for worker nodes.
	WorkerNodeTypeFlexibility *NodeTypeFlexibility `fieldmask:"worker_node_type_flexibility"`
	// Flexible node type configuration for the driver node.
	DriverNodeTypeFlexibility *NodeTypeFlexibility `fieldmask:"driver_node_type_flexibility"`
	// SSH public key contents that will be added to each Spark node in this
	// cluster. The corresponding private keys can be used to login with the user
	// name `ubuntu` on port `2200`. Up to 10 keys can be specified.
	SshPublicKeys []string `fieldmask:"ssh_public_keys"`
	// Additional tags for cluster resources. <Databricks> will tag all cluster
	// resources (e.g., AWS instances and EBS volumes) with these tags in addition
	// to `default_tags`. Notes:
	//
	// - Currently, <Databricks> allows at most 45 custom tags
	//
	// - Clusters can only reuse cloud resources if the resources' tags are a subset
	// of the cluster tags
	CustomTags map[string]string `fieldmask:"custom_tags"`
	// The configuration for delivering spark logs to a long-term storage
	// destination. Three kinds of destinations (DBFS, S3 and Unity Catalog volumes)
	// are supported. Only one destination can be specified for one cluster. If the
	// conf is given, the logs will be delivered to the destination every `5 mins`.
	// The destination of driver logs is `$destination/$clusterId/driver`, while the
	// destination of executor logs is `$destination/$clusterId/executor`.
	ClusterLogConf *ClusterLogConf `fieldmask:"cluster_log_conf"`
	// An object containing a set of optional, user-specified environment variable
	// key-value pairs. Please note that key-value pair of the form (X,Y) will be
	// exported as is (i.e., `export X='Y'`) while launching the driver and workers.
	//
	// In order to specify an additional set of `SPARK_DAEMON_JAVA_OPTS`, we
	// recommend appending them to `$SPARK_DAEMON_JAVA_OPTS` as shown in the example
	// below. This ensures that all default databricks managed environmental
	// variables are included as well.
	//
	// Example Spark environment variables: `{"SPARK_WORKER_MEMORY": "28000m",
	// "SPARK_LOCAL_DIRS": "/local_disk0"}` or `{"SPARK_DAEMON_JAVA_OPTS":
	// "$SPARK_DAEMON_JAVA_OPTS -Dspark.shuffle.service.enabled=true"}`
	SparkEnvVars map[string]string `fieldmask:"spark_env_vars"`
	// Automatically terminates the cluster after it is inactive for this time in
	// minutes. If not set, this cluster will not be automatically terminated. If
	// specified, the threshold must be between 10 and 10000 minutes. Users can also
	// set this value to 0 to explicitly disable automatic termination.
	AutoterminationMinutes *int `fieldmask:"autotermination_minutes"`
	// Autoscaling Local Storage: when enabled, this cluster will dynamically
	// acquire additional disk space when its Spark workers are running low on disk
	// space.
	EnableElasticDisk *bool `fieldmask:"enable_elastic_disk"`
	// The configuration for storing init scripts. Any number of destinations can be
	// specified. The scripts are executed sequentially in the order provided. If
	// `cluster_log_conf` is specified, init script logs are sent to
	// `<destination>/<cluster-ID>/init_scripts`.
	InitScripts []InitScriptInfo `fieldmask:"init_scripts"`
	// Custom docker image BYOC
	DockerImage *DockerImage `fieldmask:"docker_image"`
	// The optional ID of the instance pool to which the cluster belongs.
	InstancePoolId *string `fieldmask:"instance_pool_id"`
	// Single user name if data_security_mode is `SINGLE_USER`
	SingleUserName *string `fieldmask:"single_user_name"`
	// The ID of the cluster policy used to create the cluster if applicable.
	PolicyId *string `fieldmask:"policy_id"`
	// Whether to enable LUKS on cluster VMs' local disks
	EnableLocalDiskEncryption *bool `fieldmask:"enable_local_disk_encryption"`
	// The optional ID of the instance pool for the driver of the cluster belongs.
	// The pool cluster uses the instance pool with id (instance_pool_id) if the
	// driver pool is not assigned.
	DriverInstancePoolId *string          `fieldmask:"driver_instance_pool_id"`
	WorkloadType         *WorkloadType    `fieldmask:"workload_type"`
	DataSecurityMode     DataSecurityMode `fieldmask:"data_security_mode"`
	// Determines the cluster's runtime engine, either standard or Photon.
	//
	// This field is not compatible with legacy `spark_version` values that contain
	// `-photon-`. Remove `-photon-` from the `spark_version` and set
	// `runtime_engine` to `PHOTON`.
	//
	// If left unspecified, the runtime engine defaults to standard unless the
	// spark_version contains -photon-, in which case Photon will be used.
	RuntimeEngine RuntimeEngine `fieldmask:"runtime_engine"`
	Kind          ComputeKind   `fieldmask:"kind"`
	// This field can only be used when `kind = CLASSIC_PREVIEW`.
	//
	// `effective_spark_version` is determined by `spark_version` (DBR release),
	// this field `use_ml_runtime`, and whether `node_type_id` is gpu node or not.
	UseMlRuntime *bool `fieldmask:"use_ml_runtime"`
	// This field can only be used when `kind = CLASSIC_PREVIEW`.
	//
	// When set to true, <Databricks> will automatically set single node related
	// `custom_tags`, `spark_conf`, and `num_workers`
	IsSingleNode *bool `fieldmask:"is_single_node"`
	// If set, what the configurable throughput (in Mb/s) for the remote disk is.
	// Currently only supported for GCP HYPERDISK_BALANCED disks.
	RemoteDiskThroughput *int `fieldmask:"remote_disk_throughput"`
	// If set, what the total initial volume size (in GB) of the remote disks should
	// be. Supported for GCP.
	TotalInitialRemoteDiskSize *int `fieldmask:"total_initial_remote_disk_size"`
	// Controls dependency configuration for the cluster.
	DependencyMode DependencyMode `fieldmask:"dependency_mode"`
	// contains filtered or unexported fields
}

type UpdateClusterRequest_UpdateClusterResource_Size_Autoscale

type UpdateClusterRequest_UpdateClusterResource_Size_Autoscale struct {
	Autoscale AutoScale `fieldmask:"autoscale"`
}

UpdateClusterRequest_UpdateClusterResource_Size_Autoscale selects Autoscale for UpdateClusterRequest_UpdateClusterResource.Size. Parameters needed in order to automatically scale clusters up and down based on load. Note: autoscaling works best with DB runtime versions 3.0 or later.

type UpdateClusterRequest_UpdateClusterResource_Size_NumWorkers

type UpdateClusterRequest_UpdateClusterResource_Size_NumWorkers struct {
	NumWorkers int `fieldmask:"num_workers"`
}

UpdateClusterRequest_UpdateClusterResource_Size_NumWorkers selects NumWorkers for UpdateClusterRequest_UpdateClusterResource.Size. Number of worker nodes that this cluster should have. A cluster has one Spark Driver and `num_workers` Executors for a total of `num_workers` + 1 Spark nodes.

Note: When reading the properties of a cluster, this field reflects the desired number of workers rather than the actual current number of workers. For instance, if a cluster is resized from 5 to 10 workers, this field will immediately be updated to reflect the target size of 10 workers, whereas the workers listed in `spark_info` will gradually increase from 5 to 10 as the new nodes are provisioned.

type UpdateClusterResponse

type UpdateClusterResponse struct {
}

type UpdateClusterWaiter

type UpdateClusterWaiter struct {
	// contains filtered or unexported fields
}

UpdateClusterWaiter tracks the state of the operation started by UpdateCluster.

func (*UpdateClusterWaiter) Done

func (w *UpdateClusterWaiter) Done(ctx context.Context, opts ...call.Option) (bool, error)

Done polls once and reports whether the operation has reached a terminal state.

func (*UpdateClusterWaiter) GetClusterId

func (w *UpdateClusterWaiter) GetClusterId() string

GetClusterId returns the ClusterId value used to identify the operation.

func (*UpdateClusterWaiter) Wait

func (w *UpdateClusterWaiter) Wait(ctx context.Context, opts ...lro.Option) (*ClusterInfo, error)

Wait polls until the operation reaches a terminal state.

type VolumesStorageInfo

type VolumesStorageInfo struct {
	// UC Volumes destination, e.g.
	// `/Volumes/catalog/schema/vol1/init-scripts/setup-datadog.sh` or
	// `dbfs:/Volumes/catalog/schema/vol1/init-scripts/setup-datadog.sh`
	Destination *string `fieldmask:"destination"`
}

A storage location back by UC Volumes..

type WorkloadType

type WorkloadType struct {
	// defined what type of clients can use the cluster. E.g. Notebooks, Jobs
	Clients *WorkloadType_ClientsTypes `fieldmask:"clients"`
}

Cluster Attributes showing for clusters workload types..

type WorkloadType_ClientsTypes

type WorkloadType_ClientsTypes struct {
	// With notebooks set, this cluster can be used for notebooks
	Notebooks *bool `fieldmask:"notebooks"`
	// With jobs set, the cluster can be used for jobs
	Jobs *bool `fieldmask:"jobs"`
}

type WorkspaceStorageInfo

type WorkspaceStorageInfo struct {
	// wsfs destination, e.g. `workspace:/cluster-init-scripts/setup-datadog.sh`
	Destination *string
}

A storage location in Workspace Filesystem (WSFS).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL