dataquality

package
v0.0.1-dev.6 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 7, 2026 License: Apache-2.0 Imports: 21 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type AggregationGranularity 

type AggregationGranularity string

The granularity for aggregating data into time windows based on their timestamp. 

const (
	AggregationGranularity_Unspecified AggregationGranularity = ""
	// 5 minutes.
	AggregationGranularity_AggregationGranularity5Minutes AggregationGranularity = "AGGREGATION_GRANULARITY_5_MINUTES"
	// 30 minutes.
	AggregationGranularity_AggregationGranularity30Minutes AggregationGranularity = "AGGREGATION_GRANULARITY_30_MINUTES"
	// 1 hour.
	AggregationGranularity_AggregationGranularity1Hour AggregationGranularity = "AGGREGATION_GRANULARITY_1_HOUR"
	// 1 day.
	AggregationGranularity_AggregationGranularity1Day AggregationGranularity = "AGGREGATION_GRANULARITY_1_DAY"
	// 1 week.
	AggregationGranularity_AggregationGranularity1Week AggregationGranularity = "AGGREGATION_GRANULARITY_1_WEEK"
	// 2 weeks.
	AggregationGranularity_AggregationGranularity2Weeks AggregationGranularity = "AGGREGATION_GRANULARITY_2_WEEKS"
	// 3 weeks.
	AggregationGranularity_AggregationGranularity3Weeks AggregationGranularity = "AGGREGATION_GRANULARITY_3_WEEKS"
	// 4 weeks.
	AggregationGranularity_AggregationGranularity4Weeks AggregationGranularity = "AGGREGATION_GRANULARITY_4_WEEKS"
	// 1 month.
	AggregationGranularity_AggregationGranularity1Month AggregationGranularity = "AGGREGATION_GRANULARITY_1_MONTH"
	// 1 year.
	AggregationGranularity_AggregationGranularity1Year AggregationGranularity = "AGGREGATION_GRANULARITY_1_YEAR"
)

type AnomalyDetectionConfig 

type AnomalyDetectionConfig struct {
	// List of fully qualified table names to exclude from anomaly detection.
	ExcludedTableFullNames []string `fieldmask:"excluded_table_full_names"`
}

Anomaly Detection Configurations..

type CancelRefreshRequest 

type CancelRefreshRequest struct {
	// The type of the monitored object. Can be one of the following: `schema` or
	// `table`.
	ObjectType *string
	// The UUID of the request object. It is `schema_id` for `schema`, and
	// `table_id` for `table`.
	//
	// Find the `schema_id` from either: 1. The [schema_id] of the `Schemas`
	// resource. 2. In [Catalog Explorer] > select the `schema` > go to the
	// `Details` tab > the `Schema ID` field.
	//
	// Find the `table_id` from either: 1. The [table_id] of the `Tables` resource.
	// 2. In [Catalog Explorer] > select the `table` > go to the `Details` tab > the
	// `Table ID` field.
	//
	// [Catalog Explorer]: https://docs.databricks.com/aws/en/catalog-explorer/
	// [schema_id]: https://docs.databricks.com/api/workspace/schemas/get#schema_id
	// [table_id]: https://docs.databricks.com/api/workspace/tables/get#table_id
	ObjectId *string
	// Unique id of the refresh operation.
	RefreshId *int64
}

Request to cancel a refresh..

type CancelRefreshResponse 

type CancelRefreshResponse struct {
	// The refresh to cancel.
	Refresh *Refresh
}

Response to cancelling a refresh..

type Client 

type Client struct {
	// contains filtered or unexported fields
}

func NewClient 

func NewClient(ctx context.Context, opts ...client.Option) (*Client, error)

func (*Client) CancelRefresh 

func (c *Client) CancelRefresh(ctx context.Context, req CancelRefreshRequest, opts ...call.Option) (*CancelRefreshResponse, error)

Cancels a data quality monitor refresh. Currently only supported for the `table` `object_type`. The call must be made in the same workspace as where the monitor was created.

The caller must have either of the following sets of permissions: 1. **MANAGE** and **USE_CATALOG** on the table's parent catalog. 2. **USE_CATALOG** on the table's parent catalog, and **MANAGE** and **USE_SCHEMA** on the table's parent schema. 3. **USE_CATALOG** on the table's parent catalog, **USE_SCHEMA** on the table's parent schema, and **MANAGE** on the table.

func (*Client) CreateMonitor 

func (c *Client) CreateMonitor(ctx context.Context, req CreateMonitorRequest, opts ...call.Option) (*Monitor, error)

Create a data quality monitor on a Unity Catalog object. The caller must provide either `anomaly_detection_config` for a schema monitor or `data_profiling_config` for a table monitor.

For the `table` `object_type`, the caller must have either of the following sets of permissions: 1. **MANAGE** and **USE_CATALOG** on the table's parent catalog, **USE_SCHEMA** on the table's parent schema, and **SELECT** on the table 2. **USE_CATALOG** on the table's parent catalog, **MANAGE** and **USE_SCHEMA** on the table's parent schema, and **SELECT** on the table. 3. **USE_CATALOG** on the table's parent catalog, **USE_SCHEMA** on the table's parent schema, and **MANAGE** and **SELECT** on the table.

Workspace assets, such as the dashboard, will be created in the workspace where this call was made.

For the `schema` `object_type`, the caller must have either of the following sets of permissions: 1. **MANAGE** and **USE_CATALOG** on the schema's parent catalog. 2. **USE_CATALOG** on the schema's parent catalog, and **MANAGE** and **USE_SCHEMA** on the schema.

func (*Client) CreateRefresh 

func (c *Client) CreateRefresh(ctx context.Context, req CreateRefreshRequest, opts ...call.Option) (*Refresh, error)

Creates a refresh. Currently only supported for the `table` `object_type`. The call must be made in the same workspace as where the monitor was created.

The caller must have either of the following sets of permissions: 1. **MANAGE** and **USE_CATALOG** on the table's parent catalog. 2. **USE_CATALOG** on the table's parent catalog, and **MANAGE** and **USE_SCHEMA** on the table's parent schema. 3. **USE_CATALOG** on the table's parent catalog, **USE_SCHEMA** on the table's parent schema, and **MANAGE** on the table.

func (*Client) DeleteMonitor 

func (c *Client) DeleteMonitor(ctx context.Context, req DeleteMonitorRequest, opts ...call.Option) error

Delete a data quality monitor on Unity Catalog object.

For the `table` `object_type`, the caller must have either of the following sets of permissions: **MANAGE** and **USE_CATALOG** on the table's parent catalog. **USE_CATALOG** on the table's parent catalog, and **MANAGE** and **USE_SCHEMA** on the table's parent schema. **USE_CATALOG** on the table's parent catalog, **USE_SCHEMA** on the table's parent schema, and **MANAGE** on the table.

Note that the metric tables and dashboard will not be deleted as part of this call; those assets must be manually cleaned up (if desired).

For the `schema` `object_type`, the caller must have either of the following sets of permissions: 1. **MANAGE** and **USE_CATALOG** on the schema's parent catalog. 2. **USE_CATALOG** on the schema's parent catalog, and **MANAGE** and **USE_SCHEMA** on the schema.

func (*Client) DeleteRefresh 

func (c *Client) DeleteRefresh(ctx context.Context, req DeleteRefreshRequest, opts ...call.Option) error

(Unimplemented) Delete a refresh

func (*Client) GetMonitor 

func (c *Client) GetMonitor(ctx context.Context, req GetMonitorRequest, opts ...call.Option) (*Monitor, error)

Read a data quality monitor on a Unity Catalog object.

For the `table` `object_type`, the caller must have either of the following sets of permissions: 1. **MANAGE** and **USE_CATALOG** on the table's parent catalog. 2. **USE_CATALOG** on the table's parent catalog, and **MANAGE** and **USE_SCHEMA** on the table's parent schema. 3. **USE_CATALOG** on the table's parent catalog, **USE_SCHEMA** on the table's parent schema, and **SELECT** on the table.

For the `schema` `object_type`, the caller must have either of the following sets of permissions: 1. **MANAGE** and **USE_CATALOG** on the schema's parent catalog. 2. **USE_CATALOG** on the schema's parent catalog, and **USE_SCHEMA** on the schema.

The returned information includes configuration values on the entity and parent entity as well as information on assets created by the monitor. Some information (e.g. dashboard) may be filtered out if the caller is in a different workspace than where the monitor was created.

func (*Client) GetRefresh 

func (c *Client) GetRefresh(ctx context.Context, req GetRefreshRequest, opts ...call.Option) (*Refresh, error)

Get data quality monitor refresh. The call must be made in the same workspace as where the monitor was created.

For the `table` `object_type`, the caller must have either of the following sets of permissions: 1. **MANAGE** and **USE_CATALOG** on the table's parent catalog. 2. **USE_CATALOG** on the table's parent catalog, and **MANAGE** and **USE_SCHEMA** on the table's parent schema. 3. **USE_CATALOG** on the table's parent catalog, **USE_SCHEMA** on the table's parent schema, and **SELECT** on the table.

For the `schema` `object_type`, the caller must have either of the following sets of permissions: 1. **MANAGE** and **USE_CATALOG** on the schema's parent catalog. 2. **USE_CATALOG** on the schema's parent catalog, and **USE_SCHEMA** on the schema.

func (*Client) ListMonitor 

func (c *Client) ListMonitor(ctx context.Context, req ListMonitorRequest, opts ...call.Option) (*ListMonitorResponse, error)

(Unimplemented) List data quality monitors.

func (*Client) ListMonitorIter 

func (c *Client) ListMonitorIter(ctx context.Context, req ListMonitorRequest, opts ...call.Option) iter.Seq2[*Monitor, error]

ListMonitorIter returns an iterator that iterates over the results of ListMonitor.

For example: 

for item, err := range c.ListMonitorIter(ctx, ListMonitorRequest{}) {
  if err != nil {
    return err
  }
  fmt.Println(item)
}

Options opts are passed to each ListMonitor call made by the iterator under the hood.

Callers who need custom pagination logic should use ListMonitor directly.

func (*Client) ListRefresh 

func (c *Client) ListRefresh(ctx context.Context, req ListRefreshRequest, opts ...call.Option) (*ListRefreshResponse, error)

List data quality monitor refreshes. The call must be made in the same workspace as where the monitor was created.

For the `table` `object_type`, the caller must have either of the following sets of permissions: 1. **MANAGE** and **USE_CATALOG** on the table's parent catalog. 2. **USE_CATALOG** on the table's parent catalog, and **MANAGE** and **USE_SCHEMA** on the table's parent schema. 3. **USE_CATALOG** on the table's parent catalog, **USE_SCHEMA** on the table's parent schema, and **SELECT** on the table.

For the `schema` `object_type`, the caller must have either of the following sets of permissions: 1. **MANAGE** and **USE_CATALOG** on the schema's parent catalog. 2. **USE_CATALOG** on the schema's parent catalog, and **USE_SCHEMA** on the schema.

func (*Client) ListRefreshIter 

func (c *Client) ListRefreshIter(ctx context.Context, req ListRefreshRequest, opts ...call.Option) iter.Seq2[*Refresh, error]

ListRefreshIter returns an iterator that iterates over the results of ListRefresh.

For example: 

for item, err := range c.ListRefreshIter(ctx, ListRefreshRequest{}) {
  if err != nil {
    return err
  }
  fmt.Println(item)
}

Options opts are passed to each ListRefresh call made by the iterator under the hood.

Callers who need custom pagination logic should use ListRefresh directly.

func (*Client) UpdateMonitor 

func (c *Client) UpdateMonitor(ctx context.Context, req UpdateMonitorRequest, opts ...call.Option) (*Monitor, error)

Update a data quality monitor on Unity Catalog object.

For the `table` `object_type`, the caller must have either of the following sets of permissions: 1. **MANAGE** and **USE_CATALOG** on the table's parent catalog. 2. **USE_CATALOG** on the table's parent catalog, and **MANAGE** and **USE_SCHEMA** on the table's parent schema. 3. **USE_CATALOG** on the table's parent catalog, **USE_SCHEMA** on the table's parent schema, and **MANAGE** on the table.

For the `schema` `object_type`, the caller must have either of the following sets of permissions: 1. **MANAGE** and **USE_CATALOG** on the schema's parent catalog. 2. **USE_CATALOG** on the schema's parent catalog, and **MANAGE** and **USE_SCHEMA** on the schema.

func (*Client) UpdateRefresh 

func (c *Client) UpdateRefresh(ctx context.Context, req UpdateRefreshRequest, opts ...call.Option) (*Refresh, error)

(Unimplemented) Update a refresh

type CreateMonitorRequest 

type CreateMonitorRequest struct {
	// The monitor to create.
	Monitor *Monitor
}

Request to create a Monitor..

type CreateRefreshRequest 

type CreateRefreshRequest struct {
	// The refresh to create
	Refresh *Refresh
}

Request to create a refresh..

type CronSchedule 

type CronSchedule struct {
	// The expression that determines when to run the monitor. See [examples].
	//
	// [examples]: https://www.quartz-scheduler.org/documentation/quartz-2.3.0/tutorials/crontrigger.html
	QuartzCronExpression *string `fieldmask:"quartz_cron_expression"`
	// A Java timezone id. The schedule for a job will be resolved with respect to
	// this timezone. See `Java TimeZone
	// <http://docs.oracle.com/javase/7/docs/api/java/util/TimeZone.html>`_ for
	// details. The timezone id (e.g., “America/Los_Angeles“) in which to evaluate
	// the quartz expression.
	TimezoneId *string `fieldmask:"timezone_id"`
	// Read only field that indicates whether the schedule is paused or not.
	PauseStatus CronSchedulePauseStatus `fieldmask:"pause_status"`
}

The data quality monitoring workflow cron schedule..

type CronSchedulePauseStatus 

type CronSchedulePauseStatus string

The data quality monitoring workflow cron schedule pause status. 

const (
	CronSchedulePauseStatus_Unspecified CronSchedulePauseStatus = ""
	// The cron schedule is not paused.
	CronSchedulePauseStatus_CronSchedulePauseStatusUnpaused CronSchedulePauseStatus = "CRON_SCHEDULE_PAUSE_STATUS_UNPAUSED"
	// The cron schedule is paused.
	CronSchedulePauseStatus_CronSchedulePauseStatusPaused CronSchedulePauseStatus = "CRON_SCHEDULE_PAUSE_STATUS_PAUSED"
)

type DataProfilingConfig 

type DataProfilingConfig struct {
	// ID of the schema where output tables are created.
	OutputSchemaId *string `fieldmask:"output_schema_id"`
	// Field for specifying the absolute path to a custom directory to store
	// data-monitoring assets. Normally prepopulated to a default user location via
	// UI and Python APIs.
	AssetsDir *string `fieldmask:"assets_dir"`
	// (--[Create:REQ Update:REQ]--) Analysis config which is used to determine
	// analysis logic.
	AnalysisConfig isDataProfilingConfig_AnalysisConfig
	// List of column expressions to slice data with for targeted analysis. The data
	// is grouped by each expression independently, resulting in a separate slice
	// for each predicate and its complements. For example
	// `slicing_exprs=[“col_1”, “col_2 > 10”]` will generate the following
	// slices: two slices for `col_2 > 10` (True and False), and one slice per
	// unique value in `col1`. For high-cardinality columns, only the top 100 unique
	// values by frequency will generate slices.
	SlicingExprs []string `fieldmask:"slicing_exprs"`
	// Custom metrics.
	CustomMetrics []DataProfilingCustomMetric `fieldmask:"custom_metrics"`
	// Baseline table name. Baseline data is used to compute drift from the data in
	// the monitored `table_name`. The baseline table and the monitored table shall
	// have the same schema.
	BaselineTableName *string `fieldmask:"baseline_table_name"`
	// The cron schedule.
	Schedule *CronSchedule `fieldmask:"schedule"`
	// Field for specifying notification settings.
	NotificationSettings *NotificationSettings `fieldmask:"notification_settings"`
	// Whether to skip creating a default dashboard summarizing data quality
	// metrics.
	SkipBuiltinDashboard *bool `fieldmask:"skip_builtin_dashboard"`
	// Optional argument to specify the warehouse for dashboard creation. If not
	// specified, the first running warehouse will be used.
	WarehouseId *string `fieldmask:"warehouse_id"`
	// Unity Catalog table to monitor. Format: `catalog.schema.table_name`
	MonitoredTableName *string `fieldmask:"monitored_table_name"`
	// The data profiling monitor status.
	Status DataProfilingStatus `fieldmask:"status"`
	// The latest error message for a monitor failure.
	LatestMonitorFailureMessage *string `fieldmask:"latest_monitor_failure_message"`
	// Table that stores profile metrics data. Format: `catalog.schema.table_name`.
	ProfileMetricsTableName *string `fieldmask:"profile_metrics_table_name"`
	// Table that stores drift metrics data. Format: `catalog.schema.table_name`.
	DriftMetricsTableName *string `fieldmask:"drift_metrics_table_name"`
	// Id of dashboard that visualizes the computed metrics. This can be empty if
	// the monitor is in PENDING state.
	DashboardId *string `fieldmask:"dashboard_id"`
	// Represents the current monitor configuration version in use. The version will
	// be represented in a numeric fashion (1,2,3...). The field has flexibility to
	// take on negative values, which can indicate corrupted monitor_version
	// numbers.
	MonitorVersion *int64 `fieldmask:"monitor_version"`
	// The warehouse for dashboard creation
	EffectiveWarehouseId *string `fieldmask:"effective_warehouse_id"`
	// contains filtered or unexported fields
}

Data Profiling Configurations..

type DataProfilingConfig_AnalysisConfig_InferenceLog 

type DataProfilingConfig_AnalysisConfig_InferenceLog struct {
	InferenceLog InferenceLogConfig `fieldmask:"inference_log"`
}

DataProfilingConfig_AnalysisConfig_InferenceLog selects InferenceLog for DataProfilingConfig.AnalysisConfig. `Analysis Configuration` for monitoring inference log tables.

type DataProfilingConfig_AnalysisConfig_Snapshot 

type DataProfilingConfig_AnalysisConfig_Snapshot struct {
	Snapshot SnapshotConfig `fieldmask:"snapshot"`
}

DataProfilingConfig_AnalysisConfig_Snapshot selects Snapshot for DataProfilingConfig.AnalysisConfig. `Analysis Configuration` for monitoring snapshot tables.

type DataProfilingConfig_AnalysisConfig_TimeSeries 

type DataProfilingConfig_AnalysisConfig_TimeSeries struct {
	TimeSeries TimeSeriesConfig `fieldmask:"time_series"`
}

DataProfilingConfig_AnalysisConfig_TimeSeries selects TimeSeries for DataProfilingConfig.AnalysisConfig. `Analysis Configuration` for monitoring time series tables.

type DataProfilingCustomMetric 

type DataProfilingCustomMetric struct {
	// Name of the metric in the output tables.
	Name *string
	// Jinja template for a SQL expression that specifies how to compute the metric.
	// See [create metric definition].
	//
	// [create metric definition]: https://docs.databricks.com/en/lakehouse-monitoring/custom-metrics.html#create-definition
	Definition *string
	// A list of column names in the input table the metric should be computed for.
	// Can use “":table"“ to indicate that the metric needs information from
	// multiple columns.
	InputColumns []string
	// The output type of the custom metric.
	OutputDataType *string
	// The type of the custom metric.
	Type DataProfilingCustomMetricType
}

Custom metric definition..

type DataProfilingCustomMetricType 

type DataProfilingCustomMetricType string

The custom metric type. 

const (
	DataProfilingCustomMetricType_Unspecified DataProfilingCustomMetricType = ""
	// Only depend on the existing columns in the table.
	DataProfilingCustomMetricType_DataProfilingCustomMetricTypeAggregate DataProfilingCustomMetricType = "DATA_PROFILING_CUSTOM_METRIC_TYPE_AGGREGATE"
	// Only depend on previously computed aggregate metrics.
	DataProfilingCustomMetricType_DataProfilingCustomMetricTypeDerived DataProfilingCustomMetricType = "DATA_PROFILING_CUSTOM_METRIC_TYPE_DERIVED"
	// Depend on previously computed aggregate or derived metrics.
	DataProfilingCustomMetricType_DataProfilingCustomMetricTypeDrift DataProfilingCustomMetricType = "DATA_PROFILING_CUSTOM_METRIC_TYPE_DRIFT"
)

type DataProfilingStatus 

type DataProfilingStatus string

The status of the data profiling monitor. 

const (
	DataProfilingStatus_Unspecified                      DataProfilingStatus = ""
	DataProfilingStatus_DataProfilingStatusActive        DataProfilingStatus = "DATA_PROFILING_STATUS_ACTIVE"
	DataProfilingStatus_DataProfilingStatusPending       DataProfilingStatus = "DATA_PROFILING_STATUS_PENDING"
	DataProfilingStatus_DataProfilingStatusDeletePending DataProfilingStatus = "DATA_PROFILING_STATUS_DELETE_PENDING"
	DataProfilingStatus_DataProfilingStatusError         DataProfilingStatus = "DATA_PROFILING_STATUS_ERROR"
	DataProfilingStatus_DataProfilingStatusFailed        DataProfilingStatus = "DATA_PROFILING_STATUS_FAILED"
)

type DeleteMonitorRequest 

type DeleteMonitorRequest struct {
	// The type of the monitored object. Can be one of the following: `schema` or
	// `table`.
	ObjectType *string
	// The UUID of the request object. It is `schema_id` for `schema`, and
	// `table_id` for `table`.
	//
	// Find the `schema_id` from either: 1. The [schema_id] of the `Schemas`
	// resource. 2. In [Catalog Explorer] > select the `schema` > go to the
	// `Details` tab > the `Schema ID` field.
	//
	// Find the `table_id` from either: 1. The [table_id] of the `Tables` resource.
	// 2. In [Catalog Explorer] > select the `table` > go to the `Details` tab > the
	// `Table ID` field.
	//
	// [Catalog Explorer]: https://docs.databricks.com/aws/en/catalog-explorer/
	// [schema_id]: https://docs.databricks.com/api/workspace/schemas/get#schema_id
	// [table_id]: https://docs.databricks.com/api/workspace/tables/get#table_id
	ObjectId *string
}

Request to delete a Monitor..

type DeleteRefreshRequest 

type DeleteRefreshRequest struct {
	// The type of the monitored object. Can be one of the following: `schema` or
	// `table`.
	ObjectType *string
	// The UUID of the request object. It is `schema_id` for `schema`, and
	// `table_id` for `table`.
	//
	// Find the `schema_id` from either: 1. The [schema_id] of the `Schemas`
	// resource. 2. In [Catalog Explorer] > select the `schema` > go to the
	// `Details` tab > the `Schema ID` field.
	//
	// Find the `table_id` from either: 1. The [table_id] of the `Tables` resource.
	// 2. In [Catalog Explorer] > select the `table` > go to the `Details` tab > the
	// `Table ID` field.
	//
	// [Catalog Explorer]: https://docs.databricks.com/aws/en/catalog-explorer/
	// [schema_id]: https://docs.databricks.com/api/workspace/schemas/get#schema_id
	// [table_id]: https://docs.databricks.com/api/workspace/tables/get#table_id
	ObjectId *string
	// Unique id of the refresh operation.
	RefreshId *int64
}

Request to delete a ronitor..

type GetMonitorRequest 

type GetMonitorRequest struct {
	// The type of the monitored object. Can be one of the following: `schema` or
	// `table`.
	ObjectType *string
	// The UUID of the request object. It is `schema_id` for `schema`, and
	// `table_id` for `table`.
	//
	// Find the `schema_id` from either: 1. The [schema_id] of the `Schemas`
	// resource. 2. In [Catalog Explorer] > select the `schema` > go to the
	// `Details` tab > the `Schema ID` field.
	//
	// Find the `table_id` from either: 1. The [table_id] of the `Tables` resource.
	// 2. In [Catalog Explorer] > select the `table` > go to the `Details` tab > the
	// `Table ID` field.
	//
	// [Catalog Explorer]: https://docs.databricks.com/aws/en/catalog-explorer/
	// [schema_id]: https://docs.databricks.com/api/workspace/schemas/get#schema_id
	// [table_id]: https://docs.databricks.com/api/workspace/tables/get#table_id
	ObjectId *string
}

Request to get a Monitor..

type GetRefreshRequest 

type GetRefreshRequest struct {
	// The type of the monitored object. Can be one of the following: `schema` or
	// `table`.
	ObjectType *string
	// The UUID of the request object. It is `schema_id` for `schema`, and
	// `table_id` for `table`.
	//
	// Find the `schema_id` from either: 1. The [schema_id] of the `Schemas`
	// resource. 2. In [Catalog Explorer] > select the `schema` > go to the
	// `Details` tab > the `Schema ID` field.
	//
	// Find the `table_id` from either: 1. The [table_id] of the `Tables` resource.
	// 2. In [Catalog Explorer] > select the `table` > go to the `Details` tab > the
	// `Table ID` field.
	//
	// [Catalog Explorer]: https://docs.databricks.com/aws/en/catalog-explorer/
	// [schema_id]: https://docs.databricks.com/api/workspace/schemas/get#schema_id
	// [table_id]: https://docs.databricks.com/api/workspace/tables/get#table_id
	ObjectId *string
	// Unique id of the refresh operation.
	RefreshId *int64
}

Request to get a refresh..

type InferenceLogConfig 

type InferenceLogConfig struct {
	// Problem type the model aims to solve.
	ProblemType InferenceProblemType `fieldmask:"problem_type"`
	// Column for the timestamp.
	TimestampColumn *string `fieldmask:"timestamp_column"`
	// List of granularities to use when aggregating data into time windows based on
	// their timestamp.
	Granularities []AggregationGranularity `fieldmask:"granularities"`
	// Column for the prediction.
	PredictionColumn *string `fieldmask:"prediction_column"`
	// Column for the label.
	LabelColumn *string `fieldmask:"label_column"`
	// Column for the model identifier.
	ModelIdColumn *string `fieldmask:"model_id_column"`
}

Inference log configuration..

type InferenceProblemType 

type InferenceProblemType string

Inference problem type the model aims to solve. 

const (
	InferenceProblemType_Unspecified InferenceProblemType = ""
	// Classification inference problem.
	InferenceProblemType_InferenceProblemTypeClassification InferenceProblemType = "INFERENCE_PROBLEM_TYPE_CLASSIFICATION"
	// Regression inference problem.
	InferenceProblemType_InferenceProblemTypeRegression InferenceProblemType = "INFERENCE_PROBLEM_TYPE_REGRESSION"
)

type ListMonitorRequest 

type ListMonitorRequest struct {
	PageToken *string
	PageSize  *int
}

Request to list Monitors..

type ListMonitorResponse 

type ListMonitorResponse struct {
	Monitors      []Monitor
	NextPageToken *string
}

Response for listing Monitors..

type ListRefreshRequest 

type ListRefreshRequest struct {
	// The type of the monitored object. Can be one of the following: `schema` or
	// `table`.
	ObjectType *string
	// The UUID of the request object. It is `schema_id` for `schema`, and
	// `table_id` for `table`.
	//
	// Find the `schema_id` from either: 1. The [schema_id] of the `Schemas`
	// resource. 2. In [Catalog Explorer] > select the `schema` > go to the
	// `Details` tab > the `Schema ID` field.
	//
	// Find the `table_id` from either: 1. The [table_id] of the `Tables` resource.
	// 2. In [Catalog Explorer] > select the `table` > go to the `Details` tab > the
	// `Table ID` field.
	//
	// [Catalog Explorer]: https://docs.databricks.com/aws/en/catalog-explorer/
	// [schema_id]: https://docs.databricks.com/api/workspace/schemas/get#schema_id
	// [table_id]: https://docs.databricks.com/api/workspace/tables/get#table_id
	ObjectId  *string
	PageToken *string
	PageSize  *int
}

Request to list refreshes..

type ListRefreshResponse 

type ListRefreshResponse struct {
	Refreshes     []Refresh
	NextPageToken *string
}

Response for listing refreshes..

type Monitor 

type Monitor struct {
	// The type of the monitored object. Can be one of the following: `schema` or
	// `table`.
	ObjectType *string `fieldmask:"object_type"`
	// The UUID of the request object. It is `schema_id` for `schema`, and
	// `table_id` for `table`.
	//
	// Find the `schema_id` from either: 1. The [schema_id] of the `Schemas`
	// resource. 2. In [Catalog Explorer] > select the `schema` > go to the
	// `Details` tab > the `Schema ID` field.
	//
	// Find the `table_id` from either: 1. The [table_id] of the `Tables` resource.
	// 2. In [Catalog Explorer] > select the `table` > go to the `Details` tab > the
	// `Table ID` field.
	//
	// [Catalog Explorer]: https://docs.databricks.com/aws/en/catalog-explorer/
	// [schema_id]: https://docs.databricks.com/api/workspace/schemas/get#schema_id
	// [table_id]: https://docs.databricks.com/api/workspace/tables/get#table_id
	ObjectId *string `fieldmask:"object_id"`
	// Anomaly Detection Configuration, applicable to `schema` object types.
	AnomalyDetectionConfig *AnomalyDetectionConfig `fieldmask:"anomaly_detection_config"`
	// Data Profiling Configuration, applicable to `table` object types. Exactly one
	// `Analysis Configuration` must be present.
	DataProfilingConfig *DataProfilingConfig `fieldmask:"data_profiling_config"`
}

Monitor for the data quality of unity catalog entities such as schema or table..

type NotificationDestination 

type NotificationDestination struct {
	// The list of email addresses to send the notification to. A maximum of 5 email
	// addresses is supported.
	EmailAddresses []string `fieldmask:"email_addresses"`
}

Destination of the data quality monitoring notification..

type NotificationSettings 

type NotificationSettings struct {
	// Destinations to send notifications on failure/timeout.
	OnFailure *NotificationDestination `fieldmask:"on_failure"`
}

Settings for sending notifications on the data quality monitoring..

type Refresh 

type Refresh struct {
	// The type of the monitored object. Can be one of the following: `schema` or
	// `table`.
	ObjectType *string `fieldmask:"object_type"`
	// The UUID of the request object. It is `schema_id` for `schema`, and
	// `table_id` for `table`.
	//
	// Find the `schema_id` from either: 1. The [schema_id] of the `Schemas`
	// resource. 2. In [Catalog Explorer] > select the `schema` > go to the
	// `Details` tab > the `Schema ID` field.
	//
	// Find the `table_id` from either: 1. The [table_id] of the `Tables` resource.
	// 2. In [Catalog Explorer] > select the `table` > go to the `Details` tab > the
	// `Table ID` field.
	//
	// [Catalog Explorer]: https://docs.databricks.com/aws/en/catalog-explorer/
	// [schema_id]: https://docs.databricks.com/api/workspace/schemas/get#schema_id
	// [table_id]: https://docs.databricks.com/api/workspace/tables/get#table_id
	ObjectId *string `fieldmask:"object_id"`
	// Unique id of the refresh operation.
	RefreshId *int64 `fieldmask:"refresh_id"`
	// The current state of the refresh.
	State RefreshState `fieldmask:"state"`
	// An optional message to give insight into the current state of the refresh
	// (e.g. FAILURE messages).
	Message *string `fieldmask:"message"`
	// Time when the refresh started (milliseconds since 1/1/1970 UTC).
	StartTimeMs *int64 `fieldmask:"start_time_ms"`
	// Time when the refresh ended (milliseconds since 1/1/1970 UTC).
	EndTimeMs *int64 `fieldmask:"end_time_ms"`
	// What triggered the refresh.
	Trigger RefreshTrigger `fieldmask:"trigger"`
}

The Refresh object gives information on a refresh of the data quality monitoring pipeline..

type RefreshState 

type RefreshState string

The state of the refresh. 

const (
	RefreshState_Unspecified RefreshState = ""
	// The refresh is pending.
	RefreshState_MonitorRefreshStatePending RefreshState = "MONITOR_REFRESH_STATE_PENDING"
	// The refresh is running.
	RefreshState_MonitorRefreshStateRunning RefreshState = "MONITOR_REFRESH_STATE_RUNNING"
	// The refresh is successful.
	RefreshState_MonitorRefreshStateSuccess RefreshState = "MONITOR_REFRESH_STATE_SUCCESS"
	// The refresh has failed.
	RefreshState_MonitorRefreshStateFailed RefreshState = "MONITOR_REFRESH_STATE_FAILED"
	// The refresh is cancelled.
	RefreshState_MonitorRefreshStateCanceled RefreshState = "MONITOR_REFRESH_STATE_CANCELED"
)

type RefreshTrigger 

type RefreshTrigger string

The trigger of the refresh. 

const (
	RefreshTrigger_Unspecified RefreshTrigger = ""
	// The refresh has been triggered manually.
	RefreshTrigger_MonitorRefreshTriggerManual RefreshTrigger = "MONITOR_REFRESH_TRIGGER_MANUAL"
	// The refresh has been triggered from a schedule.
	RefreshTrigger_MonitorRefreshTriggerSchedule RefreshTrigger = "MONITOR_REFRESH_TRIGGER_SCHEDULE"
	// The refresh has been triggered from a data change.
	RefreshTrigger_MonitorRefreshTriggerDataChange RefreshTrigger = "MONITOR_REFRESH_TRIGGER_DATA_CHANGE"
)

type SnapshotConfig 

type SnapshotConfig struct {
}

Snapshot analysis configuration..

type TimeSeriesConfig 

type TimeSeriesConfig struct {
	// Column for the timestamp.
	TimestampColumn *string `fieldmask:"timestamp_column"`
	// List of granularities to use when aggregating data into time windows based on
	// their timestamp.
	Granularities []AggregationGranularity `fieldmask:"granularities"`
}

Time series analysis configuration..

type UpdateMonitorRequest 

type UpdateMonitorRequest struct {
	// The type of the monitored object. Can be one of the following: `schema` or
	// `table`.
	ObjectType *string
	// The UUID of the request object. It is `schema_id` for `schema`, and
	// `table_id` for `table`.
	//
	// Find the `schema_id` from either: 1. The [schema_id] of the `Schemas`
	// resource. 2. In [Catalog Explorer] > select the `schema` > go to the
	// `Details` tab > the `Schema ID` field.
	//
	// Find the `table_id` from either: 1. The [table_id] of the `Tables` resource.
	// 2. In [Catalog Explorer] > select the `table` > go to the `Details` tab > the
	// `Table ID` field.
	//
	// [Catalog Explorer]: https://docs.databricks.com/aws/en/catalog-explorer/
	// [schema_id]: https://docs.databricks.com/api/workspace/schemas/get#schema_id
	// [table_id]: https://docs.databricks.com/api/workspace/tables/get#table_id
	ObjectId *string
	// The monitor to update.
	Monitor *Monitor
	// The field mask to specify which fields to update as a comma-separated list.
	// Example value:
	// `data_profiling_config.custom_metrics,data_profiling_config.schedule.quartz_cron_expression`
	UpdateMask *types.FieldMask[Monitor]
}

Request to update a Monitor..

type UpdateRefreshRequest 

type UpdateRefreshRequest struct {
	// The type of the monitored object. Can be one of the following: `schema` or
	// `table`.
	ObjectType *string
	// The UUID of the request object. It is `schema_id` for `schema`, and
	// `table_id` for `table`.
	//
	// Find the `schema_id` from either: 1. The [schema_id] of the `Schemas`
	// resource. 2. In [Catalog Explorer] > select the `schema` > go to the
	// `Details` tab > the `Schema ID` field.
	//
	// Find the `table_id` from either: 1. The [table_id] of the `Tables` resource.
	// 2. In [Catalog Explorer] > select the `table` > go to the `Details` tab > the
	// `Table ID` field.
	//
	// [Catalog Explorer]: https://docs.databricks.com/aws/en/catalog-explorer/
	// [schema_id]: https://docs.databricks.com/api/workspace/schemas/get#schema_id
	// [table_id]: https://docs.databricks.com/api/workspace/tables/get#table_id
	ObjectId *string
	// Unique id of the refresh operation.
	RefreshId *int64
	// The refresh to update.
	Refresh *Refresh
	// The field mask to specify which fields to update.
	UpdateMask *types.FieldMask[Refresh]
}

Request to update a refresh..

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.