databrew

package
v1.1.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 20, 2026 License: MIT Imports: 20 Imported by: 0

README

Glue DataBrew

Parity grade: A · SDK aws-sdk-go-v2/service/databrew@v1.40.0 · last audited 2026-07-13 (782e2a93)

Coverage

Metric Value
Operations audited 45 (43 ok, 2 partial)
Feature families 1 (1 ok)
Known gaps 5
Deferred items 1
Resource leaks clean
Known gaps
  • CreateDataset/UpdateDataset don't accept PathOptions (S3 wildcard-path dataset config) -- optional field, not commonly exercised, silently ignored if sent (bd: TODO file if prioritized)
  • CreateProfileJob/UpdateProfileJob don't accept Configuration (ProfileConfiguration) or JobSample -- Job struct already has ProfileConfiguration/JobSample fields wired for JSON output but nothing ever populates them; would need threading through CreateJob/UpdateJob signatures (bd: TODO file if prioritized)
  • CreateRecipeJob/UpdateRecipeJob don't accept DataCatalogOutputs/DatabaseOutputs/EncryptionMode/EncryptionKeyArn/LogSubscription/ValidationConfigurations -- Job struct has matching JSON fields but they're never populated (bd: TODO file if prioritized)
  • Recipe version history is not modeled: only one working/published version is tracked per recipe (RecipeVersion flips between "0.1"/an unpublished value and "1.0" on PublishRecipe). BatchDeleteRecipeVersion/DeleteRecipeVersion/ListRecipeVersions all operate against that single version rather than a real version list; each PublishRecipe overwrites rather than appending a new version. A full fix needs a per-recipe version history data structure -- larger scope than this pass's budget (bd: TODO file if prioritized)
  • StartProjectSession/SendProjectSessionAction (the interactive project-editor session flow) are near-total no-ops beyond echoing Name -- acceptable since these model an interactive editing session tests don't poll for correctness, but flagged for completeness
Deferred
  • CSV/Excel/Json FormatOptions sub-fields (e.g. Delimiter, HeaderRow, SheetNames) are passed through as map[string]any rather than typed structs -- wire-compatible (arbitrary nested JSON round-trips byte-for-byte) but not validated

More

Documentation

Overview

Package databrew implements an in-memory AWS Glue DataBrew service backend.

Index

Constants

This section is empty.

Variables

View Source 
var (
	// ErrNotFound is returned when a requested resource does not exist.
	ErrNotFound = awserr.New("ResourceNotFoundException", awserr.ErrNotFound)
	// ErrAlreadyExists is returned when a resource already exists.
	ErrAlreadyExists = awserr.New("ConflictException", awserr.ErrAlreadyExists)
	// ErrValidation is returned when input validation fails.
	ErrValidation = awserr.New("ValidationException", awserr.ErrInvalidParameter)
)
View Source 
var ErrNilAppContext = errors.New("nil AppContext passed to DataBrew Provider.Init")

ErrNilAppContext is returned by Init when a nil AppContext is passed.

Functions

This section is empty.

Types

type DataCatalogInput 

type DataCatalogInput struct {
	DatabaseName string `json:"DatabaseName"`
	TableName    string `json:"TableName"`
}

DataCatalogInput references a Glue Data Catalog table.

type DatabaseInput 

type DatabaseInput struct {
	GlueConnectionName string `json:"GlueConnectionName"`
	DatabaseTableName  string `json:"DatabaseTableName"`
}

DatabaseInput references a database table.

type Dataset 

type Dataset struct {
	FormatOptions    DatasetFormatOptions `json:"FormatOptions,omitzero"`
	Input            DatasetInput         `json:"Input,omitzero"`
	Tags             map[string]string    `json:"Tags,omitempty"`
	Name             string               `json:"Name"`
	Arn              string               `json:"ResourceArn"`
	Format           string               `json:"Format,omitempty"`
	Source           string               `json:"Source,omitempty"`
	CreatedBy        string               `json:"CreatedBy,omitempty"`
	LastModifiedBy   string               `json:"LastModifiedBy,omitempty"`
	CreateDate       float64              `json:"CreateDate,omitempty"`
	LastModifiedDate float64              `json:"LastModifiedDate,omitempty"`
}

Dataset represents a DataBrew dataset.

type DatasetFormatOptions 

type DatasetFormatOptions struct {
	Csv   map[string]any `json:"Csv,omitempty"`
	Excel map[string]any `json:"Excel,omitempty"`
	JSON  map[string]any `json:"Json,omitempty"`
}

DatasetFormatOptions holds format-specific options for a dataset.

The JSON field's wire key is "Json" (mixed case), NOT "JSON" -- confirmed against aws-sdk-go-v2/service/databrew's deserializer (awsRestjson1_deserializeDocumentFormatOptions switches on the exact, case-sensitive key "Json"). A response emitting the Go-idiomatic "JSON" falls through that switch's default case and the client silently drops the field, so a dataset created with JSON format options would appear to have none on describe/list.

type DatasetInput 

type DatasetInput struct {
	S3InputDefinition          *S3Location       `json:"S3InputDefinition,omitempty"`
	DataCatalogInputDefinition *DataCatalogInput `json:"DataCatalogInputDefinition,omitempty"`
	DatabaseInputDefinition    *DatabaseInput    `json:"DatabaseInputDefinition,omitempty"`
}

DatasetInput holds the data source for a dataset.

type Handler 

type Handler struct {
	Backend   StorageBackend
	AccountID string
	Region    string
}

Handler is the HTTP handler for AWS Glue DataBrew operations.

func NewHandler 

func NewHandler(backend StorageBackend) *Handler

NewHandler creates a new DataBrew handler.

func (*Handler) ExtractOperation 

func (h *Handler) ExtractOperation(c *echo.Context) string

func (*Handler) ExtractResource 

func (h *Handler) ExtractResource(c *echo.Context) string

func (*Handler) GetSupportedOperations 

func (h *Handler) GetSupportedOperations() []string

func (*Handler) Handler 

func (h *Handler) Handler() echo.HandlerFunc

func (*Handler) MatchPriority 

func (h *Handler) MatchPriority() int

MatchPriority returns PriorityPathVersioned+1 so DataBrew is evaluated before IoT Analytics, which also claims /datasets at PriorityPathVersioned.

func (*Handler) Name 

func (h *Handler) Name() string

func (*Handler) Reset 

func (h *Handler) Reset()

func (*Handler) Restore 

func (h *Handler) Restore(ctx context.Context, data []byte) error

Restore implements persistence.Persistable by delegating to the backend.

func (*Handler) RouteMatcher 

func (h *Handler) RouteMatcher() service.Matcher

func (*Handler) Shutdown 

func (h *Handler) Shutdown(ctx context.Context)

Shutdown implements service.Shutdowner. It cancels in-flight job run transition goroutines and waits for them to drain, bounded by ctx.

func (*Handler) Snapshot 

func (h *Handler) Snapshot(ctx context.Context) []byte

Snapshot implements persistence.Persistable by delegating to the backend. Handler previously had no Snapshot/Restore of its own -- and neither did InMemoryBackend -- so cli.go's generic setupPersistence (which type-asserts the registered service.Registerable, i.e. the Handler, for a Snapshot/Restore pair) never picked DataBrew up at all: dead wiring, with no persistence underneath it either. This delegation (matching the codecommit/codepipeline/cleanrooms pattern) is what wires DataBrew into persistence for the first time.

func (*Handler) StartWorker 

func (h *Handler) StartWorker(_ context.Context) error

type InMemoryBackend 

type InMemoryBackend struct {
	// contains filtered or unexported fields
}

InMemoryBackend stores DataBrew state in memory.

All resource collections are nested by region (outer key = region) so that same-named resources are isolated across regions. datasets/recipes/ projects/jobs/rulesets/schedules each hold one *store.Table[T] per region, created lazily via the *Table accessors in store_setup.go — mirroring the lazy map creation the hand-rolled *Store helpers did before the Phase 3.3 pkgs/store conversion (see store_setup.go's package doc for why jobRuns is NOT converted). Callers must hold b.mu while accessing any of them.

func NewInMemoryBackend 

func NewInMemoryBackend(accountID, region string) *InMemoryBackend

NewInMemoryBackend creates a new in-memory DataBrew backend with a background lifecycle context.

func NewInMemoryBackendWithContext 

func NewInMemoryBackendWithContext(
	svcCtx context.Context,
	accountID, region string,
) *InMemoryBackend

NewInMemoryBackendWithContext creates a new in-memory DataBrew backend whose delayed lifecycle goroutines are tied to svcCtx. When svcCtx (or the backend's Shutdown) is cancelled, in-flight transition goroutines exit promptly. If svcCtx is nil, context.Background is used.

func (*InMemoryBackend) AccountID 

func (b *InMemoryBackend) AccountID() string

func (*InMemoryBackend) CreateDataset 

func (b *InMemoryBackend) CreateDataset(
	ctx context.Context,
	name, format string,
	input DatasetInput,
	formatOpts DatasetFormatOptions,
	tags map[string]string,
) (*Dataset, error)

func (*InMemoryBackend) CreateJob 

func (b *InMemoryBackend) CreateJob(
	ctx context.Context,
	name, jobType, datasetName, projectName, recipeName, roleArn string,
	outputs []Output,
	tags map[string]string,
) (*Job, error)

func (*InMemoryBackend) CreateProject 

func (b *InMemoryBackend) CreateProject(
	ctx context.Context,
	name, datasetName, recipeName, roleArn string,
	sample Sample,
	tags map[string]string,
) (*Project, error)

func (*InMemoryBackend) CreateRecipe 

func (b *InMemoryBackend) CreateRecipe(
	ctx context.Context,
	name, description string,
	steps []RecipeStep,
	tags map[string]string,
) (*Recipe, error)

func (*InMemoryBackend) CreateRuleset 

func (b *InMemoryBackend) CreateRuleset(
	ctx context.Context,
	name, description, targetArn string,
	rules []Rule,
	tags map[string]string,
) (*Ruleset, error)

func (*InMemoryBackend) CreateSchedule 

func (b *InMemoryBackend) CreateSchedule(
	ctx context.Context,
	name string,
	jobNames []string,
	cron string,
	tags map[string]string,
) (*Schedule, error)

func (*InMemoryBackend) DeleteDataset 

func (b *InMemoryBackend) DeleteDataset(ctx context.Context, name string) error

func (*InMemoryBackend) DeleteJob 

func (b *InMemoryBackend) DeleteJob(ctx context.Context, name string) error

func (*InMemoryBackend) DeleteProject 

func (b *InMemoryBackend) DeleteProject(ctx context.Context, name string) error

func (*InMemoryBackend) DeleteRecipe 

func (b *InMemoryBackend) DeleteRecipe(ctx context.Context, name string) error

func (*InMemoryBackend) DeleteRuleset 

func (b *InMemoryBackend) DeleteRuleset(ctx context.Context, name string) error

func (*InMemoryBackend) DeleteSchedule 

func (b *InMemoryBackend) DeleteSchedule(ctx context.Context, name string) error

func (*InMemoryBackend) DescribeDataset 

func (b *InMemoryBackend) DescribeDataset(ctx context.Context, name string) (*Dataset, error)

func (*InMemoryBackend) DescribeJob 

func (b *InMemoryBackend) DescribeJob(ctx context.Context, name string) (*Job, error)

func (*InMemoryBackend) DescribeJobRun 

func (b *InMemoryBackend) DescribeJobRun(ctx context.Context, name, runID string) (*JobRun, error)

func (*InMemoryBackend) DescribeProject 

func (b *InMemoryBackend) DescribeProject(ctx context.Context, name string) (*Project, error)

func (*InMemoryBackend) DescribeRecipe 

func (b *InMemoryBackend) DescribeRecipe(ctx context.Context, name string) (*Recipe, error)

func (*InMemoryBackend) DescribeRuleset 

func (b *InMemoryBackend) DescribeRuleset(ctx context.Context, name string) (*Ruleset, error)

func (*InMemoryBackend) DescribeSchedule 

func (b *InMemoryBackend) DescribeSchedule(ctx context.Context, name string) (*Schedule, error)

func (*InMemoryBackend) FindTagsByArn 

func (b *InMemoryBackend) FindTagsByArn(
	ctx context.Context,
	arnVal string,
) (map[string]string, error)

FindTagsByArn searches all resources in the request region for a specific ARN and returns its tags.

func (*InMemoryBackend) ListDatasets 

func (b *InMemoryBackend) ListDatasets(
	ctx context.Context,
	maxResults int,
	nextToken string,
) ([]*Dataset, string)

func (*InMemoryBackend) ListJobRuns 

func (b *InMemoryBackend) ListJobRuns(
	ctx context.Context,
	jobName string,
	maxResults int,
	nextToken string,
) ([]*JobRun, string, error)

func (*InMemoryBackend) ListJobs 

func (b *InMemoryBackend) ListJobs(
	ctx context.Context,
	maxResults int,
	nextToken,
	datasetName,
	projectName string,
) ([]*Job, string)

func (*InMemoryBackend) ListProjects 

func (b *InMemoryBackend) ListProjects(
	ctx context.Context,
	maxResults int,
	nextToken string,
) ([]*Project, string)

func (*InMemoryBackend) ListRecipes 

func (b *InMemoryBackend) ListRecipes(
	ctx context.Context,
	maxResults int,
	nextToken string,
) ([]*Recipe, string)

func (*InMemoryBackend) ListRulesets 

func (b *InMemoryBackend) ListRulesets(
	ctx context.Context,
	maxResults int,
	nextToken, targetArn string,
) ([]*Ruleset, string)

func (*InMemoryBackend) ListSchedules 

func (b *InMemoryBackend) ListSchedules(
	ctx context.Context,
	maxResults int,
	nextToken string,
) ([]*Schedule, string)

func (*InMemoryBackend) PublishRecipe 

func (b *InMemoryBackend) PublishRecipe(ctx context.Context, name, description string) error

func (*InMemoryBackend) Region 

func (b *InMemoryBackend) Region() string

func (*InMemoryBackend) Reset 

func (b *InMemoryBackend) Reset()

func (*InMemoryBackend) Restore 

func (b *InMemoryBackend) Restore(ctx context.Context, data []byte) error

Restore loads backend state from a JSON snapshot. It implements persistence.Persistable.

func (*InMemoryBackend) Shutdown 

func (b *InMemoryBackend) Shutdown(ctx context.Context)

Shutdown cancels the backend's lifecycle context and waits for in-flight delayed goroutines to finish, bounded by ctx. After Shutdown the backend no longer schedules state transitions.

func (*InMemoryBackend) Snapshot 

func (b *InMemoryBackend) Snapshot(ctx context.Context) []byte

Snapshot serialises the backend state to JSON. It implements persistence.Persistable.

func (*InMemoryBackend) StartJobRun 

func (b *InMemoryBackend) StartJobRun(ctx context.Context, jobName string) (*JobRun, error)

StartJobRun creates a new job run with STARTING state, transitioning to SUCCEEDED asynchronously.

func (*InMemoryBackend) StopJobRun 

func (b *InMemoryBackend) StopJobRun(ctx context.Context, name, runID string) (*JobRun, error)

func (*InMemoryBackend) UpdateDataset 

func (b *InMemoryBackend) UpdateDataset(
	ctx context.Context,
	name, format string,
	input DatasetInput,
	formatOpts DatasetFormatOptions,
) error

func (*InMemoryBackend) UpdateJob 

func (b *InMemoryBackend) UpdateJob(
	ctx context.Context,
	name, roleArn string,
	outputs []Output,
	maxCapacity, maxRetries, timeout int,
) error

func (*InMemoryBackend) UpdateProject 

func (b *InMemoryBackend) UpdateProject(
	ctx context.Context,
	name, datasetName, roleArn string,
	sample Sample,
) error

func (*InMemoryBackend) UpdateRecipe 

func (b *InMemoryBackend) UpdateRecipe(
	ctx context.Context,
	name, description string,
	steps []RecipeStep,
) error

func (*InMemoryBackend) UpdateRuleset 

func (b *InMemoryBackend) UpdateRuleset(
	ctx context.Context,
	name, description string,
	rules []Rule,
) error

func (*InMemoryBackend) UpdateSchedule 

func (b *InMemoryBackend) UpdateSchedule(
	ctx context.Context,
	name string,
	jobNames []string,
	cron string,
) error

func (*InMemoryBackend) UpdateTagsByArn 

func (b *InMemoryBackend) UpdateTagsByArn(
	ctx context.Context,
	arnVal string,
	add map[string]string,
	remove []string,
) error

UpdateTagsByArn searches all resources in the request region and applies tags additions/removals.

type Job 

type Job struct {
	ProfileConfiguration     map[string]any    `json:"ProfileConfiguration,omitempty"`
	JobSample                map[string]any    `json:"JobSample,omitempty"`
	Tags                     map[string]string `json:"Tags,omitempty"`
	RecipeReference          *RecipeRef        `json:"RecipeReference,omitempty"`
	EncryptionMode           string            `json:"EncryptionMode,omitempty"`
	EncryptionKeyArn         string            `json:"EncryptionKeyArn,omitempty"`
	DatasetName              string            `json:"DatasetName,omitempty"`
	ProjectName              string            `json:"ProjectName,omitempty"`
	Name                     string            `json:"Name"`
	CreatedBy                string            `json:"CreatedBy,omitempty"`
	RecipeName               string            `json:"-"`
	RoleArn                  string            `json:"RoleArn,omitempty"`
	LogSubscription          string            `json:"LogSubscription,omitempty"`
	Type                     string            `json:"Type,omitempty"`
	LastModifiedBy           string            `json:"LastModifiedBy,omitempty"`
	Arn                      string            `json:"ResourceArn"`
	ValidationConfigurations []map[string]any  `json:"ValidationConfigurations,omitempty"`
	DataCatalogOutputs       []map[string]any  `json:"DataCatalogOutputs,omitempty"`
	DatabaseOutputs          []map[string]any  `json:"DatabaseOutputs,omitempty"`
	Outputs                  []Output          `json:"Outputs,omitempty"`
	Timeout                  int               `json:"Timeout,omitempty"`
	MaxRetries               int               `json:"MaxRetries,omitempty"`
	MaxCapacity              int               `json:"MaxCapacity,omitempty"`
	LastModifiedDate         float64           `json:"LastModifiedDate,omitempty"`
	CreateDate               float64           `json:"CreateDate,omitempty"`
}

Job represents a DataBrew job.

type JobRun 

type JobRun struct {
	DatasetName   string  `json:"DatasetName,omitempty"`
	JobName       string  `json:"JobName"`
	RunID         string  `json:"RunId"`
	State         string  `json:"State"`
	LogGroupName  string  `json:"LogGroupName,omitempty"`
	StartedOn     float64 `json:"StartedOn,omitempty"`
	CompletedOn   float64 `json:"CompletedOn,omitempty"`
	ExecutionTime int     `json:"ExecutionTime,omitempty"`
}

JobRun represents a single execution of a DataBrew job.

type Output 

type Output struct {
	FormatOptions     map[string]any `json:"FormatOptions,omitempty"`
	Location          S3Location     `json:"Location,omitzero"`
	Format            string         `json:"Format,omitempty"`
	CompressionFormat string         `json:"CompressionFormat,omitempty"`
	PartitionColumns  []string       `json:"PartitionColumns,omitempty"`
	MaxOutputFiles    int            `json:"MaxOutputFiles,omitempty"`
	Overwrite         bool           `json:"Overwrite,omitempty"`
}

Output describes a DataBrew job output destination.

type Project 

type Project struct {
	Tags             map[string]string `json:"Tags,omitempty"`
	Name             string            `json:"Name"`
	Arn              string            `json:"ResourceArn"`
	DatasetName      string            `json:"DatasetName,omitempty"`
	RecipeName       string            `json:"RecipeName"`
	RoleArn          string            `json:"RoleArn,omitempty"`
	SessionStatus    string            `json:"SessionStatus,omitempty"`
	CreatedBy        string            `json:"CreatedBy,omitempty"`
	LastModifiedBy   string            `json:"LastModifiedBy,omitempty"`
	Sample           Sample            `json:"Sample,omitzero"`
	CreateDate       float64           `json:"CreateDate,omitempty"`
	LastModifiedDate float64           `json:"LastModifiedDate,omitempty"`
}

Project represents a DataBrew project.

type Provider 

type Provider struct{}

Provider implements service.Provider for AWS Glue DataBrew.

func (*Provider) Init 

func (p *Provider) Init(ctx *service.AppContext) (service.Registerable, error)

Init initializes the DataBrew service backend and handler.

func (*Provider) Name 

func (p *Provider) Name() string

Name returns the provider name.

type Recipe 

type Recipe struct {
	Tags             map[string]string `json:"Tags,omitempty"`
	Name             string            `json:"Name"`
	Arn              string            `json:"ResourceArn"`
	Description      string            `json:"Description,omitempty"`
	PublishedBy      string            `json:"PublishedBy,omitempty"`
	RecipeVersion    string            `json:"RecipeVersion,omitempty"`
	CreatedBy        string            `json:"CreatedBy,omitempty"`
	LastModifiedBy   string            `json:"LastModifiedBy,omitempty"`
	Steps            []RecipeStep      `json:"Steps,omitempty"`
	PublishedDate    float64           `json:"PublishedDate,omitempty"`
	CreateDate       float64           `json:"CreateDate,omitempty"`
	LastModifiedDate float64           `json:"LastModifiedDate,omitempty"`
}

Recipe represents a DataBrew recipe.

type RecipeRef 

type RecipeRef struct {
	Name          string `json:"Name"`
	RecipeVersion string `json:"RecipeVersion,omitempty"`
}

RecipeRef holds a reference to a DataBrew recipe and optional version.

type RecipeStep 

type RecipeStep struct {
	Action               map[string]any   `json:"Action,omitempty"`
	ConditionExpressions []map[string]any `json:"ConditionExpressions,omitempty"`
}

RecipeStep is one transformation step in a recipe.

type Rule 

type Rule struct {
	SubstitutionMap map[string]string `json:"SubstitutionMap,omitempty"`
	Threshold       map[string]any    `json:"Threshold,omitempty"`
	Name            string            `json:"Name"`
	CheckExpression string            `json:"CheckExpression"`
	ColumnSelectors []map[string]any  `json:"ColumnSelectors,omitempty"`
	Disabled        bool              `json:"Disabled,omitempty"`
}

Rule represents a data quality rule.

type Ruleset 

type Ruleset struct {
	Tags             map[string]string `json:"Tags,omitempty"`
	Name             string            `json:"Name"`
	Arn              string            `json:"ResourceArn"`
	Description      string            `json:"Description,omitempty"`
	TargetArn        string            `json:"TargetArn"`
	CreatedBy        string            `json:"CreatedBy,omitempty"`
	LastModifiedBy   string            `json:"LastModifiedBy,omitempty"`
	Rules            []Rule            `json:"Rules"`
	CreateDate       float64           `json:"CreateDate,omitempty"`
	LastModifiedDate float64           `json:"LastModifiedDate,omitempty"`
}

Ruleset represents a DataBrew data quality ruleset.

type S3Location 

type S3Location struct {
	Bucket string `json:"Bucket"`
	Key    string `json:"Key,omitempty"`
}

S3Location references an S3 path.

type Sample 

type Sample struct {
	Type string `json:"Type,omitempty"`
	Size int    `json:"Size,omitempty"`
}

Sample describes a data sample for a project.

type Schedule 

type Schedule struct {
	Tags             map[string]string `json:"Tags,omitempty"`
	Name             string            `json:"Name"`
	Arn              string            `json:"ResourceArn"`
	CronExpression   string            `json:"CronExpression"`
	CreatedBy        string            `json:"CreatedBy,omitempty"`
	LastModifiedBy   string            `json:"LastModifiedBy,omitempty"`
	JobNames         []string          `json:"JobNames,omitempty"`
	CreateDate       float64           `json:"CreateDate,omitempty"`
	LastModifiedDate float64           `json:"LastModifiedDate,omitempty"`
}

Schedule represents a DataBrew schedule.

type StorageBackend 

type StorageBackend interface {
	Region() string
	AccountID() string
	Reset()

	// Snapshot and Restore implement persistence.Persistable. Handler
	// delegates to them (see persistence.go) so cli.go's generic
	// setupPersistence picks DataBrew up.
	Snapshot(ctx context.Context) []byte
	Restore(ctx context.Context, data []byte) error

	// Dataset operations.
	CreateDataset(
		ctx context.Context,
		name, format string,
		input DatasetInput,
		formatOpts DatasetFormatOptions,
		tags map[string]string,
	) (*Dataset, error)
	DescribeDataset(ctx context.Context, name string) (*Dataset, error)
	ListDatasets(ctx context.Context, maxResults int, nextToken string) ([]*Dataset, string)
	UpdateDataset(
		ctx context.Context,
		name, format string,
		input DatasetInput,
		formatOpts DatasetFormatOptions,
	) error
	DeleteDataset(ctx context.Context, name string) error

	// Recipe operations.
	CreateRecipe(
		ctx context.Context,
		name, description string,
		steps []RecipeStep,
		tags map[string]string,
	) (*Recipe, error)
	DescribeRecipe(ctx context.Context, name string) (*Recipe, error)
	ListRecipes(ctx context.Context, maxResults int, nextToken string) ([]*Recipe, string)
	PublishRecipe(ctx context.Context, name, description string) error
	UpdateRecipe(ctx context.Context, name, description string, steps []RecipeStep) error
	DeleteRecipe(ctx context.Context, name string) error

	// Project operations.
	CreateProject(
		ctx context.Context,
		name, datasetName, recipeName, roleArn string,
		sample Sample,
		tags map[string]string,
	) (*Project, error)
	DescribeProject(ctx context.Context, name string) (*Project, error)
	ListProjects(ctx context.Context, maxResults int, nextToken string) ([]*Project, string)
	UpdateProject(ctx context.Context, name, datasetName, roleArn string, sample Sample) error
	DeleteProject(ctx context.Context, name string) error

	// Job operations.
	CreateJob(
		ctx context.Context,
		name, jobType, datasetName, projectName, recipeName, roleArn string,
		outputs []Output,
		tags map[string]string,
	) (*Job, error)
	DescribeJob(ctx context.Context, name string) (*Job, error)
	ListJobs(ctx context.Context, maxResults int, nextToken, datasetName, projectName string) ([]*Job, string)
	UpdateJob(
		ctx context.Context,
		name, roleArn string,
		outputs []Output,
		maxCapacity, maxRetries, timeout int,
	) error
	DeleteJob(ctx context.Context, name string) error
	StartJobRun(ctx context.Context, jobName string) (*JobRun, error)
	ListJobRuns(
		ctx context.Context,
		jobName string,
		maxResults int,
		nextToken string,
	) ([]*JobRun, string, error)
	DescribeJobRun(ctx context.Context, name, runID string) (*JobRun, error)
	StopJobRun(ctx context.Context, name, runID string) (*JobRun, error)

	// Ruleset operations.
	CreateRuleset(
		ctx context.Context,
		name, description, targetArn string,
		rules []Rule,
		tags map[string]string,
	) (*Ruleset, error)
	DescribeRuleset(ctx context.Context, name string) (*Ruleset, error)
	ListRulesets(ctx context.Context, maxResults int, nextToken, targetArn string) ([]*Ruleset, string)
	UpdateRuleset(ctx context.Context, name, description string, rules []Rule) error
	DeleteRuleset(ctx context.Context, name string) error

	// Schedule operations.
	CreateSchedule(
		ctx context.Context,
		name string,
		jobNames []string,
		cron string,
		tags map[string]string,
	) (*Schedule, error)
	DescribeSchedule(ctx context.Context, name string) (*Schedule, error)
	ListSchedules(ctx context.Context, maxResults int, nextToken string) ([]*Schedule, string)
	UpdateSchedule(ctx context.Context, name string, jobNames []string, cron string) error
	DeleteSchedule(ctx context.Context, name string) error

	// Tag operations.
	FindTagsByArn(ctx context.Context, arn string) (map[string]string, error)
	UpdateTagsByArn(ctx context.Context, arn string, add map[string]string, remove []string) error
}

StorageBackend defines the interface for all DataBrew backend operations.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.