README
¶
Amazon Redshift
Extract table metadata from Amazon Redshift clusters.
Usage
source:
name: redshift
config:
cluster_id: cluster_test
db_name: testDB
db_user: testUser
aws_region: us-east-1
exclude:
databases:
- secondaryDB
Configuration
| Key | Type | Required | Description |
|---|---|---|---|
cluster_id |
string |
Yes | Cluster ID to access the Redshift cluster |
db_name |
string |
Yes | Default database name for the Redshift cluster |
db_user |
string |
Yes | Database username for the Redshift cluster |
aws_region |
string |
Yes | AWS region of the Redshift cluster |
exclude.databases |
[]string |
No | List of databases to exclude |
Authentication uses the default AWS credential chain (environment variables, shared credentials file, or IAM role).
Entities
- Entity type:
table - Source:
redshift - URN format:
urn:redshift:{cluster_id}:table:{cluster_id}.{database}.{table}
Properties
| Property | Type | Description |
|---|---|---|
properties.columns |
array |
List of column metadata |
properties.columns[].name |
string |
Column name |
properties.columns[].data_type |
string |
Data type of the column |
properties.columns[].is_nullable |
bool |
Whether the column is nullable |
properties.columns[].description |
string |
Column label (omitted if empty) |
properties.columns[].length |
int |
Column length (omitted if 0) |
Edges
This extractor does not emit edges.
Contributing
Refer to the contribution guidelines for information on contributing to this module.
Documentation
¶
Index ¶
- type Config
- type Exclude
- type Extractor
- func (e *Extractor) Extract(_ context.Context, emit plugins.Emit) error
- func (e *Extractor) GetColumn(dbName, tableName string) ([]*redshiftdataapiservice.ColumnMetadata, error)
- func (e *Extractor) GetDBList() ([]string, error)
- func (e *Extractor) GetTables(dbName string) ([]string, error)
- func (e *Extractor) Init(ctx context.Context, config plugins.Config) error
- type Option
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Config ¶
type Config struct {
ClusterID string `json:"cluster_id" yaml:"cluster_id" mapstructure:"cluster_id" validate:"required"`
DBName string `json:"db_name" yaml:"db_name" mapstructure:"db_name" validate:"required"`
DBUser string `json:"db_user" yaml:"db_user" mapstructure:"db_user" validate:"required"`
AWSRegion string `json:"aws_region" yaml:"aws_region" mapstructure:"aws_region" validate:"required"`
Exclude Exclude `json:"exclude" yaml:"exclude" mapstructure:"exclude"`
}
Config holds the set of configuration for the redshift extractor
type Exclude ¶ added in v0.13.0
type Exclude struct {
Databases []string `json:"databases" yaml:"databases" mapstructure:"databases"`
}
Exclude contains the list of databases to skip during extraction.
type Extractor ¶
type Extractor struct {
plugins.BaseExtractor
// contains filtered or unexported fields
}
Extractor manages the extraction of data from the redshift server
func (*Extractor) Extract ¶
Extract collects metadata from the source. Metadata is collected through the emitter
func (*Extractor) GetColumn ¶
func (e *Extractor) GetColumn(dbName, tableName string) ([]*redshiftdataapiservice.ColumnMetadata, error)
GetColumn returns the column metadata of particular table in a database
type Option ¶
type Option func(*Extractor)
Option provides extension abstraction to Extractor constructor
func WithClient ¶
func WithClient(redshiftClient redshiftdataapiserviceiface.RedshiftDataAPIServiceAPI) Option
WithClient assign custom client to the Extractor constructor
Source Files