Documentation
¶
Index ¶
- Constants
- func DiagsToError(prefix string, diags hcl.Diagnostics) error
- func GetMatrixItem(ctx context.Context) map[string]interface{}
- func IsCancelled(ctx context.Context) bool
- func Logger(ctx context.Context) hclog.Logger
- func RetryHydrate(ctx context.Context, d *QueryData, hydrateData *HydrateData, ...) (interface{}, error)
- func Serve(opts *ServeOpts)
- type Column
- type ConcurrencyManager
- type Connection
- type ConnectionConfigInstanceFunc
- type ConnectionConfigSchema
- type CreatePlugin
- type DefaultConcurrencyConfig
- type ErrorPredicate
- type GetConfig
- type HydrateCall
- type HydrateConfig
- type HydrateData
- type HydrateDependencies
- type HydrateFunc
- type KeyColumn
- type KeyColumnEqualsQualMap
- type KeyColumnQualMap
- type KeyColumnQuals
- type KeyColumnSlice
- func (k KeyColumnSlice) AllEquals() bool
- func (k KeyColumnSlice) IsAnyOf() bool
- func (k KeyColumnSlice) SingleEqualsQual() *KeyColumn
- func (k KeyColumnSlice) String() string
- func (k KeyColumnSlice) StringSlice() []string
- func (k KeyColumnSlice) ToProtobuf() []*proto.KeyColumn
- func (k KeyColumnSlice) Validate() []string
- type ListConfig
- type MatrixItemFunc
- type NewPluginOptions
- type Plugin
- func (p *Plugin) Execute(req *proto.ExecuteRequest, stream proto.WrapperPlugin_ExecuteServer) (err error)
- func (p *Plugin) GetSchema() (*grpc.PluginSchema, error)
- func (p *Plugin) Initialise()
- func (p *Plugin) SetConnectionConfig(connectionName, connectionConfigString string) (err error)
- func (p *Plugin) Validate() string
- type PluginFunc
- type QueryContext
- type QueryData
- type QueryStatus
- type RetryConfig
- type RowData
- type ServeOpts
- type Table
Constants ¶
const ( Required = "required" Optional = "optional" AnyOf = "any_of" )
const ( SchemaModeStatic = "static" SchemaModeDynamic = "dynamic" )
Variables ¶
This section is empty.
Functions ¶
func DiagsToError ¶ added in v0.2.0
func DiagsToError(prefix string, diags hcl.Diagnostics) error
DiagsToError converts hcl diags into an error
func GetMatrixItem ¶ added in v0.2.0
GetMatrixItem extracts the matrix item map with the given key from the context
func IsCancelled ¶ added in v1.3.0
IsCancelled is a helper function which returns whether the context has been cancelled
func RetryHydrate ¶ added in v0.2.8
func RetryHydrate(ctx context.Context, d *QueryData, hydrateData *HydrateData, hydrateFunc HydrateFunc, retryConfig *RetryConfig) (interface{}, error)
RetryHydrate function invokes the hydrate function with retryable errors and retries the function until the maximum attemptes before throwing error
Types ¶
type Column ¶
type Column struct {
// column name
Name string
// column type
Type proto.ColumnType
// column description
Description string
// explicitly specify the function which populates this data
// - this is only needed if any of the default hydrate functions wil NOT return this column
Hydrate HydrateFunc
// the default column value
Default interface{}
// a list of transforms to generate the column value
Transform *transform.ColumnTransforms
// contains filtered or unexported fields
}
Column contains column data, in a format compatible with proto ColumnDefinition
type ConcurrencyManager ¶ added in v0.1.1
type ConcurrencyManager struct {
// contains filtered or unexported fields
}
ConcurrencyManager struct ensures that hydrate functions stay within concurrency limits
func (*ConcurrencyManager) Close ¶ added in v0.2.0
func (c *ConcurrencyManager) Close()
Close executs when the query is complete and dumps out the concurrency stats
func (*ConcurrencyManager) DisplayConcurrencyStats ¶ added in v0.2.0
func (c *ConcurrencyManager) DisplayConcurrencyStats()
DisplayConcurrencyStats displays the the summary of all the concurrent hydrate calls
func (*ConcurrencyManager) Finished ¶ added in v0.1.1
func (c *ConcurrencyManager) Finished(name string)
Finished decrements the counter for the named function
func (*ConcurrencyManager) StartIfAllowed ¶ added in v0.1.1
func (c *ConcurrencyManager) StartIfAllowed(name string, maxCallConcurrency int) (res bool)
StartIfAllowed checks whether the named hydrate call is permitted to start based on the number of running instances of that call, and the total calls in progress
type Connection ¶ added in v0.2.0
type Connection struct {
Name string
// the connection config
// NOTE: we always pass and store connection config BY VALUE
Config interface{}
}
type ConnectionConfigInstanceFunc ¶ added in v0.2.0
type ConnectionConfigInstanceFunc func() interface{}
type ConnectionConfigSchema ¶ added in v0.2.0
type ConnectionConfigSchema struct {
Schema map[string]*schema.Attribute
// function which returns an instance of a connection config struct
NewInstance ConnectionConfigInstanceFunc
}
ConnectionConfigSchema struct is used to define the connection config schema and store the config for each plugin connection
func NewConnectionConfigSchema ¶ added in v0.2.0
func NewConnectionConfigSchema() *ConnectionConfigSchema
func (*ConnectionConfigSchema) Parse ¶ added in v0.2.0
func (c *ConnectionConfigSchema) Parse(configString string) (config interface{}, err error)
Parse function parses the hcl string into a connection config struct. The schema and the struct to parse into are provided by the plugin
func (*ConnectionConfigSchema) Validate ¶ added in v0.2.0
func (c *ConnectionConfigSchema) Validate() []string
Validate validates the connection config
type CreatePlugin ¶ added in v1.6.0
type DefaultConcurrencyConfig ¶ added in v0.2.0
type DefaultConcurrencyConfig struct {
// max number of ALL hydrate calls in progress
TotalMaxConcurrency int
DefaultMaxConcurrency int
}
DefaultConcurrencyConfig contains plugin level config to define default hydrate concurrency - this is used if no HydrateConfig is specified for a specific call
type ErrorPredicate ¶
type GetConfig ¶
type GetConfig struct {
// key or keys which are used to uniquely identify rows - used to determine whether a query is a 'get' call
KeyColumns KeyColumnSlice
// the hydrate function which is called first when performing a 'get' call.
// if this returns 'not found', no further hydrate functions are called
Hydrate HydrateFunc
// a function which will return whenther to ignore a given error
ShouldIgnoreError ErrorPredicate
RetryConfig *RetryConfig
}
type HydrateCall ¶
type HydrateCall struct {
Func HydrateFunc
// the dependencies expressed using function name
Depends []string
Config *HydrateConfig
Name string
}
HydrateCall struct encapsulates a hydrate call, its config and dependencies
func (HydrateCall) CanStart ¶
func (h HydrateCall) CanStart(rowData *RowData, name string, concurrencyManager *ConcurrencyManager) bool
func (*HydrateCall) Start ¶ added in v0.1.1
func (h *HydrateCall) Start(ctx context.Context, r *RowData, d *QueryData, concurrencyManager *ConcurrencyManager)
Start starts a hydrate call
type HydrateConfig ¶ added in v0.1.1
type HydrateConfig struct {
Func HydrateFunc
MaxConcurrency int
RetryConfig *RetryConfig
ShouldIgnoreError ErrorPredicate
Depends []HydrateFunc
}
HydrateConfig defines the hydrate function configurations, Name, Maximum number of concurrent calls to be allowed, dependencies
type HydrateData ¶
type HydrateData struct {
// if there was a parent-child list call, store the parent list item
ParentItem interface{}
Item interface{}
HydrateResults map[string]interface{}
}
HydrateData contains the input data passed to every hydrate function
type HydrateDependencies ¶
type HydrateDependencies struct {
Func HydrateFunc
Depends []HydrateFunc
}
HydrateDependencies defines the hydrate function dependencies - other hydrate functions which must be run first Deprecated: used HydrateConfig
type HydrateFunc ¶
type HydrateFunc func(context.Context, *QueryData, *HydrateData) (interface{}, error)
HydrateFunc is a function which retrieves some or all row data for a single row item.
func WrapHydrate ¶ added in v0.2.8
func WrapHydrate(hydrateFunc HydrateFunc, shouldIgnoreError ErrorPredicate) HydrateFunc
WrapHydrate is a higher order function which returns a HydrateFunc which handles Ignorable errors
func (HydrateFunc) WithCache ¶ added in v1.3.0
func (hydrate HydrateFunc) WithCache(args ...HydrateFunc) HydrateFunc
WithCache is a chainable function which wraps a hydrate call with caching and checks for pending execution of the same function
type KeyColumn ¶ added in v1.3.0
KeyColumn is a struct representing the definition of a KeyColumn used to filter and Get/List call
func (*KeyColumn) InitialiseOperators ¶ added in v1.3.0
func (k *KeyColumn) InitialiseOperators()
InitialiseOperators adds a default '=' operator is no operators are set, and converts "!=" to "<>"
func (*KeyColumn) SingleEqualsQual ¶ added in v1.3.0
SingleEqualsQual returns whether this key column has a single = operator
func (*KeyColumn) ToProtobuf ¶ added in v1.3.0
ToProtobuf converts the KeyColumn to a protobuf object
type KeyColumnEqualsQualMap ¶ added in v1.3.0
KeyColumnEqualsQualMap is a map of column name to qual value, used to represent a map of any equals quals
func (KeyColumnEqualsQualMap) GetListQualValues ¶ added in v1.8.3
func (m KeyColumnEqualsQualMap) GetListQualValues() map[string]*proto.QualValueList
GetListQualValues returns a map of all qual values with a List value
func (KeyColumnEqualsQualMap) String ¶ added in v1.3.0
func (m KeyColumnEqualsQualMap) String() string
type KeyColumnQualMap ¶ added in v1.3.0
type KeyColumnQualMap map[string]*KeyColumnQuals
KeyColumnQualMap is a map of KeyColumnQuals keyed by column name
func NewKeyColumnQualValueMap ¶ added in v1.3.0
func NewKeyColumnQualValueMap(qualMap map[string]*proto.Quals, keyColumns KeyColumnSlice) KeyColumnQualMap
NewKeyColumnQualValueMap creates a KeyColumnQualMap from a qual map and a KeyColumnSlice
func (KeyColumnQualMap) GetUnsatisfiedKeyColumns ¶ added in v1.5.0
func (m KeyColumnQualMap) GetUnsatisfiedKeyColumns(columns KeyColumnSlice) KeyColumnSlice
func (KeyColumnQualMap) String ¶ added in v1.3.0
func (m KeyColumnQualMap) String() string
func (KeyColumnQualMap) ToEqualsQualValueMap ¶ added in v1.3.0
func (m KeyColumnQualMap) ToEqualsQualValueMap() map[string]*proto.QualValue
ToEqualsQualValueMap converts a KeyColumnQualMap to a column-qual value map, including only the
type KeyColumnQuals ¶ added in v1.3.0
KeyColumnQuals is a struct representing all quals for a specific column
func (KeyColumnQuals) SatisfiesKeyColumn ¶ added in v1.3.0
func (k KeyColumnQuals) SatisfiesKeyColumn(keyColumn *KeyColumn) bool
func (KeyColumnQuals) SingleEqualsQual ¶ added in v1.3.0
func (k KeyColumnQuals) SingleEqualsQual() bool
type KeyColumnSlice ¶ added in v1.3.0
type KeyColumnSlice []*KeyColumn
func AllColumns ¶
func AllColumns(columns []string) KeyColumnSlice
AllColumns creates a KeyColumnSlice based on a slice of column names, each with a single equals operator and Require=Required
func AnyColumn ¶
func AnyColumn(columns []string) KeyColumnSlice
AnyColumn Columns creates a KeyColumnSlice based on a slice of column names, each with a single equals operator and Require=AnyOf
func NewEqualsKeyColumnSlice ¶ added in v1.3.0
func NewEqualsKeyColumnSlice(columns []string, require string) KeyColumnSlice
NewEqualsKeyColumnSlice creates a KeyColumnSlice from a list of column names, each with a single equals operator
func OptionalColumns ¶ added in v1.3.0
func OptionalColumns(columns []string) KeyColumnSlice
OptionalColumns Columns creates a KeyColumnSlice based on a slice of column names, with a single equals operator and Require=Optional
func SingleColumn ¶
func SingleColumn(column string) KeyColumnSlice
SingleColumn creates a KeyColumnSlice based on a column name The created slice has a single KeyColumn using a single equals operator and Require=Required
func (KeyColumnSlice) AllEquals ¶ added in v1.3.0
func (k KeyColumnSlice) AllEquals() bool
AllEquals returns whether all KeyColumns only use equals operators
func (KeyColumnSlice) IsAnyOf ¶ added in v1.6.2
func (k KeyColumnSlice) IsAnyOf() bool
IsAnyOf returns whether all key columns have Require == AnyOf
func (KeyColumnSlice) SingleEqualsQual ¶ added in v1.3.0
func (k KeyColumnSlice) SingleEqualsQual() *KeyColumn
SingleEqualsQual determines whether this key column slice has a single qual with a single = operator and if so returns it
func (KeyColumnSlice) String ¶ added in v1.3.0
func (k KeyColumnSlice) String() string
func (KeyColumnSlice) StringSlice ¶ added in v1.3.0
func (k KeyColumnSlice) StringSlice() []string
StringSlice converts a KeyColumnSlice to a slice of strings
func (KeyColumnSlice) ToProtobuf ¶ added in v1.3.0
func (k KeyColumnSlice) ToProtobuf() []*proto.KeyColumn
ToProtobuf converts the KeyColumnSlice to a slice of protobuf KeyColumns
func (KeyColumnSlice) Validate ¶ added in v1.3.0
func (k KeyColumnSlice) Validate() []string
Validate validates all child columns
type ListConfig ¶
type ListConfig struct {
KeyColumns KeyColumnSlice
// the list function, this should stream the list results back using the QueryData object, and return nil
Hydrate HydrateFunc
// the parent list function - if we list items with a parent-child relationship, this will list the parent items
ParentHydrate HydrateFunc
ShouldIgnoreError ErrorPredicate
RetryConfig *RetryConfig
}
type MatrixItemFunc ¶ added in v0.2.0
type MatrixItemFunc func(context.Context, *Connection) []map[string]interface{}
type NewPluginOptions ¶ added in v1.6.0
type Plugin ¶
type Plugin struct {
Name string
Logger hclog.Logger
TableMap map[string]*Table
// TableMapFunc is a callback function which can be used to populate the table map
// this con optionally be provided by the plugin, and allows the connection config to be used in the table creation
// (connection config is not available at plugin creation time)
TableMapFunc func(ctx context.Context, p *Plugin) (map[string]*Table, error)
DefaultTransform *transform.ColumnTransforms
DefaultGetConfig *GetConfig
DefaultConcurrency *DefaultConcurrencyConfig
DefaultRetryConfig *RetryConfig
// every table must implement these columns
RequiredColumns []*Column
ConnectionConfigSchema *ConnectionConfigSchema
// connection this plugin is instantiated for
Connection *Connection
// object to handle caching of connection specific data
ConnectionManager *connection_manager.Manager
// is this a static or dynamic schema
SchemaMode string
Schema map[string]*proto.TableSchema
// ConnectionConfigChangedFunc is a callback function executed every time the connection config is updated
// NOTE: it is NOT executed when it is set for the first time (???)
ConnectionConfigChangedFunc func() error
// contains filtered or unexported fields
}
Plugin is an object used to build all necessary data for a given query
func (*Plugin) Execute ¶
func (p *Plugin) Execute(req *proto.ExecuteRequest, stream proto.WrapperPlugin_ExecuteServer) (err error)
Execute executes a query and stream the results
func (*Plugin) Initialise ¶ added in v0.2.0
func (p *Plugin) Initialise()
Initialise initialises the connection config map, set plugin pointer on all tables and setup logger
func (*Plugin) SetConnectionConfig ¶ added in v0.2.0
SetConnectionConfig is always called before any other plugin function it parses the connection config string, and populate the connection data for this connection it also calls the table creation factory function, if provided by the plugin
type PluginFunc ¶
type QueryContext ¶ added in v1.3.0
func NewQueryContext ¶ added in v1.3.0
func NewQueryContext(p *proto.QueryContext) *QueryContext
NewQueryContext maps from a proto.QueryContext to a plugin.QueryContext. the only difference is the representation of the limit (as protobuf does not support pointers)
type QueryData ¶
type QueryData struct {
// The table this query is associated with
Table *Table
// if this is a get call this will be populated with the quals as a map of column name to quals
// (this will also be populated for a list call if list key columns are specified -
// however this usage is deprecated and provided for legacy reasons only)
KeyColumnQuals KeyColumnEqualsQualMap
// a map of all key column quals which were specified in the query
Quals KeyColumnQualMap
// columns which have a single equals qual
// is this a 'get' or a 'list' call
FetchType fetchType
// query context data passed from postgres - this includes the requested columns and the quals
QueryContext *QueryContext
// the status of the in-progress query
QueryStatus *QueryStatus
// connection details - the connection name and any config declared in the connection config file
Connection *Connection
// Matrix is an array of parameter maps (MatrixItems)
// the list/get calls with be executed for each element of this array
Matrix []map[string]interface{}
// object to handle caching of connection specific data
ConnectionManager *connection_manager.Manager
// streaming funcs
StreamListItem func(ctx context.Context, item interface{})
// deprecated - plugins should no longer call StreamLeafListItem directly and should just call StreamListItem
// event for the child list of a parent child list call
StreamLeafListItem func(ctx context.Context, item interface{})
// contains filtered or unexported fields
}
func (*QueryData) KeyColumnQualString ¶ added in v1.3.0
KeyColumnQualString looks for the specified key column quals and if it exists, return the value as a string
func (*QueryData) ShallowCopy ¶ added in v0.2.4
ShallowCopy creates a shallow copy of the QueryData this is used to pass different quals to multiple list/get calls, when an in() clause is specified
type QueryStatus ¶ added in v1.6.0
type QueryStatus struct {
// contains filtered or unexported fields
}
func (*QueryStatus) RowsRemaining ¶ added in v1.6.0
func (s *QueryStatus) RowsRemaining(ctx context.Context) int
RowsRemaining returns how many rows are required to complete the query
- if no limit has been parsed from the query, this will return math.MaxInt32 (meaning an unknown number of rows remain)
- if there is a limit, it will return the number of rows required to reach this limit
- if the context has been cancelled, it will return zero
type RetryConfig ¶ added in v0.2.8
type RetryConfig struct {
ShouldRetryError ErrorPredicate
}
type RowData ¶
type RowData struct {
// the output of the get/list call which is passed to all other hydrate calls
Item interface{}
// if there was a parent-child list call, store the parent list item
ParentItem interface{}
// contains filtered or unexported fields
}
RowData contains the row data
func (*RowData) GetColumnData ¶
GetColumnData returns the root item, and, if this column has a hydrate function registered, the associated hydrate data
type ServeOpts ¶
type ServeOpts struct {
PluginName string
PluginFunc PluginFunc
}
ServeOpts are the configurations to serve a plugin.
type Table ¶
type Table struct {
Name string
// table description
Description string
// column definitions
Columns []*Column
List *ListConfig
Get *GetConfig
GetMatrixItem MatrixItemFunc
DefaultTransform *transform.ColumnTransforms
// the parent plugin object
Plugin *Plugin
// definitions of dependencies between hydrate functions
HydrateDependencies []HydrateDependencies
HydrateConfig []HydrateConfig
// contains filtered or unexported fields
}
Table represents a plugin table
func (Table) GetSchema ¶
func (t Table) GetSchema() *proto.TableSchema
func (*Table) ValidateColumnsExist ¶ added in v1.3.0
func (t *Table) ValidateColumnsExist(keyColumns KeyColumnSlice) []string
Source Files
¶
- column.go
- concurrency.go
- connection_config.go
- connection_config_validate.go
- context.go
- hydrate.go
- hydrate_cache.go
- hydrate_call.go
- key_column.go
- key_column_qual.go
- key_column_qual_map.go
- key_column_qual_value_map.go
- key_column_slice.go
- key_column_slice_create.go
- plugin.go
- plugin_validate.go
- query_context.go
- query_data.go
- query_status.go
- required_hydrate_calls.go
- row.go
- serve.go
- table.go
- table_column.go
- table_fetch.go
- table_schema.go
- table_validate.go
Directories
¶
| Path | Synopsis |
|---|---|
|
Transform package provides the ability to transform data from APIs.
|
Transform package provides the ability to transform data from APIs. |