Documentation
¶
Index ¶
- func NewColumnCheckOperator(manager config.ConnectionGetter) *ansisql.ColumnCheckOperator
- func NewMaterializer(fullRefresh bool) *pipeline.Materializer
- type AcceptedValuesCheck
- type BasicOperator
- type Client
- func (c *Client) CreateSchemaIfNotExist(ctx context.Context, asset *pipeline.Asset) error
- func (c *Client) GetDatabaseSummary(ctx context.Context) (*ansisql.DBDatabase, error)
- func (c *Client) Ping(ctx context.Context) error
- func (c *Client) RunQueryWithoutResult(ctx context.Context, query *query.Query) error
- func (c *Client) Select(ctx context.Context, query *query.Query) ([][]interface{}, error)
- func (c *Client) SelectWithSchema(ctx context.Context, queryObj *query.Query) (*query.QueryResult, error)
- type Config
- type PatternCheck
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func NewColumnCheckOperator ¶
func NewColumnCheckOperator(manager config.ConnectionGetter) *ansisql.ColumnCheckOperator
NewColumnCheckOperator builds the column-check runner for Sail. The generic numeric/null checks come from ansisql, but accepted_values and pattern need Spark SQL syntax (see the types below) rather than the Presto-flavored SQL the athena checks emit.
func NewMaterializer ¶
func NewMaterializer(fullRefresh bool) *pipeline.Materializer
NewMaterializer builds a materializer that renders Sail (Spark SQL) SQL.
Types ¶
type AcceptedValuesCheck ¶
type AcceptedValuesCheck struct {
// contains filtered or unexported fields
}
func NewAcceptedValuesCheck ¶
func NewAcceptedValuesCheck(conn config.ConnectionGetter) *AcceptedValuesCheck
func (*AcceptedValuesCheck) Check ¶
func (c *AcceptedValuesCheck) Check(ctx context.Context, ti *scheduler.ColumnCheckInstance) error
type BasicOperator ¶
type BasicOperator struct {
// contains filtered or unexported fields
}
func NewBasicOperator ¶
func NewBasicOperator(conn config.ConnectionGetter, extractor query.QueryExtractor, fullRefresh bool, hoister pipeline.DeclareHoister, parser *sqlparser.SQLParser) *BasicOperator
func (BasicOperator) Run ¶
func (o BasicOperator) Run(ctx context.Context, ti scheduler.TaskInstance) error
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
func (*Client) CreateSchemaIfNotExist ¶
CreateSchemaIfNotExist ensures the schema (Spark database) an asset's table lives in exists before the table is created. Sail rejects a CREATE TABLE into a database that does not exist ("Database not found: ..."), so we create it first with CREATE SCHEMA IF NOT EXISTS.
func (*Client) GetDatabaseSummary ¶
GetDatabaseSummary introspects the available schemas and tables using the ANSI information_schema.
func (*Client) RunQueryWithoutResult ¶
func (*Client) SelectWithSchema ¶
type Config ¶
type Config struct {
Host string
Port int
Username string
Password string
Token string
Database string
TLS bool
TLSSkipVerify bool
}
Config holds the connection parameters for a Sail platform.
Sail (LakeSail / PySail) exposes an Arrow Flight SQL server and speaks Spark SQL. Flight SQL is served in Go by the apache/arrow-adbc Flight SQL driver. Sail does not require authentication by default; the username/password and bearer Token fields are provided for deployments that put auth in front of it. Token and username/password are mutually exclusive.
func (Config) GetDatabase ¶
func (Config) ToDSN ¶
ToDSN builds a DSN understood by the ADBC Flight SQL database/sql driver. The driver parses the DSN as a ";"-separated list of "key=value" options, where the keys are the standard ADBC option names (uri, username, password, adbc.flight.sql.*).
The driver does not unescape values, so credentials are passed through verbatim; a ";" in a credential would split the DSN into a bogus option and fail to parse. We reject that explicitly rather than corrupt the value. ("=" is safe because the driver splits each option on its first "=" only.)
type PatternCheck ¶
type PatternCheck struct {
// contains filtered or unexported fields
}
func NewPatternCheck ¶
func NewPatternCheck(conn config.ConnectionGetter) *PatternCheck
func (*PatternCheck) Check ¶
func (c *PatternCheck) Check(ctx context.Context, ti *scheduler.ColumnCheckInstance) error
Source Files