Documentation
¶
Index ¶
- func AssetTypeToDialect(assetType pipeline.AssetType) (string, error)
- func ConnectionTypeToDialect(connectionType string) string
- func ValidateReadOnlyQuery(query, dialect string) error
- type CTE
- type ColumnLineage
- type Lineage
- type Parser
- type QueryConfig
- type SQLParser
- func (s *SQLParser) AddLimit(sql string, limit int, dialect string) (string, error)
- func (s *SQLParser) Close() error
- func (s *SQLParser) ColumnLineage(sql, dialect string, schema Schema) (*Lineage, error)
- func (s *SQLParser) ExtractSelect(sql string, dialect string) (string, error)
- func (s *SQLParser) FreezeTime(sql string, dialect string, executionTime string) (string, error)
- func (s *SQLParser) GetMissingDependenciesForAsset(asset *pipeline.Asset, pipeline *pipeline.Pipeline, ...) ([]string, error)
- func (s *SQLParser) HoistDeclares(sql string, assetType pipeline.AssetType) (string, error)
- func (s *SQLParser) HoistDeclaresList(queries []string, assetType pipeline.AssetType) ([]string, error)
- func (s *SQLParser) IsReadOnlyQuery(query, dialect string) (bool, error)
- func (s *SQLParser) IsSingleSelectQuery(sql string, dialect string) (bool, error)
- func (s *SQLParser) PrependCTEs(sql string, dialect string, ctes []CTE) (string, error)
- func (s *SQLParser) RenameTables(sql string, dialect string, tableMapping map[string]string) (string, error)
- func (s *SQLParser) SelectFromCTE(sql string, dialect string, cteName string) (string, error)
- func (s *SQLParser) Start() error
- func (s *SQLParser) UsedTables(sql, dialect string) ([]string, error)
- type Schema
- type UpstreamColumn
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func AssetTypeToDialect ¶ added in v0.11.192
func ConnectionTypeToDialect ¶ added in v0.11.644
ConnectionTypeToDialect maps a connection type identifier (e.g. "clickhouse") to the dialect string used by the SQL parser. Returns the empty string when no dialect is registered for the type.
func ValidateReadOnlyQuery ¶ added in v0.11.756
Types ¶
type CTE ¶ added in v0.11.661
CTE is a named common table expression to prepend to a query. The JSON tags match the keys the parser's add-ctes verb reads, so a []CTE serializes directly onto the wire.
type ColumnLineage ¶
type ColumnLineage struct {
Name string `json:"name"`
Upstream []UpstreamColumn `json:"upstream"`
Type string `json:"type"`
}
type Lineage ¶
type Lineage struct {
Columns []ColumnLineage `json:"columns"`
NonSelectedColumns []ColumnLineage `json:"non_selected_columns"`
Errors []string `json:"errors"`
}
type Parser ¶ added in v0.11.488
type Parser interface {
Start() error
ColumnLineage(sql, dialect string, schema Schema) (*Lineage, error)
UsedTables(sql, dialect string) ([]string, error)
RenameTables(sql, dialect string, tableMapping map[string]string) (string, error)
AddLimit(sql string, limit int, dialect string) (string, error)
IsSingleSelectQuery(sql string, dialect string) (bool, error)
GetMissingDependenciesForAsset(asset *pipeline.Asset, pipeline *pipeline.Pipeline, renderer jinja.RendererInterface) ([]string, error)
Close() error
}
type QueryConfig ¶
type SQLParser ¶
type SQLParser struct {
MaxQueryLength int
// contains filtered or unexported fields
}
SQLParser analyzes SQL in-process with pkg/sqlengine. It used to drive an embedded Python interpreter running SQLGlot; the API and behavior are unchanged.
func NewSQLParser ¶
func NewSQLParserCached ¶ added in v0.11.497
NewSQLParserCached creates a SQLParser. It is kept for API compatibility: parsing no longer needs any extracted files, so cached and uncached parsers are equivalent.
func NewSQLParserWithConfig ¶ added in v0.11.241
func (*SQLParser) ColumnLineage ¶
func (*SQLParser) ExtractSelect ¶ added in v0.11.661
ExtractSelect reduces an asset statement to the SELECT that produces its rows. A CREATE OR REPLACE VIEW / CTAS / INSERT ... SELECT is unwrapped to its inner SELECT, and a statement that is already a SELECT (with or without a WITH clause) is returned unchanged. This lets a unit test exercise the read logic of a `materialization: none` (full-DDL) asset without issuing the DDL, keeping the test a single read-only SELECT. It errors when the statement has no SELECT to test (e.g. a CREATE TABLE with a column list).
func (*SQLParser) FreezeTime ¶ added in v0.11.661
FreezeTime replaces CURRENT_TIMESTAMP / CURRENT_DATE / CURRENT_TIME in a query with literals fixed at executionTime, so a unit test of a time-dependent asset is deterministic instead of reading the warehouse clock.
func (*SQLParser) GetMissingDependenciesForAsset ¶ added in v0.11.216
func (*SQLParser) HoistDeclares ¶ added in v0.11.768
HoistDeclares moves top-level declarations ahead of other statements without regenerating their SQL. On failure it returns the original input.
func (*SQLParser) HoistDeclaresList ¶ added in v0.11.768
func (s *SQLParser) HoistDeclaresList(queries []string, assetType pipeline.AssetType) ([]string, error)
HoistDeclaresList preserves whole query entries, including their formatting.
func (*SQLParser) IsReadOnlyQuery ¶ added in v0.11.756
func (*SQLParser) IsSingleSelectQuery ¶ added in v0.11.254
func (*SQLParser) PrependCTEs ¶ added in v0.11.661
PrependCTEs adds the given CTEs to the front of a query's WITH clause (merging with any existing one, so existing CTEs can reference the prepended ones) and returns the rewritten SQL in the same dialect. It underpins connection-mode unit tests, which substitute the tables a query reads with inline fixture CTEs so the test runs as a single read-only SELECT with no DDL.
func (*SQLParser) RenameTables ¶ added in v0.11.169
func (*SQLParser) SelectFromCTE ¶ added in v0.11.661
SelectFromCTE rewrites a query so it returns all rows of one of its own named CTEs (WITH … SELECT * FROM <cteName>), keeping the other CTEs in place. It lets a unit test assert the output of an intermediate CTE. Errors if the query has no WITH clause or no CTE with that name.
Source Files