model

package
v0.1.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 13, 2026 License: Apache-2.0 Imports: 4 Imported by: 0

Documentation

Index

Constants

View Source 
const DeletionColumnName = "__deletion__"

DeletionColumnName is the target column a statement that removes rows records its edges against: `source.x -> target.__deletion__` says the value of source.x decides which rows the target loses, without pretending the target read a column of its own.

View Source 
const FileSourceName = "__file__"

FileSourceName is the source table a data-loading statement names the file it reads: LOAD DATA, COPY and COPY INTO all record their edges as `__file__ -> target`, so a file source reads the same however it was loaded.

View Source 
const ResultTableName = "__result__"

ResultTableName is the synthetic table every statement's own result rows are recorded against. An edge targeting it is temporary by definition: it names no stored object.

View Source 
const WildcardColumn = "*"

WildcardColumn is the column marker a lineage edge carries when the column itself is unknown because a wildcard was never expanded. A temporary relation's entry with this target forwards every column of its source.

Variables

This section is empty.

Functions

func FormatDiagnostics 

func FormatDiagnostics(notes []Diagnostic, omitted int) string

FormatDiagnostics renders notes as the "; "-joined text an error message carries. omitted is how many the collector's limit left out: naming the count says the result is partial without letting a statement full of unresolved references grow the text without bound.

func NormalizeIdentifier 

func NormalizeIdentifier(text string) string

NormalizeIdentifier returns an identifier's text with its quoting removed: a name written `a` or "a" names the same object as a, and a doubled quote inside it is one quote. Text that is not a quoted identifier comes back with its surrounding space trimmed and nothing else changed.

func SameTransformations 

func SameTransformations(a, b []Transformation) bool

SameTransformations reports whether two transformation lists are equal in order and content. It is how an edge's identity is decided: two edges that connect the same columns are the same edge only when the same operations produced both.

Types

type Column 

type Column struct {
	Table ObjectIdentifier
	Name  string
}

type ColumnRelation 

type ColumnRelation struct {
	Source         Column
	Target         Column
	Transformation []Transformation
	RelationType   RelationType
	// IsTemp reports whether the target is the statement's own result rather than
	// a stored object. A target is temporary exactly when it names
	// ResultTableName, so the flag is derived from the target and not a second
	// source of truth.
	IsTemp bool
}

func AnsweringLineage 

func AnsweringLineage(lineage []ColumnRelation, column string) []ColumnRelation

AnsweringLineage returns the entries of a temporary relation's lineage that answer a reference to column, in their original order. An entry whose target names the column answers it; when none does, the entries whose target is WildcardColumn do, because a body that never expanded its star forwards every column of its sources — the column behind the wildcard is unknown, the source table is not. An empty result means the lineage says nothing about the name.

Names are compared case-insensitively, the way the scope resolver compares them: a MySQL-family identifier is case-insensitive, so a reference may spell a column differently from the alias the lineage recorded.

type Diagnostic 

type Diagnostic struct {
	Category DiagnosticCategory
	// Subject names what the diagnostic is about, the way the analyzer names it:
	// the statement or clause shape it does not model ("MERGE", "WITH before
	// INSERT"), or the clause a reference was written in ("a CTE body").
	Subject string
	// Reference is the identifier a reference diagnostic could not resolve, the
	// way the SQL writes it ("t.c", "t", "c"). It is empty for a diagnostic about
	// a shape rather than a reference.
	Reference string
	// Detail carries the cause for a diagnostic that needs one, such as the
	// parser defect that makes a clause unanalyzable or the catalog error behind
	// a degraded lookup.
	Detail string
}

Diagnostic is one thing an analysis could not represent, kept as data rather than as a sentence so a caller can act on the category and point at the reference. String renders it the way an error message carries it, which is what keeps a stored version message and a caller's report the same text.

func (Diagnostic) String 

func (d Diagnostic) String() string

String renders the diagnostic as the sentence an error message carries.

type DiagnosticCategory 

type DiagnosticCategory int

DiagnosticCategory classifies what an analysis could not represent. The categories call for different action: a shape the analyzer does not model is a coverage gap in the analyzer, while a reference that does not resolve is usually the SQL naming something it cannot see. 

const (
	// DiagnosticNotModelled is a statement or clause shape the analyzer does not
	// model yet.
	DiagnosticNotModelled DiagnosticCategory = iota + 1
	// DiagnosticUnresolved is a reference that resolved to nothing.
	DiagnosticUnresolved
	// DiagnosticAmbiguous is a reference a single-valued position could not
	// choose between, which is an unqualified name several relations in scope own.
	DiagnosticAmbiguous
	// DiagnosticCatalogUnavailable is a metadata lookup that failed. The analysis
	// carried on with the relation's columns unknown, so the note is what keeps a
	// degraded result distinguishable from a complete one.
	DiagnosticCatalogUnavailable
)

type ObjectIdentifier 

type ObjectIdentifier struct {
	InstanceID string
	Database   string
	Schema     string
	Name       string
}

func StrToObjectIdentifier 

func StrToObjectIdentifier(s string) ObjectIdentifier

func (ObjectIdentifier) FullName 

func (o ObjectIdentifier) FullName() string

FullName returns the object's dotted name, omitting empty qualifiers.

func (ObjectIdentifier) GUID 

func (o ObjectIdentifier) GUID() string

type OperationType 

type OperationType string

OperationType represents the type of transformation operation. 

const (
	// OperationDelete represents a DELETE operation.
	OperationDelete OperationType = "DELETE"
	// OperationUnion represents a UNION operation.
	OperationUnion OperationType = "UNION"
	// OperationIntersect represents an INTERSECT operation.
	OperationIntersect OperationType = "INTERSECT"
	// OperationExcept represents an EXCEPT operation.
	OperationExcept OperationType = "EXCEPT"
	// OperationProject represents a projection (simple column reference or expression).
	OperationProject OperationType = "PROJECT"
	// OperationFunction represents a function call.
	OperationFunction OperationType = "FUNCTION"
	// OperationAggregate represents an aggregate function (COUNT, SUM, etc.).
	OperationAggregate OperationType = "AGGREGATE"
	// OperationWindow represents a window function.
	OperationWindow OperationType = "WINDOW"
	// OperationOperator represents an arithmetic or comparison operator.
	OperationOperator OperationType = "OPERATOR"
	// OperationCase represents a CASE expression.
	OperationCase OperationType = "CASE"
	// OperationFilter represents a predicate that decides which rows a statement
	// produces without its value flowing into any output column (WHERE, HAVING).
	OperationFilter OperationType = "FILTER"
	// OperationJoin represents a join condition.
	OperationJoin OperationType = "JOIN"
	// OperationGroupBy represents a grouping that decides which rows are
	// aggregated together. The SQL analyzers record it as AGGREGATE group keys;
	// this value carries OpenLineage's INDIRECT/GROUP_BY subtype.
	OperationGroupBy OperationType = "GROUP_BY"
	// OperationSort represents an ordering of the output rows. The SQL analyzers
	// do not produce it yet; it carries OpenLineage's INDIRECT/SORT subtype.
	OperationSort OperationType = "SORT"
)

type RelationType 

type RelationType int
const (
	RelationTypeDirect RelationType = iota + 1
	RelationTypeIndirect
	RelationTypeJoin
	RelationTypeGroup
	RelationTypeUnion
	RelationTypeIntersect
	RelationTypeExcept
	RelationTypeUnknown
)

func RelationTypeOf 

func RelationTypeOf(transform []Transformation) RelationType

RelationTypeOf infers the relation type of an edge from its transformations, which is the one rule every producer shares: a SQL analyzer, the OpenLineage ingestion and the wildcard expansion all describe an edge by the operations it carries. An edge with no transformation is direct, and an edge with several is described by the outermost one, which comes first.

type Transformation 

type Transformation struct {
	// Operation is the type of transformation operation.
	Operation OperationType `json:"operation"`

	// Expression is the text representation of the expression (used by most operation types).
	Expression string `json:"expression,omitempty"`

	// FunctionName is the name of the function (for FUNCTION, AGGREGATE, WINDOW operations).
	FunctionName string `json:"function_name,omitempty"`

	// Arguments contains function arguments (for FUNCTION operation).
	Arguments []string `json:"arguments,omitempty"`

	// GroupKeys contains GROUP BY keys (for AGGREGATE operation).
	GroupKeys []string `json:"group_keys,omitempty"`

	// PartitionBy contains PARTITION BY columns (for WINDOW operation).
	PartitionBy []string `json:"partition_by,omitempty"`

	// OrderBy contains ORDER BY columns (for WINDOW operation).
	OrderBy []string `json:"order_by,omitempty"`

	// OpType is the operator kind for an OPERATOR transformation, e.g.
	// "ADDITION", "SUBTRACTION", "EQUALS".
	OpType string `json:"op_type,omitempty"`

	// Condition is the predicate text a row-set operation carries: the DELETE
	// condition, or the WHERE / HAVING / ON expression of a FILTER or JOIN
	// influence edge.
	Condition string `json:"condition,omitempty"`

	// All records that a set operation keeps duplicate rows instead of removing
	// them, which is the difference between UNION ALL and UNION (and between
	// INTERSECT ALL / EXCEPT ALL and their plain forms). It is meaningful for
	// the set-operation kinds only.
	All bool `json:"all,omitempty"`
}

Transformation represents a data transformation operation in the lineage. It uses a tagged union pattern where Operation determines which fields are relevant.

func CombineTransformations 

func CombineTransformations(base, additional []Transformation) []Transformation

CombineTransformations joins an edge's own transformations with the ones a consumer adds on top of them, returning a list the caller owns.

Neither argument is written to and the result never shares an array with either. Appending to the first argument instead returns a list backed by the array the argument already had, and a later append into the combined list then overwrites the transformations of every other edge built from the same base.

func NewAggregateTransformation 

func NewAggregateTransformation(functionName, expression string, groupKeys []string) Transformation

NewAggregateTransformation creates a Transformation for aggregate functions.

func NewCaseTransformation 

func NewCaseTransformation(expression string) Transformation

NewCaseTransformation creates a Transformation for CASE expressions.

func NewDeleteTransformation 

func NewDeleteTransformation(condition string) Transformation

NewDeleteTransformation creates a Transformation for DELETE operations.

func NewExceptTransformation 

func NewExceptTransformation(all bool) Transformation

NewExceptTransformation creates a Transformation for an EXCEPT, or for an EXCEPT ALL when all is set.

func NewFilterTransformation 

func NewFilterTransformation(condition string) Transformation

NewFilterTransformation creates a Transformation for a predicate that selects which rows a statement produces.

func NewFunctionTransformation 

func NewFunctionTransformation(functionName, expression string, arguments []string) Transformation

NewFunctionTransformation creates a Transformation for function calls.

func NewIntersectTransformation 

func NewIntersectTransformation(all bool) Transformation

NewIntersectTransformation creates a Transformation for an INTERSECT, or for an INTERSECT ALL when all is set.

func NewJoinTransformation 

func NewJoinTransformation(condition string) Transformation

NewJoinTransformation creates a Transformation for a join condition.

func NewOperatorTransformation 

func NewOperatorTransformation(opType, expression string) Transformation

NewOperatorTransformation creates a Transformation for operator expressions.

func NewProjectTransformation 

func NewProjectTransformation(expression string) Transformation

NewProjectTransformation creates a Transformation for simple projection.

func NewUnionTransformation 

func NewUnionTransformation(all bool) Transformation

NewUnionTransformation creates a Transformation for a UNION, or for a UNION ALL when all is set.

func NewWindowTransformation 

func NewWindowTransformation(functionName, expression string, partitionBy, orderBy []string) Transformation

NewWindowTransformation creates a Transformation for window functions.

func (Transformation) Equal 

func (t Transformation) Equal(other Transformation) bool

Equal reports whether two transformations describe the same operation with the same fields, field by field.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.