Documentation
¶
Overview ¶
Package lineagevalidation re-checks ingested lineage whose endpoints the metadata registry did not know when it was stored.
Ingestion keeps such an edge because it cannot tell a table created after the last schema sync from one that never existed, so the edge is stored unchecked. A schema sync is what changes that: the relation the edge named may exist now, and waiting for the maintenance pass' six-hour interval is a poor bound on noticing. The syncer signals this runner when it has committed metadata, and the runner coalesces the signals into one pass, so a round of database syncs costs one revalidation.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Revalidator ¶
type Revalidator interface {
RevalidateContradictedColumnClaims(ctx context.Context, limit int) (revalidated, stillUnknown int, err error)
}
Revalidator re-checks ingested lineage whose column claims a relation that appeared after ingestion contradicts, exactly as the maintenance pass does on its interval.
type Runner ¶
type Runner struct {
// contains filtered or unexported fields
}
Runner revalidates ingested lineage after a schema sync commits.
func NewRunner ¶
func NewRunner(revalidator Revalidator) *Runner
NewRunner creates a revalidation runner.
Source Files