Documentation
¶
Overview ¶
Package run builds a project's models against a warehouse.
The loop is the same for every dialect and lives here exactly once: ask what is there, ask the dialect what to run, run it. Everything that differs per warehouse is behind the Dialect interface, and everything that differs per project was decided by `project` before this is called.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Built ¶
type Built struct {
Ref string
Kind model.Materialisation
// Was is what the relation had been. It is reported because "this was a
// view until today" is the first thing somebody wants to know when a
// query that worked yesterday is slow.
Was dialect.Kind
Took time.Duration
// Target is the catalog identity -- what goes on the `landed` line and
// what puts this model on `/data`.
Target string
// Watermark is the greatest value of the model's watermark column after
// the build, as the warehouse printed it. Empty unless the model is
// incremental.
//
// FOR REPORTING AND NOTHING ELSE. It is published to the run context so
// a step below reads it without a query, which is what #63 asks for --
// and it is NEVER fed back into SQL. BigQuery returns its own TIMESTAMP
// as `"1767484800.0"` over REST and then refuses that string as a
// literal; the filter is a subquery for exactly that reason, and a
// string that is only ever printed cannot bring the problem back.
Watermark string
// Rows is how many the table holds, and it is NIL FOR A VIEW.
//
// A pointer because ABSENT IS NOT ZERO, which is the engine's own rule
// for this field: "a step that does not count says nothing, and a nil
// summed into a zero would draw a table that emptied overnight". A view
// holds no rows at all, and reporting 0 for one is indistinguishable
// from a table that really is empty.
Rows *int64
}
Built is one model, after.
type Options ¶ added in v0.2.0
type Options struct {
// FullRefresh rebuilds every incremental model in the order from
// scratch, forgetting its watermark. It does nothing to a view or a
// table, which are rebuilt every time anyway.
FullRefresh bool
}
Options is what the CALLER asked for, as opposed to what the project says.
A STRUCT FOR ONE FIELD, deliberately. `Build(ctx, d, conn, p, order, true)` at a call site says nothing about what is true, and the next flag after this one would be a second bare bool beside it.
type Result ¶
type Result struct{ Built []Built }
Result is the build.
func Build ¶
func Build(ctx context.Context, d dialect.Dialect, conn dialect.Conn, p *project.Project, order []string, opt Options) (Result, error)
Build runs every model in `order`, which is already a dependency order.
ORDER IS NOT COMPUTED HERE, and that is on purpose: `project.Select` produces it, `brevis-sql graph` prints the same thing, and a second ordering in this file is a second answer to one question.
It stops at the first failure. A warehouse half built is a worse place to stand than one that stopped where the error is -- and continuing would build models on top of a relation that is not what their SQL assumed.
Source Files