build

package
v1.6.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 8, 2026 License: GPL-3.0 Imports: 32 Imported by: 0

Documentation

Index

Constants

View Source
const (
	NodeTypeModel = "model"
	NodeTypeSeed  = "seed"
)

NodeTypeModel and NodeTypeSeed are the compiled node types.

View Source
const ConfigFileName = "sling_build.yml"

ConfigFileName is the canonical config file name.

View Source
const ConfigFileNameYAML = "sling_build.yaml"

ConfigFileNameYAML is the alternate config file name.

View Source
const DefaultThreads = 4

DefaultThreads is the default parallelism for model execution.

Variables

View Source
var ValidModes = map[string]bool{
	"full-refresh": true,
	"view":         true,
	"truncate":     true,
	"incremental":  true,
	"append":       true,
}

ValidModes are the recognized materialization modes, post-normalization. Aliases (table, snapshot) are resolved by normalizeMode before this is checked.

Functions

func ExpandConfigVars added in v1.6.1

func ExpandConfigVars(text string) (string, []string)

ExpandConfigVars expands ${VAR} and ${VAR:-fallback} in sling_build.yml text.

func ExpandConfigVarsEnv added in v1.6.1

func ExpandConfigVarsEnv(text string, extra map[string]string) (string, []string)

ExpandConfigVarsEnv is ExpandConfigVars with extra names checked before the process environment.

func ExtractTableReferences

func ExtractTableReferences(sql string) []string

ExtractTableReferences extracts table references from FROM/JOIN clauses in SQL. It returns unique table references (schema.table or just table) found in the SQL, excluding CTEs and Jinja template expressions.

func FindConfigFile added in v1.6.1

func FindConfigFile(dir string) (string, bool)

FindConfigFile returns the path of sling_build.yml or sling_build.yaml in dir. Prefer .yml when both exist. Match is case-insensitive on the file name.

func GetMacrosForModel

func GetMacrosForModel(project *BuildProject, model *Model) string

GetMacrosForModel returns the concatenated macro SQL applicable to a model. Macros are ordered root-first (outermost to innermost scope).

func IsConfigFileName added in v1.6.1

func IsConfigFileName(name string) bool

IsConfigFileName reports whether name is sling_build.yml or sling_build.yaml.

func LoadSeed

func LoadSeed(seed *Seed, connName string, fullRefresh bool) (rows, bytes uint64, err error)

LoadSeed loads a seed file into the target database using the existing sling task infrastructure. This gets CSV/JSON/Parquet parsing, type inference, bulk loading, and all 30+ connectors for free. Seeds always use full-refresh.

func MakeSeedConfig

func MakeSeedConfig(seed *Seed, connName string) *sling.Config

MakeSeedConfig creates a sling.Config for loading a seed file without executing it. Useful for testing and compile mode.

func RewriteTableReferences

func RewriteTableReferences(sql string, project *BuildProject, selfName string) (string, []string, error)

RewriteTableReferences scans compiled SQL for table references matching prod-mode names of known models/seeds and rewrites them to current-mode FullTableNames. Returns the rewritten SQL and matched model/seed names (for DependsOn). An ambiguous bare name is an error.

func RunForHook

func RunForHook(path string, opts sling.HookBuildRunOptions) (map[string]any, error)

RunForHook compiles and executes a build project for a pipeline/replication hook. Returns a state map with per-node results for state.<step_id>.results.

func SplitModelSQL

func SplitModelSQL(sql, dialect string) (preStatements []string, modelQuery string, postStatements []string, err error)

SplitModelSQL splits multi-statement SQL into pre-statements, the model query, and post-statements. The model query is the single SELECT/WITH/UNION statement.

Returns an error if zero or more than one query statement is found.

func SyncBuildFailure added in v1.6.1

func SyncBuildFailure(runErr error)

SyncBuildFailure reports a build that failed before an Executor existed

func ValidateReadOnlyQuery

func ValidateReadOnlyQuery(sql string, dbType dbio.Type) error

ValidateReadOnlyQuery returns an error unless sql is a single read-only statement for the given database type. It parses with golyglot; if the parser is unavailable or cannot parse the dialect, a conservative keyword guard decides. Neither layer can catch side-effecting functions (select my_procedure()) — use a read-only database user for connections a project token can reach.

func WarmParser

func WarmParser()

WarmParser starts loading the golyglot dylib in the background so the first ValidateReadOnlyQuery does not pay the download/load cost. Init failure is non-fatal — the keyword fallback covers it.

Types

type Build

type Build struct {
	Project   *BuildProject
	DAG       *DAG
	Engine    *TemplateEngine
	Options   BuildOptions
	Selected  []string // selected node names after selector filtering
	SubBuilds []*Build // compiled sub-projects (for multi-target compile mode)

	ExecRows  uint64 // sum of model/seed rows after Execute
	ExecBytes uint64
	Results   []ExecutionResult // per-node results after Execute
	// contains filtered or unexported fields
}

Build is the main orchestrator for sling build. It loads a project, compiles templates, builds the DAG, applies selectors, and provides compile output.

func NewBuild

func NewBuild(dir string, opts BuildOptions) (*Build, error)

NewBuild creates a new Build from the given project directory and options.

func (*Build) Compile

func (b *Build) Compile() error

Compile loads the project, compiles all model templates, builds the DAG, and applies selectors. After Compile(), the Build is ready for execution or compile output.

func (*Build) CompileJSONPayload

func (b *Build) CompileJSONPayload() map[string]any

CompileJSONPayload returns the --compile --json object.

func (*Build) Compiled added in v1.6.1

func (b *Build) Compiled() *BuildConfig

Compiled returns the project config with the compile output set: the resolved target, the selected nodes in execution order, and the compiled SQL of each. Safe when Compile did not finish (e.g. a cycle): order/nodes stay empty.

func (*Build) Execute

func (b *Build) Execute() error

Execute runs the compiled build against the target database. Must be called after Compile().

func (*Build) GetModelMode

func (b *Build) GetModelMode(model *Model) string

GetModelMode returns the effective mode for a model, considering: 1. CLI --full-refresh flag overrides everything 2. Model config() block mode 3. Project defaults.mode 4. Default: full-refresh

func (*Build) GetTarget

func (b *Build) GetTarget() string

GetTarget returns the resolved target connection name.

func (*Build) PrintCompileJSON

func (b *Build) PrintCompileJSON()

PrintCompileJSON prints compile output as JSON (models, deps, SQL).

func (*Build) PrintCompileOutput

func (b *Build) PrintCompileOutput()

PrintCompileOutput prints the compile output in YAML format for each selected node.

func (*Build) PrintListJSON

func (b *Build) PrintListJSON()

PrintListJSON prints selected nodes as JSON.

func (*Build) PrintListOutput

func (b *Build) PrintListOutput()

PrintListOutput prints the selected models/seeds and exits.

func (*Build) PrintTestJSON added in v1.6.1

func (b *Build) PrintTestJSON()

PrintTestJSON prints per-node test results as JSON.

type BuildConfig

type BuildConfig struct {
	Target     string         `json:"target,omitempty" yaml:"target"`
	Dev        *DevConfig     `json:"dev,omitempty" yaml:"dev,omitempty"`
	DbtProject any            `json:"dbt_project,omitempty" yaml:"dbt_project,omitempty"`
	Vars       map[string]any `json:"vars,omitempty" yaml:"vars,omitempty"`
	Defaults   BuildDefaults  `json:"defaults,omitempty" yaml:"defaults,omitempty"`

	// Order and Nodes are the compiled output, set by Build.Compile.
	// They live on the root config only; mergeConfigs does not carry them.
	Order    []string `json:"order" yaml:"-"`
	Nodes    []Node   `json:"nodes" yaml:"-"`
	Compiled bool     `json:"compiled" yaml:"-"`
	// contains filtered or unexported fields
}

BuildConfig represents the contents of sling_build.yml.

func LoadBuildConfig added in v1.6.1

func LoadBuildConfig(content string) (cfg *BuildConfig, err error)

LoadBuildConfig parses a sling_build.yml body. It expands ${VAR} references and validates defaults.mode.

func (*BuildConfig) JSONPayload added in v1.6.1

func (c *BuildConfig) JSONPayload() map[string]any

JSONPayload renders the compile output as the --compile --json object. Node keys are omitted when empty, to keep the payload as it was.

func (*BuildConfig) ModelNames added in v1.6.1

func (c *BuildConfig) ModelNames() (names []string)

ModelNames returns the names of the compiled nodes that are models, in order.

type BuildDefaults

type BuildDefaults struct {
	Mode          string        `yaml:"mode,omitempty"`
	Schema        string        `yaml:"schema,omitempty"`
	Database      string        `yaml:"database,omitempty"` // three-part dialects only
	Tags          []string      `yaml:"tags,omitempty"`     // additive across nesting
	UniqueKey     any           `yaml:"unique_key,omitempty"`
	UpdateKey     string        `yaml:"update_key,omitempty"`
	MergeStrategy string        `yaml:"merge_strategy,omitempty"`
	Enabled       *bool         `yaml:"enabled,omitempty"`
	Hooks         sling.HookMap `yaml:"hooks,omitempty"` // additive across nesting
	DropCascade   *bool         `yaml:"drop_cascade,omitempty"`
}

BuildDefaults holds default settings for models.

type BuildModelState

type BuildModelState struct {
	Name     string `json:"name,omitempty"`
	Schema   string `json:"schema,omitempty"`
	Database string `json:"database,omitempty"`
	FullName string `json:"full_name,omitempty"`
	Mode     string `json:"mode,omitempty"`
}

BuildModelState holds model metadata available in hooks.

type BuildOptions

type BuildOptions struct {
	Target      string
	Schema      string
	Prod        bool
	Vars        map[string]any
	FullRefresh bool
	Select      []string
	Exclude     []string
	Compile     bool
	Threads     int
	FailFast    bool
	List        bool
	NoSeeds     bool
	Range       *string // CLI --range: "start,end[,step]"
	Recursive   bool    // CLI --recursive/-R: discover sling_build.yml in immediate subdirectories
	Test        bool    // CLI --test: run data tests only (no materialization)
	JSON        bool    // CLI --json: machine-readable compile/list output
	// SkipUnresolvedCheck skips the ${VAR} error for fields in effect.
	// sling validate uses this because it does not choose a run mode.
	SkipUnresolvedCheck bool
}

BuildOptions holds CLI-provided overrides.

type BuildProject

type BuildProject struct {
	Dir            string
	Config         *BuildConfig            // from sling_build.yml (nil if missing)
	Models         map[string]*Model       // keyed by unique model name
	Seeds          map[string]*Seed        // keyed by unique seed name
	Macros         []*MacroFile            // collected .macros.sql files
	Mode           string                  // "dev" or "prod"
	SchemaOverride string                  // dev mode schema
	DefaultSchema  string                  // default schema for root-level files (default: "public")
	ChildConfigs   map[string]*BuildConfig // child sling_build.yml configs keyed by relative dir
	SubProjects    []*BuildProject         // independent build projects (when no root yml)
	Recursive      bool                    // CLI --recursive: keep immediate child projects
}

BuildProject represents a sling build project discovered from a directory.

func LoadProject

func LoadProject(dir string, opts ...BuildOptions) (*BuildProject, error)

LoadProject loads a build project from the given directory.

func (*BuildProject) AllNames

func (p *BuildProject) AllNames() []string

AllNames returns a sorted list of all model and seed names.

func (*BuildProject) BuildProdNameIndex

func (p *BuildProject) BuildProdNameIndex() map[string]prodNameEntry

BuildProdNameIndex builds a lookup map from lowercased prod FullTableName (and unqualified name) to the model/seed entry. Qualified names take priority over unqualified for the same key.

func (*BuildProject) GetConfig added in v1.6.1

func (p *BuildProject) GetConfig() *BuildConfig

GetConfig returns the project root config, creating it when absent.

func (*BuildProject) GetEffectiveConfig

func (p *BuildProject) GetEffectiveConfig(dir string) *BuildConfig

GetEffectiveConfig returns the merged config for the project, applying child overrides.

func (*BuildProject) LookupFullTableName

func (p *BuildProject) LookupFullTableName(name string) (string, bool)

LookupFullTableName returns the full table name for a given model or seed name.

func (*BuildProject) ResolveName added in v1.6.1

func (p *BuildProject) ResolveName(name string) (*NameRef, error)

ResolveName returns the model or seed for name. name may be the file stem ("dim_customers") or the prod table name ("marts.dim_customers", or "DB.marts.dim_customers" on three-part projects).

func (*BuildProject) UnresolvedConfigVars added in v1.6.1

func (p *BuildProject) UnresolvedConfigVars() []string

UnresolvedConfigVars returns ${VAR} names that had no value and no fallback.

type BuildState

type BuildState struct {
	State     map[string]map[string]any `json:"state,omitempty"`
	Store     map[string]any            `json:"store,omitempty"`
	Env       map[string]any            `json:"env,omitempty"`
	Timestamp sling.DateTimeState       `json:"timestamp,omitempty"`
	Model     BuildModelState           `json:"model,omitempty"`
	Target    BuildTargetState          `json:"target,omitempty"`
}

BuildState implements sling.RuntimeState for build model hooks.

func (*BuildState) GetStore

func (bs *BuildState) GetStore() map[string]any

func (*BuildState) Marshall

func (bs *BuildState) Marshall() string

func (*BuildState) SetStateData

func (bs *BuildState) SetStateData(id string, data map[string]any)

func (*BuildState) SetStateKeyValue

func (bs *BuildState) SetStateKeyValue(id, key string, value any)

func (*BuildState) SetStoreData

func (bs *BuildState) SetStoreData(key string, value any, del bool)

func (*BuildState) StepExecution

func (bs *BuildState) StepExecution() *sling.PipelineStepExecution

func (*BuildState) TaskExecution

func (bs *BuildState) TaskExecution() *sling.TaskExecution

type BuildTargetState

type BuildTargetState struct {
	Name string `json:"name,omitempty"`
}

BuildTargetState holds target connection metadata available in hooks.

type DAG

type DAG struct {
	Nodes map[string]*DAGNode
	Order []string // topological sort result
}

DAG represents a directed acyclic graph of models and seeds.

func BuildDAG

func BuildDAG(project *BuildProject) (*DAG, error)

BuildDAG constructs a dependency graph from a project's models and seeds. Models must have been compiled first (DependsOn populated).

func (*DAG) DetectCycles

func (dag *DAG) DetectCycles() [][]string

DetectCycles finds cycles in the graph using DFS.

func (*DAG) GetDownstream

func (dag *DAG) GetDownstream(name string) []string

GetDownstream returns all transitive downstream dependents of the given node.

func (*DAG) GetDownstreamN

func (dag *DAG) GetDownstreamN(name string, n int) []string

GetDownstreamN returns downstream dependents up to N degrees from the given node.

func (*DAG) GetExecutionLevels

func (dag *DAG) GetExecutionLevels() [][]string

GetExecutionLevels groups nodes by depth for parallel execution. Each level contains nodes that can be executed concurrently.

func (*DAG) GetUpstream

func (dag *DAG) GetUpstream(name string) []string

GetUpstream returns all transitive upstream dependencies of the given node.

func (*DAG) GetUpstreamN

func (dag *DAG) GetUpstreamN(name string, n int) []string

GetUpstreamN returns upstream dependencies up to N degrees from the given node.

func (*DAG) TopologicalSort

func (dag *DAG) TopologicalSort() ([]string, error)

TopologicalSort performs Kahn's algorithm to produce a topological ordering. Returns an error if a cycle is detected.

type DAGNode

type DAGNode struct {
	Name         string
	Model        *Model // nil for seeds
	Seed         *Seed  // nil for models
	Dependencies []string
	Dependents   []string
	Depth        int
}

DAGNode represents a node in the dependency graph.

type DbtProjectConfig

type DbtProjectConfig struct {
	ModelsPath string `yaml:"models_path,omitempty"` // default: "models"
	SeedsPath  string `yaml:"seeds_path,omitempty"`  // default: "seeds"
}

DbtProjectConfig holds dbt project compatibility settings.

type DevConfig

type DevConfig struct {
	Target   string `yaml:"target,omitempty"`   // optional, falls back to top-level target
	Schema   string `yaml:"schema"`             // mandatory for dev mode
	Database string `yaml:"database,omitempty"` // optional; falls back to defaults.database
}

DevConfig holds dev-mode settings in sling_build.yml. When present, dev mode is the default (override with --prod).

type ExecutionResult

type ExecutionResult struct {
	Name      string
	NodeType  string // "seed", "model", or "test"
	Mode      string // "full-refresh", "view", "truncate", "incremental", "append"
	Duration  time.Duration
	Err       error
	Skipped   bool
	StartTime *time.Time
	// Rows is sql.Result.RowsAffected, or COUNT(*) on column-store
	// full-refresh / truncate / first-run append. Incremental and subsequent
	// append count the staging temp table. CREATE VIEW reports 0.
	Rows  uint64
	Bytes uint64
}

ExecutionResult holds the outcome of executing one node.

type Executor

type Executor struct {
	Build    *Build
	ConnName string              // resolved target connection name
	DbConn   database.Connection // database connection (nil until Connect)
	Results  []ExecutionResult   // per-node results
	RunID    string              // unique per Execute() for temp table isolation
	// contains filtered or unexported fields
}

Executor runs a compiled Build against a target database.

func NewExecutor

func NewExecutor(b *Build) (*Executor, error)

NewExecutor creates an Executor from a compiled Build.

func (*Executor) Close

func (e *Executor) Close()

Close closes the database connection.

func (*Executor) Connect

func (e *Executor) Connect() error

Connect establishes a database connection to the target.

func (*Executor) CreateSchemas

func (e *Executor) CreateSchemas() error

CreateSchemas creates all unique schemas needed by the selected nodes.

func (*Executor) Execute

func (e *Executor) Execute() error

Execute runs all selected nodes with a ready-queue scheduler. A node is dispatched as soon as its selected dependencies complete (no level barriers).

type IncrementalContext

type IncrementalContext struct {
	WhereCond     string // e.g., `"created_at" > '2024-01-01'` or "1=1"
	Value         string // e.g., `'2024-01-01'` or "null"
	IsIncremental bool   // drives is_incremental() for dbt-style models
}

IncrementalContext carries values used by CompileModel to resolve incremental functions and flags. One type serves both styles:

  • Style A (dbt): CompileModel reads IsIncremental only and passes it to the is_incremental() Jinja function. WhereCond/Value are ignored because the SQL does not call incremental_where_cond() / incremental_value().

  • Style B (sling): CompileModel reads WhereCond/Value and exposes them as the incremental_where_cond() / incremental_value() Jinja functions. is_incremental() is still registered but returns IsIncremental (typically false for sling-style).

A nil *IncrementalContext is equivalent to DefaultIncrementalContext(): first-run semantics (WhereCond=1=1, Value=null, IsIncremental=false).

func DefaultIncrementalContext

func DefaultIncrementalContext() *IncrementalContext

DefaultIncrementalContext returns a first-run context: Jinja functions resolve to "1=1"/"null" and is_incremental() returns false.

type MacroFile

type MacroFile struct {
	FilePath string // absolute path
	Dir      string // directory relative to project root ("" for root, "staging", "marts/core")
	RawSQL   string // raw file content with {% macro %} definitions
}

MacroFile represents a .macros.sql file containing Jinja macro definitions.

type Model

type Model struct {
	Name              string      // file stem, unique across the project, e.g. "events"
	FilePath          string      // absolute path
	RelPath           string      // relative path from project root
	Schema            string      // derived from first folder or override
	Database          string      // optional catalog; three-part dialects only
	FullTableName     string      // [database.]schema.name (current mode)
	ProdFullTableName string      // [database.]schema.name (always prod-mode, for SQL matching)
	RawSQL            string      // raw file content (frontmatter stripped)
	CompiledSQL       string      // after Jinja rendering
	PreStatements     []string    // SQL statements before the model query (from multi-statement splitting)
	PostStatements    []string    // SQL statements after the model query (from multi-statement splitting)
	Config            ModelConfig // from YAML frontmatter or config() block
	HasFrontmatter    bool        // true if config was set via YAML frontmatter (config() becomes no-op)
	Style             Style       // detected incremental pattern: StyleDbt or StyleSling (populated at load)
	Refs              []string    // ref() dependencies
	Sources           []string    // src() references
	DependsOn         []string    // all DAG dependencies (refs + bare refs + auto-detected)
	// contains filtered or unexported fields
}

Model represents a SQL model file in the project.

type ModelConfig

type ModelConfig struct {
	Mode          string        `yaml:"mode,omitempty"`
	Materialized  string        `yaml:"materialized,omitempty"` // dbt alias for mode
	UniqueKey     any           `yaml:"unique_key,omitempty"`   // string or []string
	MergeStrategy string        `yaml:"merge_strategy,omitempty"`
	UpdateKey     string        `yaml:"update_key,omitempty"`
	Tags          []string      `yaml:"tags,omitempty"`
	Hooks         sling.HookMap `yaml:"hooks,omitempty"`
	PreHook       string        `yaml:"pre_hook,omitempty"`  // deprecated: kept for validation only
	PostHook      string        `yaml:"post_hook,omitempty"` // deprecated: kept for validation only
	Schema        string        `yaml:"schema,omitempty"`
	Database      string        `yaml:"database,omitempty"`
	Enabled       *bool         `yaml:"enabled,omitempty"`
	Engine        string        `yaml:"engine,omitempty"`
	Range         *RangeConfig  `yaml:"range,omitempty"`
	DropCascade   *bool         `yaml:"drop_cascade,omitempty"` // default false; CASCADE on DROP when true
	Rewrite       *bool         `yaml:"rewrite,omitempty"`      // default true; set false to skip bare-name rewrite
	Tests         []any         `yaml:"tests,omitempty"`        // declarative data tests
}

ModelConfig holds configuration extracted from the config() block in a SQL model.

type ModelSQL

type ModelSQL struct {
	PreStatements  []string
	ModelQuery     string
	PostStatements []string
}

ModelSQL holds the split result of a multi-statement SQL model file.

func MakeModelSQL

func MakeModelSQL(sql string, dbType dbio.Type) (*ModelSQL, error)

MakeModelSQL splits a SQL model file into pre-statements, the model query, and post-statements. The model query is the single SELECT/WITH/UNION statement. All other statements are classified as pre (before) or post (after) the query.

type NameNotFoundError added in v1.6.1

type NameNotFoundError struct {
	Query       string
	Kind        string // "ref" or "selector"
	Suggestions []NameRef
}

NameNotFoundError is returned when no model or seed matches.

func (*NameNotFoundError) Error added in v1.6.1

func (e *NameNotFoundError) Error() string

type NameRef added in v1.6.1

type NameRef struct {
	Name          string
	RelPath       string
	FullTableName string
	Model         *Model
	Seed          *Seed
}

NameRef is a resolved model or seed.

type Node added in v1.6.1

type Node struct {
	Name         string   `json:"name" yaml:"name"`
	Type         string   `json:"type,omitempty" yaml:"type,omitempty"` // model or seed
	Table        string   `json:"table,omitempty" yaml:"table,omitempty"`
	File         string   `json:"file,omitempty" yaml:"file,omitempty"`
	Mode         string   `json:"mode,omitempty" yaml:"mode,omitempty"`                 // models only
	Dependencies []string `json:"dependencies,omitempty" yaml:"dependencies,omitempty"` // models only
	SQL          string   `json:"sql,omitempty" yaml:"sql,omitempty"`                   // models only
	Tests        []any    `json:"tests,omitempty" yaml:"tests,omitempty"`               // models only
}

Node is one selected model or seed of a compiled build project.

type Range

type Range struct {
	Chunks      []RangeChunk // ordered; 0 chunks = no-op
	UpdateState bool         // advance SLING_STATE after last chunk succeeds
	FromCLI     bool         // came from --range (print resume hint on failure)
	CLIRaw      string       // original raw --range value (for resume hint)
	Step        string       // parsed step for resume hint, may be ""
}

Range is the resolved set of chunks for a single build model execution.

type RangeChunk

type RangeChunk struct {
	Lower          string         // already-quoted SQL literal, or "" for unbounded
	Upper          string         // already-quoted SQL literal, or "" for unbounded
	LowerInclusive bool           // true → use >= for lower
	ColType        iop.ColumnType // for state writes
	LowerRaw       string         // raw display form for logs/resume hint
	UpperRaw       string         // raw display form for logs/resume hint
}

RangeChunk is a single [lower, upper) window for one merge pass.

func (RangeChunk) Describe

func (c RangeChunk) Describe(updateKey string) string

Describe is used for log lines.

func (RangeChunk) WhereCond

func (c RangeChunk) WhereCond(updateKey string, quote func(string) string) string

WhereCond returns the WHERE clause body for this chunk.

type RangeConfig

type RangeConfig struct {
	Start    string `yaml:"start,omitempty"`    // literal value; parsed lazily at execution time
	Advance  string `yaml:"advance,omitempty"`  // duration (5m, 5h, 5d, 5w, 1mo, 1y) — per-run forward advance
	Lookback string `yaml:"lookback,omitempty"` // duration
}

RangeConfig holds the range block from model front-matter. It drives the unified incremental / lookback / paged-backfill behavior (sling style only).

func (*RangeConfig) HasAdvance

func (r *RangeConfig) HasAdvance() bool

HasAdvance returns true if paged-advance mode is enabled.

func (*RangeConfig) HasLookback

func (r *RangeConfig) HasLookback() bool

HasLookback returns true if a lookback window is configured.

type Seed

type Seed struct {
	Name              string // file stem, unique across the project
	FilePath          string // absolute path
	RelPath           string // relative path from project root
	Schema            string
	Database          string // optional catalog; three-part dialects only
	FullTableName     string // [database.]schema.name (current mode)
	ProdFullTableName string // [database.]schema.name (always prod-mode, for SQL matching)
	Format            string // csv, json, parquet
}

Seed represents a seed file (CSV, JSON, Parquet) in the project.

type Selector

type Selector struct {
	Includes []string
	Excludes []string
	Project  *BuildProject
}

Selector filters DAG nodes based on include/exclude patterns.

func NewSelector

func NewSelector(includes, excludes []string) *Selector

NewSelector creates a new selector from include and exclude patterns.

func (*Selector) Apply

func (s *Selector) Apply(dag *DAG) ([]string, error)

Apply filters DAG nodes based on the selector patterns, returning names in DAG order.

type Style

type Style int

Style identifies which incremental pattern a model uses.

const (
	// StyleDbt is the dbt-compatible pattern: models use is_incremental() and {{ this }}
	// to write their own WHERE clauses. This is the zero value and the harmless default
	// for non-incremental models.
	StyleDbt Style = iota
	// StyleSling is the sling-native pattern: models use incremental_where_cond() and/or
	// incremental_value() Jinja functions, and sling owns the WHERE clause / watermark.
	StyleSling
)

type TableIdentity added in v1.6.1

type TableIdentity struct {
	Name     string // file stem
	Schema   string
	Database string // optional; three-part dialects only
}

TableIdentity is the resolved warehouse location of a model or seed.

func (TableIdentity) FullName added in v1.6.1

func (t TableIdentity) FullName() string

FullName returns [database.]schema.name.

type TemplateEngine

type TemplateEngine struct {
	// contains filtered or unexported fields
}

TemplateEngine compiles SQL model templates using Jinja-like syntax.

func NewTemplateEngine

func NewTemplateEngine(project *BuildProject, vars map[string]any) *TemplateEngine

NewTemplateEngine creates a new template engine for the given project.

func (*TemplateEngine) CompileAll

func (te *TemplateEngine) CompileAll(incCtx *IncrementalContext) error

CompileAll compiles all models in the project using the provided incremental context. A nil incCtx is equivalent to DefaultIncrementalContext().

func (*TemplateEngine) CompileModel

func (te *TemplateEngine) CompileModel(model *Model, incCtx *IncrementalContext) (string, error)

CompileModel compiles a model's SQL template, extracting config and resolving references. The incCtx parameter carries incremental-pattern values:

  • Style A (dbt) models read incCtx.IsIncremental via the is_incremental() Jinja function.
  • Style B (sling) models read incCtx.WhereCond / incCtx.Value via incremental_where_cond() and incremental_value() Jinja functions.

A nil incCtx is equivalent to DefaultIncrementalContext() — first-run semantics.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL