pipelinebuilder

package
v0.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 16, 2026 License: Apache-2.0 Imports: 13 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func DeserializeStateWithIssues

func DeserializeStateWithIssues(
	input []byte,
) (State, []ValidationIssue, error)

func InlineSQLFrom

func InlineSQLFrom(operationInline map[string]any) string

func LayoutGraph

func LayoutGraph(spec *PipelineSpec) map[string]Position

LayoutGraph computes deterministic positions for all nodes in the builder graph.

Algorithm: 1. Assign each step to a fixed column by kind (query/transform/sink) 2. Within each column, preserve pipeline spec order (pipeline.steps array) 3. Assign Y by position within the kind bucket using fixed row spacing

func PipelineFormatFromPath

func PipelineFormatFromPath(path string) (model.PipelineFormat, error)

func ResolvePipelineExportFormat

func ResolvePipelineExportFormat(context Context) model.PipelineFormat

func ResolveSinkBackend

func ResolveSinkBackend(operationType string) model.SinkBackend

ResolveSinkBackend returns the catalog-defined sink backend for a builder operation type.

func SerializePipelineSpec

func SerializePipelineSpec(input State, format model.PipelineFormat) ([]byte, error)

func SerializeStateCanonical

func SerializeStateCanonical(input State) ([]byte, error)

SerializeStateCanonical emits a stable JSON representation for diffs/reviews. We canonicalize and validate first so all writers produce the same output shape.

func SerializeStateComparable

func SerializeStateComparable(input State) ([]byte, error)

func ToDraftPipeline

func ToDraftPipeline(state State) (*pipelinespec.Pipeline, error)

ToDraftPipeline reconstructs a draft Pipeline spec from builder graph state.

func ToPipeline

func ToPipeline(state State) (*pipelinespec.Pipeline, error)

ToPipeline reconstructs a Pipeline spec from builder graph state. The authored pipeline step order is preserved while graph bindings update dependencies.

func TransformInputsForSources

func TransformInputsForSources(
	existing map[string]pipelinespec.StepInput,
	sources []slug.Slug,
) (map[string]pipelinespec.StepInput, error)

func ValidatePipelineSpec

func ValidatePipelineSpec(format model.PipelineFormat, input []byte) error

func ValidateState

func ValidateState(input State) error

ValidateState enforces structural integrity before save/export. We fail fast on invalid slugs or broken references to prevent corrupt graph state.

Types

type Binding

type Binding struct {
	FromStep slug.Slug `json:"from_step" validate:"required,slug"`
	FromPort string    `json:"from_port" validate:"required,min=1"`
	ToStep   slug.Slug `json:"to_step"   validate:"required,slug"`
	ToPort   string    `json:"to_port"   validate:"required,min=1"`
}

type Context

type Context struct {
	ExpectedVersionID     *string               `json:"expected_spec_hash,omitempty"`
	CurrentVersionID      *string               `json:"current_version_spec_hash,omitempty"`
	DraftBasedOnVersionID *string               `json:"draft_based_on_spec_hash,omitempty"`
	SourceFormat          *model.PipelineFormat `json:"source_format,omitempty"`
	ExportFormat          *model.PipelineFormat `json:"export_format,omitempty"`
	Deterministic         bool                  `json:"deterministic"`
	TimeoutSecs           *int                  `json:"timeout_seconds,omitempty"`
	RecordMeta            bool                  `json:"record_run_metadata"`
	TargetOrgSlug         string                `json:"target_org_slug,omitempty"`
	TargetProjectSlug     string                `json:"target_project_slug,omitempty"`
	TargetEnvironmentSlug string                `json:"target_environment_slug,omitempty"`
}

type Edge

type Edge struct {
	FromStep slug.Slug `json:"from_step" validate:"required,slug"`
	ToStep   slug.Slug `json:"to_step"   validate:"required,slug"`
}

type Graph

type Graph struct {
	Nodes    []GraphNode `json:"nodes"    validate:"dive"`
	Edges    []Edge      `json:"edges"    validate:"dive"`
	Bindings []Binding   `json:"bindings" validate:"dive"`
}

type GraphNode

type GraphNode struct {
	StepSlug slug.Slug `json:"step_slug" validate:"required,slug"`
}

type PipelineSpec

type PipelineSpec struct {
	Metadata pipelinespec.Metadata `json:"metadata" validate:"required"`
	Steps    []pipelinespec.Step   `json:"steps"    validate:"dive"`
}

type Position

type Position struct {
	X int
	Y int
}

Position represents X,Y coordinates for a node in the builder graph.

type State

type State struct {
	PipelineSpec PipelineSpec `json:"pipeline_spec"     validate:"required"`
	Graph        Graph        `json:"graph"             validate:"required"`
	Context      Context      `json:"context,omitempty"`
}

func CanonicalizeState

func CanonicalizeState(input State) State

CanonicalizeState normalizes values and sorts unordered graph collections. Pipeline step and graph node order are authored source state and must remain stable across a visual round trip.

func DeserializeState

func DeserializeState(input []byte) (State, error)

DeserializeState loads state from JSON and normalizes it immediately. Canonicalization on read keeps round-trips deterministic across UI/code boundaries.

func DeserializeStatePermissive

func DeserializeStatePermissive(input []byte) (State, error)

func FromPipeline

func FromPipeline(pipeline *pipelinespec.Pipeline) (State, error)

FromPipeline maps a validated pipeline spec into builder graph state. This establishes the visual load path for supported step kinds in a deterministic form.

func FromPipelineSpec

func FromPipelineSpec(path string, data []byte) (State, error)

FromPipelineSpec loads a pipeline spec and converts it into builder graph state. This is the spec-to-visual entry point used by import flows.

type ValidationIssue

type ValidationIssue struct {
	Field   string
	Code    string
	Message string
}

func ValidateStateIssues

func ValidateStateIssues(input State) []ValidationIssue

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL