Help us shape the future of Go! Take the 2026 Go Developer Survey

sdb

package
v0.13.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 22, 2026 License: MIT Imports: 19 Imported by: 0

Documentation

Overview

Package sdb is the API surface handed to Go scripts. A script is a plain Go file (package main) defining:

func Run(s *sdb.S) error

It can query any configured connection, loop with parameters, push results into the results view, and export in any supported format.

Index

Constants

View Source
const (
	KindSource    = pipeline.KindSource
	KindTransform = pipeline.KindTransform
	KindSink      = pipeline.KindSink
	KindAction    = pipeline.KindAction

	FieldString   = pipeline.FieldString   // one line
	FieldText     = pipeline.FieldText     // several lines
	FieldInt      = pipeline.FieldInt      // an integer
	FieldBool     = pipeline.FieldBool     // true/false
	FieldDuration = pipeline.FieldDuration // "30s", "5m"
	FieldEnum     = pipeline.FieldEnum     // one of Field.Enum
	FieldConn     = pipeline.FieldConn     // a connection name: the picker
	FieldTable    = pipeline.FieldTable    // a table on the node's conn
	FieldColumns  = pipeline.FieldColumns  // column names, comma- or line-separated
	FieldSQL      = pipeline.FieldSQL      // SQL, in a SQL editor
	FieldGo       = pipeline.FieldGo       // Go, in a Go editor
)

The four kinds of plugin, and the field types, for a plugin file's descriptor.

Variables

This section is empty.

Functions

func IsCanceled

func IsCanceled(err error) bool

IsCanceled reports whether an error from Query or Exec is the user stopping the run, as opposed to a genuine failure — so a script can unwind quietly:

if err != nil {
	if sdb.IsCanceled(err) {
		return nil
	}
	return err
}

Errors from Copy, Reader and Writer carry context.Canceled instead: the etl package does not depend on db, so both are recognised here.

Types

type Batch added in v0.13.0

type Batch = pipeline.Batch

Batch is the rows between two nodes of a fragment: columns and rows, with helpers to find, add, drop and filter.

func NewBatch added in v0.13.0

func NewBatch(cols []Col, rows [][]any) *Batch

NewBatch makes a batch of cols with rows; ColsOf pairs names with the source's type names ("" for unknown). For a go.source's first batch.

type Cfg added in v0.13.0

type Cfg = pipeline.Config

Cfg is a node's settings, every value a string: sdb.Cfg{"conn": "prod"}.

type Col added in v0.13.0

type Col = pipeline.Col

Col is one column of a Batch: its name and the source's type name.

func ColsOf added in v0.13.0

func ColsOf(names, dbTypes []string) []Col

type CopyOpts added in v0.9.0

type CopyOpts = etl.CopyOptions

CopyOpts shapes s.Copy: destination name, a Where or a whole Query as the source, a column subset, Create/Truncate of the destination, a per-row Transform. The zero value copies the whole table into an existing table of the same name. Set ProgressEvery to have the script output report the running row count.

type CopyStats added in v0.9.0

type CopyStats = etl.CopyStats

CopyStats reports a finished copy; its String() is a one-line summary.

type Env added in v0.13.0

type Env = pipeline.Env

Env is what a Go node runs in: the session (S), the parameters, the values earlier fragments published, and a logger.

type Field added in v0.13.0

type Field = pipeline.Field

Field is one setting of a plugin. It has a Name, a Type (what the inspector draws), Doc, Default, Required and, for an enum, its values.

type FieldType added in v0.13.0

type FieldType = pipeline.FieldType

FieldType is what a Field holds. It decides what the inspector draws.

type Fragment added in v0.13.0

type Fragment = pipeline.FragmentBuilder

type FragmentStats added in v0.13.0

type FragmentStats = pipeline.FragmentStats

FragmentStats and NodeStats are the parts of RunStats.

type JobPipeline added in v0.13.0

type JobPipeline struct {
	ID string
	RunStats
}

JobPipeline is one pipeline of a job's run: its step id in the job and its stats (Status, Fragments, Rows(), …). A step skipped because one before it failed has status skipped.

type JobRun added in v0.13.0

type JobRun struct {
	ID        string
	Job       string
	Status    string // succeeded, failed, canceled
	Error     string
	Started   time.Time
	Ended     time.Time
	Pipelines []JobPipeline
}

JobRun is a job's run as s.RunJob returns it: its id, its outcome, and each pipeline's stats in the job's order.

type JobRunner added in v0.13.0

type JobRunner func(s *S, name string, params Params) (*JobRun, error)

JobRunner runs a job for RunJob and waits for it. The host provides it.

var DefaultJobRunner JobRunner

DefaultJobRunner runs s.RunJob for a session whose host set none. Package jobs sets it when it is linked in (every dbc binary): a private engine over the session's connections and paths.

type Kind added in v0.13.0

type Kind = pipeline.Kind

Kind is what a plugin builds. It is one of KindSource, KindTransform, KindSink or KindAction.

type NodeStats added in v0.13.0

type NodeStats = pipeline.NodeStats

type Params added in v0.13.0

type Params = map[string]string

Params are a run's parameter overrides: sdb.Params{"days": "7"}.

type Paths added in v0.13.0

type Paths struct {
	ScriptsDir   string
	PipelinesDir string
	JobsDir      string
	RunsDir      string
	FilesDir     string
}

Paths are where a session finds scripts, pipelines and jobs by name, puts a job run's record, and resolves a pipeline's relative file paths. The host sets them: script.run, RunPipelineNamed and RunJob read the first three, RunsDir is where a job run there leaves its record, and FilesDir (the config's files_dir) is PipelineOpts.FilesDir's default.

type Pipeline added in v0.13.0

type Pipeline = pipeline.Builder

Pipeline is a pipeline being built in Go (sdb.NewPipeline); Fragment is one of its fragments being filled.

func NewPipeline added in v0.13.0

func NewPipeline(name string) *Pipeline

NewPipeline starts a pipeline called name, to fill with Fragment and run with s.RunPipeline.

type PipelineOpts added in v0.13.0

type PipelineOpts = pipeline.Options

PipelineOpts shape a run: Params, a Log line sink, Progress, one Fragment only, or PreviewRows for a preview that writes nothing.

type PipelineSpec added in v0.13.0

type PipelineSpec = pipeline.Spec

PipelineSpec is a pipeline as data — what a pipeline file holds.

func ParsePipeline added in v0.13.0

func ParsePipeline(text string) (*PipelineSpec, error)

ParsePipeline reads a pipeline from its JSON text.

type Plan

type Plan = explain.Plan

Plan is an explained statement: its steps (Root, Nodes), what it cost or took, and dbc's findings about it (Insights). See package explain.

type PlanText

type PlanText = explain.TextOptions

PlanText shapes Plan.Text: width, the metric bars are sized by, color, and whether the findings are appended.

type Plugin added in v0.13.0

type Plugin = pipeline.Plugin

Plugin describes a kind of node. A plugin file in plugins_dir declares one as var Plugin = sdb.Plugin{…} — Name ("mask.email"), Kind, Label, Doc and Fields — beside plain funcs the Kind calls for (Apply for a transform, Next for a source, Write for a sink, Run for an action); the loader leaves New and Check to itself. Fields are the node's settings, drawn as the inspector's form and read in the funcs as e.Cfg.

type Reader added in v0.9.0

type Reader = etl.Reader

Reader streams a query's rows: for rd.Next() { row := rd.Row() }, then rd.Err(). Close it (defer rd.Close()) if you stop before the end.

type Result

type Result = model.Result

Result is the tabular result type scripts receive from Query.

type RunStats added in v0.13.0

type RunStats = pipeline.RunStats

RunStats is a pipeline's run: each fragment's status, rows and node counters, and the outcome. Its String is a one-line summary.

type S

type S struct {
	// contains filtered or unexported fields
}

S is the session handle passed to a script's Run function.

func New

func New(mgr *db.Manager, show func(*model.Result), print func(string)) *S

New builds a script session. show receives results pushed via Show; print receives Print output (results table / log pane in the TUI, stdout in headless mode).

func (*S) Canceled

func (s *S) Canceled() bool

Canceled reports whether the user has stopped this run.

func (*S) CatalogChanged added in v0.9.1

func (s *S) CatalogChanged(conn string) bool

CatalogChanged reports whether the session has run a statement that may change conn's catalog (sqlsplit.ChangesCatalog: CREATE, ALTER, DROP, RENAME, COMMENT, ATTACH, DETACH) — through Query, Exec, or what Copy and Writer run to ready a destination. The host asks it once the script has returned, to decide whether the sidebar should list the tables again. Statements run through DB's raw handle are out of its sight, as they are out of the DDL log's.

func (*S) Conns

func (s *S) Conns() []string

Conns returns the configured connection names.

func (*S) Copy added in v0.9.0

func (s *S) Copy(src, dst, table string, opt CopyOpts) (CopyStats, error)

Copy copies table from connection src into connection dst:

st, err := s.Copy("prod", "local", "public.orders", sdb.CopyOpts{
	Create: true, Truncate: true, Where: "created_at >= now() - interval '7 days'",
})
s.Print("%s", st) // copied 48213 rows prod:public.orders → local:public.orders in 1.4s (direct COPY)

Postgres to Postgres without a Transform or Args streams COPY to COPY and never decodes a row; every other pairing goes row by row. Either way the destination is loaded in one transaction (with its Create and Truncate), so a failed or stopped copy leaves it as it was.

func (*S) Ctx

func (s *S) Ctx() context.Context

Ctx returns the session's context. Long-running scripts can select on s.Ctx().Done() (or check s.Canceled()) to bail out early.

func (*S) DB

func (s *S) DB(conn string) (*sql.DB, error)

DB exposes the raw *database/sql.DB for a connection — the escape hatch for transactions, prepared statements, or anything else database/sql can do.

func (*S) ETLConn added in v0.13.0

func (s *S) ETLConn(name string) (etl.Conn, error)

ETLConn resolves a connection name to the etl package's view of it: the pool, opened on first use under the run's context, and its SQL dialect. Its Trace feeds the DDL log (LogDDL), so the CREATE TABLE a Copy makes or a Writer's Setup runs is logged as a script's own Exec of it would be.

func (*S) Exec

func (s *S) Exec(conn, stmt string, args ...any) (int64, error)

Exec runs a non-query statement and returns the rows affected. Its server notices are printed as Query's are.

func (*S) Explain

func (s *S) Explain(conn, stmt string, analyze bool) (*Plan, error)

Explain describes how conn's database runs stmt, as the TUI's Ctrl+X and `dbc explain` do. analyze runs the statement to measure it: on Postgres a write is analyzed inside a transaction that is rolled back; on the other engines a write is explained but not run. It runs on its own session, not one the script shares with Query.

p, err := s.Explain("pg", "SELECT * FROM orders WHERE user_id = 42", true)
s.Print("%s", p.Text(sdb.PlanText{Insights: true}))
for _, in := range p.Insights { if in.Severity != "info" { … } }

func (*S) Export

func (s *S) Export(r *Result, format, path string) error

Export renders a result in the named format (csv|tsv|markdown|html|json|text) and writes it to path — or to the system clipboard when path is empty. A relative path is in files_dir, as s.Path has it: "report.csv" is one file whether a shell, dbc web or a schedule runs the script. Missing directories are created, as csv.write creates them.

func (*S) ForFiles added in v0.13.0

func (s *S) ForFiles(dir string) pipeline.Host

ForFiles is the session as a pipeline run whose files_dir is dir sees it: the same session — its connections, context and output, and the Readers, Writers and catalog changes it keeps for the host (shared) — with Paths.FilesDir set to dir. pipeline.Run hands it to the run's nodes (pipeline.FilesHost), so a go.action or script.run node's s.Path and s.Export resolve where the run's file nodes do. The host's, not a script's: the session itself is returned when dir is already its own.

func (*S) LoadPipeline added in v0.13.0

func (s *S) LoadPipeline(name string) (*PipelineSpec, error)

LoadPipeline finds a pipeline the way RunPipelineNamed does and parses it.

func (*S) LogDDL added in v0.9.0

func (s *S) LogDDL() *S

LogDDL turns on the DDL log for the rest of the session: every data definition statement it runs (sqlsplit.IsDDL: CREATE, ALTER, DROP, TRUNCATE, RENAME, COMMENT, GRANT, REVOKE) is written to the script output, as Print does, the moment before it runs:

DDL warehouse: CREATE INDEX orders_user_idx ON orders (user_id)

That covers a script's Query and Exec and what Copy and Writer run to ready a destination (a Create's CREATE TABLE, Truncate, a Writer's Setup). Statements a script runs through DB's raw handle are out of its sight.

Logging before the statement, as Postgres's log_statement = 'ddl' does, rather than after it, puts a long CREATE INDEX or an ALTER stuck behind a lock in the log while it is still running. A Query or Exec that then fails adds a "DDL <conn> failed" line, since a script may carry on past the error; a failed Copy or Writer stops with an error naming the statement anyway.

script.Run turns this on for every script, whichever UI runs it. The session's other host, dbc copy, leaves it off. There is deliberately no way to turn it off again, so a script cannot opt out of its own log.

func (*S) Manager added in v0.13.0

func (s *S) Manager() *db.Manager

Manager is the connections the session runs on. For the host's helpers (a job runner builds its sessions over it), not for scripts.

func (*S) Path added in v0.13.0

func (s *S) Path(p string) string

Path is where a script's file p is, a relative p being in files_dir as a file node's is: os.Create(s.Path("out.csv")). "~/…" is under the home directory and an absolute path is left as written; a relative one is joined to the host's files_dir (Paths.FilesDir), so "out.csv" is one file whether the script runs from `dbc script` in a shell, the TUI, dbc web, a schedule or dbc.app — not one in each process's working directory. A script opens every file of its own through it (os.Create, os.Open, os.ReadFile, …); s.Export already does.

One result, not Env.Path's (string, error), so it goes straight into an os call: the one failure, a "~" path with no home directory to find, leaves p as written, and the open then fails naming it. With no files_dir set (a host that sets no Paths, a test) a relative p is left relative to the working directory, as before files_dir. In a go.action or script.run node, s is the run's session as the run sees it (ForFiles): its files_dir is the run's, so the node's own files land with the run's file nodes', even when a script ran the pipeline with a files_dir of its own (PipelineOpts.FilesDir).

func (*S) Paths added in v0.13.0

func (s *S) Paths() Paths

Paths is where the session resolves script and pipeline names.

func (*S) Print

func (s *S) Print(format string, args ...any)

Print writes a Printf-style message to the script output.

func (*S) Query

func (s *S) Query(conn, query string, args ...any) (*Result, error)

Query runs a statement with parameters on the named connection and returns the result set. Use the placeholder style of the target driver ($1 for postgres/bytdb, ? for mysql/sqlite).

Server notices the statement raises (Postgres RAISE NOTICE and its kin) are written to the script output, as Print does, before Query returns — whether or not the statement failed.

func (*S) Reader added in v0.9.0

func (s *S) Reader(conn, query string, args ...any) (*Reader, error)

Reader runs query on conn and streams its rows. Use the placeholder style of the connection's driver ($1 postgres/bytdb, ? mysql/sqlite).

func (*S) Release added in v0.9.0

func (s *S) Release()

Release closes every Reader and rolls back every Writer the script left open. The host calls it when the script's Run returns (or panics): a forgotten Writer would otherwise hold a connection and an open transaction — and its locks — for the rest of the process. It rolls back rather than commits, because only an explicit Close says the load is complete. Both are no-ops on what the script already closed.

func (*S) RunJob added in v0.13.0

func (s *S) RunJob(name string, params Params) (*JobRun, error)

RunJob runs the job called name — a file in the jobs directory (.json may be left off) or a built-in example — with params overriding its own, and waits for it to end. Stop (Ctrl+K) cancels it as it does a query. The run comes back however it ended; the error is non-nil when it did not succeed.

func (*S) RunPipeline added in v0.13.0

func (s *S) RunPipeline(p *Pipeline, opt PipelineOpts) (*RunStats, error)

RunPipeline runs a pipeline built with NewPipeline. Every fragment runs in order, each sink committing at its fragment's end; the stats come back complete even on error. Stop (Ctrl+K) cancels the run as it does a query.

func (*S) RunPipelineNamed added in v0.13.0

func (s *S) RunPipelineNamed(name string, opt PipelineOpts) (*RunStats, error)

RunPipelineNamed runs a saved pipeline: a file path, a name in the pipelines directory (.json optional), or a built-in example.

func (*S) RunPipelineSpec added in v0.13.0

func (s *S) RunPipelineSpec(spec *PipelineSpec, opt PipelineOpts) (*RunStats, error)

RunPipelineSpec runs a pipeline given as data (ParsePipeline, or a literal). A relative path in its file nodes is in the host's files_dir (Paths.FilesDir) unless opt.FilesDir names another directory, so a pipeline run from a script finds the files it finds when scheduled.

${run.trigger} is "script" unless opt.Run says otherwise, and ${run.id} a fresh id of the run's own (pipeline.RunID), as the jobs engine's runs have; no run record is written for it.

func (*S) Show

func (s *S) Show(r *Result)

Show pushes a result to the results view (TUI) or prints it (headless).

func (*S) WithContext

func (s *S) WithContext(ctx context.Context) *S

WithContext attaches a cancellation context to the session and returns it. The host — not the script — sets this: when the user stops a run, the query in flight is aborted and every later Query or Exec fails immediately, so a looping script unwinds through its own error handling.

func (*S) WithJobs added in v0.13.0

func (s *S) WithJobs(r JobRunner) *S

WithJobs sets the runner s.RunJob uses: the host's engine. The host sets it; a script does not.

func (*S) WithPaths added in v0.13.0

func (s *S) WithPaths(p Paths) *S

WithPaths tells the session where scripts and pipelines live, for running them by name. The host sets it; a script does not.

func (*S) Writer added in v0.9.0

func (s *S) Writer(conn, table string, cols []string, opt WriteOpts) (*Writer, error)

Writer starts a load into table on conn; rows passed to Write line up with cols. Nothing is visible to other sessions until Close commits.

type Stats added in v0.13.0

type Stats = pipeline.Stats

Stats is a sink's or an action's report. It holds the rows loaded (or affected), a note, and Vars for later fragments (${frag.<fragment>.<name>}); a go.sink's Commit or a plugin's Run returns one.

type WriteOpts added in v0.9.0

type WriteOpts = etl.WriteOptions

WriteOpts shapes s.Writer: Setup statements and Truncate run first, in the load's transaction; BatchSize tunes INSERT batching off Postgres.

type Writer added in v0.9.0

type Writer = etl.Writer

Writer loads rows into one table in one transaction: w.Write(row) per row, then w.Close() to commit. defer w.Abort() rolls back on any early return; it is a no-op after Close.

Directories

Path Synopsis
Package sdbapi describes the sdb package — what a script can call on s *sdb.S, and the fields and methods of the types it hands back — as data, for the script editor's completion and hover in dbc web.
Package sdbapi describes the sdb package — what a script can call on s *sdb.S, and the fields and methods of the types it hands back — as data, for the script editor's completion and hover in dbc web.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL