scriptrun

package
v1.121.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 18, 2026 License: Apache-2.0 Imports: 20 Imported by: 0

Documentation

Overview

Package scriptrun is the managed-script execution engine: an embedded Starlark interpreter, the curated host stdlib scripts are allowed to call, and the static validator the authoring loop answers with.

Starlark (go.starlark.net) is the engine because determinism is a property of the LANGUAGE rather than of a blocklist the platform has to maintain: the language has no ambient clock, randomness, filesystem, or network, iteration order is specified, and unbounded loops and recursion are off by default. A script can only affect the world through bindings this package chooses to predeclare, which is why the surface here is small and enumerable — an enumerable surface is what makes a capability review meaningful.

The determinism contract this engine supports, stated exactly:

same script version + same parameters + same underlying data => same output

It is not "identical forever". The warehouse changes between runs, and that is the point of re-running. What the platform eliminates is every source of variation it controls: no clock or RNG is predeclared, the fire time arrives as a pinned parameter rather than a clock read, and tool results reaching a script carry no semantic enrichment (which varies with catalog state).

Resource limits, honestly: starlark-go bounds CPU with an execution-step limit and wall-clock through thread cancellation, and this package adds hard row and byte caps on every host result plus bounded log capture. Neither starlark-go nor any comparable embedded interpreter offers a hard MEMORY cap, so a pathological script can still grow the process heap. That residual risk is recorded in docs/scripts/security.md rather than papered over.

Index

Constants

View Source
const (
	CapabilityQuery  = script.CapabilityQuery
	CapabilityExport = script.CapabilityExport
)

Capability names. A capability is one host binding, and the set is closed: review means being able to enumerate everything a script can reach, which is only possible while the surface stays small enough to list. Nondeterministic tools (search, memory, catalog mutation) are deliberately absent — they have no place inside an automation whose value proposition is reproducibility.

The names live in the domain (pkg/script) because a grant is written in them: the vocabulary a reviewer approves and the vocabulary this engine enforces have to be one list, not two that agree by convention.

View Source
const (
	// DraftMaxSteps caps interpreter execution steps for a draft run.
	DraftMaxSteps = 2_000_000

	// DraftTimeout caps wall-clock time for a draft run, including the time
	// spent inside host calls.
	DraftTimeout = 60 * time.Second

	// DraftMaxRows caps the rows one platform.query may return to a draft.
	DraftMaxRows = 5_000

	// DraftMaxResultBytes caps the serialized size of one platform.query result.
	DraftMaxResultBytes = 8 << 20

	// MaxLogBytes caps captured print output. Anything a script needs to emit
	// that is larger than a log is an output asset, not a log line.
	MaxLogBytes = 64 << 10
)

Draft-execution limits. A draft runs interactively, under its author's own identity, while they iterate — so it is bounded more tightly than an approved run will be: the author is waiting for the answer, and a runaway draft should fail fast with a legible message instead of occupying a serving replica.

View Source
const (
	// ApprovedMaxSteps caps interpreter execution steps for an approved run.
	ApprovedMaxSteps = 20_000_000

	// ApprovedTimeout caps wall-clock time for one approved run, including the
	// time spent inside host calls. It matches the ceiling trino_export already
	// applies to a synchronous export.
	ApprovedTimeout = 10 * time.Minute

	// ApprovedMaxRows caps the rows one platform.query may return.
	ApprovedMaxRows = 20_000

	// ApprovedMaxResultBytes caps the serialized size of one query result.
	ApprovedMaxResultBytes = 32 << 20
)

Approved-execution limits. An approved run is looser than a draft on every axis, because nobody is waiting at a prompt for it and its code has been reviewed — but it is still bounded on all of them, because the one resource no embedded interpreter of this class can cap is memory, and every limit here is part of what keeps a runaway script from taking the process with it.

View Source
const (
	SeverityError   = "error"
	SeverityWarning = "warning"
)

Finding severities. An error means the script cannot run as written; a warning means it will run but a reviewer should look.

Variables

View Source
var (
	ErrStepLimit = errors.New("script exceeded its execution-step limit")
	ErrTimeout   = errors.New("script exceeded its time limit")
)

ErrStepLimit marks a run stopped by the execution-step limit, and ErrTimeout a run stopped by the wall-clock limit. Both are script-side failures: the same script on the same inputs will hit them again, so a caller must never retry them.

View Source
var Capabilities = script.Capabilities

Capabilities is the full host surface, in the order help and validate report it.

Functions

This section is empty.

Types

type Caller

type Caller interface {
	// CallTool invokes the named tool and returns its structured content. A
	// tool that reports an error returns a non-nil error carrying the tool's
	// message; the script sees it as a Starlark error and the run fails.
	CallTool(ctx context.Context, name string, args map[string]any) (map[string]any, error)
}

Caller issues one platform tool call on behalf of a running script and returns the tool's structured result.

Every host binding goes through this one seam, and the production implementation drives the fully assembled MCP server over an in-memory session — so persona and connection authorization, rate limiting, and audit all apply to a script's calls exactly as they apply to an agent's, with no second implementation to keep in step. Binding host functions straight onto narrow Go interfaces would be faster per call and would mean re-implementing authorization for user-authored code, which is the drift this platform's single-funnel design exists to prevent.

type ExportRecord

type ExportRecord struct {
	Name string `json:"name"`
	// Destination is the name the script wrote, so a run that sends one result
	// to two places reads as two records rather than as a repeat.
	Destination string `json:"destination"`
	Format      string `json:"format"`
	RowCount    int    `json:"row_count"`
	Bytes       int    `json:"bytes"`
	// Preview is true when nothing was persisted.
	Preview      bool   `json:"preview"`
	AssetID      string `json:"asset_id,omitempty"`
	AssetVersion int    `json:"asset_version,omitempty"`
	Bucket       string `json:"bucket,omitempty"`
	Key          string `json:"key,omitempty"`
}

ExportRecord is what one platform.export call did, in call order on the run's Result. A record with Preview set measured the output and wrote nothing, which is what a draft run does; otherwise it names the asset version written or the object delivered.

type ExportRequest

type ExportRequest struct {
	// Name identifies the output across runs. It is the stable half of the
	// output's identity: the same name from the same script maps to one asset
	// whose versions are its run history.
	Name string
	// Format is one of the formats exportFormats admits.
	Format string
	// Columns is the column order the script wrote, read from the rows before
	// they became order-free Go maps. A tabular format writes its columns in
	// this order; see columnOrder.
	Columns []string
	// Rows is the list of row dicts to write.
	Rows []any
	// Destination is the granted destination the output goes to, already
	// resolved from the name the script wrote to the address its approval
	// pinned. A draft carries only the name, because a draft writes nothing and
	// there is no approval yet to resolve against.
	Destination script.Destination
	// Key is the object key the script asked for beneath the destination's
	// granted prefix, empty when it named none and never set for the portal.
	Key string
}

ExportRequest is one output as the script produced it.

type ExportResult

type ExportResult struct {
	AssetID      string
	AssetVersion int
	Bucket       string
	Key          string
	// Bytes is the serialized size actually written, which the writer knows
	// exactly and the engine can only estimate.
	Bytes int
}

ExportResult is where one output landed. A portal output reports the asset version it created; a delivered one reports the object it wrote.

type Exporter

type Exporter interface {
	// Export writes one output and reports where it landed. An error fails the
	// run: a report whose output did not persist did not happen.
	Export(ctx context.Context, req ExportRequest) (*ExportResult, error)
}

Exporter persists one script output. The engine holds it behind this interface so nothing here knows what an asset, a bucket, or a portal is: the interpreter's job ends at "these rows, under this name, in this format".

type Finding

type Finding struct {
	Severity string `json:"severity" example:"error"`
	Line     int    `json:"line,omitempty" example:"12"`
	Message  string `json:"message"`
	// Hint is the corrective action. It carries most of the value of this
	// validator: an author who writes Python at a Starlark interpreter needs to
	// be told what to write instead, not merely that the parser disagreed.
	Hint string `json:"hint,omitempty"`
}

Finding is one thing the validator noticed, addressed to the author.

type Options

type Options struct {
	// Source is the Starlark source to execute, and Name labels it in tracebacks.
	Source string
	Name   string

	// RunID, FireTime and Params populate the frozen run dict — the script's
	// only source of time and of caller input.
	RunID    string
	FireTime time.Time
	Params   map[string]any

	// Caller issues the script's platform tool calls. A nil Caller leaves the
	// platform module predeclared but every call on it fails, which is what a
	// syntax-only execution wants.
	Caller Caller

	// Grants, when non-nil, is the approved capability set this run is confined
	// to: the host bindings it may call, the connections it may query, and the
	// destinations it may write. nil means no grant layer applies, which is the
	// draft case — a draft executes as its own author, with that author's
	// authority, so there is nothing to narrow.
	//
	// A non-nil grant with empty lists denies everything, which is the correct
	// reading of "approved with nothing granted" and the reason this is a
	// pointer rather than a value.
	Grants *script.Grants

	// Exporter persists what platform.export produces. nil previews instead,
	// which is what a draft run does.
	Exporter Exporter

	// Limits. Zero means the draft default.
	MaxSteps       uint64
	Timeout        time.Duration
	MaxRows        int
	MaxResultBytes int
	MaxLogBytes    int
}

Options configures one script execution.

func ApprovedLimits

func ApprovedLimits() Options

ApprovedLimits returns the limit set an approved run executes under, so a caller configures them by naming the policy rather than by copying four numbers it would then have to keep in step.

type Report

type Report struct {
	OK       bool      `json:"ok"`
	Findings []Finding `json:"findings"`
	// Capabilities is the set of host bindings the source references,
	// Connections the connection names it names literally, and Destinations
	// the output destinations it names literally. Together they are the raw
	// material for the grants diff a reviewer will be shown: what the code
	// reaches, next to what it was granted.
	Capabilities []string `json:"capabilities"`
	Connections  []string `json:"connections"`
	Destinations []string `json:"destinations"`
	// DynamicConnections is true when a platform.query call computes its
	// connection instead of naming one literally, and DynamicDestinations when
	// a platform.export call computes its destination, so the list in question
	// is known to be incomplete. Reporting the gap is the point: a reviewer
	// reading a list that silently omitted a computed name would be reading a
	// false statement.
	DynamicConnections  bool `json:"dynamic_connections"`
	DynamicDestinations bool `json:"dynamic_destinations"`
}

Report is the result of validating one script's source: whether it can run, what it would reach if it did, and everything the author or a reviewer should know first.

func Validate

func Validate(source string) Report

Validate parses and resolves a script without executing it, and reports what it would reach. It is the fast half of the authoring loop: an author gets interpreter-accurate errors, the Python-isms their instincts produce get a specific correction, and the capability set is extracted for review — all without a query running or a row moving.

type Result

type Result struct {
	// Log is the captured print output, truncated at the log cap.
	Log string `json:"log"`
	// LogTruncated is true when output was dropped at the cap.
	LogTruncated bool `json:"log_truncated"`
	// Steps is the interpreter step count the run consumed, and Duration its
	// wall-clock time. Both are the raw material for sizing an approved run's
	// limits against a draft that already works.
	Steps    uint64        `json:"steps"`
	Duration time.Duration `json:"-"`
	// Queries counts the platform.query calls the run issued.
	Queries int `json:"queries"`
	// Exports lists what every platform.export call did, in call order.
	Exports []ExportRecord `json:"exports"`
}

Result reports one completed execution.

func Run

func Run(ctx context.Context, opts Options) (*Result, error)

Run executes a script and returns its result. The error is non-nil when the script itself failed — a Starlark error, a refused host call, or a limit — and the returned Result still carries whatever log and metrics the run produced before failing, because that log is exactly what the author needs.

Failures here are deterministic by construction: the same source on the same inputs fails the same way. Callers must not retry them.

type SessionCaller

type SessionCaller struct {
	// contains filtered or unexported fields
}

SessionCaller issues a script's platform calls over one in-memory MCP session against the fully assembled server. It is the production Caller: every host binding a script invokes becomes an ordinary tool call, crossing the same authentication, authorization, gate, rate-limit, and audit middleware an agent's call crosses, with no second implementation to keep in step.

func Connect

func Connect(ctx context.Context, server *mcp.Server, label string) (*SessionCaller, func(), error)

Connect opens an in-memory MCP session against server and returns the Caller that drives it, plus the teardown for both ends.

The identity the session authenticates as comes from ctx, which the caller has already decorated: a draft run carries its author's own identity, an approved run carries the script principal and the roles its approval bound. This function deliberately establishes no identity of its own — there is one place a script's authority is decided, and it is not here. label names the client in the handshake so the two run kinds are distinguishable in logs.

func (*SessionCaller) CallTool

func (c *SessionCaller) CallTool(ctx context.Context, name string, args map[string]any) (map[string]any, error)

CallTool invokes one tool and returns its structured content.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL