engine

package
v0.16.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 31, 2026 License: Apache-2.0 Imports: 20 Imported by: 0

Documentation

Overview

Package engine is the anfra engine's public Go API: with the shared packages hosts build on (shared/apperr, appstep and apptracing, which classify its errors; shared/apikit, httpkit and requestid, the API framework its operations are served by), the only packages outside internal/ that another module may import.

It exists so that a separate application — a server, a hosted product, a different front end — can run anfra commands in-process, with the same registry and therefore the same semantics as the `anfra` CLI. Everything reaches commands through Dispatch, so the two surfaces cannot drift.

What this package is for

Keeping the surface small is the point, not an accident. Everything the engine knows how to do is reachable through Dispatch, and everything else here exists only to build one call: connect to sidecars, open a repo, state what the caller may see. If a symbol below is not needed for that, it should not be here — and if something in internal/ is needed and is not here, adding it is a deliberate act rather than an import away.

What this package is NOT for

The engine makes no access decisions. There is no principal, no role, no policy, and no notion of a tenant anywhere in this API. A host decides whether a caller may run a command before it calls Dispatch; what crosses this boundary is the consequence of that decision — the data restrictions that apply — and never its inputs. See DataPerms and Attribution, which look alike and are emphatically not the same thing.

Stability

This package is versioned with the module. Everything under internal/ is not part of it and may change in any release; if you find yourself wanting something from there, open an issue rather than a fork.

Index

Constants

View Source
const (
	// StatusOK reports a command that ran and produced a valid result.
	StatusOK = app.StatusOK
	// StatusInvalid reports a command that ran correctly and produced a result
	// that is not valid — validation diagnostics, typically. It is an outcome,
	// not a failure to run.
	StatusInvalid = app.StatusInvalid
)
View Source
const (
	ArgString      = command.ArgString
	ArgBool        = command.ArgBool
	ArgStringArray = command.ArgStringArray
	ArgInt         = command.ArgInt
	ArgObject      = command.ArgObject
	ArgObjectArray = command.ArgObjectArray
)
View Source
const (
	DataPermModeGoverned   = datasource.DataPermModeGoverned
	DataPermModeUngoverned = datasource.DataPermModeUngoverned
)

The modes a connection is asked for in (DataPermMode).

Variables

View Source
var (
	// UnknownCommand: the request names no registered command.
	UnknownCommand = errcode.UnknownCommand
	// DataPermsMissing: the Invocation's DataPerms were never decided.
	DataPermsMissing = errcode.DataPermsMissing
	// DataPermsUnenforceable: the Invocation's DataPerms are Governed, and the
	// query cannot be governed (raw SQL). A host that authorized the caller on
	// the data source states Ungoverned.
	DataPermsUnenforceable = errcode.DataPermsUnenforceable
	// SidecarUnavailable: a sidecar the command needs did not respond, or is
	// not connected.
	SidecarUnavailable = errcode.SidecarUnavailable
	// QueryInvalid: a query that cannot run or compile, with its diagnostics.
	QueryInvalid = query.QueryInvalid
	// QueryFailed: the data source did not run the query — the database
	// rejecting it, the data source unreachable, or its credentials refused —
	// with its own message. User scope: the query or the data source is the
	// user's to fix. Which of these it is is not told apart yet.
	QueryFailed = errcode.QueryFailed
	// ExportsUnavailable: an export, on an Invocation with no Exports store.
	ExportsUnavailable = errcode.ExportsUnavailable
	// ResultTooLarge: a query whose result has more rows than the Invocation's
	// QueryLimits allow; it fails rather than being cut short.
	ResultTooLarge = query.ResultTooLarge
	// PageTooLarge: a page of a query's result, answered whole, larger in bytes
	// than the Invocation's QueryLimits allow; it fails rather than being cut
	// short, so that page numbers stay exact.
	PageTooLarge = query.PageTooLarge
	// ErrUnknownDataSource is what DataSources.Connection answers for a name the
	// repo has no data source by.
	ErrUnknownDataSource = datasource.ErrUnknown

	// LocalRepoNotAllowed: an Invocation on a local repo (OpenRepo) without
	// AllowLocalRepo. A host mistake, never declared by an op.
	LocalRepoNotAllowed = errcode.LocalRepoNotAllowed
)

The codes Dispatch fails with, beside apperr.ValidationFailed for args it refuses (an arg unknown, missing, malformed, conflicting with another, or not applicable; its violations name each one). They carry a scope and no status: a host translates them to its own codes where it calls Dispatch (apperr.Translate), and treats anything else Dispatch returns as an internal error.

View Source
var (
	QueryOp         = apikit.OpRef[QueryRunInput]{Name: "core.query"}
	QueryCompileOp  = apikit.OpRef[QueryRunInput]{Name: "core.query.compile"}
	QueryValidateOp = apikit.OpRef[QueryInput]{Name: "core.query.validate"}
	QueryExportOp   = apikit.OpRef[QueryExportInput]{Name: "core.query.export"}
	ShowOp          = apikit.OpRef[ShowInput]{Name: "core.show"}
)

Refs to the ops a host authorizes on, with their input types: what a host mounts them by (apikit.Mount), so that a derivation for another type does not compile.

View Source
var Namespace = errcode.NS

Namespace is the engine's namespace, for its codes and its steps: anfra.

Functions

func Commands

func Commands() []string

Commands returns the names of every registered command, in registry order.

Names only: what a command *requires* of a caller is not the engine's question, so no permission vocabulary crosses this boundary. This exists so a host that maintains its own command-to-permission mapping can test that the mapping still covers the registry after an anfra core upgrade — and fail its build rather than silently exposing a new command.

func ErrorCodes

func ErrorCodes() []apperr.Code

ErrorCodes lists the engine's own codes Dispatch can fail with — every one but the generic validation_failed — with its scope and details type: for a host to test that it translates each of them, and fail its build on one added by an anfra core upgrade.

func Ops

Ops is every command as an apikit op, core.<command>, admitted from an Invocation: the core API, for a host to serve as it is. A host mounts each op it serves (apikit.Mount, by its ref above or apikit.Untyped) under its own admission, building the Invocation for the caller it admitted; the ops' names, schemas, handlers and codes are anfra core's, so a client of the core API meets the same contract on every host. Each call returns a new registry.

Types

type ArgSpec

type ArgSpec = app.ArgSpec

ArgSpec is one of a command's args as an API caller sees it.

type ArgType

type ArgType = command.ArgType

ArgType is an arg's type: ArgString, ArgBool, ArgStringArray, ArgInt, ArgObject or ArgObjectArray.

type Attributes

type Attributes = dataperm.Attributes

Attributes are caller attributes that the repo's data rules are evaluated against — a region, a department, an email. They are decision-bearing: the compiler reads them, and changing one changes which rows come back.

type Attribution

type Attribution = attribution.Fields

Attribution names the caller of one invocation for logs, spans and audit records — a tenant, a user, a session, a request id, whatever you can attribute by.

It is tracing baggage and nothing else. No command reads it and it never reaches the compiler. If a value must change which rows a query returns, it belongs in Attributes, and the engine enforces that separation rather than trusting this comment.

type Clients

type Clients = command.Clients

Clients are the sidecar clients commands run against. Build one with Connect.

func Connect

func Connect(ctx context.Context, nodeURL, canalQueryURL string, opts ...ConnectOption) (Clients, io.Closer, error)

Connect dials sidecars that something else is running — the compose or Kubernetes topology, where anfra-node and canal-query are separate containers. Both URLs are required; this never starts a process.

The returned Closer releases the clients. It does not stop the sidecars, because this process does not own them.

Spawning sidecars is deliberately not exposed. It is how the local CLI works, where one short-lived process owns its children, and it is the wrong shape for a server: a long-lived host wants sidecars with their own lifecycle, restart policy and scaling.

type CommandSpec

type CommandSpec = app.CommandSpec

CommandSpec is a command as an API caller sees it.

func Describe

func Describe() []CommandSpec

Describe returns every registered command, in registry order, in the shape an API caller sees: its args with their types, which are required and which are closed to a set of values, the groups of which exactly one must be set, the type of its answer and whether it can be invalid, and the codes it can fail with. A host publishes each as an operation (an OpenAPI path, an MCP tool, a usage page) without re-declaring what the engine knows.

It is the API's view, not the CLI's: aliases, shorthands and stdin are left out. Dispatch enforces what it says, and every field is derived from the command's definition, so it cannot drift from what the command does. The specs are copies.

type CompiledQuery added in v0.11.1

type CompiledQuery = command.CompiledQuery

CompiledQuery is a query as it is sent to its data source.

type ConnectOption

type ConnectOption func(*connectConfig)

ConnectOption configures Connect.

func WithTransport

func WithTransport(wrap func(sidecar string, rt http.RoundTripper) http.RoundTripper) ConnectOption

WithTransport wraps each sidecar client's HTTP transport, e.g. with otelhttp.NewTransport, so a host traces calls to the sidecars and propagates its trace context into them. wrap receives the engine's own transport, so its settings (timeouts, pooling) are kept, and the sidecar's name ("anfra-node", "canal-query") to label spans by. It applies once a sidecar is ready: readiness polling is not wrapped.

The engine takes no tracing dependency for this: the host brings the wrapper.

type DataPermMode added in v0.11.1

type DataPermMode = datasource.DataPermMode

DataPermMode is how the SQL a connection carries reads the data, as the engine states it when it asks DataSources.Connection: governed, compiled from a dataset with the invocation's DataPerms in it; or ungoverned, as written. A host may refuse a mode it did not grant the call on that data source — a second gate, behind its authorization and DataPerms, which must be right on their own.

type DataPerms

type DataPerms = dataperm.Set

DataPerms is how a caller's queries may read the data, for one invocation: governed, through the semantic layer with the repo's rules applied; or ungoverned, raw SQL included. It cannot be built with a struct literal: use Governed or Ungoverned, so that "ungoverned" is something a host states rather than something it defaults into by leaving a field blank.

func Governed added in v0.11.1

func Governed(attrs Attributes) DataPerms

Governed states that this caller's queries reach the data only through the semantic layer, with the repo's rules evaluated against attrs: raw SQL is refused, even with no attributes.

func Ungoverned added in v0.11.1

func Ungoverned() DataPerms

Ungoverned states that this caller's queries may read the data as written: raw SQL runs, and no rule applies. It is an answer, not a default: Dispatch refuses an Invocation whose DataPerms nobody set, so a host that has not thought about permissions cannot accidentally get ungoverned access.

type DataSource added in v0.9.0

type DataSource = datasource.DataSource

DataSource is one data source: its name, its type (the SQL dialect), its connection, which never leaves the host, and its settings for the query engine (timeouts, SSL and the like; none for the defaults).

type DataSourceSet added in v0.9.0

type DataSourceSet = datasource.Set

DataSourceSet is DataSources held in memory, by name: for a host that already holds them all.

type DataSources added in v0.9.0

type DataSources = datasource.DataSources

DataSources is how the engine reaches a repo's data sources, as a host supplies them on an Invocation (Invocation.DataSources): every one's dialect, to compile; one connection, only to run a query on the data source it targets; neither, to validate. A host implements it over its own store, where Connection is the one place a data source's connection is retrieved and its use authorized. Required for a managed repo, which has no data_sources.yml, for the commands that compile or run a query.

type Dialect added in v0.9.0

type Dialect = datasource.Dialect

Dialect is a data source as compiling needs it: its name and type, no connection.

type Export

type Export = command.Export

Export is one file an ExportStore is writing.

type ExportLink = command.ExportLink

ExportLink is an export's answer: where to download its file, until when.

type ExportStore

type ExportStore = command.ExportStore

ExportStore is where exports go, set on an Invocation as Exports: a file written once, then a link to it that expires. A host provides its own (object storage and presigned URLs, say); without one, an export fails with ExportsUnavailable. See docs/designs/exports.md.

type FinishedQuery added in v0.11.1

type FinishedQuery = command.FinishedQuery

FinishedQuery is how a query ended: its rows, bytes scanned, timings and error.

type Invocation

type Invocation = command.CommandContext

Invocation is everything a command runs against other than its arguments. It is the trusted half of a call: build it server-side and never decode it from a request body, or a caller could assert their own permissions.

type Observers added in v0.11.1

type Observers = command.Observers

Observers are who the engine tells of what a command does as it does it, one per capability, set on an Invocation as Observers; each nil when nobody listens. A host that records them sets the ones it records.

type QueryExportInput added in v0.11.1

type QueryExportInput = querycmd.ExportInput

QueryExportInput is core.query.export's input.

type QueryID added in v0.11.1

type QueryID = command.QueryID

QueryID matches a query's calls to its QueryObserver.

type QueryInput added in v0.11.1

type QueryInput = querycmd.QueryInput

QueryInput is core.query.validate's input, and the query and its target, in the others'.

type QueryLimits added in v0.11.1

type QueryLimits = query.Limits

QueryLimits bound what a query may cost the host, set on an Invocation as QueryLimits. The zero value is the defaults (a million result rows).

type QueryObserver added in v0.11.1

type QueryObserver = command.QueryObserver

QueryObserver is told of each query a command runs on a warehouse: compiled, on the warehouse, finished.

type QueryRunInput added in v0.11.1

type QueryRunInput = querycmd.QueryRunInput

QueryRunInput is core.query's and core.query.compile's input.

type Repo

type Repo = repo.Repo

Repo is the AML repo an invocation acts on: a directory (OpenRepo), or a managed repo, which the platform keeps and the engine reads at a revision (ManagedRepo). It exposes no directory: a command that needs one (the search catalog) refuses a managed repo, rather than reading an empty path.

func ManagedRepo added in v0.9.0

func ManagedRepo(id, revision string) (Repo, error)

ManagedRepo is a repo the platform manages, read at one revision: id is its stable identity, which anfra-node keys its incremental compile cache by and which must not change from one revision to the next; revision is "<repo>@<sha>", the commit's full sha. anfra-node must be configured to read revisions (its git service). It has no directory, so no data_sources.yml: the host supplies its data sources on the Invocation.

func OpenRepo

func OpenRepo(dir string) Repo

OpenRepo resolves an AML repo directory to its identity and state layout. It creates no directories and does not validate the contents. A command on it is refused unless the Invocation sets AllowLocalRepo: a platform serving managed repos (ManagedRepo) leaves it unset, so a directory cannot be served by mistake.

Unlike the CLI, which derives a repo from the working directory, a server has no meaningful working directory and must be told.

type Request

type Request = app.Request

Request is a command invocation: a name plus its arguments. It mirrors a CLI invocation, and it is the untrusted half of a call — decode it from a request body if you like.

type Response

type Response = app.Response

Response is the envelope every command returns: an explicit status plus a result payload.

func Dispatch

func Dispatch(ctx context.Context, inv Invocation, req Request) (Response, error)

Dispatch runs one command and returns its Response.

It refuses an Invocation whose DataPerms nobody decided — every command, with no exemptions. That is not an authorization decision: the engine does not know who is calling or what they are entitled to. It is the guarantee that nothing runs until the host has said what the caller may see.

type ShowInput added in v0.11.1

type ShowInput = show.ShowInput

ShowInput is core.show's input.

type Status

type Status = app.Status

Status is a command's explicit outcome. A command that ran correctly but produced an invalid result (a failed validation, say) reports StatusInvalid, which is not an error.

type TimedStep added in v0.11.1

type TimedStep = command.TimedStep

TimedStep is when one step of a run started.

type Timings added in v0.11.1

type Timings = command.Timings

Timings are a run's timeline: when each of its steps started, and when it ended. A query's are its FinishedQuery's.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL