anfranode

package
v0.10.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 16, 2026 License: Apache-2.0 Imports: 17 Imported by: 0

Documentation

Overview

Package anfranode is the anfra-node sidecar, the AML/AQL engine: its supervisor, which spawns one or connects to an external one, and its JSON-RPC client.

Index

Constants

View Source
const EnvPrefix = "ANFRA_NODE_ENV_"

EnvPrefix sets anfra-node's own settings from anfra's environment, for the one anfra spawns: ANFRA_NODE_ENV_NODE_OPTIONS=--max-old-space-size=4096 is its NODE_OPTIONS (sidecar.ProcSpec.EnvPrefix).

View Source
const Name = "anfra-node"

Name is the sidecar's name, in its logs, its errors and its binary's.

View Source
const RPCInvalidParams = -32602

RPCInvalidParams is the JSON-RPC code anfra-node answers a request it refuses as invalid input with (its InvalidInputError).

Variables

This section is empty.

Functions

func CompileDataSources

func CompileDataSources(dialects []datasource.Dialect) map[string]CompileDataSource

CompileDataSources is the repo's data sources as the sidecar takes them: a name and a dialect each, without their connections.

Types

type AQLDiagnostic

type AQLDiagnostic struct {
	Message  string `json:"message"`
	Severity string `json:"severity"`
	Line     *int   `json:"line,omitempty"`
	Column   *int   `json:"column,omitempty"`
}

AQLDiagnostic is one AQL type-check finding. Severity is "error" or "warning"; Line/Column are 1-based positions when known.

type CatalogContext added in v0.9.0

type CatalogContext struct {
	RepoRef
	CanalQueryBaseURL string `json:"canalQueryBaseUrl"`
}

CatalogContext is the repo a catalog call is about, as anfra-node needs it: the repo, named as every repo-scoped call names it (a directory, or a revision a host keeps), and the canal-query to reach its context sources through. anfra-node reads the context sources from the repo itself (.anfra/context_sources.yml), as it reads its AML, so a managed repo, which has no directory, needs no path either. catalog.ingest, catalog.search and catalog.lineage all take it, so search and lineage can ingest on their own and need no catalog.ingest first.

func CatalogContextOf added in v0.9.0

func CatalogContextOf(r repo.Repo, canalQueryBaseURL string) CatalogContext

CatalogContextOf is r's catalog context, its context sources reached through the canal-query at canalQueryBaseURL.

type CatalogIngestRequest

type CatalogIngestRequest struct {
	Context CatalogContext `json:"context"`
	Source  string         `json:"source,omitempty"`
}

CatalogIngestRequest mirrors anfra-node's catalog.ingest params: the catalog's context, and optionally the one context source to ingest. anfra-node owns source parsing and ingestion internals.

type CatalogLineageRequest added in v0.9.0

type CatalogLineageRequest struct {
	Context CatalogContext  `json:"context"`
	Targets []LineageTarget `json:"targets"`
	Limit   int             `json:"limit,omitzero"`
	Offset  int             `json:"offset,omitzero"`
}

CatalogLineageRequest mirrors anfra-node's catalog.lineage params: the catalog's context, the fields to trace, and the page of what each one needs.

type CatalogLineageResult added in v0.9.0

type CatalogLineageResult struct {
	Results []TargetDiagram `json:"results"`
}

CatalogLineageResult is catalog.lineage's answer: one per target, in the order of the request's.

type CatalogSearchRequest

type CatalogSearchRequest struct {
	Context CatalogContext `json:"context"`
	Query   string         `json:"query"`
}

CatalogSearchRequest mirrors anfra-node's catalog.search params. The Go host supplies the catalog's context; anfra-node owns search parsing, planning, and result shaping.

type CatalogSearchResult

type CatalogSearchResult map[string]any

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client talks to an anfra-node over JSON-RPC. It is address-based and owns no process, so it works against a host-spawned sidecar (Unix socket) or an external one reachable over TCP (docker-compose / k8s).

func NewClientHTTP

func NewClientHTTP(baseURL string) *Client

NewClientHTTP connects to an anfra-node reachable over TCP, e.g. a docker-compose / k8s service. No process is owned.

func NewClientUnix

func NewClientUnix(socketPath string) *Client

NewClientUnix dials a host-spawned sidecar over its Unix socket.

func (*Client) Call

func (c *Client) Call(ctx context.Context, method string, params any, out any) error

Call invokes a JSON-RPC method and unmarshals the result into out (if non-nil).

func (*Client) CatalogLineage added in v0.9.0

func (c *Client) CatalogLineage(ctx context.Context, req CatalogLineageRequest) (CatalogLineageResult, error)

CatalogLineage traces fields through the catalog of the request's context.

func (*Client) CompileToSQL

func (c *Client) CompileToSQL(ctx context.Context, req CompileToSQLRequest) (CompileToSQLResult, error)

CompileToSQL compiles an AQL query against a dataset into dialect SQL.

func (*Client) IngestCatalog

func (c *Client) IngestCatalog(ctx context.Context, req CatalogIngestRequest) error

IngestCatalog builds the local search catalog through anfra-node. The anfra-node method is a void RPC: success means the catalog was written, while failures are returned through the JSON-RPC error channel.

func (*Client) Ping

func (c *Client) Ping(ctx context.Context) (map[string]any, error)

Ping is the liveness check.

func (*Client) SearchCatalog

func (c *Client) SearchCatalog(ctx context.Context, req CatalogSearchRequest) (CatalogSearchResult, error)

SearchCatalog searches the active local catalog through anfra-node.

func (*Client) ShowAML

func (c *Client) ShowAML(ctx context.Context, req ShowRequest) (ShowResult, error)

ShowAML shows an object of the repo, read from its compiled AML.

func (*Client) ValidateAML

func (c *Client) ValidateAML(ctx context.Context, req ValidateAMLRequest) (ValidateAMLResult, error)

ValidateAML validates an AML repo (compile + validator suite) for the selected paths and returns the findings.

func (*Client) ValidateAQL

func (c *Client) ValidateAQL(ctx context.Context, req CompileToSQLRequest) (ValidateAQLResult, error)

ValidateAQL type-checks a single AQL query against a dataset (same inputs as CompileToSQL) and returns its diagnostics instead of throwing on the first error.

func (*Client) WaitReady

func (c *Client) WaitReady(ctx context.Context) error

WaitReady polls /health until the sidecar answers or the deadline passes.

type CompileDataSource

type CompileDataSource struct {
	Name   string `json:"name"`
	DBType string `json:"dbtype"`
}

CompileDataSource is the name->dialect entry the sidecar needs to compile SQL. Connection/credentials are NOT part of this — they stay host-side.

type CompileError

type CompileError struct {
	FilePath string `json:"filePath,omitempty"`
	Message  string `json:"message"`
	Row      *int   `json:"row,omitempty"`
	Col      *int   `json:"col,omitempty"`
}

CompileError is an AML file that failed to compile (interpret). Row/Col are 1-based when a location is known.

type CompileOptions

type CompileOptions struct {
	TimezoneRegion string `json:"timezoneRegion,omitempty"`
	UseV2Compiler  bool   `json:"useV2Compiler"`
}

CompileOptions are the compile options anfra sets. TimezoneRegion is an IANA zone, for relative dates and date truncation. UseV2Compiler picks the AQL v2 pipeline for compiling and type-checking; anfra-node defaults it to true, so it is always sent.

type CompileToSQLRequest

type CompileToSQLRequest struct {
	RepoRef
	DatasetFqn  string                       `json:"datasetFqn"`
	AQL         string                       `json:"aql"`
	DataSources map[string]CompileDataSource `json:"dataSources"`
	// Input is the query's Query Input, applied by rewriting the AQL before it
	// compiles. Nil: none.
	Input *QueryTransforms `json:"input,omitempty"`
	// Pagination asks for one page of rows, compiled into LIMIT/OFFSET. Nil: every
	// row. Refused for a pivot query.
	Pagination *Pagination     `json:"pagination,omitempty"`
	Options    *CompileOptions `json:"options,omitempty"`
}

CompileToSQLRequest / Result mirror the sidecar's aql.compile_to_sql method.

type CompileToSQLResult

type CompileToSQLResult struct {
	SQL string `json:"sql"`
	// AQL is the query that compiled: the AQL with its Query Input applied.
	AQL        string            `json:"aql"`
	DataSource CompileDataSource `json:"dataSource"` // the data source the SQL targets (for execution routing)
	// Columns describes each output column of an explore query; absent for other
	// query shapes.
	Columns []ExploreColumn `json:"columns,omitempty"`
}

type Diagram added in v0.9.0

type Diagram struct {
	Version int            `json:"version" enum:"2"`
	Query   string         `json:"query" doc:"the catalog search the lineage is"`
	Total   int            `json:"total" doc:"how many entities the search matched, over every page"`
	Matched []string       `json:"matched" doc:"the ids of this page's matched entities, in search order"`
	Schema  DiagramSchema  `json:"schema"`
	Layers  []DiagramLayer `json:"layers" doc:"the cards in layers, upstream first"`
	Cards   []DiagramCard  `json:"cards" doc:"every card, in layer order"`
}

Diagram is a field's lineage, laid out to be drawn: cards (the entities drawn at the top level, each with its children nested in sections), in layers from upstream to downstream, and the links between them. The core API passes it through as anfra-node formats it.

type DiagramCard added in v0.9.0

type DiagramCard struct {
	DiagramNode
	Layer    int  `json:"layer"`
	Detached bool `json:"detached,omitzero" doc:"its type is not a top-level one, but its container is not in the lineage"`
}

DiagramCard is a node drawn at the top level, in its layer.

type DiagramLayer added in v0.9.0

type DiagramLayer struct {
	Index   int      `json:"index"`
	Types   []string `json:"types" doc:"the entity types of its cards"`
	CardIDs []string `json:"card_ids"`
}

DiagramLayer is one layer of cards.

type DiagramLink struct {
	Type  string         `json:"type" doc:"the edge type, such as refers or includes"`
	To    string         `json:"to" doc:"the linked entity's id"`
	Props map[string]any `json:"props,omitempty" doc:"the edge's properties, when it has some"`
}

DiagramLink is an edge from a node.

type DiagramNode added in v0.9.0

type DiagramNode struct {
	ID       string           `json:"id" doc:"the catalog's id of the entity"`
	Type     string           `json:"type" doc:"the catalog's entity type, such as aml.model or aml.metric"`
	Label    string           `json:"label"`
	Ref      string           `json:"ref,omitempty" doc:"the entity this node is a copy of"`
	Tags     []string         `json:"tags,omitempty"`
	Props    map[string]any   `json:"props,omitempty" doc:"the entity's properties"`
	Links    []DiagramLink    `json:"links,omitempty" doc:"the node's links; a link is on its edge's from entity"`
	Sections []DiagramSection `json:"sections,omitempty" doc:"the node's children, grouped by type"`
}

DiagramNode is an entity of the lineage, with its children.

type DiagramSchema added in v0.9.0

type DiagramSchema struct {
	Edges map[string]string            `json:"edges" doc:"edge type → what it means"`
	Types map[string]DiagramSchemaType `json:"types" doc:"entity type → how it connects"`
}

DiagramSchema is which entity types the diagram has, and how they connect, read from the diagram itself.

type DiagramSchemaType added in v0.9.0

type DiagramSchemaType struct {
	TopLevel bool                `json:"top_level" doc:"no entity of the type is a child in a section"`
	Edges    map[string][]string `json:"edges" doc:"edge type → the entity types it links to"`
}

DiagramSchemaType is how the entities of one type connect.

type DiagramSection added in v0.9.0

type DiagramSection struct {
	Type  string        `json:"type"`
	Nodes []DiagramNode `json:"nodes"`
}

DiagramSection is a node's children of one type.

type ExploreColumn

type ExploreColumn struct {
	Name        string `json:"name" doc:"the column's key, as in fields"`
	FieldName   string `json:"fieldName" doc:"the field it draws; for an adhoc column, its name"`
	ModelID     string `` /* 126-byte string literal not displayed */
	Label       string `json:"label"`
	Adhoc       bool   `json:"adhoc" doc:"a query-local expression, not a field the dataset defines"`
	IsMeasure   bool   `json:"isMeasure"`
	Aggregation string `json:"aggregation,omitempty" doc:"an aggregated field's aggregation, such as sum or count distinct"`
	// Type is the AMQL type the type checker gives the column's expression
	// (number, text, date, ...), not the type its field declares: a count of a
	// text field is a number. Absent from an anfra-node before 0.4.0.
	Type string `json:"type,omitempty"`
}

ExploreColumn is one column of a query's answer: the key its values come back under, and the dataset field it draws, in the names AQL uses in the dataset. anfra-node describes an explore query's; the core API passes them through in these names, amql's and the Anfra SDK's.

type LineageTarget added in v0.9.0

type LineageTarget struct {
	Parent string `json:"parent" minLength:"1" doc:"the fqn of what holds the field: a model, or a dataset"`
	Child  string `json:"child" minLength:"1" doc:"the field's name in its parent: a dimension, a measure or a metric"`
}

LineageTarget is one field to trace: its parent, and its name in it.

type Pagination

type Pagination struct {
	Page     int `json:"page"`
	PageSize int `json:"pageSize"`
}

Pagination is one 1-based page of rows.

type QueryCondition

type QueryCondition struct {
	Expr string `json:"expr" doc:"an AQL condition"`
}

type QueryDateDrill

type QueryDateDrill struct {
	Field string `json:"field" doc:"model.field: a date field"`
	Grain string `json:"grain" doc:"the grain, such as month"`
}

type QueryFilter

type QueryFilter struct {
	Field       string `json:"field" doc:"model.field for a dataset field, or the name of a dataset metric"`
	Operator    string `json:"operator" doc:"the operator, such as is, contains, between, last"`
	Values      []any  `` /* 297-byte string literal not displayed */
	Modifier    string `json:"modifier,omitempty" doc:"the date unit of a relative operator, such as day"`
	Aggregation string `json:"aggregation,omitempty" doc:"the condition applies to this aggregate of field, such as sum"`
}

type QuerySort

type QuerySort struct {
	Field     string `json:"field" doc:"a result column's name"`
	Direction string `json:"direction" enum:"asc,desc" doc:"the direction"`
}

type QueryTransforms

type QueryTransforms struct {
	Filters    []QueryFilter    `json:"filters,omitempty" doc:"conditions on fields, ANDed with the query's own filters"`
	Conditions []QueryCondition `json:"conditions,omitempty" doc:"AQL conditions ANDed with the query's own filters"`
	Sorts      []QuerySort      `json:"sorts,omitempty" doc:"sorts by result column, replacing the query's own"`
	DateDrills []QueryDateDrill `json:"dateDrills,omitempty" doc:"date fields redrawn at another grain, their columns' names kept"`
}

QueryTransforms is a query's Query Input: the structured transforms it carries on one run (a Data App's control filters, cross-filter conditions, sorts and date drills), applied by anfra-node rewriting its AQL before it compiles. The core API passes it through in these names, amql's and the Anfra SDK's: anfra-node checks the values (an operator, a grain).

type RPCError

type RPCError struct {
	Method  string
	Code    int
	Message string
	Data    jsontext.Value
}

RPCError is an error anfra-node answered a call with. Data is the error's JSON-RPC data, when it carries one: for invalid input, the {path} of what is wrong, e.g. a Query Input entry.

func (*RPCError) Error

func (e *RPCError) Error() string

func (*RPCError) Path

func (e *RPCError) Path() (path string, ok bool)

Path is where invalid input is wrong, from the error's data, e.g. a Query Input entry's "filters[2].operator"; "" for the input as a whole. ok is false when the error does not say: not invalid input, or no path in its data.

type RepoRef added in v0.9.0

type RepoRef struct {
	RepoID   string `json:"repoId"`
	RepoPath string `json:"repoPath,omitempty"`
	Revision string `json:"revision,omitempty"`
}

RepoRef names a request's repo to anfra-node: its compile-cache identity, and exactly one of where its files are, a directory (RepoPath) or a revision a host keeps (Revision, "<repo>@<sha>"). RefTo builds it from a repo; every repo-scoped call checks it before it leaves the process, so a caller that sends neither or both fails here, with a stack that names it, rather than with an error off the wire.

anfra-node holds no repo identity of its own (one process can serve many repos), so RepoID is required. It is the repo's stable identity, never the revision: anfra-node's compile cache is an incremental seed keyed by it, made exactly the requested files on every request, so a stable key keeps commits incremental and the content keeps it correct.

func RefTo added in v0.9.0

func RefTo(r repo.Repo) RepoRef

RefTo is r as anfra-node names it.

type ReportTrace

type ReportTrace struct {
	Kind string          `json:"kind"`
	Name string          `json:"name"`
	Loc  *ReportTraceLoc `json:"loc,omitempty"`
}

type ReportTraceLoc

type ReportTraceLoc struct {
	FilePath string `json:"filePath"`
	Line     *int   `json:"line,omitempty"`
	Column   *int   `json:"column,omitempty"`
}

ReportTraceLoc / ReportTrace locate a validator finding within the AML source.

type ShowDataset

type ShowDataset struct {
	Kind        string      `json:"kind" enum:"dataset"`
	Fqn         string      `json:"fqn" doc:"the dataset's fully qualified name, as AQL and the catalog name it"`
	Name        string      `json:"name"`
	Label       string      `json:"label,omitempty"`
	Description string      `json:"description,omitempty"`
	Models      []ShowModel `json:"models,omitempty" doc:"the models the dataset uses; absent in outline"`
	Metrics     []ShowField `json:"metrics,omitempty" doc:"the dataset's metrics; absent in outline"`
}

ShowDataset is a dataset: in outline, who it is; in full, also its models and metrics. Its interface, not its implementation: what a caller needs to write a query against it or read a query's result, nothing of how it is built.

type ShowField

type ShowField struct {
	Fqn              string `json:"fqn" doc:"the catalog's identity: <model fqn>.<name>, or <dataset fqn>.<name> for a dataset-level field or a metric"`
	Name             string `json:"name" doc:"AQL writes <model fqn>.<name>; a metric, its name alone"`
	Label            string `json:"label,omitempty"`
	Description      string `json:"description,omitempty"`
	Role             string `json:"role" enum:"dimension,measure,param,metric"`
	Type             string `json:"type" doc:"the compiled data type, such as number, text, date, datetime, truefalse"`
	Aggregation      string `json:"aggregation,omitempty" doc:"a measure's aggregation, such as sum or count distinct; custom for a custom one"`
	Hidden           bool   `json:"hidden"`
	Format           any    `json:"format,omitzero" doc:"how a value displays, as the AML declares it"`
	Definition       string `json:"definition,omitempty" doc:"the definition of a field or metric written in AQL"`
	DefinedInDataset bool   `json:"definedInDataset,omitzero" doc:"a dimension the dataset declares on one of its models"`
}

ShowField is a field or a metric.

type ShowModel

type ShowModel struct {
	Fqn         string      `json:"fqn" doc:"the model's fully qualified name: what AQL writes before a field"`
	Name        string      `json:"name"`
	Label       string      `json:"label,omitempty"`
	Description string      `json:"description,omitempty"`
	Fields      []ShowField `json:"fields"`
}

ShowModel is a model as a dataset uses it, with the fields AQL can reference on it.

type ShowObject

type ShowObject struct {
	Repo    *ShowRepo
	Dataset *ShowDataset
}

ShowObject is one object of the repo, tagged by its kind: the repo, or a dataset. Exactly one of its fields is set. More kinds come (a model, a field, a metric), so a client must accept a kind it does not know.

func (ShowObject) MarshalJSONTo

func (o ShowObject) MarshalJSONTo(enc *jsontext.Encoder) error

MarshalJSONTo encodes the object set, within the encoder it is in, so the encoder's options reach inside it (httpkit.JSONOptions).

func (ShowObject) Schema

func (ShowObject) Schema(r huma.Registry) *huma.Schema

Schema publishes the union, discriminated by kind.

func (*ShowObject) UnmarshalJSON

func (o *ShowObject) UnmarshalJSON(b []byte) error

type ShowRepo

type ShowRepo struct {
	Kind     string        `json:"kind" enum:"repo"`
	Datasets []ShowDataset `` /* 151-byte string literal not displayed */
}

ShowRepo is the repo: the datasets it offers, each in full, or in outline when it cannot be shown in full (a diagnostic says why).

type ShowRequest

type ShowRequest struct {
	RepoRef
	DataSources map[string]CompileDataSource `json:"dataSources"`
	Type        string                       `json:"type,omitempty"`
	Fqn         string                       `json:"fqn,omitempty"`
}

ShowRequest mirrors the sidecar's aml.show params: what to show, as the semantic catalog identifies it, an entity type and its fqn; both empty for the repo. DataSources are what compiling a dataset resolves its dialect from.

type ShowResult

type ShowResult struct {
	Object      ShowObject     `json:"object"`
	Diagnostics []CompileError `` /* 214-byte string literal not displayed */
}

ShowResult is aml.show's answer, which the core API passes through: the object shown, and what is wrong in the repo for it.

type Sidecar

type Sidecar struct {
	// contains filtered or unexported fields
}

Sidecar supervises a host-spawned anfra-node. It owns the process lifecycle and exposes a Client pointed at it; callers do their RPC through Client() (which also works against external sidecars, so consumers don't depend on host-spawning).

func New

func New(cfg sidecar.Config) *Sidecar

New is anfra-node's supervisor, not yet started.

func (*Sidecar) Client

func (a *Sidecar) Client() *Client

Client returns the RPC client for the spawned sidecar.

func (*Sidecar) Close

func (a *Sidecar) Close()

Close stops the sidecar and removes its socket file. A no-op for an external sidecar, which this process never owned.

func (*Sidecar) Start

func (a *Sidecar) Start(ctx context.Context) error

Start connects to the sidecar and waits until it reports healthy: to an external one when Config.NodeURL is set, otherwise to one it spawns.

type TargetDiagram added in v0.9.0

type TargetDiagram struct {
	Target  LineageTarget `json:"target"`
	Query   string        `json:"query"`
	Diagram *Diagram      `json:"diagram,omitempty"`
	Error   string        `json:"error,omitempty"`
}

TargetDiagram is one target's lineage, or why it has none.

type ValidateAMLRequest

type ValidateAMLRequest struct {
	RepoRef
	Paths []string `json:"paths,omitempty"`
}

ValidateAMLRequest mirrors the sidecar's aml.validate params. Paths are file/dir/glob selectors relative to the repo; empty validates the whole repo. No data sources are involved — this type-checks AML.

type ValidateAMLResult

type ValidateAMLResult struct {
	CompileErrors []CompileError     `json:"compileErrors"`
	Reports       []ValidationReport `json:"reports"`
}

ValidateAMLResult mirrors the sidecar's aml.validate result: files that failed to compile, plus the validator suite's findings.

type ValidateAQLResult

type ValidateAQLResult struct {
	Diagnostics []AQLDiagnostic `json:"diagnostics"`
}

type ValidationReport

type ValidationReport struct {
	Validator string        `json:"validator"`
	FilePath  string        `json:"filePath"`
	Type      string        `json:"type"`
	Fqn       string        `json:"fqn"`
	Severity  string        `json:"severity"`
	Message   string        `json:"message"`
	Code      any           `json:"code,omitzero"`
	Trace     []ReportTrace `json:"trace,omitempty"`
}

ValidationReport is one finding from the validator suite. Severity is "error", "warning", or "info".

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL