Documentation
¶
Overview ¶
elpspath — positional-arg path operations
The elpspath API addresses locations inside nested data structures with positional path steps: each step is an ordinary ELPS value — no mini-language to learn and no runtime string parsing. (parse-path is the deliberate exception: it converts a string path that arrives as data into those steps, once, so the operations themselves stay parse-free.) (A legacy jq-string path DSL is still spoken by builtins downstream in luthersystems/substrate, over the parser in selector.go; see below.)
Builtins ¶
All functions take a data structure as the first argument, followed by zero or more path steps. Functions ending in "!" mutate in place; ? reads without copying. The copying writes rebuild maps, lists and vectors; tagged and quote wrappers are rebuilt recursively. Other values, including functions, are opaque leaves shared by reference. Further steps into a leaf fail with its type and location. Out-of-range integer writes leave the document unchanged. In-place list element edits remain unsupported. Containers, including those inside tagged and quote wrappers, must be acyclic, and arrays must be one-dimensional. Opaque internals are not walked.
(elpspath:? val &rest steps) ; get (elpspath:?set! val &rest steps-and-value) ; set (mutating) (elpspath:?set val &rest steps-and-value) ; set (copy) (elpspath:?del! val &rest steps) ; delete (mutating) (elpspath:?del val &rest steps) ; delete (copy) (elpspath:?nil! val &rest steps) ; nil (mutating) (elpspath:?nil val &rest steps) ; nil (copy) (elpspath:parse-path selector) ; string path -> steps
For ?set! and ?set the last variadic argument is always the new value; everything before it is treated as path steps.
Path step types ¶
Type Meaning Example jq analogue ───────────── ─────── ─────── ─────────── string map key "foo" .foo int array index 0, -1 [0], [-1] symbol '* iterate all '* [] list (range a b) array slice '(range 1 3) [1:3]
Examples ¶
Basic access:
(elpspath:? obj "name") ; get a key (elpspath:? obj "address" "city") ; nested keys (elpspath:? obj "items" 0) ; array index (elpspath:? obj "items" -1) ; last element
Iterators:
(elpspath:? users '* "name") ; all user names (elpspath:? obj '* "tags" 0) ; first tag from each item (elpspath:? org "teams" '* "members" '* "name") ; double iterate (flattens)
Ranges:
(elpspath:? scores '(range 1 3)) ; elements [1,3)
Mutations:
(elpspath:?set! user "profile" "bio" "hello") ; mutating set (set new (elpspath:?set user "profile" "bio" "hello")) ; copy set (elpspath:?set! data "users" '* "active" true) ; set via iterator
Deletes:
(elpspath:?del! user "tmp-field") ; mutating delete (set clean (elpspath:?del obj "metadata" "internal-id")) ; copy delete (elpspath:?del! records '* "cache") ; delete via iterator
Nils:
(elpspath:?nil! record "deprecated") ; null a field in place (set redacted (elpspath:?nil patient "ssn")) ; null with copy
Dynamic paths (steps are plain values, so no string splicing is needed):
(defun get-field (obj field) (elpspath:? obj field)) (defun get-nth-result (resp n) (elpspath:? resp "results" n "value"))
Legacy jq-string DSL ¶
The deprecated legacy BUILTINS (get-path, set-path!, etc.), which encode paths as jq-style strings, did not move here: they remain downstream in luthersystems/substrate, whose loader composes them into this same lisp-visible elpspath package. The positional-arg API is ~3-4x faster because it skips regex-based string parsing — path steps are dispatched by type switch.
The PARSER those builtins are built on does live here, as the Go-level ParseSelector in selector.go (issue #564): it is pure translation of a selector string into the exported Path constructors, so leaving it downstream meant one repository owning the syntax of a path language whose semantics live in another. The one lisp-visible way to reach it is parse-path, which converts a selector string into path steps rather than operating on a document with it.
Index ¶
- Constants
- func BuiltinParsePath(env *lisp.LEnv, args *lisp.LVal) *lisp.LVal
- func BuiltinQueryDelete(env *lisp.LEnv, args *lisp.LVal) *lisp.LVal
- func BuiltinQueryDeleteMutate(env *lisp.LEnv, args *lisp.LVal) *lisp.LVal
- func BuiltinQueryGet(env *lisp.LEnv, args *lisp.LVal) *lisp.LVal
- func BuiltinQueryNil(env *lisp.LEnv, args *lisp.LVal) *lisp.LVal
- func BuiltinQueryNilMutate(env *lisp.LEnv, args *lisp.LVal) *lisp.LVal
- func BuiltinQuerySet(env *lisp.LEnv, args *lisp.LVal) *lisp.LVal
- func BuiltinQuerySetMutate(env *lisp.LEnv, args *lisp.LVal) *lisp.LVal
- func LoadPackage(env *lisp.LEnv) *lisp.LVal
- func SelectorSteps(selector string) ([]*lisp.LVal, error)
- type Path
Constants ¶
const DefaultPackageName = "elpspath"
DefaultPackageName is the package name used by LoadPackage.
Variables ¶
This section is empty.
Functions ¶
func BuiltinParsePath ¶ added in v1.60.0
BuiltinParsePath implements (elpspath:parse-path selector).
It returns a LIST and not a vector because apply takes a list, and splicing the steps into a ? call is the entire point of the function.
IT IS STRICTER THAN ParseSelector, in exactly one way, and this is the layer that difference belongs at. A bracket-led selector is cut at its first newline and the tail dropped in silence (see ParseSelector's second wart), so `.[0]\n.password` converts to the single step 0 -- a PREFIX of the path that was asked for, with no error anywhere. Through the pattern this function exists for,
(apply ?set (concat 'list (list obj) (parse-path sel) (list v)))
that replaces the whole of element 0 instead of its "password" field. It is the same failure mode the empty step list has -- a shorter path is a LIVE path, not a dead one -- which is why a malformed selector already raises here rather than returning no steps.
So a selector whose tail would be discarded is REFUSED. This is new surface with no downstream counterpart, so nothing depends on the wart; the two Go functions keep it, agree with each other about it, and stay one grammar that FuzzParseSelector can hold to a single acceptance rule.
The check costs one IndexByte over the selector on the happy path, and only for a selector that leads with a bracket at all.
func BuiltinQueryDelete ¶
BuiltinQueryDelete implements (elpspath:?del val &rest steps).
func BuiltinQueryDeleteMutate ¶
BuiltinQueryDeleteMutate implements (elpspath:?del! val &rest steps).
func BuiltinQueryGet ¶
BuiltinQueryGet implements (elpspath:? val &rest steps).
func BuiltinQueryNil ¶
BuiltinQueryNil implements (elpspath:?nil val &rest steps).
func BuiltinQueryNilMutate ¶
BuiltinQueryNilMutate implements (elpspath:?nil! val &rest steps).
func BuiltinQuerySet ¶
BuiltinQuerySet implements (elpspath:?set val &rest steps-and-value). The last vararg is the new value; all preceding varargs are path steps.
func BuiltinQuerySetMutate ¶
BuiltinQuerySetMutate implements (elpspath:?set! val &rest steps-and-value). The last vararg is the new value; all preceding varargs are path steps.
func LoadPackage ¶
LoadPackage adds the elpspath package to env.
DefinePackage is get-or-create, and AddBuiltins refuses only exact symbol-name collisions, so an embedder may compose additional builtins into the same lisp-visible "elpspath" package with its own loader (as substrate does with its legacy jq-string operations).
func SelectorSteps ¶ added in v1.60.0
SelectorSteps translates a jq-style selector string into the positional path steps the ? family takes, as ordinary lisp values.
SelectorSteps(`.users[0]["full name"]`) => "users", 0, "full name"
SelectorSteps(".items[].id") => "items", '*, "id"
SelectorSteps(".items[1:3]") => "items", '(range 1 3)
SelectorSteps(".items[1:]") => "items", '(range 1)
SelectorSteps(".") => no steps
It exists so a path that ARRIVES AS A STRING -- from inside a document, a client request, or a persisted envelope -- can be converted once and then applied many times through the positional API, instead of being re-parsed on every operation. The identity selector yielding no steps is what makes that uniform: applying an empty step list is the identity, exactly as (? obj) is.
It shares selectorPaths with ParseSelector, so the two agree on the grammar by construction rather than by test -- including the newline wart described there, which this function therefore also has: a bracket-led selector is CUT at its first newline and the tail is dropped in silence. The agreement is asserted (TestSelectorGrammarPathologies, and FuzzParseSelector on every input it accepts), so the strictness that wart needs lives one layer up, in BuiltinParsePath, rather than being bolted on to one of the two functions. A Go caller wanting it should reject selectors containing a newline before calling.
What the steps mean is checked anyway: TestSelectorStepsMatchParseSelector applies both routes to documents and requires the same answer.
The open-ended range is why this can be lossless. Before it had a step spelling, "[1:]" had no positional form, so a conversion would have silently dropped or mis-rendered exactly the selectors that persist.
Types ¶
type Path ¶
type Path interface {
// Get evaluates a get path operation on an elps LVal.
Get(*lisp.LVal) (*lisp.LVal, error)
// SetMutate evaluates a mutating set path operation on an elps LVal.
SetMutate(*lisp.LVal, *lisp.LVal) (*lisp.LVal, error)
// Set evaluates a set path operation on an elps LVal, and returns a
// newly constructed LVal.
Set(*lisp.LVal, *lisp.LVal) (*lisp.LVal, error)
// DeleteMutate evaluates a mutating delete path operation on an elps LVal.
DeleteMutate(*lisp.LVal) (*lisp.LVal, error)
// Delete evaluates a delete path operation on an elps LVal, and returns
// a newly constructed LVal.
Delete(*lisp.LVal) (*lisp.LVal, error)
// Nil sets elements at the end of a path to be null. If the path references
// multiple elements (i.e. a range) then all of the elements in that range
// are set to null.
Nil(*lisp.LVal) (*lisp.LVal, error)
// NilMutate mutates elements at the end of a path to be null. If the path
// references multiple elements (i.e. a range) then all of the elements in
// that range are set to null.
NilMutate(*lisp.LVal) (*lisp.LVal, error)
// String returns a string representation of the path.
String() string
}
Path represents an operation on a path.
A Path's methods have no environment, so they neither charge steps nor poll a context for the work an iterator does (budget.go, issue #722): a '* over a document that shares a subtree along many paths can do exponential work inside one call. An embedder that runs paths a program supplied should go through the Builtin* functions with the program's environment -- SelectorSteps converts a selector string to their arguments -- rather than call these methods directly.
func ArgsToPath ¶
ArgsToPath converts positional ELPS args into a Path. Each arg is dispatched by type:
- LString → Dot(key)
- LInt → Index(i)
- LSymbol "*" → Iter()
- LSExpr (range from to) → Range(from, to, false)
- LSExpr (range from) → Range(from, 0, true), the end resolved against the input length at evaluation time (issue #563)
More than maxPathSteps iterator steps are refused before compiling nested iterators, which would otherwise recurse over program-built values.
A step copies at MaxValueDepth unless the operation is run through setPath, deletePath or nilPath, which carry the caller's own limit; the query builtins below take that from env.Runtime.ValueDepthLimit().
func ParseSelector ¶ added in v1.60.0
ParseSelector translates a jq-style selector string into a Path.
It is named ParseSelector rather than Parse because this repository has a lisp reader, and an exported Parse in a lisp package reads as that one.
p, err := libelpspath.ParseSelector(`.users[0]["full name"]`) v, err := p.Get(doc)
The grammar, and how each form lands on a Path:
. => Root(Chain()) the whole document
.foo => Dot("foo") bare keys only: [A-Za-z_][A-Za-z_0-9]*
["$foo"] => Dot("$foo") any key, as a Go-quoted string literal
[0] [-1] => Index(n) negative counts from the end
[1:3] => Range(1, 3, false)
[:3] => Range(0, 3, false) an absent "from" is 0, not implicit
[1:] [:] => Range(n, 0, true) the end is resolved against the document
[] => Iter() every element
Whitespace is permitted around and inside brackets, and a selector must start with "." -- ".[0]" is accepted, a bare "[0]" is not. It holds no state at all, so this may be called from any goroutine.
The Path it returns is an ordinary one: nothing distinguishes a parsed path from one assembled by hand or by ArgsToPath, and the same seven operations apply. A caller that is about to hand a document to one of them should run okSimpleType over the document first, which is what the builtins do -- see that function for why it is not optional.
ONE WART. The jq optional-selector suffix "?" is ACCEPTED AND DISCARDED. In jq, ".a?" suppresses the error a non-object .a would raise; here every reader consumes a trailing "?" and none of them records it, so ".a?" is exactly ".a" and the error is raised. Nothing in the engine implements error-suppressing steps, so honouring the suffix would be a feature, not a fix.
A SECOND WART, and this one can lose you data. A selector that leads with a bracket is cut at its first NEWLINE and the rest is discarded in silence:
ParseSelector(".[0]\n.password") => .[0] -- the tail is dropped
ParseSelector(".items[0]\n.id") => .["items"][0]["id"]
The two differ because only the bracket-led form goes through selectorBody's leading-bracket rule; see that function for why the behaviour is kept. It matters because the truncated path is a PREFIX of what was asked for, so a write through it lands on the wrong node rather than failing: `.[0]\n.password` sets the whole element, not its field.
This function keeps the wart for parity with the v1 jq-string builtins downstream. Everything else does not: the parse-path builtin REFUSES a selector whose tail would be discarded, and a Go caller converting selector text that came from outside the program should do the same -- rejecting any selector containing a newline is sufficient and is what parse-path amounts to.
Two properties worth stating because they are easy to break:
The identity selector "." returns Root(Chain()), which prints "." and matches what ArgsToPath builds for an empty step list. Chain() alone behaves identically -- rootPath proxies all seven operations and adds only a leading "." to String() -- but prints the empty string, which this parser cannot read back. TestParseSelectorRootSpelling pins the agreement.
A quoted key ends at the first UNESCAPED quote, which is why scanStringLiteral skips two bytes after a backslash. Ending it at the LAST quote in the selector instead -- which is what the regexp this scanner replaced did before issue #566, its body reading as "any character at all" -- limits a selector to ONE bracketed key and makes String()'s own output unreadable, since it brackets every map key. TestParseSelectorTwoQuotedKeys covers the grammar and the round-trip test carries multi-key selectors.
Source Files