Help us shape the future of Go! Take the 2026 Go Developer Survey

aql

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 19, 2026 License: MIT Imports: 14 Imported by: 0

Documentation

Overview

Package aql parses Artifactory Query Language and runs it (decision 0012). It covers the items domain: the shape the JFrog CLI generates for every file specification (research 09) and what users write by hand. That is $and and $or, field comparisons, $match and $nmatch globs, @property and property.key/property.value criteria, $msp, the stat domain, and the include, sort, offset, limit, transitive and distinct suffixes with Artifactory's output rules (research 22 §1). The builds and build.promotions domains are served too, and every domain reaches the others through relation paths such as artifact.module.build.name (see Joined), which is how the JFrog CLI finds a build's files; an include may name those domains' fields too, which Nest fills in (nest.go). It also parses the two actions, items.delete and properties.update (research 22 §1.5), whose criteria select items the same way; the server carries them out. Every repository's root is an items row (Roots).

A parsed Query is run two ways that must agree row for row: Match and Compare are the reference semantics (the in-memory store uses them), and Compile turns the query into SQL for SQLite or PostgreSQL. The store contract suite runs the same queries on every backend and expects the same ordered rows. Before either, Resolve rewrites repo criteria against the configured repositories, so remote and virtual keys match what they stand for.

Index

Constants

View Source
const FieldVirtualRepos = "virtual_repos"

FieldVirtualRepos lists the virtual repositories that contain an item's repository. It can be included but not searched on, and it needs repo, path and name in the output (research 22 §1.2).

View Source
const MaxLength = 6000

MaxLength is the longest query accepted, in characters: Artifactory's artifactory.aql.search.query.size.limit default. ErrTooLong's text names it, as Artifactory's message names the configured limit.

Variables

View Source
var (
	// ErrSyntax reports a query that does not parse.
	ErrSyntax = errors.New("aql: syntax error")
	// ErrUnsupported reports valid AQL that artifactweave does not serve:
	// another domain, a secondary-domain field, or an operator outside the
	// supported set.
	ErrUnsupported = errors.New("aql: unsupported")
	// ErrTooLong reports a query above MaxLength characters. Its text is
	// Artifactory's message (aql-performance).
	ErrTooLong = errors.New("AQL query is too long; please reduce the query length to less than " +
		strconv.Itoa(MaxLength) + " chars")
	// ErrRestricted reports a query a non-admin caller may not run.
	ErrRestricted = errors.New("aql: restricted")
	// ErrEmpty reports a request with no query at all.
	ErrEmpty = errors.New("aql: empty query")
	// ErrInvalid reports a query that parses but breaks an output rule: a
	// sort on a field the output leaves out, or virtual_repos without repo,
	// path and name. It also reports a suffix the query's action does not
	// take, and a properties.update without .keys() or .newValue().
	ErrInvalid = errors.New("aql: invalid query")
)

Functions

This section is empty.

Types

type Action

type Action int

Action is what a query does with the items its criteria select.

const (
	Find Action = iota
	DeleteItems
	UpdateProperties
)

Actions. Find returns the items; DeleteItems is items.delete, which removes them, and UpdateProperties is properties.update, which replaces the values of the query's Keys on them.

func (Action) String

func (a Action) String() string

String is the action as a query spells it.

type And

type And []Node

And holds when every child holds; an empty And always holds.

type BuildHit

type BuildHit struct {
	Build     types.Build
	Promotion *types.BuildPromotion
}

BuildHit is one row a builds or build.promotions query found: the run, without its modules, properties or document, and for build.promotions the promotion.

func SelectBuilds

func SelectBuilds(q *Query, o Options, c *Corpus) []BuildHit

SelectBuilds runs a builds or build.promotions query over a corpus, as the SQL compiler's output does: filter, sort (by ID when the query does not sort, and by ID within equal sort keys), then offset and limit. A promotion of a run the corpus lacks is skipped, as the SQL join skips it.

type Compiled

type Compiled struct {
	Where string
	Tail  string
	Args  []any
}

Compiled is a query as SQL over the store's tables (internal/sqlstore migrations): the caller selects its columns FROM the domain's table, items, builds or build_promotions, unaliased, then appends " WHERE " + Where and Tail, binding Args in order. Placeholders are "?"; a PostgreSQL caller rebinds them.

type Corpus

type Corpus struct {
	Items      []Hit
	Builds     []types.Build
	Promotions []types.BuildPromotion
	// contains filtered or unexported fields
}

Corpus is everything the reference evaluator may find or join: the items with their properties and statistics, the build runs with their properties and modules, and the promotions. It is read, never changed; SelectItems and SelectBuilds index it on first use, so it must not change between calls.

func (*Corpus) ItemsWithSHA1

func (c *Corpus) ItemsWithSHA1(_ context.Context, sha1 string) ([]Hit, error)

ItemsWithSHA1 implements Source over the corpus.

func (*Corpus) Run

func (c *Corpus) Run(_ context.Context, id int64) (types.Build, error)

Run implements Source over the corpus.

func (*Corpus) RunPromotions

func (c *Corpus) RunPromotions(_ context.Context, id int64) ([]types.BuildPromotion, error)

RunPromotions implements Source over the corpus.

func (*Corpus) RunsWithFile

func (c *Corpus) RunsWithFile(
	_ context.Context,
	dependency bool,
	sha1 string,
) ([]types.Build, error)

RunsWithFile implements Source over the corpus.

type Dialect

type Dialect int

Dialect selects the SQL the compiler writes.

const (
	SQLite Dialect = iota
	PostgreSQL
)

Dialects.

type Domain

type Domain int

Domain is a query's primary domain (research 22 §1.1).

const (
	DomainItems Domain = iota
	DomainBuilds
	DomainPromotions
)

Primary domains. The zero value is items, so a Query built by hand searches items.

func (Domain) String

func (d Domain) String() string

type Field

type Field string

Field is an item field name as AQL spells it.

const (
	FieldRepo         Field = "repo"
	FieldPath         Field = "path"
	FieldName         Field = "name"
	FieldType         Field = "type"
	FieldSize         Field = "size"
	FieldDepth        Field = "depth"
	FieldCreated      Field = "created"
	FieldCreatedBy    Field = "created_by"
	FieldModified     Field = "modified"
	FieldModifiedBy   Field = "modified_by"
	FieldUpdated      Field = "updated"
	FieldOriginalMD5  Field = "original_md5"
	FieldActualMD5    Field = "actual_md5"
	FieldOriginalSHA1 Field = "original_sha1"
	FieldActualSHA1   Field = "actual_sha1"
	FieldSHA256       Field = "sha256"
)

Item fields. Checksums are the actual ones unless named original.

type FieldCriterion

type FieldCriterion struct {
	Field  Field
	Op     Op
	Text   string
	Number int64
	Time   time.Time
	Span   Span
}

FieldCriterion compares an item field. Exactly one of Text, Number and Time is meaningful, by the field's kind; Before and Last carry their span in Span instead.

type Hit

type Hit struct {
	Item         types.Item
	Properties   types.Properties
	Stats        *types.ItemStats
	VirtualRepos []string
}

Hit is one item a query found, with its properties and statistics when the query includes them. Stats is nil for an item that has no statistics record. VirtualRepos is filled by the caller, which knows the repository configurations, when the query includes virtual_repos.

func RootHit

func RootHit(r RepoRef) Hit

RootHit is the row of a repository root that has no record: a folder with path and name ".", ID 0, and no times, properties or statistics.

func Select

func Select(q *Query, o Options, candidates []Hit) []Hit

Select runs q over candidates the way the SQL compiler's output does: filter, sort (by ID when the query does not sort, and by ID within equal sort keys), then offset and limit. Properties and statistics are dropped unless the query includes them. It is the reference the SQL backends are tested against; candidates must carry their properties and statistics. Relation paths reach no builds here; SelectItems joins them.

func SelectItems

func SelectItems(q *Query, o Options, c *Corpus) []Hit

SelectItems is Select over a corpus: the candidates are c.Items and the roots of q that no item of c records, and the relation paths in q's criteria join them to c's builds and promotions.

type Joined

type Joined struct {
	Criteria And
}

Joined holds the criteria of a query's root that name relation paths. A row matches when one combination of the rows its paths reach satisfies them all at once, as when Artifactory's SQL joins each related table once per query: {"artifact.module.build.name":"a","artifact.module.build.number":"1"} needs one build named a with number 1, not a build named a and another numbered 1. A path that reaches no row takes the null row an outer join would, on which no criterion holds, so an alternative in an $or can still match. Parse gathers every root conjunct that names a path into one Joined; criteria on the found row alone stay outside it.

type MSP

type MSP []*PropertyCriterion

MSP is $msp, match on a single property: it holds when one property of the item satisfies every criterion at once. Inside it, a negative operator tests that one property ($ne is a different value, $nmatch a value that does not match), not the absence of every property that would match.

type Nesting

type Nesting struct {
	// Now resolves $before and $last in the relation-path criteria; zero
	// means time.Now.
	Now time.Time
	// ItemVisible and RunVisible, when set, report whether the caller may
	// see a related item, or a run with its modules, files and promotions.
	// A row they refuse is left out with every row below it.
	ItemVisible func(types.Item) bool
	RunVisible  func(types.Build) bool
}

Nesting is how Nest renders related rows for one caller.

type Node

type Node interface {
	// contains filtered or unexported methods
}

Node is a criteria tree node: And, Or, *FieldCriterion, *PropertyCriterion, *StatCriterion, MSP, the RepoSet that Resolve puts in place of repo criteria, or a criterion of the builds domains and relation paths: *PathCriterion, *PathProperty, *PathMSP and the *Joined that gathers those naming paths.

type Op

type Op string

Op is a comparison operator.

const (
	OpEq     Op = "$eq"
	OpNe     Op = "$ne"
	OpGt     Op = "$gt"
	OpGte    Op = "$gte"
	OpLt     Op = "$lt"
	OpLte    Op = "$lte"
	OpMatch  Op = "$match"
	OpNMatch Op = "$nmatch"
	OpBefore Op = "$before"
	OpLast   Op = "$last"
)

Operators. Before and Last are relative times, resolved against the caller's clock when the query runs.

type Options

type Options struct {
	// Now resolves $before and $last; zero means time.Now.
	Now time.Time
	// Max is the most rows any query returns, its own limit included: the
	// search hard limit (decision 0012). Zero means no cap.
	Max int
}

Options are the caller's settings for running a query.

type Or

type Or []Node

Or holds when any child holds; an empty Or never holds.

type PathCriterion

type PathCriterion struct {
	Path   string
	Field  string
	Op     Op
	Text   string
	Number int64
	Time   time.Time
	Span   Span
	// contains filtered or unexported fields
}

PathCriterion compares a field of the row a relation path reaches from the row a query finds (research 22 §1.1): in an items query, Path "artifact.module.build" with Field "name" is "artifact.module.build.name". In a builds or build.promotions query, Path "" compares the found row's own fields. Exactly one of Text, Number and Time is meaningful, by the field's kind; Before and Last carry Span.

type PathMSP

type PathMSP struct {
	Path string
	MSP  MSP
	// contains filtered or unexported fields
}

PathMSP is $msp on the properties of the row Path reaches.

type PathProperty

type PathProperty struct {
	Path      string
	Criterion *PropertyCriterion
	// contains filtered or unexported fields
}

PathProperty is a property criterion on the item, module or build Path reaches; Path "" in a builds query is the build's own properties.

type PropertyCriterion

type PropertyCriterion struct {
	Part  PropertyPart
	Key   string
	Op    Op
	Value string
}

PropertyCriterion matches items by property. For PartPair, Key "*" is any key and Value "*" with $eq (or $match) is any value. For PartKey and PartValue, Value is what the key or the value is compared with, and "*" with $eq is any. A negative operator ($ne, $nmatch) holds when the item has no property that matches the positive form, so items without the key match it too. Each criterion is tested on its own: two of them may be satisfied by different properties of the item, unless they sit in an MSP.

type PropertyPart

type PropertyPart int

PropertyPart says which part of a property a PropertyCriterion tests.

const (
	PartPair PropertyPart = iota
	PartKey
	PartValue
)

Property parts: "@key" tests a key's values, "property.key" the keys alone and "property.value" the values alone.

type Query

type Query struct {
	// Domain is the primary domain: what the query finds. Actions are on
	// items.
	Domain Domain
	// Action is Find unless the query is items.delete or properties.update.
	Action Action
	// DryRun is an action's .dryRun(), true unless it says false, so an
	// action that leaves it out changes nothing. Find has none.
	DryRun bool
	// Keys and NewValue are properties.update's .keys() and .newValue().
	Keys     []string
	NewValue string
	// Criteria is the find criteria; it already carries the implicit
	// type=file that applies when the query names no type.
	Criteria Node
	// Include is the include list as given, in order.
	Include []string
	// Sort is nil when the query does not sort.
	Sort *Sort
	// Offset and Limit are the suffixes; HasOffset and HasLimit tell
	// .offset(0) and .limit(0) from none. Like Sort, they count result rows,
	// which are items whatever the include: properties and statistics render
	// inside their item's row, so they never add rows to skip or count.
	// Artifactory counts its joined item-property rows instead (DV-074).
	Offset    int
	HasOffset bool
	Limit     int
	HasLimit  bool
	// Transitive and Distinct are accepted for compatibility and have no
	// effect: items rows are always distinct (properties and statistics
	// render inside their item), and artifactweave has no Smart Remote
	// search to make transitive.
	Transitive bool
	Distinct   bool
	// contains filtered or unexported fields
}

Query is a parsed query, or an action over items.

func Parse

func Parse(src string) (*Query, error)

Parse parses an items, builds or build.promotions query, items.delete or properties.update. Errors wrap ErrEmpty, ErrSyntax, ErrUnsupported, ErrInvalid or ErrTooLong.

func (*Query) BuildResponse

func (q *Query) BuildResponse(hits []BuildHit) Response

BuildResponse builds the body for hits, which SelectBuilds or a store returned for a builds or build.promotions query.

func (*Query) CheckRestricted

func (q *Query) CheckRestricted() error

CheckRestricted applies decision 0012's rule for non-admin callers: every items result must carry repo, path and name, and every builds result name, number and repo, so each row can be checked against the caller's permissions. Whether the caller may query builds at all is the caller's to decide, from its token's scope.

func (*Query) Compare

func (q *Query) Compare(a, b types.Item) int

Compare orders two items by the query's sort, then by ID, as the SQL ORDER BY does. A missing time sorts first ascending and last descending. IDs tie only between roots without a record.

func (*Query) Compile

func (q *Query) Compile(d Dialect, o Options) (Compiled, error)

Compile writes q as SQL for d. Every value is bound, never interpolated.

func (*Query) Fields

func (q *Query) Fields() []Field

Fields returns the item fields each result row carries, in render order: the included item fields, or the default fields when the include names none. Including only properties or statistics keeps the defaults, as Artifactory does.

func (*Query) ForNesting

func (q *Query) ForNesting() *Query

ForNesting is the query to run for q's rows when q relates: a copy whose item rows carry every property and statistic, which the relation-path criteria Nest evaluates again may read. q itself still renders the response, so they show only as q includes them.

func (*Query) Joins

func (q *Query) Joins() bool

Joins reports whether q's criteria name relation paths, so running it needs the rows of the domains they reach.

func (*Query) Match

func (q *Query) Match(h Hit, now time.Time) bool

Match reports whether the hit's item, with its properties and statistics, satisfies the criteria. Relation paths reach no rows here.

func (*Query) MissingRoots

func (q *Query) MissingRoots(o Options, recorded func(RepoRef) bool) []Hit

MissingRoots returns the roots of q that recorded does not hold, as RootHit makes them, and that satisfy q's criteria at o's clock.

func (*Query) Nest

func (q *Query) Nest(ctx context.Context, src Source, opt Nesting, hits []Hit) (Related, error)

Nest finds the rows of the other domains q's include names for each hit. The hits must carry the properties and statistics q's criteria read. Rows src cannot load make the error; a query whose include reaches too many combinations of related rows wraps ErrInvalid.

func (*Query) NestBuilds

func (q *Query) NestBuilds(
	ctx context.Context,
	src Source,
	opt Nesting,
	hits []BuildHit,
) (Related, error)

NestBuilds is Nest for the hits of a builds or build.promotions query.

func (*Query) Properties

func (q *Query) Properties() bool

Properties reports whether result rows carry the item's properties: an include of "property", "property.*", "property.key", "property.value", "@", "@*" or "@<key>", or a properties.update, whose rows show the keys it changes.

func (*Query) PropertyKeys

func (q *Query) PropertyKeys() []string

PropertyKeys returns the keys an "@<key>" include limits the rendered properties to, in include order; nil means every key, which an include of the whole property domain also asks for.

func (*Query) Relates

func (q *Query) Relates() bool

Relates reports whether q's include names fields of other domains, which Nest or NestBuilds then fills in.

func (*Query) Resolve

func (q *Query) Resolve(repos Repositories)

Resolve rewrites q's repo criteria against the configured repositories (decision 0012, research 22 §1.4). The JFrog CLI downloads from a remote by naming the remote's own key, so a key is matched by what it stands for, not only by the key results report:

  • a remote's key, and its "<key>-cache" alias, match the remote's cache;
  • a virtual repository's key matches its members, recursively: local members' items and remote members' caches;
  • a local key matches its own items, even one that ends in "-cache";
  • a key no repository has keeps the plain comparison.

$eq and $ne match the expansion of their key and its complement. $match holds for an item whose reported key matches the pattern or that lies in the expansion of a configured key the pattern matches, and $nmatch holds when $match does not. Ordering operators compare the reported key as before. A query whose $eq names a virtual repository returns virtual_repos on every row, as Artifactory does.

An items query parsed from AQL text also learns the repositories' roots (Roots).

func (*Query) Response

func (q *Query) Response(hits []Hit) Response

Response builds the body for hits, which Select or a store returned for q and the caller gave their virtual repositories when q includes them.

func (*Query) Roots

func (q *Query) Roots() []RepoRef

Roots returns the repository roots an items query's rows include beside the stored items, by key: every repository that stores items has a root, a remote one in its cache, and Artifactory's items domain lists each as a folder with path and name "." (#445). A virtual repository stores nothing and has none. A root with a record (the store's EnsureRoot) is a stored item already; one without is the row RootHit makes. Only Resolve sets them, and only for a query Parse read from AQL text: the queries the GET searches build in code list no root.

func (*Query) Stats

func (q *Query) Stats() []StatField

Stats returns the statistic fields result rows carry, in render order: all of them for "stat" or "stat.*", or those named "stat.<field>".

func (*Query) VirtualRepos

func (q *Query) VirtualRepos() bool

VirtualRepos reports whether result rows carry virtual_repos: the query includes them, or its criteria name a virtual repository (Resolve).

func (*Query) Window

func (q *Query) Window(o Options) (offset, limit int)

Window returns the offset and the row limit to apply; a negative limit means none. The cap trims a query's own limit too.

func (*Query) WithRoots

func (q *Query) WithRoots(
	o Options,
	roots []Hit,
	search func(*Query, Options) ([]Hit, error),
) ([]Hit, error)

WithRoots runs search for q and merges roots, rows search cannot return (MissingRoots), into its rows in q's order, then applies q's offset and limit to the whole, as Select does. search runs a query without offset whose limit covers the rows the window needs.

type Related struct {
	// contains filtered or unexported fields
}

Related holds, for each row a query found, the rows of other domains its include names, which Response.WithRelated nests in the response.

type RepoClass

type RepoClass int

RepoClass is what a repository key stands for in a repo criterion.

const (
	RepoLocal RepoClass = iota
	// RepoRemote holds only its cache, reported as "<key>-cache".
	RepoRemote
	// RepoVirtual holds nothing itself and stands for its members.
	RepoVirtual
)

Repository classes. Every class that stores its own items (local, federated, release bundles) is RepoLocal.

type RepoRef

type RepoRef struct {
	Key    string
	Cached bool
}

RepoRef is one stored location of items: a repository key and whether it is that repository's remote cache, the items table's (repo_key, cached).

type RepoSet

type RepoSet struct {
	Refs   []RepoRef
	Negate bool
}

RepoSet holds when an item is stored in one of Refs, or, with Negate, in none of them. Resolve puts it in place of repo criteria; Parse never produces it.

type Repositories

type Repositories map[string]Repository

Repositories are the configured repositories, by key.

func (Repositories) Expand

func (r Repositories) Expand(key string) []RepoRef

Expand returns the stored locations key stands for, as a repo criterion's $eq resolves it: a remote's cache, a virtual repository's members through nested virtual repositories, or the key's own items.

type Repository

type Repository struct {
	Class RepoClass
	// Members are a virtual repository's members, in order.
	Members []string
}

Repository is one configured repository as a repo criterion resolves it.

type Response

type Response struct {
	Rows []Row
	// Start is the offset; Total the number of rows returned.
	Start, Total int
	// Limit is the query's limit, rendered only when it had one.
	Limit    int
	HasLimit bool
	// Notification explains a capped result; empty renders nothing.
	Notification string
}

Response is the body of POST /api/search/aql: {"results":[...],"range":{"start_pos","end_pos","total","limit"?,"notification"?}}. It is marshalled by hand so rows keep a fixed field order.

func (Response) MarshalJSON

func (r Response) MarshalJSON() ([]byte, error)

MarshalJSON renders the response.

func (Response) WithRelated

func (r Response) WithRelated(rel Related) Response

WithRelated nests rel's rows in the response's rows, which must be those of the hits Nest or NestBuilds was given, in the same order.

type Row

type Row struct {
	// contains filtered or unexported fields
}

Row is one result: the query's fields of an item and, when included, its virtual repositories, properties and statistics; or the query's fields of a run or a promotion.

func (Row) MarshalJSON

func (r Row) MarshalJSON() ([]byte, error)

MarshalJSON renders the row alone.

type Sort

type Sort struct {
	Desc   bool
	Fields []Field
}

Sort orders results by Fields, all ascending or all descending.

type Source

type Source interface {
	// RunsWithFile returns the runs with an artifact, or with dependency a
	// dependency, whose SHA-1 is sha1, by ID.
	RunsWithFile(ctx context.Context, dependency bool, sha1 string) ([]types.Build, error)
	// ItemsWithSHA1 returns the files whose actual SHA-1 is sha1, by ID,
	// each with Stats set only when it has a statistics record.
	ItemsWithSHA1(ctx context.Context, sha1 string) ([]Hit, error)
	// Run returns the run with id.
	Run(ctx context.Context, id int64) (types.Build, error)
	// RunPromotions returns the promotions of the run with id, by ID.
	RunPromotions(ctx context.Context, id int64) ([]types.BuildPromotion, error)
}

Source loads the rows of other domains that Nest follows relation paths to. Runs come with their modules and properties, and items with their properties, because the query's criteria may read them, and with their statistics, which an include may render.

type Span

type Span struct {
	N    int
	Unit string
}

Span is a relative time for $before and $last: N units back from now.

type StatCriterion

type StatCriterion struct {
	Field  StatField
	Op     Op
	Null   bool
	Text   string
	Number int64
	Time   time.Time
	Span   Span
}

StatCriterion compares a statistic field. Null with $eq holds for an item that has no value for the field (never downloaded), and with $ne for one that has; otherwise a missing value compares as SQL NULL and no comparison holds.

type StatField

type StatField string

StatField is a field of the stat domain, which criteria name "stat.<field>". Its values come from the item's download statistics; an item that was never downloaded has none, which compares as SQL NULL.

const (
	StatDownloaded         StatField = "downloaded"
	StatDownloads          StatField = "downloads"
	StatDownloadedBy       StatField = "downloaded_by"
	StatRemoteDownloads    StatField = "remote_downloads"
	StatRemoteDownloaded   StatField = "remote_downloaded"
	StatRemoteDownloadedBy StatField = "remote_downloaded_by"
	StatRemoteOrigin       StatField = "remote_origin"
	StatRemotePath         StatField = "remote_path"
)

Statistic fields (research 22 §1.1).

Directories

Path Synopsis
Package aqltest holds the AQL fixtures every search backend must agree on (decision 0012): seed items chosen to trip the dialect traps (a GLOB character class, LIKE wildcards in names, a multi-byte character, a missing time, a remote cache, an item without statistics beside one whose statistics are zero), the repositories they live in, the query shapes the JFrog CLI generates, and the rows each query returns, in order.
Package aqltest holds the AQL fixtures every search backend must agree on (decision 0012): seed items chosen to trip the dialect traps (a GLOB character class, LIKE wildcards in names, a multi-byte character, a missing time, a remote cache, an item without statistics beside one whose statistics are zero), the repositories they live in, the query shapes the JFrog CLI generates, and the rows each query returns, in order.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL