Documentation
¶
Overview ¶
Package aql parses Artifactory Query Language and runs it (decision 0012). It covers the items domain: the shape the JFrog CLI generates for every file specification (research 09) and what users write by hand. That is $and and $or, field comparisons, $match and $nmatch globs, @property and property.key/property.value criteria, $msp, the stat domain, and the include, sort, offset, limit, transitive and distinct suffixes with Artifactory's output rules (research 22 §1). The builds and build.promotions domains are served too, and every domain reaches the others through relation paths such as artifact.module.build.name (see Joined), which is how the JFrog CLI finds a build's files; an include may name those domains' fields too, which Nest fills in (nest.go). It also parses the two actions, items.delete and properties.update (research 22 §1.5), whose criteria select items the same way; the server carries them out. Every repository's root is an items row (Roots).
A parsed Query is run two ways that must agree row for row: Match and Compare are the reference semantics (the in-memory store uses them), and Compile turns the query into SQL for SQLite or PostgreSQL. The store contract suite runs the same queries on every backend and expects the same ordered rows. Before either, Resolve rewrites repo criteria against the configured repositories, so remote and virtual keys match what they stand for.
Index ¶
- Constants
- Variables
- type Action
- type And
- type BuildHit
- type Compiled
- type Corpus
- func (c *Corpus) ItemsWithSHA1(_ context.Context, sha1 string) ([]Hit, error)
- func (c *Corpus) Run(_ context.Context, id int64) (types.Build, error)
- func (c *Corpus) RunPromotions(_ context.Context, id int64) ([]types.BuildPromotion, error)
- func (c *Corpus) RunsWithFile(_ context.Context, dependency bool, sha1 string) ([]types.Build, error)
- type Dialect
- type Domain
- type Field
- type FieldCriterion
- type Hit
- type Joined
- type MSP
- type Nesting
- type Node
- type Op
- type Options
- type Or
- type PathCriterion
- type PathMSP
- type PathProperty
- type PropertyCriterion
- type PropertyPart
- type Query
- func (q *Query) BuildResponse(hits []BuildHit) Response
- func (q *Query) CheckRestricted() error
- func (q *Query) Compare(a, b types.Item) int
- func (q *Query) Compile(d Dialect, o Options) (Compiled, error)
- func (q *Query) Fields() []Field
- func (q *Query) ForNesting() *Query
- func (q *Query) Joins() bool
- func (q *Query) Match(h Hit, now time.Time) bool
- func (q *Query) MissingRoots(o Options, recorded func(RepoRef) bool) []Hit
- func (q *Query) Nest(ctx context.Context, src Source, opt Nesting, hits []Hit) (Related, error)
- func (q *Query) NestBuilds(ctx context.Context, src Source, opt Nesting, hits []BuildHit) (Related, error)
- func (q *Query) Properties() bool
- func (q *Query) PropertyKeys() []string
- func (q *Query) Relates() bool
- func (q *Query) Resolve(repos Repositories)
- func (q *Query) Response(hits []Hit) Response
- func (q *Query) Roots() []RepoRef
- func (q *Query) Stats() []StatField
- func (q *Query) VirtualRepos() bool
- func (q *Query) Window(o Options) (offset, limit int)
- func (q *Query) WithRoots(o Options, roots []Hit, search func(*Query, Options) ([]Hit, error)) ([]Hit, error)
- type Related
- type RepoClass
- type RepoRef
- type RepoSet
- type Repositories
- type Repository
- type Response
- type Row
- type Sort
- type Source
- type Span
- type StatCriterion
- type StatField
Constants ¶
const FieldVirtualRepos = "virtual_repos"
FieldVirtualRepos lists the virtual repositories that contain an item's repository. It can be included but not searched on, and it needs repo, path and name in the output (research 22 §1.2).
const MaxLength = 6000
MaxLength is the longest query accepted, in characters: Artifactory's artifactory.aql.search.query.size.limit default. ErrTooLong's text names it, as Artifactory's message names the configured limit.
Variables ¶
var ( // ErrSyntax reports a query that does not parse. ErrSyntax = errors.New("aql: syntax error") // ErrUnsupported reports valid AQL that artifactweave does not serve: // another domain, a secondary-domain field, or an operator outside the // supported set. ErrUnsupported = errors.New("aql: unsupported") // ErrTooLong reports a query above MaxLength characters. Its text is // Artifactory's message (aql-performance). ErrTooLong = errors.New("AQL query is too long; please reduce the query length to less than " + strconv.Itoa(MaxLength) + " chars") // ErrRestricted reports a query a non-admin caller may not run. ErrRestricted = errors.New("aql: restricted") // ErrEmpty reports a request with no query at all. ErrEmpty = errors.New("aql: empty query") // ErrInvalid reports a query that parses but breaks an output rule: a // sort on a field the output leaves out, or virtual_repos without repo, // path and name. It also reports a suffix the query's action does not // take, and a properties.update without .keys() or .newValue(). ErrInvalid = errors.New("aql: invalid query") )
Functions ¶
This section is empty.
Types ¶
type Action ¶
type Action int
Action is what a query does with the items its criteria select.
Actions. Find returns the items; DeleteItems is items.delete, which removes them, and UpdateProperties is properties.update, which replaces the values of the query's Keys on them.
type BuildHit ¶
type BuildHit struct {
Build types.Build
Promotion *types.BuildPromotion
}
BuildHit is one row a builds or build.promotions query found: the run, without its modules, properties or document, and for build.promotions the promotion.
func SelectBuilds ¶
SelectBuilds runs a builds or build.promotions query over a corpus, as the SQL compiler's output does: filter, sort (by ID when the query does not sort, and by ID within equal sort keys), then offset and limit. A promotion of a run the corpus lacks is skipped, as the SQL join skips it.
type Compiled ¶
Compiled is a query as SQL over the store's tables (internal/sqlstore migrations): the caller selects its columns FROM the domain's table, items, builds or build_promotions, unaliased, then appends " WHERE " + Where and Tail, binding Args in order. Placeholders are "?"; a PostgreSQL caller rebinds them.
type Corpus ¶
type Corpus struct {
Items []Hit
Builds []types.Build
Promotions []types.BuildPromotion
// contains filtered or unexported fields
}
Corpus is everything the reference evaluator may find or join: the items with their properties and statistics, the build runs with their properties and modules, and the promotions. It is read, never changed; SelectItems and SelectBuilds index it on first use, so it must not change between calls.
func (*Corpus) ItemsWithSHA1 ¶
ItemsWithSHA1 implements Source over the corpus.
func (*Corpus) RunPromotions ¶
RunPromotions implements Source over the corpus.
type Domain ¶
type Domain int
Domain is a query's primary domain (research 22 §1.1).
Primary domains. The zero value is items, so a Query built by hand searches items.
type Field ¶
type Field string
Field is an item field name as AQL spells it.
const ( FieldRepo Field = "repo" FieldPath Field = "path" FieldName Field = "name" FieldType Field = "type" FieldSize Field = "size" FieldDepth Field = "depth" FieldCreated Field = "created" FieldCreatedBy Field = "created_by" FieldModified Field = "modified" FieldModifiedBy Field = "modified_by" FieldUpdated Field = "updated" FieldOriginalMD5 Field = "original_md5" FieldActualMD5 Field = "actual_md5" FieldOriginalSHA1 Field = "original_sha1" FieldActualSHA1 Field = "actual_sha1" FieldSHA256 Field = "sha256" )
Item fields. Checksums are the actual ones unless named original.
type FieldCriterion ¶
FieldCriterion compares an item field. Exactly one of Text, Number and Time is meaningful, by the field's kind; Before and Last carry their span in Span instead.
type Hit ¶
type Hit struct {
Item types.Item
Properties types.Properties
Stats *types.ItemStats
VirtualRepos []string
}
Hit is one item a query found, with its properties and statistics when the query includes them. Stats is nil for an item that has no statistics record. VirtualRepos is filled by the caller, which knows the repository configurations, when the query includes virtual_repos.
func RootHit ¶
RootHit is the row of a repository root that has no record: a folder with path and name ".", ID 0, and no times, properties or statistics.
func Select ¶
Select runs q over candidates the way the SQL compiler's output does: filter, sort (by ID when the query does not sort, and by ID within equal sort keys), then offset and limit. Properties and statistics are dropped unless the query includes them. It is the reference the SQL backends are tested against; candidates must carry their properties and statistics. Relation paths reach no builds here; SelectItems joins them.
type Joined ¶
type Joined struct {
Criteria And
}
Joined holds the criteria of a query's root that name relation paths. A row matches when one combination of the rows its paths reach satisfies them all at once, as when Artifactory's SQL joins each related table once per query: {"artifact.module.build.name":"a","artifact.module.build.number":"1"} needs one build named a with number 1, not a build named a and another numbered 1. A path that reaches no row takes the null row an outer join would, on which no criterion holds, so an alternative in an $or can still match. Parse gathers every root conjunct that names a path into one Joined; criteria on the found row alone stay outside it.
type MSP ¶
type MSP []*PropertyCriterion
MSP is $msp, match on a single property: it holds when one property of the item satisfies every criterion at once. Inside it, a negative operator tests that one property ($ne is a different value, $nmatch a value that does not match), not the absence of every property that would match.
type Nesting ¶
type Nesting struct {
// Now resolves $before and $last in the relation-path criteria; zero
// means time.Now.
Now time.Time
// ItemVisible and RunVisible, when set, report whether the caller may
// see a related item, or a run with its modules, files and promotions.
// A row they refuse is left out with every row below it.
ItemVisible func(types.Item) bool
RunVisible func(types.Build) bool
}
Nesting is how Nest renders related rows for one caller.
type Node ¶
type Node interface {
// contains filtered or unexported methods
}
Node is a criteria tree node: And, Or, *FieldCriterion, *PropertyCriterion, *StatCriterion, MSP, the RepoSet that Resolve puts in place of repo criteria, or a criterion of the builds domains and relation paths: *PathCriterion, *PathProperty, *PathMSP and the *Joined that gathers those naming paths.
type Op ¶
type Op string
Op is a comparison operator.
const ( OpEq Op = "$eq" OpNe Op = "$ne" OpGt Op = "$gt" OpGte Op = "$gte" OpLt Op = "$lt" OpLte Op = "$lte" OpMatch Op = "$match" OpNMatch Op = "$nmatch" OpBefore Op = "$before" OpLast Op = "$last" )
Operators. Before and Last are relative times, resolved against the caller's clock when the query runs.
type Options ¶
type Options struct {
// Now resolves $before and $last; zero means time.Now.
Now time.Time
// Max is the most rows any query returns, its own limit included: the
// search hard limit (decision 0012). Zero means no cap.
Max int
}
Options are the caller's settings for running a query.
type PathCriterion ¶
type PathCriterion struct {
Path string
Field string
Op Op
Text string
Number int64
Time time.Time
Span Span
// contains filtered or unexported fields
}
PathCriterion compares a field of the row a relation path reaches from the row a query finds (research 22 §1.1): in an items query, Path "artifact.module.build" with Field "name" is "artifact.module.build.name". In a builds or build.promotions query, Path "" compares the found row's own fields. Exactly one of Text, Number and Time is meaningful, by the field's kind; Before and Last carry Span.
type PathProperty ¶
type PathProperty struct {
Path string
Criterion *PropertyCriterion
// contains filtered or unexported fields
}
PathProperty is a property criterion on the item, module or build Path reaches; Path "" in a builds query is the build's own properties.
type PropertyCriterion ¶
type PropertyCriterion struct {
Part PropertyPart
Key string
Op Op
Value string
}
PropertyCriterion matches items by property. For PartPair, Key "*" is any key and Value "*" with $eq (or $match) is any value. For PartKey and PartValue, Value is what the key or the value is compared with, and "*" with $eq is any. A negative operator ($ne, $nmatch) holds when the item has no property that matches the positive form, so items without the key match it too. Each criterion is tested on its own: two of them may be satisfied by different properties of the item, unless they sit in an MSP.
type PropertyPart ¶
type PropertyPart int
PropertyPart says which part of a property a PropertyCriterion tests.
const ( PartPair PropertyPart = iota PartKey PartValue )
Property parts: "@key" tests a key's values, "property.key" the keys alone and "property.value" the values alone.
type Query ¶
type Query struct {
// Domain is the primary domain: what the query finds. Actions are on
// items.
Domain Domain
// Action is Find unless the query is items.delete or properties.update.
Action Action
// DryRun is an action's .dryRun(), true unless it says false, so an
// action that leaves it out changes nothing. Find has none.
DryRun bool
// Keys and NewValue are properties.update's .keys() and .newValue().
Keys []string
NewValue string
// Criteria is the find criteria; it already carries the implicit
// type=file that applies when the query names no type.
Criteria Node
// Include is the include list as given, in order.
Include []string
// Sort is nil when the query does not sort.
Sort *Sort
// Offset and Limit are the suffixes; HasOffset and HasLimit tell
// .offset(0) and .limit(0) from none. Like Sort, they count result rows,
// which are items whatever the include: properties and statistics render
// inside their item's row, so they never add rows to skip or count.
// Artifactory counts its joined item-property rows instead (DV-074).
Offset int
HasOffset bool
Limit int
HasLimit bool
// Transitive and Distinct are accepted for compatibility and have no
// effect: items rows are always distinct (properties and statistics
// render inside their item), and artifactweave has no Smart Remote
// search to make transitive.
Transitive bool
Distinct bool
// contains filtered or unexported fields
}
Query is a parsed query, or an action over items.
func Parse ¶
Parse parses an items, builds or build.promotions query, items.delete or properties.update. Errors wrap ErrEmpty, ErrSyntax, ErrUnsupported, ErrInvalid or ErrTooLong.
func (*Query) BuildResponse ¶
BuildResponse builds the body for hits, which SelectBuilds or a store returned for a builds or build.promotions query.
func (*Query) CheckRestricted ¶
CheckRestricted applies decision 0012's rule for non-admin callers: every items result must carry repo, path and name, and every builds result name, number and repo, so each row can be checked against the caller's permissions. Whether the caller may query builds at all is the caller's to decide, from its token's scope.
func (*Query) Compare ¶
Compare orders two items by the query's sort, then by ID, as the SQL ORDER BY does. A missing time sorts first ascending and last descending. IDs tie only between roots without a record.
func (*Query) Fields ¶
Fields returns the item fields each result row carries, in render order: the included item fields, or the default fields when the include names none. Including only properties or statistics keeps the defaults, as Artifactory does.
func (*Query) ForNesting ¶
ForNesting is the query to run for q's rows when q relates: a copy whose item rows carry every property and statistic, which the relation-path criteria Nest evaluates again may read. q itself still renders the response, so they show only as q includes them.
func (*Query) Joins ¶
Joins reports whether q's criteria name relation paths, so running it needs the rows of the domains they reach.
func (*Query) Match ¶
Match reports whether the hit's item, with its properties and statistics, satisfies the criteria. Relation paths reach no rows here.
func (*Query) MissingRoots ¶
MissingRoots returns the roots of q that recorded does not hold, as RootHit makes them, and that satisfy q's criteria at o's clock.
func (*Query) Nest ¶
Nest finds the rows of the other domains q's include names for each hit. The hits must carry the properties and statistics q's criteria read. Rows src cannot load make the error; a query whose include reaches too many combinations of related rows wraps ErrInvalid.
func (*Query) NestBuilds ¶
func (q *Query) NestBuilds( ctx context.Context, src Source, opt Nesting, hits []BuildHit, ) (Related, error)
NestBuilds is Nest for the hits of a builds or build.promotions query.
func (*Query) Properties ¶
Properties reports whether result rows carry the item's properties: an include of "property", "property.*", "property.key", "property.value", "@", "@*" or "@<key>", or a properties.update, whose rows show the keys it changes.
func (*Query) PropertyKeys ¶
PropertyKeys returns the keys an "@<key>" include limits the rendered properties to, in include order; nil means every key, which an include of the whole property domain also asks for.
func (*Query) Relates ¶
Relates reports whether q's include names fields of other domains, which Nest or NestBuilds then fills in.
func (*Query) Resolve ¶
func (q *Query) Resolve(repos Repositories)
Resolve rewrites q's repo criteria against the configured repositories (decision 0012, research 22 §1.4). The JFrog CLI downloads from a remote by naming the remote's own key, so a key is matched by what it stands for, not only by the key results report:
- a remote's key, and its "<key>-cache" alias, match the remote's cache;
- a virtual repository's key matches its members, recursively: local members' items and remote members' caches;
- a local key matches its own items, even one that ends in "-cache";
- a key no repository has keeps the plain comparison.
$eq and $ne match the expansion of their key and its complement. $match holds for an item whose reported key matches the pattern or that lies in the expansion of a configured key the pattern matches, and $nmatch holds when $match does not. Ordering operators compare the reported key as before. A query whose $eq names a virtual repository returns virtual_repos on every row, as Artifactory does.
An items query parsed from AQL text also learns the repositories' roots (Roots).
func (*Query) Response ¶
Response builds the body for hits, which Select or a store returned for q and the caller gave their virtual repositories when q includes them.
func (*Query) Roots ¶
Roots returns the repository roots an items query's rows include beside the stored items, by key: every repository that stores items has a root, a remote one in its cache, and Artifactory's items domain lists each as a folder with path and name "." (#445). A virtual repository stores nothing and has none. A root with a record (the store's EnsureRoot) is a stored item already; one without is the row RootHit makes. Only Resolve sets them, and only for a query Parse read from AQL text: the queries the GET searches build in code list no root.
func (*Query) Stats ¶
Stats returns the statistic fields result rows carry, in render order: all of them for "stat" or "stat.*", or those named "stat.<field>".
func (*Query) VirtualRepos ¶
VirtualRepos reports whether result rows carry virtual_repos: the query includes them, or its criteria name a virtual repository (Resolve).
func (*Query) Window ¶
Window returns the offset and the row limit to apply; a negative limit means none. The cap trims a query's own limit too.
func (*Query) WithRoots ¶
func (q *Query) WithRoots( o Options, roots []Hit, search func(*Query, Options) ([]Hit, error), ) ([]Hit, error)
WithRoots runs search for q and merges roots, rows search cannot return (MissingRoots), into its rows in q's order, then applies q's offset and limit to the whole, as Select does. search runs a query without offset whose limit covers the rows the window needs.
type Related ¶
type Related struct {
// contains filtered or unexported fields
}
Related holds, for each row a query found, the rows of other domains its include names, which Response.WithRelated nests in the response.
type RepoClass ¶
type RepoClass int
RepoClass is what a repository key stands for in a repo criterion.
type RepoRef ¶
RepoRef is one stored location of items: a repository key and whether it is that repository's remote cache, the items table's (repo_key, cached).
type RepoSet ¶
RepoSet holds when an item is stored in one of Refs, or, with Negate, in none of them. Resolve puts it in place of repo criteria; Parse never produces it.
type Repositories ¶
type Repositories map[string]Repository
Repositories are the configured repositories, by key.
func (Repositories) Expand ¶
func (r Repositories) Expand(key string) []RepoRef
Expand returns the stored locations key stands for, as a repo criterion's $eq resolves it: a remote's cache, a virtual repository's members through nested virtual repositories, or the key's own items.
type Repository ¶
type Repository struct {
Class RepoClass
// Members are a virtual repository's members, in order.
Members []string
}
Repository is one configured repository as a repo criterion resolves it.
type Response ¶
type Response struct {
Rows []Row
// Start is the offset; Total the number of rows returned.
Start, Total int
// Limit is the query's limit, rendered only when it had one.
Limit int
HasLimit bool
// Notification explains a capped result; empty renders nothing.
Notification string
}
Response is the body of POST /api/search/aql: {"results":[...],"range":{"start_pos","end_pos","total","limit"?,"notification"?}}. It is marshalled by hand so rows keep a fixed field order.
func (Response) MarshalJSON ¶
MarshalJSON renders the response.
func (Response) WithRelated ¶
WithRelated nests rel's rows in the response's rows, which must be those of the hits Nest or NestBuilds was given, in the same order.
type Row ¶
type Row struct {
// contains filtered or unexported fields
}
Row is one result: the query's fields of an item and, when included, its virtual repositories, properties and statistics; or the query's fields of a run or a promotion.
func (Row) MarshalJSON ¶
MarshalJSON renders the row alone.
type Source ¶
type Source interface {
// RunsWithFile returns the runs with an artifact, or with dependency a
// dependency, whose SHA-1 is sha1, by ID.
RunsWithFile(ctx context.Context, dependency bool, sha1 string) ([]types.Build, error)
// ItemsWithSHA1 returns the files whose actual SHA-1 is sha1, by ID,
// each with Stats set only when it has a statistics record.
ItemsWithSHA1(ctx context.Context, sha1 string) ([]Hit, error)
// Run returns the run with id.
Run(ctx context.Context, id int64) (types.Build, error)
// RunPromotions returns the promotions of the run with id, by ID.
RunPromotions(ctx context.Context, id int64) ([]types.BuildPromotion, error)
}
Source loads the rows of other domains that Nest follows relation paths to. Runs come with their modules and properties, and items with their properties, because the query's criteria may read them, and with their statistics, which an include may render.
type StatCriterion ¶
type StatCriterion struct {
Field StatField
Op Op
Null bool
Text string
Number int64
Time time.Time
Span Span
}
StatCriterion compares a statistic field. Null with $eq holds for an item that has no value for the field (never downloaded), and with $ne for one that has; otherwise a missing value compares as SQL NULL and no comparison holds.
type StatField ¶
type StatField string
StatField is a field of the stat domain, which criteria name "stat.<field>". Its values come from the item's download statistics; an item that was never downloaded has none, which compares as SQL NULL.
const ( StatDownloaded StatField = "downloaded" StatDownloads StatField = "downloads" StatDownloadedBy StatField = "downloaded_by" StatRemoteDownloads StatField = "remote_downloads" StatRemoteDownloaded StatField = "remote_downloaded" StatRemoteDownloadedBy StatField = "remote_downloaded_by" StatRemoteOrigin StatField = "remote_origin" StatRemotePath StatField = "remote_path" )
Statistic fields (research 22 §1.1).
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package aqltest holds the AQL fixtures every search backend must agree on (decision 0012): seed items chosen to trip the dialect traps (a GLOB character class, LIKE wildcards in names, a multi-byte character, a missing time, a remote cache, an item without statistics beside one whose statistics are zero), the repositories they live in, the query shapes the JFrog CLI generates, and the rows each query returns, in order.
|
Package aqltest holds the AQL fixtures every search backend must agree on (decision 0012): seed items chosen to trip the dialect traps (a GLOB character class, LIKE wildcards in names, a multi-byte character, a missing time, a remote cache, an item without statistics beside one whose statistics are zero), the repositories they live in, the query shapes the JFrog CLI generates, and the rows each query returns, in order. |