Documentation
¶
Overview ¶
Package graphqldocs reads every raw GraphQL document this repository writes out of the source and judges each one against a GitLab schema.
The validating test transport catches a document the moment a test sends it, which covers most of them and cannot cover all of them: a document reachable by no test still ships, and a document reached only on an error branch is exercised by nobody. This package reads them out of the source instead, so the coverage of the check stops depending on the coverage of the tests.
It loads the whole program with go/packages rather than matching the source with a regular expression, because four of this repository's documents are assembled by concatenating a shared fragment constant and only the type checker knows what the assembled value is. Constants are folded during type checking, so a document written as three pieces is judged as the one string GitLab would receive.
A document that lives in a .graphql file rather than a Go constant is read straight off disk, because an embedded variable is not a constant and folds to nothing, so moving a long document into its own file would otherwise drop it out of the inventory without a word.
What it cannot do is check variables: a document read out of the source has no request behind it, so nothing says which variables a handler will send or what they will hold. That half belongs to the test transport, which sees a real request, and to the recorded request inventory it writes.
What it reads, and what it does not ¶
Collect reads DefaultPatterns, which holds every document this repository writes, and a count from it is this repository's rather than the server's whole GraphQL surface. The other half of that surface is client-go's own 42 documents, for the achievement, work item, saved view, security attribute, security category, scan profile, target branch rule and Terraform state services, and SDKDocuments reads those out of the module directory with the same walk. They stay a separate entry point because the answer is a separate claim: a document refused there is fixed upstream and not here, so its caller reports where the caller of Collect gates.
Six of the 42 are not written as sendable text: client-go writes the work item documents as text/template shells and the Terraform state queries as printf format strings, so the folded value carries a placeholder where a value belongs.
Shells, and what renders them ¶
A document written with holes in it is rendered wherever the call that fills them settles what the text is, and the walk records how in Assembly. A format string is passed through fmt.Sprintf with each constant argument's own value and a stand-in of its type for the rest. A text/template is executed inside the set it is parsed into, read back through the package variables the set is cloned from, provided nothing it reaches reads data, since a template that does renders what client-go sends for one caller and not for every one. Five of client-go's six render that way, and a test holds each rendering byte for byte to what client-go posts. The work item list does not, because its fragment is parsed inside a function from the fields a caller asks for and its variables are the filters a caller sets: it keeps its holes, IsTemplate still tells it apart, and its assembly says why. SDKDirectory finds the client-go the module requires.
Who calls it ¶
Five commands call it, and what they ask of the inventory splits them in two.
Two of them judge these documents against a schema, deliberately: cmd/audit_graphql_documents renders the result as the text of a standalone gate, and cmd/audit_1to1 folds the same result into the R-PATH dimension, where a document GitLab refuses is one of the three ways a registered action cannot reach the endpoint it names.
The other three want the inventory for a question of their own.
cmd/audit_readonly_graphql asks what operation type each document carries, and used to find them with a walk of its own that read constants and package-level variables and nothing else. Two detectors of the same thing disagree by construction, and this one reads .graphql files that one could not, so it builds its index from FromPackages and Standalone. It used to re-read the text with a rule of its own as well, which disagreed with LooksLikeDocument about two shapes and so judged a narrower set than the one it printed; it asks DefinesMutation now, so the rule that says what a document is and the rule that says whether it writes are one rule. Both read a document that parses through gqlparser and only a shell through the line rule, so a mutation after another definition on the same line, or behind a comma, a byte order mark or a hole, is in the inventory and answered as the mutation it is (issue 1321). What it cannot resolve, a document with no Document.Object that its own body walk did not record either, it reports rather than skips.
cmd/audit_graphql_shapes pairs each document it finds with the Go struct that decodes the response, and parses those pairs against the schema itself to get a selection set it can walk. It reads this inventory for the other half of that question: a document here that no pairing carries is one handed to something that audit does not follow, and it is named rather than passed over in silence.
cmd/gen_action_grants reads client-go's documents through SDKDocuments as the requests a client-go method sends, which the fine-grained derivation joins to what GitLab demands of each element they reach. A shell rendered here is read as its rendering, which is why the work item get, create and update documents and the Terraform state queries need no declaration there; one still a shell (IsTemplate) is a request the walk cannot read, which a declaration answers, and DefinesMutation puts a method's documents in the order it posts them. A change to the rendering therefore moves that command's derivation as well as the counts the two judging commands print.
Index ¶
- Constants
- Variables
- func DefaultPatterns() []string
- func DefinesMutation(document string) bool
- func ForeignModuleEnv() []string
- func IsTemplate(document Document) bool
- func LooksLikeDocument(value string) bool
- func SDKDirectory(dir string) (string, error)
- func SDKPatterns() []string
- type Assembly
- type Document
- type Options
- type Refusal
- type Result
Constants ¶
const ( // AssembledByFormat is a document written as the format of a fmt.Sprintf // call. AssembledByFormat = "fmt.Sprintf" // AssembledByTemplate is a document parsed into a text/template set. AssembledByTemplate = "text/template" )
The calls a shell's holes are filled by, which is what Assembly.By names.
const SDKImportPath = "gitlab.com/gitlab-org/api/client-go/v3"
SDKImportPath is the import path of client-go's root package, the one the handlers of this repository compile against and the one whose directory SDKDirectory finds.
Variables ¶
var ErrNoDocuments = errors.New("no GraphQL documents were found, which means this audit is looking at the wrong thing")
ErrNoDocuments is returned when an audit found nothing to judge, which means it is looking at the wrong tree. A clean exit there would be the silence this package exists to remove, so it is an error rather than an empty result.
Functions ¶
func DefaultPatterns ¶
func DefaultPatterns() []string
DefaultPatterns are the packages an audit loads. Every GraphQL document this repository writes lives under them.
It is a function rather than a variable because a caller that appended to a shared slice would change what every later caller audits, and an audit that silently looks at the wrong tree is the failure this package exists to remove.
func DefinesMutation ¶ added in v3.1.0
DefinesMutation reports whether a document carries a mutation operation, which is the half of the question cmd/audit_readonly_graphql asks of every document it reads.
It lives here rather than there because the two answers have to be made of the same rule. What a mutation looks like is part of what a document looks like, and the moment the two are written twice they drift: a shape LooksLikeDocument admits and a separate mutation rule does not match is a mutation classified as a read, which is exactly the silence that audit gates against. What stays that audit's own is what the answer means, which is that an action classified ReadOnly must not be able to reach one.
A document that parses is answered by gqlparser, the reading GitLab gives it: it carries a mutation when one of its operations is one, wherever on a line the keyword sits, and not when the keyword only opens a line of a block string. Issue 1321 is the case that settled it: "query Q {...} mutation P {...}" on one line was answered as a read. A text that does not parse is a shell, a template or a format string with its holes still in it, and is answered by the line rule, which reads the keyword after what [lineLead] lists. The comments are not skipped first there: a comment line opens with "#", which is not on that list.
func ForeignModuleEnv ¶ added in v3.1.0
func ForeignModuleEnv() []string
ForeignModuleEnv is the environment a load of a module this process does not own has to run under, appended to the caller's own.
Both entries are about the module cache rather than about GraphQL, and both were found by a load that failed outright rather than by reasoning.
GOWORK=off, because client-go ships a go.work in its module zip that names ./config, a directory the zip does not carry: in workspace mode the toolchain refuses to load anything there at all. Nothing about this repository's own build is affected, since a workspace of ours would be nobody's business during a load of somebody else's module.
GOFLAGS carries -mod=readonly, because a module cache directory is read-only and -mod=mod asks the toolchain for permission to write a go.mod it cannot. A developer or a build machine with -mod=mod exported (which is one way to run this repository's own tooling) would otherwise see this load fail with a message about workspace mode that says nothing about either cause.
That one setting is substituted into whatever GOFLAGS already said rather than written over it. The toolchain honors the last assignment of a variable, so appending "GOFLAGS=-mod=readonly" would silently take away every other flag the caller set — a machine exporting "-mod=mod -buildvcs=false" would lose the second one here and keep it everywhere else in the same run, which is a difference nobody would think to look for. The module cache makes -mod this package's business and nothing else's.
func IsTemplate ¶ added in v3.1.0
IsTemplate reports whether a document is a shell with holes in it rather than the text GitLab receives.
client-go writes six of its documents in two shapes that are not sendable text as written: the work item get, create, update and list documents are text/template shells, and the Terraform state queries are printf format strings that interpolate the project path and the state name with %q. The collector renders a shell wherever the call that fills it says what the text is (see Assembly), so five of the six arrive here rendered and this answers false for them. The work item list stays a shell: its fragment is parsed from the fields a caller asks for and its variable list is the filters a caller sets, so no rendering of it exists without a caller, and what folds still carries `{{ if .Decls }}` where that list belongs.
No schema can judge a shell, and one would be refused for the same reason on every single run. Reporting it as a refusal would teach a reader to skip the refusal list, which is the one list here that must never be skipped, so it is counted apart, named with the reason its assembly gives, and left unjudged.
func LooksLikeDocument ¶ added in v3.1.0
LooksLikeDocument reports whether a constant string is a GraphQL document rather than ordinary Go text that happens to contain braces.
This is the one definition of what counts as a document, and every caller asks it rather than carrying a rule of its own. The inventory is built on it, so a second rule anywhere is a second inventory: cmd/audit_readonly_graphql used to classify the text it read with a rule that recognized shapes this one refused, which left it printing "no read-only action reaches a mutation" while having read fewer documents than its own rule describes. A gate that narrows silently is the failure this package exists to remove, so the read-against-write question that audit does own now starts here, in DefinesMutation.
A pre-filter is needed because every constant string in the repository passes through here and handing gqlparser an error message would report a parse failure about a string nobody ever sends. The filter is deliberately crude and deliberately generous: it asks for braces and an operation definition or a selection set, and everything that gets past it is judged by the schema, so a false positive costs a reviewable finding rather than a silent skip.
Beside the line rules it admits whatever gqlparser parses from a definition's keyword, which is the reading GitLab itself gives the text. That reaches what the line rules leave out on purpose, a comma inside a definition among them, and it is what keeps this set and DefinesMutation's one set: every text that answers as a mutation is either a document gqlparser parses or a shell the widened line rule matches.
func SDKDirectory ¶ added in v3.3.0
SDKDirectory returns the directory of the client-go module the Go module in dir builds against, which is the version go.mod requires and the one SDKDocuments is pointed at.
It asks the toolchain for the root package's files and nothing more, since a caller that only needs to know where the module lives should not pay for type-checking it. A module that does not require client-go is refused rather than answered with an empty directory, which a caller would hand to SDKDocuments and have refused there under a reason that names nothing.
The lookup runs under -mod=readonly whatever GOFLAGS the caller exported. Under -mod=mod the toolchain resolves a package no requirement provides by adding the newest version it can find, so a module that requires no client-go would be answered with whichever one the module cache holds last, and this one with a version go.mod never named.
func SDKPatterns ¶ added in v3.1.0
func SDKPatterns() []string
SDKPatterns are the packages of client-go a document can live in, which is all of them: the operations sit beside the service methods that send them, in the module's root package, and the module is small enough that naming the files would be a list to keep current for no gain.
Types ¶
type Assembly ¶ added in v3.3.0
type Assembly struct {
// By names the call that fills the holes: [AssembledByFormat] or
// [AssembledByTemplate], and "" when no call this walk reads takes the
// document.
By string
// Shell is the document as the source writes it, holes included.
Shell string
// Unrendered says why [Document.Text] is still the shell, and is "" when it
// is the rendering.
Unrendered string
}
Assembly says how a document the source writes with holes in it becomes the text GitLab receives.
A document with no hole carries none. One with holes carries one whether or not it was rendered, so a reader holding a document that is still a shell learns why it is one rather than only that it is.
type Document ¶
type Document struct {
// Package is the import path of the package that declares it.
Package string
// Name is the constant or variable it is declared as, the file name when
// it is a standalone .graphql document, or "" when it is written inline at
// the point of use.
Name string
// Object is the constant or variable the document is declared as, and nil
// for the two shapes no declaration names: a standalone .graphql file and a
// document written inline where it is used.
//
// It is here for a caller that reasons about the source through the type
// checker rather than about the text: cmd/internal/actionrequests resolves
// a handler's call graph, for cmd/audit_readonly_graphql and for the
// request derivation, and asks what each object it names holds, so the
// object is the only thing that joins this inventory to that walk. A
// document with none cannot be attributed to the handler that sends it,
// which is a fact the readonly audit reports rather than one it can work
// around.
Object types.Object
// Position is where a reader will find it.
Position token.Position
// Text is the folded value, with any shared fragment already spliced in,
// and for a document the source writes with holes in it, the rendering
// the call that fills them produces wherever this walk can say what that
// is.
Text string
// Assembly says how a document written with holes in it becomes the text
// GitLab receives, and is nil for one written whole. See [Assembly].
Assembly *Assembly
}
Document is one GraphQL document found in the source.
func Collect ¶
Collect loads the packages named by patterns, rooted at dir, and returns every GraphQL document they declare.
The load itself, including the refusal of a package that did not type-check, belongs to goprogram.Load. The overlay is passed straight through: it is how a test supplies source that is not on disk, so a fixture package written in the test file itself is type-checked like any other and the folding of a document assembled from a fragment is exercised for real rather than mocked. Production passes nil.
func FromPackages ¶
FromPackages gathers the documents declared in packages that are already loaded, which is the half of the inventory that lives in Go source.
It is separate from Collect for the caller that has done the load itself: cmd/internal/actionrequests type-checks the same tree to build a call graph, for cmd/audit_readonly_graphql and for the request derivation, and loading it twice would double the seconds a gate costs for an inventory it already holds in memory. The positions come from the packages' own file set, so a caller that mixes these documents with its own walk of the same packages compares positions that mean the same thing.
A document written with holes in it is rendered once every package is walked, since the variable a template set is cloned from may be declared in a file the walk reaches after the call that parses the document. See Assembly.
func SDKDocuments ¶ added in v3.1.0
SDKDocuments reads every GraphQL document the client-go module in dir builds, folded to the string GitLab receives.
It is the same walk Collect runs over this repository, pointed at the module directory the audit already resolves for other questions, and it exists because that repository-only walk is a measured blind spot: the documents for the achievement, work item, saved view, security attribute, security category, scan profile, target branch rule and Terraform state services are assembled inside the SDK, sent through us, and judged by nothing except whichever of them a test happens to drive.
Two differences from Collect, each for a reason particular to reading a dependency.
The standalone .graphql walk is left out. The only such files client-go ships are its own test fixtures under testdata, which nothing sends; counting them would put seven documents in the inventory that no caller can reach and, since several duplicate an operation the Go source also holds, would report the same operation twice under two provenances.
A load failure is returned rather than absorbed, but every caller here reports instead of gating: this reads a module cache whose state it does not own, and a directory that has been evicted or a toolchain that refuses it is a fact about the machine and not about the server.
func Standalone ¶
Standalone gathers the documents that live in .graphql files rather than in Go constants.
A constant is the only shape this repository uses today, and it is not the only shape it may use tomorrow: moving a long document into its own file and pulling it in with an embed directive is the obvious next step for readability, and an embedded variable is not a constant, so the type checker folds nothing and the document would leave the inventory without a word. Reading the files directly closes that door before anybody walks through it.
Every failure below the root is propagated rather than skipped. A directory this cannot read is a directory whose documents go unjudged, which is the silence the whole command exists to remove; a root that is not there at all is a question about the patterns, and the package loader answers that one.
The walk goes through os.Root, as cmd/format_md_tables does, so a read is scoped to the tree being audited rather than to whatever a symlink in it points at.
type Options ¶
type Options struct {
// Dir is the repository root to audit.
Dir string
// Patterns are the load patterns; empty means [DefaultPatterns].
Patterns []string
// SchemaPath names an SDL file to judge the documents against instead of
// the pinned one. It is how the live re-probe works: cmd/gen_graphql_schema
// writes today's schema into a temporary directory and this reads it, so a
// field GitLab narrowed since the pin is reported as a failure rather than
// waiting for the next re-pin. It is also how a document meant for a
// particular self-managed release can be checked against that release.
SchemaPath string
// Schema is a schema the caller already has, judged in preference to both
// the pin and SchemaPath. It exists for the live re-probe, which
// introspects an instance itself rather than writing SDL to a file first,
// because it also has to refuse an answer too short to be a GitLab schema
// and report where the pin and that instance disagree, and both of those
// need the schema as a value rather than as a path.
Schema *ast.Schema
// Provenance names what Schema is, for the line a reader of a refusal
// needs. Required with Schema and ignored without it.
Provenance string
// Overlay supplies source that is not on disk, which is how a test hands
// the audit a fixture package instead of the repository. Production passes
// nil.
Overlay map[string][]byte
}
Options is one configured audit: where to look, what to look at, and what to judge it against.
type Refusal ¶
type Refusal struct {
// Document is the document that was refused.
Document Document
// Reasons are the schema's objections, one per line a report prints.
//
// A failure that is not a refusal at all, such as a pin that will not
// load, arrives here as its single message rather than as an empty list,
// because a finding with nothing under it reads as a document nobody could
// explain.
Reasons []string
}
Refusal is one document the schema will not accept.
type Result ¶
type Result struct {
// Provenance is the one line saying whose opinion judged these documents:
// a schema pinned on a recorded day, or one fetched today.
Provenance string
// Documents are every document the audit read, in the order a reader walks
// the repository.
Documents []Document
// Refusals are the documents the schema would not accept, a subset of
// Documents in the same order.
Refusals []Refusal
}
Result is what one audit found.
func Audit ¶
Audit reads every document under the configured patterns and judges each one.
A refused document is a finding in the result, not an error: the error return is for the audit failing to run at all, which is the case a caller must not report as a pass.
func Judge ¶ added in v3.1.0
Judge puts documents a caller already holds through the schema the options name, and is the half of Audit that does not read the tree.
It is separate for the caller that collected its documents somewhere else: the client-go documents come from a module directory rather than from a pattern under this repository, and reading them is SDKDocuments's business while judging them is exactly this. Both go through one validation so a document is refused on the same terms wherever it was found.
An empty set is ErrNoDocuments here as it is there, for the same reason: a clean exit over nothing is the silence this package exists to remove.
Source Files