Documentation
¶
Overview ¶
Package cidigest computes the CI config content digest the platform and the CLI compare to decide whether a branch may reuse the platform snapshot's resolved config: a content-keyed (not branch-keyed) identity for a project's CI config, made of the root file plus every LOCAL include statically reachable from it.
A branch that does not touch its CI config digests equal to the snapshot's resolution anchor and evaluates against the snapshot's merged_yaml with no extra call; a branch that does change it digests differently and gets its own resolution.
This package is a deliberate BYTE-IDENTICAL port of the platform's platform/backend/cidigest (monorepo, ADR-0034). The digest is compared across the two implementations, so its exact byte construction (see Compute) and its traversal rules (see Traverse) must never diverge under Version "1". Any change to the construction is a new digest_version; versions never mix in one comparison. The golden vectors in digest_test.go are the shared pin: they were computed independently of either implementation and both must reproduce them.
The package is pure: no network, no API client. Compute hashes a file set it is handed; Traverse discovers that file set through a caller-supplied fetch function. FetchFromDir supplies the CLI's own file access, reading the job checkout from disk.
Index ¶
- Constants
- Variables
- func AbortReason(err error) string
- func Compute(files map[string][]byte) string
- func ComputeForCheckout(dir, ciConfigPath string) (string, error)
- func FetchFromDir(dir string) func(string) ([]byte, error)
- func RootPath(ciConfigPath string) (string, error)
- func Traverse(root string, fetch func(path string) ([]byte, error)) (map[string][]byte, error)
Constants ¶
const DefaultRootPath = ".gitlab-ci.yml"
DefaultRootPath is the CI config file GitLab uses when a project sets no ci_config_path.
const MaxFiles = 50
MaxFiles bounds the number of distinct files a single Traverse call will visit, root included ("capped at 50 files"). It exists to keep a maliciously deep or wide local-include graph from turning one digest computation into an unbounded number of reads. Traversal is breadth-first in include-list order, so which files fall inside the cap is deterministic across identical inputs (a requirement of the wire-stable digest), not an artifact of map iteration or scheduling.
Exceeding this cap ABORTS the whole computation (see ErrTooManyFiles) rather than silently stopping discovery and returning what was found so far. The omission rule would be a false-verdict path: two configs identical in their first 50 traversal files but differing beyond the cap would compare digest-equal.
const Version = "1"
Version is the digest_version paired with every digest this package computes ("digest_version: 1"). A caller comparing digests must also compare Version; digests computed under different versions are not comparable even if they happen to be equal-length hex strings.
Variables ¶
var Absent = []byte("cidigest:absent")
Absent is the sentinel value for a files map entry whose content could not be obtained: the referenced local include does not exist, or (for the root itself) the CI config file is missing on this ref. Traverse assigns exactly this value, by reference, for such entries.
Compute recognizes Absent by IDENTITY (same backing array), not by content: isAbsent compares the address of the first byte, not the bytes themselves. This means a real file whose content happens to consist of the exact same bytes as Absent, but was independently allocated (a fresh []byte, a copy, a different string literal), is still digested as present content, not as ABSENT. Callers building a files map by hand for an absent entry must assign this exact value (files[path] = Absent), not a copy of it.
var ErrExternalRootConfig = errors.New("cidigest: ci_config_path points at another project")
ErrExternalRootConfig reports that the project's ci_config_path points at a file in ANOTHER project ("path/file.yml@group/project"). The root of the include graph is then not in this checkout at all, so no digest over the checkout can describe what GitLab would merge. Callers treat it like any other abort: no digest, always-divergent, resolve through the platform.
var ErrNotFound = errors.New("cidigest: file not found")
ErrNotFound is the sentinel a fetch function passed to Traverse must return for a referenced local include (or the root) that genuinely does not exist. Traverse treats this, and only this, as the "does not exist" case: the path is recorded as Absent and traversal continues.
Any other error is treated as an infra failure, not a content fact. That distinction is load-bearing: a read error, a permission failure or a transient provider error must never be silently folded into ABSENT, because that would produce a valid-looking digest over data that was never actually observed. Traverse aborts on any error other than ErrNotFound.
Fetch implementations should use errors.Is / wrap so errors.Is(err, ErrNotFound) is true for a genuine not-found; Traverse checks with errors.Is, not equality, so a wrapped ErrNotFound is still recognized.
var ErrTooManyFiles = errors.New("cidigest: traversal exceeds the file cap")
ErrTooManyFiles is the sentinel Traverse's returned error wraps when traversal would visit more than MaxFiles distinct files. This is deliberately an ABORT, not a truncation: a digest computed over only the first MaxFiles files would compare equal for two configs that are identical up to the cap but differ beyond it, silently evaluating merged-yaml-dependent controls against the wrong config on exactly the divergent-branch case this digest exists to protect.
Callers must treat this exactly like any other Traverse abort: no digest is produced, and a MISSING digest is treated as ALWAYS-DIVERGENT rather than trusted as a valid comparison key. On the wire that means omitting config_digest and digest_version from the resolve request.
Functions ¶
func AbortReason ¶
AbortReason classifies a non-nil error returned by Traverse into a fixed two-value vocabulary for diagnostics: "overflow" when err wraps ErrTooManyFiles, and "read_failure" for any other abort (a fetch failure other than ErrNotFound - Traverse never returns ErrNotFound itself as an abort, see its doc comment). Defined once here so every computation site classifies an abort the same way.
func Compute ¶
Compute returns the hex-encoded digest_v1 of files:
sha256(
"plumber-ci-digest/v1\n"
+ for each path, in byte-wise sorted order:
path + "\x00" + hex(sha256(content)) + "\x00"
)
with the ABSENT marker substituted for the hex hash on any entry whose content is the Absent sentinel.
files is the complete file set to digest: normally the map Traverse returns (the root CI config plus every reachable local include). Compute itself does not traverse or interpret includes; it only hashes what it is given. Go's map iteration order is randomized, which is exactly why the byte-wise sort below exists: without it, the same file set could hash differently from one call to the next, breaking the wire-stability the digest exists for.
func ComputeForCheckout ¶
ComputeForCheckout is the CLI's whole digest computation: resolve the traversal root from ciConfigPath, walk the local includes through the checkout at dir, and hash the resulting file set.
It returns ("", err) for every case that must NOT produce a digest - an external root config, a traversal that overflowed MaxFiles, or a read failure - and callers must treat a missing digest as ALWAYS-DIVERGENT rather than as a comparison key. A missing root file is not one of those cases: it is an honest ABSENT entry and still yields a digest, which is what lets two branches that both lack a CI config compare equal.
func FetchFromDir ¶
FetchFromDir returns a Traverse fetch function that reads repo-relative paths from the checkout rooted at dir - the CLI's side of the shared digest, where the platform reads the same paths through the git host's API.
The semantics each case must match on both sides:
- A path that is not a clean repo-relative path - absolute, or still escaping the root with "../" after normalization - is NEVER resolved against the filesystem and returns ErrNotFound, so it contributes ABSENT. The git host answers 404 for the same paths (its file API addresses repo-relative paths only), so both sides agree on the file set. Refusing here is also what keeps a hostile include out of the surrounding filesystem.
- A symlink is read as its TARGET PATH, not as the file it points at. That is exactly what the git host serves: git stores a symlink as a blob whose content is the target path, so the raw-file API returns that string. Reading through the link would both diverge from the platform and let an include reach outside the checkout.
- A directory returns ErrNotFound, matching the git host's 404 for a path that is not a blob.
- A missing file returns ErrNotFound: an honest ABSENT.
- Any other read failure is returned as-is and ABORTS the traversal. See maxFileBytes for why an over-cap file is a failure, not an absence.
func RootPath ¶
RootPath resolves the traversal root from a project's ci_config_path setting: the setting itself when set, DefaultRootPath otherwise.
GitLab's ci_config_path has three forms. A plain repo-relative path is the normal case. A path carrying a "@group/project" suffix names a config in a DIFFERENT project - the include graph's root is not in this checkout, so RootPath returns ErrExternalRootConfig rather than digesting a file that is not the real root. A "?ref=" query suffix pins a ref on such an external config and only ever appears alongside the "@" form, so it is covered by the same check.
func Traverse ¶
Traverse performs the static local-include scan starting at root: it recursively follows every `include: local` entry (the map form, the bare-string shorthand, and both forms mixed into an array), using fetch to obtain each file's content. Remote, template, component and cross-project include entries are deliberately NOT followed: their content is not part of the digest; only their textual reference in the including file is, and that reference is already part of the including file's own content once hashed.
The scan is cycle-safe: a path already visited is never re-fetched or re-queued, so an include cycle (A includes B, B includes A) terminates.
If traversal would need to visit a distinct, not-yet-visited path after MaxFiles have already been recorded, Traverse ABORTS the whole computation and returns (nil, err) with err wrapping ErrTooManyFiles - it never returns a truncated, first-MaxFiles-only file map.
fetch's error return is part of the contract, not incidental: return ErrNotFound (or an error that wraps it, checked with errors.Is) for a path that genuinely does not exist, including the degenerate case where root itself is missing; Traverse records that path as Absent and continues. Any OTHER error aborts the whole traversal immediately: Traverse returns (nil, err) with err wrapping the failing path, so an infra failure never produces a digest that looks valid but was computed over incomplete data.
The returned map, when err is nil, uses the same path keys fetch is called with and is suitable as-is for Compute.
Types ¶
This section is empty.
Source Files