compiler

package
v0.1.5-alpha Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 24, 2026 License: AGPL-3.0 Imports: 16 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func AggregateManifestDigest 

func AggregateManifestDigest(blobs []BlobRef) [32]byte

AggregateManifestDigest computes the manifest digest from a path-sorted blob list. Encoding per blob: 

u64-BE(len(path)) || path || digest[32]

Length-prefixing the path keeps concatenation unambiguous (two distinct (path, digest) sequences cannot collide via boundary shifting). Big-endian to match the convention in Invocation.CacheKey.

Path bytes are ASCII-lowercased before mixing in so case-insensitive filesystems (Windows NTFS, macOS APFS by default) produce the same manifest digest as case-sensitive ones (Linux ext4, APFS case- sensitive). A Windows client whose `/showIncludes` emits `src\foo.h` and a Linux client whose `gcc -M` emits `src/Foo.h` pointing at the byte-identical file will compute the same key and cross-developer-hit. BlobRef.Path stays original-case for the worker — materialization preserves what the compile expects to open (`Foo.h`, not `foo.h`), so case-sensitive Linux projects with genuinely distinct `Foo.h` and `foo.h` files keep working: both blobs go into the manifest with different content digests, so the aggregate digest still distinguishes the "has both" case from the "has only one" case. Those projects just can't cross-platform-hit a Windows version of themselves (which makes sense, since Windows can't represent both files anyway).

Callers must sort blobs by Path before calling. Exported so the worker can re-verify a client-supplied manifest using the exact same algorithm BuildManifest uses on the client side; drift between the two would silently break cross-developer cache hits.

func CacheKeyFlagsBytes 

func CacheKeyFlagsBytes(inv *Invocation) []byte

CacheKeyFlagsBytes returns the canonical byte encoding of the cache-key-relevant flags for inv — the same bytes the cache key's flags chunk is built from. Exported for the explain package, which hashes them separately so a "flags changed" diff is recoverable from the prior compile's record. Production code outside that callsite should stick with CacheKey/ComputeHash; this returns the raw bytes, not a hash.

func CollectDepEmissionExtras 

func CollectDepEmissionExtras(inv *Invocation) map[string][]byte

CollectDepEmissionExtras reads the dep-emission output files the compiler just produced (one per path returned by ExtractDepEmissionPaths) off disk, returning them keyed by their as-spelled argv path. Used by every callsite that runs the user's compiler with `-Wp,-MMD,…` / `-MF` still in argv (i.e. expects the .d files on disk afterwards) and wants to round-trip them through the cache so warm hits replay the same .d files the cold compile produced.

Returns nil if argv carries no dep-emission flags. Returns an (otherwise non-nil) map even when individual files are missing — a flag like `-Wp,-MMD,<path>` may produce no output for a source with no #includes; that's not an error, just nothing to cache for that entry.

Relative paths are resolved against inv.Cwd. Read errors other than ENOENT are silently skipped on the same theory: an extras blob is best-effort, the primary artifact is the contract.

func ExtractDepEmissionPaths 

func ExtractDepEmissionPaths(args []string) []string

ExtractDepEmissionPaths returns the list of dep-file output paths the argv asks the compiler to write — i.e. the `<path>` in any `-Wp,-MMD,<path>` / `-Wp,-MD,<path>` / `-Wp,-MF,<path>` form, plus the separate `-MF <path>` form. The caller treats these as expected side-effect outputs to capture after running the compiler (PREPROCESSED dispatch) or expects them back from the worker (CAS, via RewriteDepEmissionForCAS).

Pure read; doesn't modify args. Paths are returned in argv order and as-spelled (no normalization), so the caller can use them as keys/paths against the same cwd the compiler ran in.

func FindProjectRoot 

func FindProjectRoot(startDir string) string

FindProjectRoot walks up the directory tree starting at startDir, returning the first directory that contains a `.hpcc` marker file. Returns empty string if no marker is found before reaching the filesystem root.

Errors are swallowed silently — discovery is best-effort and the caller falls back to absolute paths. A non-existent or unreadable path is treated the same as "no marker found": cross-developer cache hits don't fire until the user adds the marker, but nothing breaks.

func HasUncapturedSideEffectFlag 

func HasUncapturedSideEffectFlag(args []string) bool

HasUncapturedSideEffectFlag reports whether argv carries a flag known to produce output files alongside the primary -o artifact that the dispatch/cache layer can't currently round-trip. Hitting any of these short-circuits Cacheable / DispatchableUnderCAS to false so the user's invocation falls through to a direct local Compiler.Invoke — they get exactly the files they asked for, just without remote dispatch or caching for that TU.

The list is curated, not exhaustive: gcc has dozens of dump/trace flags and chasing them generically (parse every gcc option, know which write files) would be brittle across compiler versions. We catch the categories that actually surface in real builds and leave room to add more as they're reported. Common-case dep emission (-MD/-MMD/-MF) is handled by RewriteDepEmissionForCAS + the CompileResponse.extra_outputs pipeline, not by this opt-out.

Recognised:

  • -save-temps [=cwd|=obj]: dumps .i, .s, .o intermediates
  • -fdump-*: every -fdump-tree-/rtl-/ipa-/passes/etc. variant
  • -fcallgraph-info [=…]: writes <output>.ci
  • -fprofile-generate [=…], -fprofile-arcs, -ftest-coverage, --coverage: gcov instrumentation writes .gcno at compile time
  • -gsplit-dwarf: writes <output>.dwo alongside the .o
  • -fdiagnostics-format=sarif-file / =json-file: writes a structured diagnostics file

func InjectReproducibilityFlags 

func InjectReproducibilityFlags(args []string, family enum.Family, srcRoot string) []string

InjectReproducibilityFlags appends compiler-family-specific flags that make `.obj` / `.pdb` output bytes deterministic across Execs. Without these the worker stages each compile under a per-Exec subdirectory (`C:\src\<exec-id>\...` or `/src/<exec-id>/...`) and the compiler embeds that path into its output — two Execs of the same source produce byte-different outputs, defeating reproducibility audits and content-addressed output caching.

MSVC: `/d1trimfile:<srcRoot>` strips srcRoot from embedded source paths in the `.obj`. `/PDBSourcePath:<srcRoot>` does the same for the PDB symbol records.

GNU / Clang: `-ffile-prefix-map=<srcRoot>=.` rewrites embedded paths so `/src/<exec>/foo.c` becomes `./foo.c` in `.o` debug info and `__FILE__` expansions. `-Werror=date-time` makes any use of `__TIME__` / `__DATE__` a build error rather than baking a wall-clock timestamp in.

srcRoot is the platform-neutral `/src` token; the worker's runtime translator (runtime.rewriteRoot) rewrites it to the concrete per-Exec staging dir at Exec time, so by the time the compiler sees the flag the prefix is fully resolved. Both `/` and `=` are path boundaries the translator recognises, so the trailing `=.` in the GNU form doesn't block the substitution.

Append-only, never duplicates: MSVC accepts `/d1trimfile:` multiple times and applies each prefix in order; GCC processes multiple `-ffile-prefix-map` left-to-right with each pair applying independently. So a user-supplied prefix-map for their project root composes cleanly with our staging-dir strip.

func RewriteDepEmissionForCAS 

func RewriteDepEmissionForCAS(args []string, outRoot string) (newArgs []string, paths []string)

RewriteDepEmissionForCAS rewrites every dep-emission path in args to live under outRoot inside the container, returning the new argv and the list of /out-relative paths the worker will produce. The client uses the second return to know which side-effect files to expect back in CompileResponse.extra_outputs and where to write them on the host (joined against inv.Cwd, which has the same relative structure).

Rewrites two forms:

  • `-Wp,-MMD,<path>` and the rest of the `-Wp,-M*,<path>` family (the kernel build uses these on every TU).
  • Separate-form `-MF <path>` (the standalone version of the same).

Bare `-MD` / `-MMD` (no explicit path) is intentionally not handled — gcc derives the default path from `-o`, and the rules are platform-specific (`<output_basename>.d` in the same dir). Easy to add when something asks for it.

func SetExecutor 

func SetExecutor(c Compiler, e Executor)

SetExecutor configures the Executor for compilers returned by Detect. The wrapper/daemon path leaves the default LocalExecutor in place; the worker side calls this with a Task.Exec-backed executor before dispatching a compile. No-op for compilers not built by this package.

func ValidateNoHostPaths 

func ValidateNoHostPaths(args []string) error

ValidateNoHostPaths returns an error if any argv element contains a substring that looks like a developer-machine path. Intended to run on the worker side as a sanity check after the client should have rewritten paths via RewritePathPrefix or RewriteForPreprocessed.

Catches the common failure modes:

  • Client forgot to call the rewrite for the chosen source mode.
  • Client rewrote some paths but missed flags (e.g. -isystem on the separate-value form).
  • Misconfigured client sent raw argv straight from a CMake build dir.

False-positive case: a user truly has a system dir literally named "Users" (very rare on Linux) — they can't run hpcc anyway. False negatives: paths under /opt, /var/ci-runner, /scratch — accepted on the assumption that those rarely originate on the dev side.

Types

type BlobRef 

type BlobRef struct {
	Path   string   // path as seen by the compiler; not normalized here
	Digest [32]byte // BLAKE3-256 of the file's bytes
	Size   int64    // file size in bytes; advisory (worker re-checks on fetch)
}

BlobRef names a single file in a source-closure manifest by path and content digest. CAS-mode dispatch (docs/plan/phase-4-distributed.md §4.5 / docs/plan/cas.md) ships these to the worker; the worker materializes the closure inside the VM by fetching each blob by digest and writing it to Path.

type CacheBackend 

type CacheBackend interface {
	Lookup(inv *Invocation, tenantID string) (*InvocationResult, error)
	Store(inv *Invocation, res *InvocationResult, tenantID string) error
}

CacheBackend is the structural shape of a cache facade. It lives in the compiler package — rather than importing the cache package directly — because the cache package needs to reference Invocation/InvocationResult, and pulling cache into compiler would form an import cycle. Any concrete cache (e.g. cache.CompileCache) satisfies this interface via duck typing.

type Category 

type Category int

Category is what kind of flag this is, semantically. The parser uses it to decide which Invocation field to populate. 

const (
	CatUnknown       Category = iota
	CatMode                   // -c, -E, -S, -M, -MM
	CatOutput                 // -o
	CatInclude                // -I
	CatSystemInclude          // -isystem, -iquote
	CatDefine                 // -D
	CatUndefine               // -U
	CatLibrary                // -l
	CatLibraryDir             // -L
	CatStandard               // -std=
	CatOptim                  // -O
	CatDebug                  // -g
	CatWarning                // -W
	CatFeature                // -f
	CatMachine                // -m
	CatLanguage               // -x
	CatPassthrough            // -Wl,..., -Xlinker, -include, -MD, etc.
)

type Compiler 

type Compiler interface {
	Name() string                                          // "gcc", "clang"
	Family() enum.Family                                   // GNU | MSVC
	Parse(args []string) (*Invocation, error)              // argv -> structured form
	Preprocess(inv *Invocation) (*PreprocessResult, error) // run -E, capture source+digest+stderr
	Invoke(inv *Invocation) (*InvocationResult, error)     // run the actual compile
	FindDependencies(inv *Invocation) ([]string, error)    // run -M, capture dependency file list
	Identity() ([]byte, error)                             // path + binary hash, for cache key

	// RewriteForPreprocessed returns a copy of inv whose argv compiles
	// the preprocessed source at srcPath. Includes, defines, and other
	// preprocess-only flags are stripped (preprocessing already inlined
	// them); the input file is replaced with srcPath; -x cpp-output (or
	// the C++ equivalent) is added so the compiler skips its own
	// preprocessor. Used client-side when shipping a PreprocessedDescriptor.
	RewriteForPreprocessed(inv *Invocation, srcPath string) (*Invocation, error)
}

Compiler

func Detect 

func Detect(argv0 string) (Compiler, error)

Detect picks a Compiler implementation based on argv[0]. The argument can be a bare command ("clang"), a relative path ("./clang.exe"), or absolute ("/usr/bin/clang") — only the basename (minus a ".exe" suffix) is used to dispatch.

The returned Compiler stores argv0 verbatim. Resolution to a real binary path happens lazily in Invoke/Preprocess/Identity so callers (and tests) don't need the toolchain installed just to parse args.

type Context 

type Context struct {
	Config   *config.Config
	Compiler Compiler
	Cache    CacheBackend

	// IdentityOverride, when non-nil, replaces Compiler.Identity() in
	// cache-key derivation. Worker-side compiles set this to the image
	// digest of the toolchain container — the toolchain isn't on the
	// worker host's filesystem (Compiler.Identity reads ./clang etc.),
	// and the image digest is what actually pins the toolchain version
	// for cache-key purposes anyway.
	IdentityOverride []byte
}

Context bundles everything the runner / cache layer needs about a single invocation: which compiler is being wrapped, the loaded config that controls cache/dispatch behavior, and the cache facade.

type Executor 

type Executor interface {
	// cwd, when non-empty, is the working directory the spawned process
	// should run from. Empty means inherit the parent's cwd. The
	// runtime/in-VM executor ignores cwd because the FC VM has its own
	// fixed staging paths, but the daemon-side LocalExecutor needs it
	// so spawned compilers see relative `-Iinclude` / `-MF foo.d`
	// arguments resolve the same way they would if the user had run
	// the compiler directly.
	Run(bin string, args, extraEnv []string, cwd string) (stdout, stderr []byte, exitCode int, err error)

	// ReadOutput returns the bytes of a file produced by a previous Run.
	// Local: os.ReadFile from the host fs. Task.Exec: read from the
	// host-side mount of the container's output volume (§4.4 /out).
	ReadOutput(path string) ([]byte, error)
}

Executor abstracts how a compiler invocation is dispatched. The host- local path uses os/exec; the worker-side path will use containerd's Task.Exec to run the same argv inside a per-tenant microVM. Both shapes return stdout, stderr, and exit code as data — a non-nil err means hpcc itself couldn't dispatch (binary missing, containerd unreachable, etc.); a non-zero exit code is normal cacheable data.

type FlagSpec 

type FlagSpec struct {
	Name     string
	Value    ValueMode
	Category Category
}

type Invocation 

type Invocation struct {
	Mode enum.InvocationMode

	Inputs []string // positional input files (.c/.cpp/.o/.a/...)
	Output string   // -o value, if any

	Includes       []string          // -I
	SystemIncludes []string          // -isystem, -iquote
	Defines        map[string]string // -D NAME[=VALUE]
	Undefines      []string          // -U
	Libraries      []string          // -l
	LibraryDirs    []string          // -L
	Std            string            // -std=...
	Optim          string            // -O...
	Debug          string            // -g... ("" if absent, "0"/"2"/etc otherwise)
	Warnings       []string          // -W... (e.g. "all", "no-foo")
	Features       []string          // -f... (e.g. "PIC", "no-rtti")
	Machine        []string          // -m... (e.g. "avx2", "arch=x86-64")
	Language       string            // -x value (forced language)

	// Passthrough holds flags we recognize but do nothing structured with —
	// linker/assembler/preprocessor passthrough, dep-info flags, -include, etc.
	Passthrough []string

	// Unknown holds flags that started with "-" but didn't match any spec.
	// They must be re-emitted verbatim when invoking the real compiler.
	Unknown []string

	// RawArgs is the argv we were given, after @file expansion.
	RawArgs []string

	// Cwd is the working directory the compiler should run from.
	// Empty means "inherit hpcc's own cwd" — correct for the in-process
	// `hpcc wrap` path (the wrapper is invoked from the right place by
	// make/etc.). The daemon sets this to the client's cwd so spawned
	// gcc/cl invocations resolve joined-form `-Iinclude`, auto-derived
	// depfile paths, and other relative arguments the way they would
	// if the user had run the compiler directly. NOT part of CacheKey
	// — only file content matters, not where it was compiled from.
	Cwd string

	// PreprocessedDigest, if non-nil, is the BLAKE3-256 of the
	// already-preprocessed source bytes. CacheKey treats it as a
	// substitute for running the preprocessor: skip FindDependencies,
	// skip Preprocess, mix this digest in directly. Set by the worker
	// for PREPROCESSED-mode requests where the client shipped
	// preprocessed bytes inline; nil otherwise.
	PreprocessedDigest *[32]byte

	// ManifestDigest, if non-nil, is the BLAKE3-256 of a source-closure
	// manifest (see Manifest.Digest in manifest.go). CacheKey treats it
	// as a substitute for running the preprocessor or walking the dep
	// closure locally: skip FindDependencies, skip Preprocess, mix this
	// digest in directly. Set by the worker for CAS-mode requests where
	// the client shipped a CasDescriptor; nil otherwise. Takes priority
	// over PreprocessedDigest when both are set (CAS-mode requests
	// can't also carry preprocessed bytes, but the precedence keeps
	// the contract explicit).
	ManifestDigest *[32]byte
}

Invocation is the parsed form of one compile command. It is pure data, produced by a parser and consumed by a Compiler implementation.

func NewInvocation 

func NewInvocation() *Invocation

NewInvocation returns an Invocation with maps initialized.

func ParseGNU 

func ParseGNU(args []string) (*Invocation, error)

ParseGNU parses a GCC/Clang-style argv into an Invocation.

The parser is lenient: anything that starts with "-" but doesn't match a known flag is recorded in Invocation.Unknown so we can re-emit it verbatim. Anything that doesn't start with "-" (or "-" alone) is treated as an input file.

"@file" response files are expanded recursively before parsing. Tokenization inside a response file is whitespace-only — backslash escapes and quoted strings are not honored. That covers ~all real-world build systems; if it ever bites us we can do proper shell-lexing then.

func ParseMSVC 

func ParseMSVC(args []string) (*Invocation, error)

ParseMSVC parses a cl.exe-style argv into an Invocation. Same Invocation shape as ParseGNU; only the grammar differs.

func RewriteForCAS 

func RewriteForCAS(inv *Invocation, projectRoot, srcRoot, outRoot string) *Invocation

RewriteForCAS prepares an Invocation for CAS-mode dispatch. It rewrites:

  • All paths matching projectRoot to live under srcRoot ("/src" inside the container). System paths (e.g. /usr/include/...) stay absolute.
  • The output to outRoot + "/" + basename(output) ("/out/<base>"), so the worker can collect the artifact from a known location. The -o flag in RawArgs is patched in-place to match.

Unlike RewriteForPreprocessed, this does NOT drop -I/-D — CAS-mode compiles still run the preprocessor on the worker side, so include paths and macro defines have to ride along. The path rewrite handles the in-container locations.

projectRoot must be absolute and stripped of any trailing slash; pass it as returned by FindProjectRoot. srcRoot and outRoot are the in-container mount points (typically "/src" and "/out").

func RewritePathPrefix 

func RewritePathPrefix(inv *Invocation, hostPrefix, vmPrefix string) *Invocation

RewritePathPrefix returns a shallow copy of inv with every host-side path prefix replaced by the in-container prefix. Used client-side when shipping a CAS-mode CompileRequest: argv references like `-I/home/alice/proj/include` become `-I/src/include` (with hostPrefix = "/home/alice/proj", vmPrefix = "/src").

The substitution is a substring replace on each argv entry, not a flag-aware rewrite — so it handles joined forms (-I<path>), separate forms (-isystem <path>), and bare positional inputs uniformly. The trade-off is false positives in the rare case where a non-path flag value happens to contain hostPrefix (e.g. -DPATH=/home/alice/proj/x). Callers should keep their hostPrefix specific enough that a stray match is unlikely.

Trailing slashes on either prefix are normalized away. An exact boundary match (host path with no trailing component) also rewrites, so a positional input that is itself the project root maps cleanly.

func (*Invocation) CacheKey 

func (inv *Invocation) CacheKey(ctx *Context) ([]byte, error)

CacheKey computes the cache-key seed for this invocation: a 32-byte BLAKE3-256 digest mixing source content, compiler identity, and the cache-key-relevant flags.

Source-content input is one of (in priority order):

  • ManifestDigest: a precomputed source-closure manifest digest. The worker uses this on CAS-mode requests after re-verifying the client-supplied manifest. Mixed in directly; no preprocessor or dep walk runs.
  • PreprocessedDigest: the BLAKE3 of preprocessed source bytes the caller already produced. The worker uses this on PREPROCESSED-mode requests. Mixed in directly.
  • Config.SourceMode = SourceModeCAS (default): run FindDependencies locally, hash each input/dep into a Manifest (manifest.go), mix the manifest digest in. Same encoding as the ManifestDigest short-circuit, so a CAS-mode worker and a local-mode daemon compute identical keys for the same TU.
  • Config.SourceMode = SourceModePreprocessed: run the preprocessor locally and hash the resulting source bytes (via the digest already computed in PreprocessResult — single pass over the bytes, not two). Pairs with PREPROCESSED dispatch on the wire.

Each chunk written to the hasher is length-prefixed so concatenation can't collide ("ab"+"c" hashes differently from "a"+"bc").

func (*Invocation) CacheKeyWithManifest 

func (inv *Invocation) CacheKeyWithManifest(ctx *Context) ([]byte, *Manifest, error)

CacheKeyWithManifest is CacheKey, but also returns the source-closure Manifest when one was built locally as part of cache-key computation (i.e. the CAS-default branch). The returned manifest is nil for every other path: the worker's ManifestDigest / PreprocessedDigest short-circuits, and the PREPROCESSED-mode local path.

Surfaced for callers (today: the daemon's `hpcc explain` recorder) that want the per-file digests the manifest already has, without paying a second BuildManifest pass.

func (*Invocation) Cacheable 

func (inv *Invocation) Cacheable() bool

Cacheable reports whether this invocation can be served from / recorded into the hpcc V1 cache. The callers (runner.Run and the daemon's compile handler) gate cache lookup and store on it so we never produce or consume entries that the cache shape can't represent correctly.

False for:

  • Non-compile modes (link/preprocess/dep-only/assemble). Link inputs are .o/.a files that depend on too much external state to key on source hash; the rest are either cheap or write to stdout/temp files that don't fit a single-output cache entry.

  • Stdin-source compiles ("-" anywhere in Inputs, or the /dev/stdin synonym). The cache key incorporates preprocessed bytes via Preprocess(), which consumes stdin — by the time Invoke runs the real compile, the FD is at EOF. Caching stdin compiles would also be unsound: the bytes the key captures are the bytes we couldn't read again, so any later invocation with the same argv but different stdin content would have no way to detect the mismatch.

  • Multi-input compiles. gcc/cl write one object per input; the CompileCache shape stores one output blob per entry, so caching here would silently drop all but one .o.

All three of these paths fall through to a direct Compiler.Invoke, which is correct: the wrapper passes the user's argv to the real compiler verbatim, the compiler writes whatever files it would write without hpcc in the picture, and the user sees normal behavior.

func (*Invocation) ComputeHash 

func (inv *Invocation) ComputeHash(ctx *Context) (string, error)

ComputeHash runs the preprocessor and returns a hex-encoded BLAKE3-256 digest of the preprocessed source. The digest is computed during preprocessing (PreprocessResult.Digest) so this is a single pass over the bytes, not two.

func (*Invocation) ComputeHashWithManifest 

func (inv *Invocation) ComputeHashWithManifest(ctx *Context) (string, *Manifest, error)

ComputeHashWithManifest is ComputeHash plus the source-closure Manifest built during cache-key computation, or nil when none was built (worker shortcut paths, PREPROCESSED-mode local path).

func (*Invocation) DispatchableUnderCAS 

func (inv *Invocation) DispatchableUnderCAS() bool

DispatchableUnderCAS reports whether this invocation can be remotely dispatched in CAS mode. Same shape rules as Cacheable — single-input compile, no stdin, no /dev/null probe — but accepts assembly inputs because CAS ships the full source closure to the worker, so .incbin'd files are present at assemble time.

Callers should reach for this in the daemon's dispatch gate when the configured source mode is CAS. The local-cache check (Cacheable) and the dispatch gate are intentionally separate: the local cache key for a .S input is still unsound (it would hash preprocessed text without the .incbin'd bytes), so the daemon dispatches without local caching when only this check returns true.

func (*Invocation) ReadsStdin 

func (inv *Invocation) ReadsStdin() bool

ReadsStdin reports whether the invocation reads its source from the wrapper's stdin (`-` or `/dev/stdin` as an input). The runner uses this to keep stdin-reading compiles on the in-process path — the daemon protocol carries parsed argv but no stdin bytes, so a daemon-dispatched stdin invocation would silently see an empty file. The clearest symptom is the Linux kernel's `scripts/cc-version.sh` probe (`gcc -E -P -x c -` with the C source heredoc'd in) producing empty preprocessor output and the build aborting with "unknown C compiler."

type InvocationResult 

type InvocationResult struct {
	Output   []byte
	Stdout   []byte
	Stderr   []byte
	ExitCode int
	Duration time.Duration
	Err      error
	Extras   map[string][]byte
}

InvocationResult is the captured output of a single compile run. It is what the cache stores on a miss and replays on a hit; the duration is recorded for metadata only and is not part of the cache key.

Extras carries side-effect output files the compile produced besides the primary -o artifact — typically .d dep files written by -Wp,-MMD,<path> for incremental-build dep tracking. Keyed by path (relative to the per-RPC output staging dir). The cache stores it alongside Output so warm hits replay the same dep files the cold compile produced; without that, deleting a .d on disk would leave `make` re-firing the rule forever on cache-hit responses that returned the .o but no .d.

type LocalExecutor 

type LocalExecutor struct{}

LocalExecutor runs binaries on the host with os/exec and reads files from the host filesystem. This is the default for the wrapper / daemon path; the worker swaps in a containerd-backed executor.

func (LocalExecutor) ReadOutput 

func (LocalExecutor) ReadOutput(path string) ([]byte, error)

func (LocalExecutor) Run 

func (LocalExecutor) Run(bin string, args, extraEnv []string, cwd string) (stdout, stderr []byte, exitCode int, err error)

type Manifest 

type Manifest struct {
	Digest [32]byte
	Blobs  []BlobRef
}

Manifest is the source-closure summary for a single compile. Digest is a stable function of the (path, content-digest) pairs of every blob; two Manifests with the same set of (path, content) pairs produce the same Digest regardless of FindDependencies' output ordering, because Blobs is sorted by Path before aggregation.

func BuildManifest 

func BuildManifest(inv *Invocation, ctx *Context) (*Manifest, error)

BuildManifest runs the compiler's dependency discovery, hashes every file in the closure (inputs + headers), and returns a Manifest. The manifest digest is what CAS-mode cache keys mix in instead of preprocessed source bytes.

Path handling: if `inv.Cwd` is set and a `.hpcc` marker file is found by walking up from it (see `FindProjectRoot`), paths under that root are rewritten relative to the root before being mixed into the manifest digest. Paths outside the project tree (typically toolchain headers under `/usr/include`, `/opt/...`) stay absolute. Files keyed by content digest hash the same bytes either way — only `BlobRef.Path` differs.

No marker → no normalization → absolute paths. Cross-developer cache hits don't fire in that case, but the same-developer behaviour is preserved.

type PreprocessResult 

type PreprocessResult struct {
	Source   []byte
	Digest   [32]byte
	Stderr   []byte
	ExitCode int
}

PreprocessResult is everything downstream needs after the preprocessor runs: the preprocessed source bytes, their BLAKE3-256 digest (so the hasher doesn't recompute it), the compiler's stderr, and its exit code.

A non-zero ExitCode is NOT returned as a Go error — the preprocessor failing (e.g. missing header, syntax error) is a normal cacheable outcome, not an hpcc-side error. A non-nil error means hpcc itself couldn't run the binary (path missing, exec failure).

type ValueMode 

type ValueMode int

ValueMode describes how a flag carries its value (if any). 

const (
	NoValue          ValueMode = iota // -c, -E (pure switch)
	JoinedValue                       // -O2, -Wall, -std=c++17 (value glued on)
	SeparateValue                     // -o foo, -isystem /path (value is next arg)
	JoinedOrSeparate                  // -I (accepts either form)
)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.