Documentation
¶
Overview ¶
Package explain records per-compile outcome metadata so `hpcc explain <source>` can answer "why did my last compile not hit the cache." Phase 5 §5.3.
On every compile the daemon writes a Record (one JSON file under the user's cache dir, keyed by sha256 of the source's abspath) describing the inputs the cache key was built from: compiler identity, canonical flags, source-content hash, per-header content hashes, image digest (if dispatched remotely), and the outcome (local_hit, remote, local_invoke, bypass, error).
On a later compile of the same source path, comparing the new inputs against the prior record yields a structured Diff that names exactly what changed — "this header's bytes changed," "this flag was added," "the compiler binary was rebuilt." That diff is the user-facing payload `hpcc explain` prints.
The Record schema is intentionally a superset of what the cache key cares about: a hit doesn't need any of this, but the *next* miss needs the hit's record to have something to compare against. So we write on every compile, not just on misses.
Index ¶
Constants ¶
const DefaultMaxEntries = 10000
DefaultMaxEntries caps the on-disk record count. Each record is a few KB; 10k records ≈ tens of MB worst case. Matches the "10k records" choice in docs/plan/phase-5-observability.md §5.3.
Variables ¶
This section is empty.
Functions ¶
func DefaultDir ¶
DefaultDir returns the user's standard explain-store location: `<os.UserCacheDir>/hpcc/explain`. The directory is not created here — NewDiskStore mkdirs on first Put.
func HashBytes ¶
HashBytes returns hex-encoded SHA-256(data). Helper so call sites don't reach into crypto/sha256 directly.
func HashFile ¶
HashFile reads p and returns its hex SHA-256. Returns "" + an error the caller is expected to log and move past — explain is best-effort; failing here must not break the compile.
func HashSourcePath ¶
HashSourcePath returns the on-disk record file name for sourcePath. Exported so the CLI can resolve the file without instantiating a Store (useful for `hpcc explain --record-path foo.c`-style diagnostics if we ever add one).
Types ¶
type Diff ¶
type Diff struct {
Kind DiffKind `json:"kind"`
Path string `json:"path,omitempty"` // header path; empty for non-DiffHeader
Before string `json:"before,omitempty"`
After string `json:"after,omitempty"`
}
Diff is one named reason the cache key changed. For headers, Path names the specific file; Before/After are hex hashes (or "" when the header was added or removed).
type DiffKind ¶
type DiffKind string
DiffKind enumerates the categories `hpcc explain` reports. Stable strings; matched on by the CLI's renderer.
type DiskStore ¶
DiskStore stores one JSON file per source path under Dir. Filenames are sha256(abspath(source))+".json" so paths with separators or odd characters don't collide and can't escape the dir.
Concurrent-safe across goroutines in the same process via atomic rename; concurrent across processes is fine — last writer wins and a partially-written file is never observed because we write to a tempfile first.
func NewDiskStore ¶
NewDiskStore returns a DiskStore writing under dir, creating the dir if needed. maxEntries == 0 picks the package default.
type Outcome ¶
type Outcome string
Outcome is what the daemon did with this compile. Stable strings so the on-disk records remain readable across versions.
type Record ¶
type Record struct {
Version int `json:"version"`
Timestamp time.Time `json:"ts"`
SourcePath string `json:"source_path"` // abspath
OutputPath string `json:"output_path,omitempty"` // abspath; empty when none
Outcome Outcome `json:"outcome"`
// CacheKey is the full hex-encoded cache key the daemon computed
// for this compile. Recorded so the user can correlate an
// explain record with the corresponding cache entry.
CacheKey string `json:"cache_key,omitempty"`
// Sub-hashes the diff engine pulls apart on the next miss. Any
// of these may be empty when the daemon couldn't compute it
// (bypass paths skip work, remote-dispatched compiles don't
// always materialise headers locally, etc.).
CompilerIdentityHash string `json:"compiler_identity_hash,omitempty"`
FlagsHash string `json:"flags_hash,omitempty"`
SourceContentHash string `json:"source_content_hash,omitempty"`
HeaderHashes map[string]string `json:"header_hashes,omitempty"` // header path → hex digest under HeaderHashAlgo
HeaderHashAlgo string `json:"header_hash_algo,omitempty"` // "blake3" (from manifest) | "sha256" (recomputed) | "" (no headers)
ImageDigest string `json:"image_digest,omitempty"`
// Human-readable context for `hpcc explain`'s output.
CompilerBinary string `json:"compiler_binary,omitempty"`
Args []string `json:"args,omitempty"`
// Diffs is the set of named reasons this compile's cache key
// differs from the previous compile's cache key for the same
// source. Computed at write time by the daemon (where both
// records are in hand) and embedded here so `hpcc explain` is
// a pure read of the latest record — no need to retain the
// prior. Empty when this is the first compile for the source,
// when the outcome was a hit, or when nothing observable
// changed.
Diffs []Diff `json:"diffs,omitempty"`
}
Record is one compile's worth of explain-relevant metadata.
All hash fields are hex-encoded SHA-256 (chosen for human readability and for the ability to grep records by digest; the cache itself uses BLAKE3 for its key, recorded verbatim in CacheKey).
func (*Record) CompareTo ¶
CompareTo returns the structured set of differences between r (the new compile) and prior (the previous compile for the same source). nil prior returns nil — the caller treats that as "no prior record; this source has never been compiled by this daemon."
Returned diffs are sorted (compiler → flags → source → headers → image) for stable rendering.
type Store ¶
type Store interface {
// Get returns the latest record for sourcePath, or (nil, nil) if
// none. Errors propagate only for I/O failures the daemon should
// surface; "missing" is not an error.
Get(sourcePath string) (*Record, error)
// Put writes r. Triggers eviction when the on-disk record count
// exceeds the store's cap.
Put(r *Record) error
}
Store is the explain-record persistence interface. Two methods so a future remote-explain (worker-side records joined into the daemon view) is a drop-in replacement.
Source Files