pipeline

package
v0.9.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 6, 2026 License: MIT Imports: 46 Imported by: 0

Documentation

Overview

Package pipeline — caller_kind.go classifies the SCOPE that emitted a CALLS edge so harness metrics can stratify precision by caller-shape.

2026-05-02 plateau-2 plan, Step 3. The aggregate F1 is dominated by edges whose CALLER is a real function or method body. A growing share of false-positives, however, are emitted from PACKAGE-LEVEL scopes — file blocks, var initializers, type declarations, init() functions. These "ghost callers" correlate strongly with low-quality resolutions (the resolver has no tight enclosing-function context to constrain candidates with).

Today they're invisible: the headline F1 number lumps every CALLS edge together, and the per-project breakdown (PR #129) shows per-subset variance but not per-caller-shape variance. After this change every CALLS edge carries a `caller_node_kind` property; the harness can compute per-kind precision and the share of FPs that originate from non-function scopes.

Classification source: the resolver already routes calls through `resolveCallEdge` (CBM extractor, function/method body) and through `collectLSPResolvedEdges` (LSP cross-file resolution, function/method body). When `EnclosingFuncQN == ""` the resolver synthesizes a module-level caller and tags the edge `CALLS_PSEUDO`; that's the "package-block" signal. The label of the caller node (looked up in the FunctionRegistry) discriminates Function-body vs Method-body. Test bodies are detected by the `IsTest` flag on the Definition, which the CBM extractor sets from the parser-level test-file + test-name heuristic.

This file plumbs information the resolver already has. No new AST traversal, no new C-side extractor work.

Package pipeline — candidate_set.go exposes the resolver's pre-tie-break candidate-set cardinality on every emitted CALLS-family edge so harness metrics can stratify precision by call-site ambiguity.

2026-05-02 plateau-2 plan, Step 5 — the largest of the five instrumentation changes. Mirrors caller_kind.go (Step 3) and resolver_rule.go (Step 4); this is the third edge property added by the plateau-2 sequence.

Why this exists --------------- Step 2's LLM-Judge taxonomy (knowledge-base PR #360) found that 60% of judged FPs are `same_named_method_disambiguation` — the resolver picked the wrong receiver type when the method name resolved to >=2 candidates. Step 4's `resolver_rule` exposes WHICH rule fired (cross-package-import-map, cross-package-suffix, receiver-qualified, etc.) but cannot say HOW AMBIGUOUS each individual call site was — a rule can be "right" 95% of the time when the call site has one candidate and "right" 40% of the time when it has six.

This step exposes the pre-tie-break candidate set: at every call site, when the resolver picked one of N candidates, record N. Multiple emitted edges can share a call site (rare; resolver is currently single-target), and a call site can produce zero edges if resolution failed.

Storage ------- Stored as a number in the edge's `candidate_set_size` property. Indexed via the `candidate_set_size_gen` generated column on the edges table (see internal/store/store.go), mirroring the existing `confidence_tier_gen`, `caller_node_kind_gen`, and `resolver_rule_gen` columns. Cypher exposes it as `r.candidate_set_size`.

Per-call-site vs per-edge ------------------------- Conceptually candidate-set-size is a CALL SITE property, not an edge property. Today the resolver is single-target — one chosen edge per call site — so each emitted edge inherits its site's cardinality without ambiguity. If the resolver ever emits multiple edges per site (interface-dispatch with explicit alternates) every emitted edge would carry the SAME cardinality, which is correct semantically: every alternate edge "knew about" the same N candidates.

LSP-resolved calls ------------------ LSP returns ONE definite resolved target with a confidence score; the LSP path does not expose the alternates it considered. LSP-resolved edges therefore carry candidate_set_size=1 BY DEFINITION — not because no other candidates existed, but because we cannot enumerate them without invasive C-side refactoring. Documented as a known limitation. The Janusian signal lives in the Go-side registry strategies (which explicitly enumerate via FunctionRegistry.byName) and in the type-dispatch paths.

Confidence: VERIFIED for all registry-strategy emit sites — every ResolutionResult already carries CandidateCount today. INFERRED for the LSP-resolved path's =1 default (LSP is single-target by design).

Definitions and calls passes: symbol registry, call resolution, edge flushing.

Split from pipeline.go without behaviour changes.

HTTP link pass: route matching, infra URL sites, JSON config discovery, file hashing.

Split from pipeline.go without behaviour changes.

Imports pass: relative-import normalization and IMPORTS edge emission.

Split from pipeline.go without behaviour changes.

Structure pass: directory, package, and file nodes with CONTAINS edges.

Split from pipeline.go without behaviour changes.

Incremental indexing: change classification, dependent expansion, hash bookkeeping.

Split from pipeline.go without behaviour changes.

Pass orchestration: full, incremental, and post-flush pass sequencing.

Split from pipeline.go without behaviour changes.

Package pipeline — resolver_rule.go classifies which resolver rule emitted a given CALLS-family edge. Mirrors caller_kind.go (Step 3 of the 2026-05-02 plateau-2 plan); this is Step 4.

Why this exists --------------- Step 2's LLM-Judge taxonomy (knowledge-base PR #360) found that the dominant FP class on the Go fixture is `same_named_method_disambiguation` (60% of judged sample) and the dominant FN class is `cross_package_heuristic_overreach` (83% of FNs — all 5 of the `Server.handleIndexRepository → tools.*` misses). Today these FPs and FNs are aggregate noise: the headline F1 lumps every CALLS edge together, the per-project breakdown shows per-subset variance, and the Step-3 caller-kind cut shows per-caller-shape variance — but the resolver_rule cut is the dimension that exposes WHICH RULE chose the wrong target.

Every CALLS-family emit site picks ONE rule it represents. Rule labels are mutually exclusive at emit time — if a site is genuinely ambiguous, pick the dominant one and document why (resolver entanglement is its own signal worth surfacing rather than papering over).

Modal upgrades (CALLS → CALLS_EXTERNAL when the target is an LSP stub; CALLS_PSEUDO for synthetic module-level callers) override the original resolver-rule label since the modal classification is the dominant signal a power user filters on. The override is applied in `pipeline.go::buildEdgesFromResults` once the stub-target check has run; the resolver picks the original rule, the modal-upgrade path overrides to `modal-external`/`modal-pseudo` if applicable.

Confidence: VERIFIED for the static rules below from reading the resolver source (resolver.go, pipeline_cbm.go, pipeline.go). LSP strategy → rule mapping VERIFIED from internal/cbm/lsp_test.go which pins the strategy strings used by the C-side LSP path.

Index

Constants

View Source
const (
	// CallerKindFunction — caller is a free function body (Go func, Python
	// def at module level, Rust fn, etc.). Default for non-method callable.
	CallerKindFunction = "function-body"

	// CallerKindMethod — caller is a method body (struct receiver, Python
	// class method, Rust impl block fn, Java/Kotlin class method).
	CallerKindMethod = "method-body"

	// CallerKindPackageInit — caller is an explicit package initializer
	// (Go `func init()`, Python `__init__.py` module body's init-time
	// statements, Rust `#[ctor]`-style). The resolver currently lumps
	// these into the synthetic-module-caller path; we use this kind when
	// the caller QN's last segment is literally "init".
	CallerKindPackageInit = "package-init-block"

	// CallerKindFileBlock — caller is the synthetic module-level caller
	// (`EnclosingFuncQN == ""` at the C-side extractor, resolved to
	// moduleQN + edgeType=CALLS_PSEUDO on the Go side). Top-level
	// function-call statements that aren't inside any callable scope.
	CallerKindFileBlock = "file-block"

	// CallerKindTypeDecl — caller is a type declaration (Go method on a
	// type, Rust impl block header, Python class body before any def).
	// Currently rare; reserved for future extractor enhancements.
	CallerKindTypeDecl = "type-decl"

	// CallerKindVarInit — caller is a package-level variable initializer
	// (`var x = expensive()` in Go, module-level `X = compute()` in
	// Python). Detected via the same module-level synthetic-caller path
	// as file-block; distinguishing requires extractor support that
	// doesn't yet exist, so for now var-init flows through
	// CallerKindFileBlock. Reserved for resolver enhancement.
	CallerKindVarInit = "var-init"

	// CallerKindTest — caller is a test function body. Detected via
	// IsTest on the caller's Definition (set by the CBM extractor from
	// `_test.go` / TestXxx / pytest discovery).
	CallerKindTest = "test-body"

	// CallerKindClosure — caller is an anonymous function / lambda /
	// closure expression. Reserved; the CBM extractor does not currently
	// emit a stable QN for closures, so this kind is unused today.
	CallerKindClosure = "closure"

	// CallerKindUnknown — fallback when no classification rule fires.
	// Should be 0% on healthy inputs; non-zero counts indicate either
	// a registry-lookup miss or a new caller shape we haven't covered.
	CallerKindUnknown = "unknown"
)

CallerKind enumerates the AST scope an emitted CALLS edge originates from. Stored as a string in the edge's `caller_node_kind` property for serialization friendliness (Cypher exposes it as an edge attribute, and json_extract('$.caller_node_kind') keeps the SQL path simple).

The resolver classifies each emitted edge into exactly one bucket. Buckets are stable identifiers — never rename them; harness baselines reference them by string.

View Source
const (
	// CandidateSetPropertyName is the JSON key in edge properties that
	// stores the resolver's pre-tie-break candidate-set cardinality.
	// Pinned string — harness baselines and the generated column
	// definition reference it by name.
	CandidateSetPropertyName = "candidate_set_size"

	// CandidateSetSizeUnknown sentinel — used when a code path emits an
	// edge without going through a resolution step that exposes its
	// candidate count. Stored as -1 (rather than 0 or NULL) to keep the
	// "unknown" signal greppable and distinguishable from "1 candidate"
	// in queries. NULL would alias with pre-migration rows; 0 would
	// alias with "resolver found nothing but emitted anyway" which is a
	// real bug class we want surfaced. -1 is the explicit sentinel.
	CandidateSetSizeUnknown = -1

	// CandidateSetSizeLSPDefault — LSP-resolved calls carry size=1 by
	// definition. The C-side LSP returns one resolved target per call
	// site; we cannot enumerate alternates without invasive C-side
	// refactoring. Documented as a limitation in the package comment.
	CandidateSetSizeLSPDefault = 1
)
View Source
const (
	// NixServiceOptionPrefixEnv overrides the option-set prefix that service
	// declarations live under (default "services" → options.services.<name>).
	NixServiceOptionPrefixEnv = "CODE_GRAPH_NIX_SERVICE_OPTION_PREFIX"
	// NixPkgsPrefixEnv overrides the package-set prefix used to detect the
	// binary a service runs (default "pkgs" → ${pkgs.<pkg>}/bin/<binary>).
	NixPkgsPrefixEnv = "CODE_GRAPH_NIX_PKGS_PREFIX"
)
View Source
const (
	// ResolverRuleExactQN — direct fully-qualified-name match. Today only
	// the LSP-resolved path (collectLSPResolvedEdges) when the LSP
	// strategy is neither receiver- nor interface-dispatch labels here.
	// Strategy strings: anything from cbm.ResolvedCall.Strategy that
	// doesn't start with "lsp_type_", "lsp_embed_", or "lsp_interface_".
	ResolverRuleExactQN = "exact-qn-match"

	// ResolverRuleReceiverQualified — method call resolved via receiver
	// type. LSP strategies "lsp_type_dispatch" and "lsp_embed_dispatch"
	// route here (the C-side LSP resolves the obj's type then looks up
	// method on the type, embedded type included).
	ResolverRuleReceiverQualified = "receiver-qualified"

	// ResolverRuleInterfaceDispatch — Go interface-satisfaction or
	// type-dispatch path. LSP strategies "lsp_interface_dispatch" and
	// "lsp_interface_resolve" route here. Also: the Go-side
	// `resolveCallWithTypes` "type_dispatch" strategy (TypeMap lookup
	// produces classQN+method that exists in the registry).
	ResolverRuleInterfaceDispatch = "interface-dispatch"

	// ResolverRuleSelfMethod — method call on the same instance, e.g.
	// Python's `self.x()` resolved against the enclosing class QN. Fires
	// in resolveCallEdge when calleeName starts with "self.".
	ResolverRuleSelfMethod = "self-method"

	// ResolverRuleSamePackageShadow — same-package symbol resolution.
	// Registry "same_module" strategy: callee was found at moduleQN +
	// "." + name (the caller's own module).
	ResolverRuleSamePackageShadow = "same-package-shadow"

	// ResolverRuleCrossPackageImportMap — registry "import_map" strategy.
	// Resolved through an EXPLICIT import: the imported alias was looked
	// up in the per-file import-map and the alias' definition was found.
	// This is the precise sub-bucket of the cross-package family;
	// 2026-05-06 baselines show 0.88-0.95 precision on Go fixtures.
	ResolverRuleCrossPackageImportMap = "cross-package-import-map"

	// ResolverRuleCrossPackageUniqueName — registry "unique_name" strategy.
	// Resolved by project-wide unique-name lookup (callee's simple name
	// has exactly one definition project-wide). Distinct from
	// import-map because it doesn't require the call site to import the
	// definition's module — the uniqueness of the name is the only
	// resolution signal.
	ResolverRuleCrossPackageUniqueName = "cross-package-unique-name"

	// ResolverRuleCrossPackageSuffix — registry "suffix_match" or
	// "import_map_suffix" strategies. The DANGEROUS sub-bucket: the
	// resolver matched the callee's simple name against the suffix of a
	// project-wide qualified name. This is the fall-through path that
	// produced the PR #165 phantom regression class (155+ phantom edges
	// from normalized Rust `Foo::new` matching against unrelated `.new`
	// methods). Drop-on-no-match is the targeted Rec 1 fix for this
	// sub-bucket. 2026-05-06 baselines show this bucket has 0.07-0.23
	// precision on Python adversarial fixtures (essentially noise).
	ResolverRuleCrossPackageSuffix = "cross-package-suffix"

	// ResolverRuleCrossPackageHeuristic — DEPRECATED 2026-05-06. The
	// original lumped bucket was split into ImportMap / UniqueName /
	// Suffix because per-fixture precision varied by an order of
	// magnitude (0.07 on flask vs 0.95 on code-graph), which the lumped
	// bucket couldn't surface. New code should NOT emit this string;
	// existing code paths that historically referenced it have been
	// updated to use the appropriate sub-bucket. Constant retained as
	// a compile-time anchor for legacy baselines (which carry the old
	// string in their JSON dumps) and as the canonical name for the
	// FAMILY across all three sub-buckets in helper predicates like
	// `isCrossPackageRule`.
	ResolverRuleCrossPackageHeuristic = "cross-package-heuristic"

	// ResolverRulePackageBlockFallback — emission where the caller is a
	// package- or file-level block rather than a real function scope.
	// Today this case is fully subsumed by ResolverRuleModalPseudo
	// (which fires when edgeType == CALLS_PSEUDO). Reserved for future
	// extractor enhancements (e.g. var-init or type-decl emission paths
	// that don't go through CALLS_PSEUDO).
	ResolverRulePackageBlockFallback = "package-block-fallback"

	// ResolverRuleFuzzyResolve — last-resort name match via
	// FunctionRegistry.FuzzyResolve (registry "fuzzy" strategy). Fires
	// when neither the type-dispatch path nor the structured registry
	// strategies produced a candidate — the resolver falls back to
	// matching purely on simple name and picks the best candidate by
	// import distance. Lowest pre-emit confidence; a high share of FPs
	// here is a known failure mode.
	ResolverRuleFuzzyResolve = "fuzzy-resolve"

	// ResolverRuleModalExternal — CALLS_EXTERNAL emission (LSP-resolved
	// external symbol; target is a synthesized stub). Set in
	// buildEdgesFromResults when the stub-target check upgrades a CALLS
	// edge to CALLS_EXTERNAL. Overrides the original rule because the
	// "external" modal classification is the dominant signal.
	ResolverRuleModalExternal = "modal-external"

	// ResolverRuleModalPseudo — CALLS_PSEUDO emission (synthetic
	// module-default caller). Set when the resolver substitutes
	// moduleQN for an empty EnclosingFuncQN. Overrides whatever the
	// underlying registry/LSP rule would have chosen because the
	// pseudo-caller property is the dominant signal.
	ResolverRuleModalPseudo = "modal-pseudo"

	// ResolverRuleUnresolvedEmitted — emitted despite no confident
	// resolution path firing. Reserved for diagnostics; today the
	// emission paths all bail out before reaching this state (no
	// ambiguous emits). Non-zero counts in production indicate a
	// resolver bug.
	ResolverRuleUnresolvedEmitted = "unresolved-emitted"

	// ResolverRuleUnknown — fallback when no classification rule fires.
	// Should be 0% on healthy inputs; non-zero counts indicate either a
	// new emit path we haven't covered or an unexpected strategy string
	// from CBM/LSP. Acts as the safe default for future extractor
	// changes.
	ResolverRuleUnknown = "unknown"
)

ResolverRule enumerates the resolver pathway that emitted a CALLS-family edge. Stored as a string in the edge's `resolver_rule` property — Cypher exposes it as an edge attribute, json_extract('$.resolver_rule') keeps the SQL path simple, and a generated column on the edges table (`resolver_rule_gen`) gives indexed access for harness queries.

Buckets are stable identifiers — never rename them; harness baselines reference them by string.

View Source
const (
	RoleAuthBoundary        = "auth_boundary"
	RoleInputEntryPoint     = "input_entry_point"
	RoleSensitiveSink       = "sensitive_sink"
	RoleCryptoOperation     = "crypto_operation"
	RolePrivilegeEscalation = "privilege_escalation"
	RoleSessionManagement   = "session_management"
	RoleAuditLogging        = "audit_logging"
	RoleSanitizer           = "sanitizer"
)

Security role constants.

View Source
const (
	// input_entry_point subtypes
	SubtypeHTTPHandler      = "http_handler"
	SubtypeCLIEntry         = "cli_entry"
	SubtypeGRPCHandler      = "grpc_handler"
	SubtypeWebSocketHandler = "websocket_handler"

	// sensitive_sink subtypes
	SubtypeSQLQuery    = "sql_query"
	SubtypeShellExec   = "shell_exec"
	SubtypeFileWrite   = "file_write"
	SubtypeNetworkSend = "network_send"
	SubtypeHardwareIO  = "hardware_io"

	// crypto_operation subtypes
	SubtypeEncryption    = "encryption"
	SubtypeHashing       = "hashing"
	SubtypeSigning       = "signing"
	SubtypeKeyGeneration = "key_generation"

	// auth_boundary subtypes
	SubtypeAuthCheck = "auth_check"

	// sanitizer subtypes
	SubtypeInputValidation = "input_validation"
	SubtypeTypeCheck       = "type_check"
	SubtypeEscapeEncode    = "escape_encode"
	SubtypeBoundsCheck     = "bounds_check"
)

Security subtype constants — granular classification within each role.

View Source
const EnrichmentVersion = "2026.03.16.1"

EnrichmentVersion tracks the version of post-flush enrichment passes. Bump this manually whenever enrichment logic changes (new security roles, community detection params, HTTP link patterns, etc.). The index_status tool compares the stored version against this constant to detect stale enrichment.

Variables

View Source
var ErrHeapPressure = errors.New("pipeline aborted: heap pressure exceeded CODE_GRAPH_HEAP_LIMIT_MB")

ErrHeapPressure is returned by checkCancel when the in-memory graph buffer + extraction caches push HeapAlloc past CODE_GRAPH_HEAP_LIMIT_MB. Pass orchestration treats it like a context cancellation — the pipeline aborts cleanly between passes rather than racing toward OOM. The returned error includes the configured limit and the observed allocation so operators can size the limit appropriately.

Functions

func IsolationEnabled added in v0.9.1

func IsolationEnabled() bool

IsolationEnabled resolves CODE_GRAPH_EXTRACT_ISOLATION: "on" and "off" are explicit; "auto" (default) is on for the real binary, because a native crash inside an MCP server or a long index run is the failure mode this exists for. Under `go test` (a *.test executable) auto resolves to off: the test binary cannot serve as its own worker, and tests that exercise isolation opt in with "on" plus SetIsolationCommandFactory.

func IsolationFileTimeout added in v0.9.1

func IsolationFileTimeout() time.Duration

IsolationFileTimeout resolves CODE_GRAPH_EXTRACT_FILE_TIMEOUT_S (default 30s).

func IsolationStats added in v0.9.1

func IsolationStats() isolate.Stats

IsolationStats reports supervisor counters for doctor/health output; zero values when isolation never started.

func NewVoyageClient

func NewVoyageClient() embed.Embedder

NewVoyageClient returns the configured embedding provider, or nil when embeddings are disabled (no VOYAGE_API_KEY). Kept for callers that use the nil check as the "embeddings available?" signal; new code should call embed.Default and check embed.IsDisabled.

func ProjectNameFromPath

func ProjectNameFromPath(absPath string) string

ProjectNameFromPath derives a unique project name from an absolute path by replacing path separators with dashes and trimming the leading dash.

func ResetIsolationForTests added in v0.9.1

func ResetIsolationForTests()

ResetIsolationForTests closes the shared worker pool and forgets the command factory so the next extraction starts fresh. Tests call it so goroutine-leak checks see no supervisor goroutines after the run.

func SetIsolationCommandFactory added in v0.9.1

func SetIsolationCommandFactory(f isolate.CommandFactory)

SetIsolationCommandFactory overrides how worker processes are launched. Tests use it to run the test binary as the worker. It must be called before the first extraction in the process.

Types

type CallContext

type CallContext struct {
	// Phase 1 — present today.
	CalleeName string
	CallerQN   string
	ModuleQN   string
	ImportMap  map[string]string

	// Phase 2+ — reserved, populated as later phases ship.
	ReceiverType   string
	ImportBindings map[string]string
	Aliases        map[string]string

	// Language identifies the source language of the call site. Populated
	// by CALLS-edge resolution paths (resolveCallEdge -> resolveCallWithTypes
	// -> ResolveCtx) so language-specific drop policies can apply. Empty
	// string means "language unknown" — the drop policies below treat
	// unknown as "do not drop" (preserve legacy behavior on Resolve()
	// callers that don't set this field). Added 2026-05-06 to support
	// CG-1 (Python drop-on-no-match for cross-package-suffix bucket).
	Language lang.Language
}

CallContext bundles every signal a resolver strategy may consult when choosing among bare-name candidates. Phase 1 of the registry.Resolve consolidation: introduce the surface, forward to legacy implementation, no behavior change.

Phase 1 fields (consumed today):

  • CalleeName, CallerQN, ModuleQN, ImportMap match the legacy Resolve(calleeName, moduleQN, importMap) signature plus an explicit CallerQN slot. CallerQN is unused today and is reserved for Phase 3+ when receiver-type discrimination needs the caller's enclosing function QN to look up type bindings.

Phase 2+ fields (reserved, always empty today):

  • ReceiverType — set by callers when a receiver-type can be inferred from the call site (e.g. PR #149's PerFuncTypeMap). Used by Phase 3a.
  • ImportBindings — bare-name → qualified-target from `use` statements in scope at the caller. Used by Phase 3b.
  • Aliases — `use X as alias` mappings. Always empty until ACC-004 (bench/accuracy/FOLLOWUPS.md) ships the import-table tracking that populates aliases. Reserved here so the struct shape doesn't churn.

The shape is finalized at Phase 1 so Phase 2 only changes how the strategies CONSUME these fields, not the struct itself.

type CargoMetadataResult

type CargoMetadataResult struct {
	// ExternalCrates: deps with a non-empty `source` field (crates.io,
	// git, registry-other). Workspace members are excluded — a sibling
	// crate referenced via path dep would have empty source AND appear
	// in WorkspaceMembers.
	ExternalCrates map[string]bool
	// WorkspaceMembers: the workspace's own packages. Used to override
	// the "external" classification when a workspace member happens to
	// share a name with an external crate.
	WorkspaceMembers map[string]bool
}

CargoMetadataResult captures the external-vs-workspace classification derived from `cargo metadata --no-deps`. Consumed by the chain walker (Tier-2 v0.1) to mark chain roots whose crate is external — those chains drop instead of fuzzy-resolving the bare callee into an in-graph candidate.

Crate names are normalized via normalizeCargoCrateName: `-` → `_` matches Rust identifier conventions (callers refer to crates as `foo_bar` even when the cargo package name is `foo-bar`).

type ChangeCoupling

type ChangeCoupling struct {
	FileA         string
	FileB         string
	CoChangeCount int
	TotalChangesA int
	TotalChangesB int
	CouplingScore float64
}

ChangeCoupling represents a pair of files that change together.

type ChangedFile

type ChangedFile struct {
	Status  string // M, A, D, R (modified, added, deleted, renamed)
	Path    string
	OldPath string // non-empty only for renames
}

ChangedFile represents a file with a status from git diff --name-status.

func ParseGitDiffFiles

func ParseGitDiffFiles(repoPath string, scope DiffScope, baseBranch string) ([]ChangedFile, error)

ParseGitDiffFiles runs git diff --name-status and returns changed files.

func ParseGitDiffFilesBetween

func ParseGitDiffFilesBetween(repoPath, from, to string) ([]ChangedFile, error)

ParseGitDiffFilesBetween runs `git diff --name-status from..to` and returns the list of changed files. Used by diff_graph to compute the symbol-level delta between two arbitrary git revisions.

Both from and to are validated against validRevName to block argument injection via a crafted SHA/branch string.

func ParseNameStatusOutput

func ParseNameStatusOutput(output string) []ChangedFile

ParseNameStatusOutput parses the raw output of git diff --name-status.

type ChangedHunk

type ChangedHunk struct {
	Path      string
	StartLine int
	EndLine   int
}

ChangedHunk represents a changed region within a file.

func ParseGitDiffHunks

func ParseGitDiffHunks(repoPath string, scope DiffScope, baseBranch string) ([]ChangedHunk, error)

ParseGitDiffHunks runs git diff --unified=0 and extracts changed line ranges.

func ParseHunksOutput

func ParseHunksOutput(output string) []ChangedHunk

ParseHunksOutput parses the raw output of git diff --unified=0.

type CommitFiles

type CommitFiles struct {
	Hash  string
	Files []string
}

CommitFiles holds the files changed in a single commit.

type Dependency

type Dependency struct {
	Name    string
	Version string
}

Dependency represents a third-party package parsed from a lockfile.

type DiffScope

type DiffScope string

DiffScope controls which changes to include.

const (
	DiffUnstaged DiffScope = "unstaged"
	DiffStaged   DiffScope = "staged"
	DiffAll      DiffScope = "all"
	DiffBranch   DiffScope = "branch"
)

type EnvBinding

type EnvBinding struct {
	Key      string
	Value    string
	FilePath string // relative path where found
}

EnvBinding represents an extracted environment variable with a URL value.

func ScanProjectEnvURLs

func ScanProjectEnvURLs(rootPath string) []EnvBinding

ScanProjectEnvURLs walks the project root, scanning all non-ignored files for env var assignments where the value looks like a URL.

type FieldTypeMap

type FieldTypeMap map[string]string

FieldTypeMap maps "<structQN>.<fieldName>" -> field-type-class QN. Populated by walking struct/enum/union field declarations across every extracted file in the project. Used by the resolver to walk chains like `obj.field.method()` — once obj's type is known via the per-function TypeMap, FieldTypeMap supplies field's type so the resolver can look up `.method` on the field's type.

type FunctionRegistry

type FunctionRegistry struct {
	// contains filtered or unexported fields
}

FunctionRegistry indexes all Function, Method, and Class nodes by qualified name and simple name for fast call resolution.

func NewFunctionRegistry

func NewFunctionRegistry() *FunctionRegistry

NewFunctionRegistry creates an empty registry.

func (*FunctionRegistry) Exists

func (r *FunctionRegistry) Exists(qualifiedName string) bool

Exists returns true if a qualified name is registered. Uses RLock for concurrent read safety.

func (*FunctionRegistry) FindByName

func (r *FunctionRegistry) FindByName(name string) []string

FindByName returns all qualified names with the given simple name.

func (*FunctionRegistry) FindEndingWith

func (r *FunctionRegistry) FindEndingWith(suffix string) []string

FindEndingWith returns all qualified names ending with ".suffix".

func (*FunctionRegistry) FuzzyResolve

func (r *FunctionRegistry) FuzzyResolve(calleeName, moduleQN string, importMap map[string]string) (ResolutionResult, bool)

FuzzyResolve is the legacy fuzzy entry point. Builds a CallContext and forwards to FuzzyResolveCtx. Existing callers (pipeline_cbm.go:462) work unchanged.

Unlike Resolve(), this does not require prefix/import agreement — it purely matches on the function name.

func (*FunctionRegistry) FuzzyResolveCtx

func (r *FunctionRegistry) FuzzyResolveCtx(ctx CallContext) (ResolutionResult, bool)

FuzzyResolveCtx is the CallContext-shaped fuzzy resolver. As of Phase 2 this is the primary fuzzy path; FuzzyResolve forwards here. See ResolveCtx for the consolidation rationale.

Phase 2 still does NOT consume ctx.ReceiverType or ctx.ImportBindings — those land in Phase 3. The fuzzy path is the most likely to benefit from receiver-type discrimination since CandidateCount > 1 is common here.

func (*FunctionRegistry) IsClassLike

func (r *FunctionRegistry) IsClassLike(qualifiedName string) bool

IsClassLike returns true if `qualifiedName` is registered as a class-like type (Class, Struct, Enum, Trait, Interface). Used by the chain walker (CG-2) to gate type_dispatch emission: even if `currentType.method` exists exactly, we should not emit type_dispatch when `currentType` itself isn't a registered class. This catches edge cases where the chain walker would otherwise bypass the `applyReceiverTypeFilter` safety net (e.g. when `currentType` is a Module name or other non-class entity that happens to share a qualified-name prefix with a method).

func (*FunctionRegistry) LabelOf

func (r *FunctionRegistry) LabelOf(qualifiedName string) string

LabelOf returns the node label for a qualified name, or "" if not registered.

func (*FunctionRegistry) Register

func (r *FunctionRegistry) Register(name, qualifiedName, nodeLabel string)

Register adds a node to the registry.

func (*FunctionRegistry) RegisterTraitImpl

func (r *FunctionRegistry) RegisterTraitImpl(structQN, traitQN string)

RegisterTraitImpl records that structQN implements traitQN. Populated by Pipeline.buildTraitImplMap from CBM ImplTraits data.

func (*FunctionRegistry) Resolve

func (r *FunctionRegistry) Resolve(calleeName, moduleQN string, importMap map[string]string) ResolutionResult

Resolve is the legacy entry point. Builds a CallContext and forwards to ResolveCtx. Existing callers (decorates.go, pipeline.go, pipeline_cbm.go references/exceptions/variables/types paths, tests) continue to work unchanged. Phase 4 may migrate or remove this wrapper.

func (*FunctionRegistry) ResolveCtx

func (r *FunctionRegistry) ResolveCtx(ctx CallContext) ResolutionResult

ResolveCtx is the CallContext-shaped entry point. As of Phase 2 of the registry.Resolve consolidation (bench/research/registry-resolve-consolidation-plan.md), this is the PRIMARY resolver path: every strategy receives the full CallContext. The legacy Resolve(calleeName, moduleQN, importMap) signature now builds a CallContext and forwards here, so existing callers see no behavior change.

Phase 2 still does NOT consume ctx.ReceiverType, ctx.ImportBindings, or ctx.Aliases — those land in Phase 3 (discrimination ladder). The strategies receive them so Phase 3 can add discrimination at each strategy's CandidateCount > 1 branch without further signature churn.

Forwarding-equivalence is pinned by TestResolveCtx_ForwardsToLegacy.

func (*FunctionRegistry) Size

func (r *FunctionRegistry) Size() int

Size returns the number of entries in the registry.

type GraphBuffer

type GraphBuffer struct {
	// contains filtered or unexported fields
}

GraphBuffer holds all nodes and edges in memory during the buffered indexing phase. Assigns temporary IDs (sequential counter) that are remapped to real SQLite-assigned IDs during FlushTo. All IDs within the buffer are valid only for cross-referencing nodes↔edges within the buffer.

func (*GraphBuffer) FindEdgesBySourceAndType

func (b *GraphBuffer) FindEdgesBySourceAndType(sourceID int64, edgeType string) []*store.Edge

FindEdgesBySourceAndType returns edges from sourceID with the given type.

func (*GraphBuffer) FindNodeByID

func (b *GraphBuffer) FindNodeByID(id int64) *store.Node

FindNodeByID returns the node with the given temp ID, or nil.

func (*GraphBuffer) FindNodeByQN

func (b *GraphBuffer) FindNodeByQN(qn string) *store.Node

FindNodeByQN returns the node with the given qualified name, or nil.

func (*GraphBuffer) FindNodeIDsByQNs

func (b *GraphBuffer) FindNodeIDsByQNs(qns []string) map[string]int64

FindNodeIDsByQNs returns a map of QN → tempID for the given qualified names.

func (*GraphBuffer) FindNodeLabelsByQNs

func (b *GraphBuffer) FindNodeLabelsByQNs(qns []string) map[string]string

FindNodeLabelsByQNs returns a map of QN → label for the given qualified names. Mirrors store.FindNodeLabelsByQNs so the CALLS pass can filter non-callable targets whether running through the buffer or the store.

func (*GraphBuffer) FindNodesByLabel

func (b *GraphBuffer) FindNodesByLabel(label string) []*store.Node

FindNodesByLabel returns all nodes with the given label.

func (*GraphBuffer) FindNodesByQNSuffix

func (b *GraphBuffer) FindNodesByQNSuffix(suffix string) []*store.Node

FindNodesByQNSuffix returns nodes whose QN ends with "."+suffix. Used by the IMPORTS resolver to find targets in nested source layouts (e.g., a project's `src/flask/ctx.py` module matched by suffix `flask.ctx`).

func (*GraphBuffer) FlushTo

func (b *GraphBuffer) FlushTo(ctx context.Context, s *store.Store) error

FlushTo writes all buffered nodes and edges to the SQLite store. Drops indexes before bulk insert and recreates them after for O(N) index builds. On a fresh DB (no existing data), skips the expensive DROP INDEX + DELETE steps.

func (*GraphBuffer) InsertEdge

func (b *GraphBuffer) InsertEdge(e *store.Edge) int64

InsertEdge inserts an edge with dedup by (sourceID, targetID, type). On conflict, merges properties. Returns the edge ID.

func (*GraphBuffer) InsertEdgeBatch

func (b *GraphBuffer) InsertEdgeBatch(edges []*store.Edge)

InsertEdgeBatch inserts multiple edges.

func (*GraphBuffer) UpsertNode

func (b *GraphBuffer) UpsertNode(n *store.Node) int64

UpsertNode inserts or updates a node. Returns the temp ID. Properties are JSON-round-tripped to normalize types (e.g., []string → []any), matching the behavior of SQLite serialization/deserialization.

func (*GraphBuffer) UpsertNodeBatch

func (b *GraphBuffer) UpsertNodeBatch(nodes []*store.Node) map[string]int64

UpsertNodeBatch upserts multiple nodes and returns a QN → tempID map.

type IndexDelta

type IndexDelta struct {
	Mode            string
	FilesDiscovered int
	FilesChanged    int
	FilesDeleted    int
	FilesUnchanged  int
}

IndexDelta describes which lifecycle path the most recent Run selected. It reports source classification, not inferred timing, so callers can distinguish a true no-op from a fast incremental or full rebuild.

type IndirectCallEdge

type IndirectCallEdge struct {
	SourceQN     string // qualified name of the caller (the function containing the dispatch site)
	TargetQN     string // qualified name of the resolved callee
	DispatchKind string // "executor_submit" | "getattr" | "decorator" | "fn_pointer" | ...
	Confidence   string // "high" | "medium" | "low" | "speculative"
	Properties   map[string]any
}

IndirectCallEdge is a candidate INDIRECT_CALLS edge surfaced by the indirect-dispatch analyzer. The pipeline converts these to store edges after deduping with the regular CALLS pass.

func AnalyzePythonIndirectCalls

func AnalyzePythonIndirectCalls(filePath string, source []byte) []IndirectCallEdge

AnalyzePythonIndirectCalls scans a Python file's AST for indirect-dispatch patterns and returns candidate INDIRECT_CALLS edges. Currently handles only `executor.submit(fn, ...)` (v0.1).

TODO(v0.1): walk the AST for `Call(Attribute(Name(executor), "submit"))`. TODO(v0.1): verify `executor` is bound to a `ThreadPoolExecutor` /

`ProcessPoolExecutor` via local scope assignment lookup.

TODO(v0.1): resolve the first arg (a Name node) to a def in the same

module via the existing fqn.Compute infrastructure.

TODO(v0.1): emit one IndirectCallEdge per resolved dispatch site.

Returns: empty slice in v0.1 stub. Test (TestAnalyzePythonIndirectCalls_executor_submit) verifies that the function exists and returns the expected shape; the actual edge count assertion is `>= 0` until v0.1 implementation lands.

type LanguageResolverConfig

type LanguageResolverConfig struct {
	// DropLooseCrossPackage drops emissions in the
	// cross-package-unique-name and cross-package-suffix
	// resolver-rule sub-buckets. Production default: unset (emit).
	// Flip via RESOLVER_DROP_LOOSE_CROSS_PACKAGE=<any non-empty>.
	// The eval harness sets this for Python fixtures (per CLAUDE.md
	// "Resolver env vars") to suppress catastrophic-precision
	// emissions without affecting Go. No language scoping at the
	// read site today; the struct preserves that.
	DropLooseCrossPackage bool

	// RequireImportsForLooseCrossPackage drops candidates that
	// aren't import-reachable from the call site's module via an
	// explicit IMPORTS edge. Default by language: Rust=true,
	// others=false (Phase F, 2026-05-09).
	// RESOLVER_REQUIRE_IMPORTS_FOR_LOOSE_CROSS_PACKAGE=<non-empty>
	// forces true for all languages.
	RequireImportsForLooseCrossPackage bool

	// DropFuzzyJanusianChains drops fuzzy resolutions whose
	// candidate set matches the empirical Janusian-co-hallucination
	// signature. Default by language: Python=true, others=false
	// (Phase E, 2026-05-14, after PSM Rust assetman regressed
	// -2.2pp F1 under global default-on).
	// RESOLVER_DROP_FUZZY_JANUSIAN_CHAINS=1/true/yes forces ON for
	// all languages; =0/false/no forces OFF for all languages;
	// unset = language default.
	DropFuzzyJanusianChains bool

	// EmitEnumVariantAsParent rewrites Enum::Variant call sites
	// (where the variant child node doesn't exist in the registry)
	// to emit CALLS edges targeting the parent Enum's QN. Default
	// false (off); opt-in via RESOLVER_EMIT_ENUM_VARIANT_AS_PARENT=
	// <non-empty> (Phase A””-2 opt-in, 2026-05-14). No language
	// scoping today.
	EmitEnumVariantAsParent bool
}

LanguageResolverConfig holds the per-language gates the resolver consults during call resolution. Built once per (language, env) pair by ResolverConfigFor; consult the struct's bool fields at resolve time instead of scattered os.Getenv calls.

PR1 of the resolver consolidation arc — INTRODUCES this type and the loader, with bit-equivalence tests proving the new path produces the same outputs as the existing scattered env reads. PR2 will migrate the 17 call sites in resolver.go + pipeline_cbm.go to consume the struct. Until then, this type is dead code from production's perspective; only tests exercise it.

Backward-compatibility contract: the four env vars listed below remain the public configuration API. Operator runbooks, CI configs, and the Phase A””-2 / E playbooks continue to work unchanged. Internal call sites just stop reading os.Getenv inline.

func ResolverConfigFor

func ResolverConfigFor(language lang.Language) LanguageResolverConfig

ResolverConfigFor returns the effective resolver config for the given language, applying per-language defaults and any operator env-var overrides. Read sites that today call os.Getenv inline should (in PR2) be migrated to call this once per resolve pass and consult the returned struct's fields.

IMPORTANT: this function MUST produce the same boolean values as the existing scattered env reads for every (env, language) pair — that invariant is pinned by TestResolverConfig_BitEquivalence_* tests below. Drift would silently change indexing behavior and invalidate the defended Loc-Bench baseline (CLAUDE.md). Any future change here that intentionally diverges from the legacy helpers must also update those helpers in the same PR and update the test matrix.

type PerFuncTypeMap

type PerFuncTypeMap map[string]TypeMap

PerFuncTypeMap maps a function/method QN to the local-variable TypeMap visible inside its body (including parameters, `self`, and `let` bindings). Keys are caller QNs; values are per-scope name->type lookups. Empty key "" is the module-scope TypeMap (used for free-fn callers and CALLS_PSEUDO sites that have no enclosing function).

type Pipeline

type Pipeline struct {
	Store       *store.Store
	RepoPath    string
	ProjectName string
	Mode        discover.IndexMode
	// Progress is called between pipeline phases to report indexing progress.
	// May be nil if no progress reporting is needed.
	Progress ProgressCallback
	// LastNodeCount and LastEdgeCount carry the post-write node and edge
	// counts populated by Run(). Callers (e.g. handleIndexRepository) read
	// these instead of issuing a fresh CountNodes/CountEdges query —
	// post-bulk-write / post-WAL-checkpoint reads via a fresh `st` reference
	// were observed to return 0 even when 5,640 nodes had committed (code-
	// search 2026-05-26). The inner CountNodes inside Run() returns the
	// real counts because it reuses the same connection state as the writes;
	// exposing those values via fields propagates the truth instead of
	// re-querying. Zero when Run() has not been called (or returned early
	// on a no-op incremental).
	LastNodeCount  int
	LastEdgeCount  int
	LastIndexDelta IndexDelta
	// SCIPStatus reports whether the optional compiler-index precision tier was
	// applied and how much of the project's function graph it covered. It is
	// populated by passSCIPIngest and intentionally kept separate from the
	// generic node/edge counts so callers cannot mistake a partially-covered
	// SCIP index for compiler-grade coverage of the whole project.
	SCIPStatus SCIPIngestStatus
	// contains filtered or unexported fields
}

Pipeline orchestrates the 3-pass indexing of a repository.

func New

func New(ctx context.Context, s *store.Store, repoPath string, mode discover.IndexMode) *Pipeline

New creates a new Pipeline.

func (*Pipeline) ConfigureSCIP

func (p *Pipeline) ConfigureSCIP(path, source string)

ConfigureSCIP binds a per-project compiler index to this pipeline run. An empty path explicitly selects the heuristic tier and suppresses the legacy process-wide environment fallback.

func (*Pipeline) Run

func (p *Pipeline) Run() error

Run executes the full 3-pass pipeline within a single transaction. If file hashes from a previous run exist, only changed files are re-processed.

func (*Pipeline) SkippedFiles added in v0.9.1

func (p *Pipeline) SkippedFiles() []store.SkippedFile

SkippedFiles returns the files the supervisor skipped during this run.

type ProgressCallback

type ProgressCallback func(phase string, pct int, detail string)

ProgressCallback is called between pipeline phases to report indexing progress. phase: "discover", "structure", "definitions", "calls", "flush", "tests",

"communities", "http_links", "security_tags", "opa_linker", "lockfile_deps", "complete"

pct: 0-100 overall percent estimate. detail: human-readable status string.

type ResolutionResult

type ResolutionResult struct {
	QualifiedName  string
	Strategy       string  // "import_map", "import_map_suffix", "same_module", "unique_name", "suffix_match", "fuzzy", "type_dispatch"
	Confidence     float64 // 0.0–1.0
	CandidateCount int     // how many candidates were considered

	// DiscriminationApplied — populated when CandidateCount > 1 and a
	// tiebreaker fired. Empty string means the candidate set was unique
	// or no discrimination was needed. Populated by Phase 3+ of the
	// registry.Resolve consolidation
	// (bench/research/registry-resolve-consolidation-plan.md). Today
	// always empty; reading this field is a forward-compat hook.
	DiscriminationApplied string
}

ResolutionResult carries the resolved QN plus quality metadata. Initial confidence values are estimates — recalibrate after measuring precision per strategy on real repos.

type ResolveAsClassReason

type ResolveAsClassReason string

ResolveAsClassReason names the specific failure mode when resolveAsClassWithReason returns an empty QN. Used by Phase D instrumentation (2026-05-08) to split the previously-aggregate `traitQN-empty` skip reason in implements.go into its three downstream causes — without changing any caller behavior.

const (
	// ResolveOK — the name resolved to a class-like QN via the existing
	// 9 resolver strategies. No failure, no fallback fired.
	ResolveOK ResolveAsClassReason = ""
	// ResolveOKViaFallbackFromEmpty — Phase A (2026-05-08, plan
	// 2026-05-08-d-implement-actix-extension). The existing 9 strategies
	// returned no QN, but the byName + class-like-label filter found
	// exactly one match. Tracks how many of PR #262's 736 resolve-empty
	// cases this fallback closes.
	ResolveOKViaFallbackFromEmpty ResolveAsClassReason = "ok:fallback-from-empty"
	// ResolveOKViaFallbackFromMismatch — Phase A. Existing strategies
	// returned a QN with a non-class-like label (e.g., Variable for a
	// TS story export named "Default"); the byName fallback found a
	// class-like-labeled candidate instead. Tracks how many of PR #262's
	// 153 label-mismatch cases this fallback closes.
	ResolveOKViaFallbackFromMismatch ResolveAsClassReason = "ok:fallback-from-mismatch"
	// ResolveOKViaFallbackFromExternal — Phase A2 (2026-05-08, plan
	// 2026-05-08-external-crate-trait-registry). After all 9 strategies
	// + PR #265's label-aware project-wide fallback returned empty, the
	// curated SyntheticInterfaceRegistry matched the trait name against
	// stdlib/tier-1-prelude entries (From, Display, Debug, Iterator,
	// Send, Sync, Serialize, Deserialize, etc.) and returned a synthetic
	// `_external.<crate>.<trait>` QN. Tracks how many of PR #265's 722
	// remaining resolve-empty cases this 11th strategy closes.
	ResolveOKViaFallbackFromExternal ResolveAsClassReason = "ok:fallback-from-external"
	// ResolveEmpty — registry.Resolve returned no QN AND the byName
	// fallback found zero or multiple class-like candidates (so we
	// can't disambiguate without making something up).
	ResolveEmpty ResolveAsClassReason = "resolve-empty"
	// ResolveLabelMissing — registry.Resolve returned a QN but
	// registry.exact has no label entry for that QN. Should be rare;
	// indicates a registry-population gap.
	ResolveLabelMissing ResolveAsClassReason = "label-missing"
	// ResolveLabelMismatch — registry.Resolve returned a QN with a
	// non-class-like label, AND the byName fallback found zero or
	// multiple class-like candidates.
	ResolveLabelMismatch ResolveAsClassReason = "label-mismatch"
)

type ReturnTypeMap

type ReturnTypeMap map[string]string

ReturnTypeMap maps function QN to the return type name.

type SCIPIngestStatus

type SCIPIngestStatus struct {
	State                  string  `json:"state"`
	Source                 string  `json:"source,omitempty"`
	Documents              int     `json:"documents"`
	DriftedDocuments       int     `json:"drifted_documents"`
	ProjectFunctions       int     `json:"project_functions"`
	CoveredFunctions       int     `json:"covered_functions"`
	CoveragePercent        float64 `json:"coverage_percent"`
	HeuristicEdgesReplaced int     `json:"heuristic_edges_replaced"`
	SCIPCallsInserted      int     `json:"scip_calls_inserted"`
	IndexSHA256            string  `json:"index_sha256,omitempty"`
	Error                  string  `json:"error,omitempty"`
}

SCIPIngestStatus is the machine-readable outcome of the optional SCIP precision pass. Coverage is function-level over the graph's existing Function/Method nodes, not a claim that every language construct is covered.

type TypeMap

type TypeMap map[string]string

TypeMap maps variable names to their resolved class QN.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL