Documentation
¶
Index ¶
- Constants
- func ChunkText(text string, maxTokens int) []string
- func CreateProjectNode(src *types.Source) types.Node
- func DetectProject(dir string) *types.Source
- func ExtractAll(root string, files []string, provider types.LLMProvider, src *types.Source, ...) ([]types.Node, []types.Edge, error)
- func ExtractAncora(dbPath string, provider types.LLMProvider, maxTokens int, ...) ([]types.Node, []types.Edge, error)
- func ExtractContract(root string, files []string, src *types.Source) ([]types.Node, []types.Edge, error)
- func ExtractDoc(path string, relFile string, provider types.LLMProvider, maxChunkTokens int) ([]types.Node, []types.Edge, error)
- func ExtractGoEdges(parsed *parsedGoFile, _ []byte, relFile string) []types.Edge
- func ExtractGoNodes(parsed *parsedGoFile, _ []byte, relFile string) []types.Node
- func ExtractPDF(path string, relFile string, provider types.LLMProvider, maxChunkTokens int) ([]types.Node, []types.Edge, error)
- func ExtractPythonEdges(root *sitter.Node, src []byte, relFile string) []types.Edge
- func ExtractPythonNodes(root *sitter.Node, src []byte, relFile string) []types.Node
- func ExtractTSEdges(root *sitter.Node, src []byte, relFile string) []types.Edge
- func ExtractTSNodes(root *sitter.Node, src []byte, relFile string) []types.Node
- func IsContractFile(path string) bool
- func ParseGoFile(path string) (*parsedGoFile, []byte, error)
- func ParsePythonFile(path string) (*sitter.Node, []byte, error)
- func ParseReferences(title, content string) []types.ObsReference
- func ParseTSFile(path string) (*sitter.Node, []byte, error)
- func PrefixedNodeID(project, fileRelPath, symbol string) string
- func ProjectNodeID(name string) string
- func RelPath(root, path string) string
- type LLMExtractor
Constants ¶
const ( EvidenceTypeOpenAPI = "openapi" EvidenceTypeProto = "proto" )
Evidence type tags specific to the contract layer.
const ( MetaLayer = "layer" MetaEvidenceType = "evidence_type" MetaEvidenceConfidence = "evidence_confidence" MetaEvidenceSourceArtifact = "evidence_source_artifact" )
Metadata keys stamped onto extracted nodes and edges so that downstream layers (graph insertion, chunking, retrieval, evidence attribution) have a uniform, layer-aware contract independent of which extractor produced them.
Canonical identity is still assigned by ExtractAll (project prefixing, file node IDs); extractors only produce layer-local metadata here — they must not invent graph-level identity.
const ( EvidenceTypeAST = "ast" EvidenceTypeDocLLM = "doc-llm" EvidenceTypePDFLLM = "pdf-llm" EvidenceTypeFilesystem = "filesystem" EvidenceTypeProject = "project" )
Evidence type tags used when stamping extraction output. These correspond to the Evidence.Type field in pkg/types and let the orchestrator attribute retrieval hits back to the extractor that produced them.
Variables ¶
This section is empty.
Functions ¶
func ChunkText ¶
ChunkText splits text into chunks of at most maxTokens (estimated via charsPerToken). Each chunk overlaps with the previous by chunkOverlapRatio of the previous chunk's length to preserve context at boundaries. If the entire text fits within maxTokens, a single-element slice is returned.
func CreateProjectNode ¶
CreateProjectNode creates the root node for a codebase extraction. All file/function nodes link to this via "belongs_to" edges.
func DetectProject ¶
DetectProject extracts project metadata from a directory. Priority:
- Git remote URL → extract repo name (e.g., "github.com/Syfra3/vela" → "vela")
- Fall back to directory basename
Returns a Source with Type=codebase, Name, Path, and Remote (if git).
func ExtractAll ¶
func ExtractAll( root string, files []string, provider types.LLMProvider, src *types.Source, maxChunkTokens ...int, ) ([]types.Node, []types.Edge, error)
ExtractAll dispatches each file to the appropriate extractor and returns the merged set of nodes and edges.
src describes which project this extraction belongs to. If nil, DetectProject is called using root. All returned nodes carry src in Node.Source and have their IDs prefixed with "<project>:". A project root node and one file node per extracted file are also created and linked via "contains" edges.
provider may be nil; if so, doc files are skipped. maxChunkTokens controls chunking for local LLMs (0 = default 8000).
func ExtractAncora ¶
func ExtractAncora( dbPath string, provider types.LLMProvider, maxTokens int, progress func(done, total int, current string), ) ([]types.Node, []types.Edge, error)
ExtractAncora reads all observations from the ancora SQLite database and returns nodes + edges for the knowledge graph.
Graph structure:
Hierarchy nodes: Workspace → Visibility → Organization (when present) Observation nodes: one per observation, linked into the hierarchy Reference edges: explicit references stored in observations.references LLM edges: semantic relations inferred by the LLM provider (optional)
provider may be nil — in that case only structural edges are produced.
func ExtractContract ¶ added in v1.4.0
func ExtractContract(root string, files []string, src *types.Source) ([]types.Node, []types.Edge, error)
ExtractContract parses the supplied set of files, keeping only those recognised as contract artifacts, and returns contract-layer nodes + edges stamped with declared evidence.
src is the repo/workspace context the artifacts were discovered in — the extractor emits a workspace-consumable binding edge (`declared_in`) from each service to the project node so workspace routing can pick up declared services without re-reading the artifact. If src is nil no binding edges are emitted (the contract graph is still produced).
func ExtractDoc ¶
func ExtractDoc(path string, relFile string, provider types.LLMProvider, maxChunkTokens int) ([]types.Node, []types.Edge, error)
ExtractDoc reads a markdown or text file, chunks it if necessary, and sends each chunk to the LLM provider for NER + RE extraction.
func ExtractGoEdges ¶
ExtractGoEdges extracts function call edges from a parsed Go AST.
func ExtractGoNodes ¶
ExtractGoNodes extracts named function declarations, struct type declarations, and interface type declarations from a parsed Go file.
func ExtractPDF ¶
func ExtractPDF(path string, relFile string, provider types.LLMProvider, maxChunkTokens int) ([]types.Node, []types.Edge, error)
ExtractPDF reads a PDF file, converts it to plain text, chunks it, and sends each chunk to the LLM provider for NER + RE extraction.
func ExtractPythonEdges ¶
ExtractPythonEdges extracts call edges from Python function bodies.
func ExtractPythonNodes ¶
ExtractPythonNodes extracts function_definition → function nodes and class_definition → struct nodes from a Python AST.
func ExtractTSEdges ¶
ExtractTSEdges extracts call edges from TypeScript/JavaScript files.
func ExtractTSNodes ¶
ExtractTSNodes extracts functions, arrow functions, classes and interfaces.
func IsContractFile ¶ added in v1.4.0
IsContractFile returns true when a path looks like a declared contract artifact supported by this extractor.
func ParseGoFile ¶
ParseGoFile parses a .go source file using go/ast (pure Go, no CGO).
func ParsePythonFile ¶
ParsePythonFile parses a .py source file and returns the root AST node.
func ParseReferences ¶
func ParseReferences(title, content string) []types.ObsReference
ParseReferences extracts explicit references from observation text. It is intentionally conservative — false positives create noise in the graph.
func ParseTSFile ¶
ParseTSFile parses a TypeScript or JavaScript file using tree-sitter. Pass the appropriate language based on the file extension.
func PrefixedNodeID ¶
PrefixedNodeID returns a node ID prefixed with the project name. Format: "<project>:<file>:<symbol>" or "<project>:<file>" for file nodes.
func ProjectNodeID ¶
ProjectNodeID returns the canonical node ID for a project root node. Format: "project:<name>"
Types ¶
type LLMExtractor ¶
type LLMExtractor struct {
// contains filtered or unexported fields
}
LLMExtractor uses an LLM to find implicit references in an observation's content that are not captured by the explicit reference parser. It runs as a worker pool consuming from the Patcher's LLM queue.
func NewLLMExtractor ¶
func NewLLMExtractor(cfg *types.LLMConfig, workerCount int, onRefsFound func(int64, []types.ObsReference)) (*LLMExtractor, error)
NewLLMExtractor creates an extractor using the provided LLM client. workerCount controls how many parallel extraction goroutines are run.
func (*LLMExtractor) Start ¶
func (e *LLMExtractor) Start(ctx context.Context, queue <-chan types.ObservationNode)
Start launches worker goroutines that consume from queue. It returns when ctx is cancelled or queue is closed.
Source Files