Documentation
¶
Overview ¶
Package semantic is the opt-in LLM enrichment stage. It runs between extract.Resolve and cluster.Cluster: an LLM reads each prose note and emits concept/rationale nodes plus cites / conceptually_related_to / semantically_similar_to edges, so communities reflect concepts and otherwise isolated notes gain edges into their real cluster.
Enrichment is ADDITIVE ONLY. It never mutates or removes a deterministic AST/markdown node or edge: deterministic nodes win on id collisions, and every emitted edge is forced to INFERRED|AMBIGUOUS confidence so the deterministic core stays byte-identical when the stage is off. A content-hash cache means only changed notes re-pay tokens, and the sensitive-file skip plus a size cap run before any content reaches a backend.
Index ¶
Constants ¶
const ConceptType = "semantic_concept"
ConceptType is the file_type for semantic concept nodes. It is distinct from extract's "concept" (external-dependency) nodes so the report's God-Node and Surprising-Connection analysis can recognise LLM-derived concepts as real abstractions without changing how external-dependency nodes are treated — and so corpora built without --semantic never produce this type, keeping their output byte-identical.
const MaxNoteBytes = 200 * 1024
MaxNoteBytes caps the prose sent to a backend per note. Notes larger than this are skipped (mirroring the extract path's size discipline) so a single pathological file cannot blow up token cost.
const RationaleType = "rationale"
RationaleType is the file_type for rationale nodes a backend may emit (a short explanation of why two things relate). Analysed as a real entity, like ConceptType.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Backend ¶
type Backend interface {
// Name identifies the backend (e.g. "bedrock"), for diagnostics.
Name() string
// Extract returns nodes and edges inferred from one note. A non-nil error
// is treated as a per-note failure: that note contributes nothing, but the
// run continues.
Extract(ctx context.Context, n Note) (nodes []model.Node, edges []model.Edge, err error)
}
Backend is an LLM concept-extraction provider. Implementations read one note's prose and return the additive concept/rationale nodes and the edges (from the note, into concepts or other nodes) that the model inferred. Designing this as an interface lets an internal OpenAI-compatible gateway be added later without touching the enrichment plumbing.
type BedrockBackend ¶
type BedrockBackend struct {
// contains filtered or unexported fields
}
BedrockBackend is a Backend backed by Anthropic Claude on Amazon Bedrock.
func NewBedrockBackend ¶
func NewBedrockBackend(ctx context.Context, region string) (*BedrockBackend, error)
NewBedrockBackend builds a Bedrock-backed concept extractor for the given AWS region, resolving credentials through the standard AWS chain (AWS_PROFILE, env, instance role). region defaults to us-east-1 when empty.
func (*BedrockBackend) Extract ¶
Extract asks Claude to record the concepts the note is about and turns the tool call into concept nodes and edges. A note with no usable tool call yields nothing rather than an error.
func (*BedrockBackend) Name ¶
func (b *BedrockBackend) Name() string
type Cache ¶
Cache maps a note's graph ID to its content hash and the semantic fragments last extracted from it, so an unchanged note reuses its result instead of re-paying tokens. It mirrors the internal/cache content-hash pattern.
func Enrich ¶
func Enrich(ctx context.Context, cfg Config, base model.Extraction, notes []Note, prev Cache) (model.Extraction, Cache, error)
Enrich runs the semantic stage over notes and merges the result additively into base. It returns the enriched extraction and the cache to persist.
prev is the previous run's cache (nil for a cold run); notes whose content hash matches prev reuse their cached fragments and the backend is not called. Sensitive files and notes over MaxNoteBytes are skipped before any content reaches the backend. A per-note backend error is logged and skipped — it never aborts the run or touches the deterministic core.
type Config ¶
type Config struct {
Backend Backend
}
Config carries the knobs for one enrichment run.
Source Files