Documentation
¶
Overview ¶
Package extract distils conversation prose into durable, classified facts without an LLM — a port of mempalace's heuristic extractor. It pulls the decisions, preferences, and problems worth keeping out of a short-term intake capture, as durable memories that earn trust on recall. Marker-driven and deliberately conservative: a miss just means no extra memory, never a wrong one. Both the backfill importer and write-time extraction share this logic.
Index ¶
Constants ¶
const ClassifyMaxChars = 400
ClassifyMaxChars bounds a classifiable write, in runes: one durable fact is terse; longer content is session history even when it contains decision markers. Runes, not bytes — a byte bound would put non-ASCII prose over the ceiling at a third of the nominal length, silently denying it a durable tier.
const MaxEntities = 12
MaxEntities caps the entities extracted from one memory, keeping the first (reading-order) mentions so one long transcript can't flood the entity index.
const MaxPerExchange = 5
MaxPerExchange caps extractions from a single exchange so one rambling turn can't flood the store.
const MinConfidence = 0.3
MinConfidence gates an extraction: confidence is min(1, score/5) where score is the count of distinct markers that match plus a length bonus, so a lone marker on a short segment (0.2) is dropped while two distinct markers, or one in a longer segment, pass.
const MinFactChars = 20
MinFactChars floors an extractable segment and a classifiable write, in runes: below this there is too little text to be a fact. Runes for the same reason as ClassifyMaxChars — on bytes, non-ASCII prose clears a floor that the same amount of ASCII prose does not.
Exported because it bounds the useful range of a classify ceiling: a ceiling under MinFactChars can admit nothing, so config rejects that range rather than letting it read as a tight bound and act as an off switch.
Variables ¶
This section is empty.
Functions ¶
func Entities ¶ added in v0.5.5
Entities extracts salient named-entity keys from a memory's content without an LLM: maximal runs of capitalized tokens ("Charlotte's Web", "Black Friday", "Postgres"), normalized to lowercase with possessives stripped. Deliberately precision-first — noisy entities poison every consumer — so it only trusts capitalization evidence:
- a sentence-initial span counts only when it has ≥2 tokens, its lead word is also seen capitalized mid-sentence in the same text, or the word carries its own casing signal (internal uppercase or digits, "iPhone", "Qwen3");
- stopwords (function words, greetings, months/weekdays) never join a span; they split it instead;
- purely numeric or single-letter tokens never start an entity.
Lowercase concept nouns ("pottery", "the retry policy") are out of scope by design: without a POS tagger they cannot be told from ordinary prose.
Types ¶
type Kind ¶
type Kind string
Kind labels an extracted memory; it doubles as the memory's first tag.
func Classify ¶ added in v0.5.4
Classify labels an entire write whose caller picked no tier, bounded by the built-in ClassifyMaxChars. See ClassifyWith.
func ClassifyWith ¶ added in v0.7.3
ClassifyWith labels an entire write whose caller picked no tier, bounding it at maxChars runes. It is stricter than Typed because the verdict covers the full text rather than an extracted segment: the content must be short, prose-shaped, unhedged, not a transcript, and clear the same marker-confidence gate. ok=false means no confident call — callers fall back to the working-intake default. maxChars <= 0 declines every write, since a zero-length ceiling admits nothing.
Source Files