Help us shape the future of Go! Take the 2026 Go Developer Survey

memstore

package
v2.6.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 30, 2026 License: Apache-2.0 Imports: 16 Imported by: 0

Documentation

Overview

Package memstore is the memory engine behind X-Ray, for a Go program that wants the same store, search and ranking in its own process.

An Item is a MemoryCell spec plus a few in-process fields. A host brings its own rules through small seams (a visibility predicate, a retention function, ranking callbacks, a kind vocabulary) and its own lock; the package reads no global state and knows no Kubernetes client. The cell index and the embedder are in the subpackage index, the change hub in hub.

Index

Constants

View Source 
const (
	// KindStableFact is established truth about a subject, e.g. a verified root
	// cause. It needs high confidence.
	KindStableFact = "stable_fact"
	// KindOperationalPattern is a recurring behaviour worth remembering.
	KindOperationalPattern = "operational_pattern"
	// KindWatchItem is something being watched; it ranks higher while active.
	KindWatchItem = "watch_item"
	// KindHypothesis is a claim not yet confirmed; it decays quickly.
	KindHypothesis = "hypothesis"
	// KindUnknown is an open question; it decays.
	KindUnknown = "unknown"
	// KindEvent is something that happened at a time of its own (EventDate).
	KindEvent = "event"
)

Kinds the engine itself knows by name. Every other kind belongs to the host, which declares it through Vocabulary.

View Source 
const (
	// OriginObserved: written by code from facts it measured.
	OriginObserved = "observed"
	// OriginDerived: written by a model. A prior claim, never evidence. A cell
	// with no origin is derived.
	OriginDerived = "derived"
	// OriginHuman: written by an identified person. Outranks derived.
	OriginHuman = "human"
	// OriginVerified: derived, then passed a check.
	OriginVerified = "verified"
)

Origin values stored in MemoryCellSpec.Origin.

View Source 
const (
	StatTotal               = "search.total"                // total HybridSearch calls
	StatKeywordHits         = "search.keyword_hits"         // calls where the lexical pass ran (always, on every call)
	StatSemanticCalls       = "search.semantic_calls"       // calls where lexical was not confident, so semantic search ran
	StatSemanticFallbacks   = "search.semantic_fallbacks"   // semantic search calls that errored and fell back to lexical-only
	StatExhaustiveFallbacks = "search.exhaustive_fallbacks" // calls rescued from a zero-result starvation by the full exact-cosine scan
	StatKWTrigramHits       = "search.kw_trigram_hits"      // keyword searches where the trigram index contributed a candidate
	StatKWVoiceHits         = "search.kw_voice_hits"        // calls served by the confident lexical fast path (no embedding call)
	StatKWVoiceMisses       = "search.kw_voice_misses"      // calls that fell through to semantic RRF fusion

)

The counter names. Durations are summed in nanoseconds beside a count, so the mean is read as a quotient and never stored.

View Source 
const (
	DefaultHypothesisStaleAfter = 24 * time.Hour
	DefaultUnknownStaleAfter    = 7 * 24 * time.Hour
)

DefaultHypothesisStaleAfter and DefaultUnknownStaleAfter are the deadlines stamped onto a model-authored hypothesis or unknown that carries none: the a day and a week.

View Source 
const AuthorTagPrefix = "author/"

AuthorTagPrefix is a reserved tag ("author/<identity>") a write path stamps to record the authenticated caller that last authored a cell, so a reader can treat the cell as that identity's claim rather than trusted derived memory. Machine writers never stamp it; StripAuthorTags removes it when a machine rewrites the cell.

View Source 
const ExactIDMatchScore = 100.0

ExactIDMatchScore is the Score of an exact cell ID hit. It lies outside Rerank's scale (base score at most 1.0, largest multiplier 1.7) so it is never mistaken for a relevance score. Order is the slice position, not this value.

View Source 
const MaxModelLinks = 12

MaxModelLinks caps the ThoughtLinks one cell accumulates through AddLink. Past the cap a link is admitted only if its Weight out-weighs the lowest-weight link, which is evicted. Replacing an existing (ToID, Relation) pair is exempt.

View Source 
const MemoryNearDupThreshold = 0.3

MemoryNearDupThreshold is the Jaccard token-set similarity at or above which two comparable memory items are treated as one fact restated rather than two distinct facts. It must sit strictly above the similarity two items merely discussing the same subject can share by vocabulary alone, and at or below the similarity any genuine paraphrase of one fact shares with another. It is the only number this file tunes; the fixture in neardup_test.go is what tunes it.

View Source 
const NearDupMinTokens = 8

NearDupMinTokens is the token-set size below which Jaccard similarity stops being a meaningful signal: two short, unrelated items can share enough of their (few) tokens to clear MemoryNearDupThreshold by coincidence rather than by restating the same fact. Below this size (the smaller of the two token sets), only near-identical text (memoryNearDupShortTextThreshold) counts as a duplicate — the ordinary threshold and the containment rule below both require both sets to have reached this size first.

View Source 
const PartitionAll = "*"

PartitionAll is the partition argument meaning no partition filter: matches from every partition. It is not "", which the host may resolve to a default partition. Use it only on read-only operator surfaces, never on a write, ingest or decision path.

View Source 
const PartitionTagPrefix = "tenant/"

PartitionTagPrefix is the tag prefix that carries an item's partition, e.g. "tenant/alpha": the unit a host isolates memory by. The "/" separator keeps it apart from tags that use ":". The spelling is part of the stored cells.

View Source 
const PrivateOwnerTagPrefix = "user:"

PrivateOwnerTagPrefix tags a private item with its owner's identity hash. Subject names the instance that holds the item, not a person, so it cannot carry ownership: the tag is the only place a private item's owner lives.

View Source 
const RefutedDataKey = "refuted"

RefutedDataKey is the spec.data entry Refute writes (a RefutedRecord as base64 JSON).

View Source 
const ScopePrivate = "private"

ScopePrivate marks an item as isolated to its owner: visible to that one caller and nowhere else, not in shared search, not on the change stream. Any other Scope value is shared.

View Source 
const ScopePublic = "public"

ScopePublic marks an item as shared: visible on the graph and in search.

View Source 
const ServiceTagPrefix = "service/"

ServiceTagPrefix tags an item with the service or entity it concerns, e.g. "service/auth". It is the tag-side home of that association, so Subject can hold the instance identity.

View Source 
const SharedTag = "shared"

SharedTag is the literal tag that marks a cell as deliberately visible to every partition. An untagged cell is private to its own partition; this mark is the one way a host lets it out. It combines with a partition tag: that means "belongs to this partition and is also shared with the others". Only a trusted writer should be able to set it; a write path strips it from other callers' tags first, as it does author/ and partition tags.

View Source 
const TopicTagPrefix = "topic/"

TopicTagPrefix tags an item with a free-form value a model supplied for Subject that was not accepted as a real subject, e.g. "topic/memory": the generic counterpart of ServiceTagPrefix.

Variables

View Source 
var ValidThoughtLinkRelationsOrdered = []string{
	"triggers", "caused_by", "correlates_with", "precedes", "explains", "contradicts",
}

ValidThoughtLinkRelationsOrdered mirrors validThoughtLinkRelations in a fixed order for rejection messages, since map order is undefined.

Functions

func AuthorTagValue 

func AuthorTagValue(item Item) (string, bool)

AuthorTagValue returns the identity from an item's first author/<identity> tag, if any.

func BetterRoutingFact 

func BetterRoutingFact(a, b Item) bool

BetterRoutingFact reports whether a should be trusted over b: the higher confidence, then the more trusted origin, then the newer, then the smaller ID, so two restatements never tie.

func ClusterNearDuplicates 

func ClusterNearDuplicates(items []Item) [][]Item

ClusterNearDuplicates groups items into near-duplicate clusters with a single greedy pass that preserves input order: each item joins the first existing cluster whose FIRST member it near-duplicates, else it starts a new cluster of its own. Comparing only against each cluster's first member (not every member already in it) keeps the pass linear in the number of clusters rather than quadratic in cluster size, and gives every cluster a stable representative — its first member — for a caller that needs to pick one item to keep.

func ContainsString 

func ContainsString(ss []string, s string) bool

func DedupComparable 

func DedupComparable(a, b Item) bool

DedupComparable reports whether a and b are eligible to be scored for near-duplication at all. Two items must share a Kind, a Scope and a Subject (compared case-insensitively), and must carry the same partition tag (PartitionFromTags). An item with no partition tag only compares against another untagged item: "" is a distinct identity, not a wildcard. When both items carry a service/ tag (ServiceTagValue) the values must match; when at most one side carries the tag the items are still comparable, because a service-scoped restatement of an unscoped fact is exactly the case the read-side collapse needs to catch. A model-written b is never comparable to a human-written a, so a restated claim cannot absorb a person's note.

a is the existing side of the comparison in every caller: FindNearDuplicate scores each candidate (a) against the incoming item (b), and ClusterNearDuplicates scores a cluster's first member (a) against each item tested for membership (b). So a is where "is this a safe merge target" lives.

A candidate (a) tagged SharedTag is never comparable, so it is never a merge target. SharedTag is a trust mark set by a privileged writer; it makes a cell visible to every partition, and an ordinary write must never rewrite that content in place through dedup. Such a cell can still be updated by naming its ID, which goes through the explicit existing-ID path rather than a near-duplicate scan.

A candidate (a) tagged author/<identity> is comparable only against an incoming item (b) carrying the identical author/<identity> tag. The tag marks the content as that identity's own claim, so only a restatement from the same identity may fold into it. An incoming item with no author tag, or a different one, always mints its own cell. A machine rewrite strips author tags from its own items before they reach this comparison.

func DedupTokens 

func DedupTokens(item Item) map[string]struct{}

DedupTokens returns the normalized token set used to compare item's title+content against another item's for near-duplication. Stop words are dropped, and so are tokens shorter than three characters unless they carry a digit: "q3" and "q4" are what tell two otherwise identical plans apart.

func EventTimeAnnotation 

func EventTimeAnnotation(eventDate string) string

EventTimeAnnotation renders a compact suffix for an item's EventDate, so a reader consuming search results as a model's context sees when a remembered item happened without parsing a separate field. Models answer temporal questions measurably worse when the dates are absent from the render. It returns "" when eventDate is empty or fails to parse under either accepted layout, so a bad or legacy value degrades to no annotation rather than garbage.

It renders the date part only ("2023-05-30") when the parsed time is exactly UTC midnight, the shape a bare YYYY-MM-DD write produces, and the full RFC3339 timestamp otherwise.

Append it to the Title, never the Content: a caller that truncates Content to fit a prompt would cut the annotation off the tail.

func HasSharedTag 

func HasSharedTag(tags []string) bool

HasSharedTag reports whether tags carries SharedTag.

func IsPrivateItem 

func IsPrivateItem(it Item) bool

IsPrivateItem reports whether it is scoped private.

func IsSharedItem 

func IsSharedItem(item Item) bool

IsSharedItem reports whether item carries SharedTag.

func KwTokens 

func KwTokens(q string) []string

KwTokens splits a normalized query into its content-bearing tokens: stopwords and single characters are dropped, except that a single character with a digit ("q3" is two characters; "3" is one) stays. A query made only of stopwords keeps all its words, since filtering everything would leave the lexical channel with nothing to say.

func NearDuplicateMemory 

func NearDuplicateMemory(a, b Item) (score float64, dup bool)

NearDuplicateMemory scores a and b for near-duplication. It returns (0, false) when the two items are not comparable (DedupComparable) or when either has an empty token set. Otherwise score is the Jaccard similarity of their token sets.

When the smaller token set is below NearDupMinTokens, dup is set only by memoryNearDupShortTextThreshold — near-identical text — since ordinary similarity on a handful of tokens is not a reliable signal.

Otherwise dup is true once score reaches MemoryNearDupThreshold. A pure Jaccard score under-counts one specific shape beyond that: a short restatement of a long fact, where the union is dominated by the longer side's extra tokens and the ratio never clears the threshold even though the short side is wholly contained in the long one. So dup is also set when at least 90% of the smaller token set is contained in the larger set.

func NewTrigramIndex 

func NewTrigramIndex() *trigramIndex

func NormalizeQuery 

func NormalizeQuery(q string) string

NormalizeQuery lowercases, splits CamelCase and PascalCase boundaries ("FooBAR" becomes "foo bar"), and turns separators and punctuation into single spaces, so "what is payment-api?" reduces to "what is payment api". Without the punctuation step a token keeps the mark glued to it ("api?") and misses an exact match against "api".

func ParseEventDate 

func ParseEventDate(s string) (time.Time, bool)

ParseEventDate parses s against eventDateLayouts, reporting whether it parsed. Empty input never parses; callers check for "" separately, since "no event date" (omit the annotation silently) and "bad event date" (skip it) need different handling.

func PartitionFromTags 

func PartitionFromTags(tags []string) string

PartitionFromTags returns the partition named by the tags, or "" when none is.

func PrivateOwner 

func PrivateOwner(it Item) string

PrivateOwner returns the owner hash carried in the item's user: tag, or "" when there is none.

func RerankBaseScore 

func RerankBaseScore(rank int) float64

RerankBaseScore converts a 0-based input rank into the base score the multipliers scale: 1/(1+rank*rerankRankDecay).

func RestoredOrigin 

func RestoredOrigin(item Item) string

RestoredOrigin is the origin a cell read back from a MemoryCell carries: the one it has, else human when it bears an author tag, else derived.

func SanitizeTagValue 

func SanitizeTagValue(v string) string

SanitizeTagValue makes v safe to use as the value segment of a tag: it trims whitespace and turns commas (the separator tag lists are split on) and newlines into a single space, so a supplied value cannot be re-split into several tags on a later round trip or break a one-line render.

func ServiceTagValue 

func ServiceTagValue(item Item) (string, bool)

ServiceTagValue returns the value of an item's service/<name> tag, if any. Lookups that filter by subject check this in addition to Subject, so they match both items whose Subject holds the service name directly and items that carry it as a tag.

func StampAuthorTag 

func StampAuthorTag(item *Item, author string)

StampAuthorTag leaves item.Tags with a single author/<author> tag. It first removes every existing author/ tag, so a caller-supplied tag list cannot assert a different author than the one the write path resolved, then appends the new one. It is a no-op when author is "".

It is idempotent: stamping the same author twice yields the same Tags, which keeps the cell to memory round trip free of spurious writes.

func StampKnowledgeRetention 

func StampKnowledgeRetention(item *Item, retention func(partition string) int)

StampKnowledgeRetention stamps a computed StaleAfter onto an item that has none. A non-zero StaleAfter is never overwritten. stable_fact and operational_pattern get retention(partition) days, the partition read from the item's tenant/ tag so a per-partition rule applies. A hypothesis or unknown without a deadline gets the compiled default instead of being refused by the base rules. Other kinds are untouched.

func StripAuthorTags 

func StripAuthorTags(tags []string) []string

StripAuthorTags returns tags without any author/ entry, keeping the order of the rest. It is nil-safe.

func ThoughtLinkRejection 

func ThoughtLinkRejection(fromID string, link memoryv1.ThoughtLink) string

ThoughtLinkRejection reports why AddLink would reject link, or "". It covers only the checks that depend on the caller's values (empty ToID, invalid Relation, self-link), not the state-dependent ones (missing cell, cell at MaxModelLinks). AddLink calls it so the write path and callers such as a handler that validates a link before writing it agree.

func TokenGrams 

func TokenGrams(toks []string) [][]string

TokenGrams precomputes each token's trigrams once for a whole query, so a scan over N candidate items does not recompute them N times. Callers that score many items against one token set should hoist this out of their loop (rankAndLimit does, via queryOverlapScores).

func ValidEventDate 

func ValidEventDate(s string) bool

ValidEventDate reports whether s is an event date the store can render back.

func ValidateBase 

func ValidateBase(v Vocabulary, item Item) string

ValidateBase is the Store method of the same name for a caller that has a vocabulary but no store.

func VisibleToUser 

func VisibleToUser(it Item, userHash string) bool

VisibleToUser reports whether it is visible to a caller identified by userHash. A shared item is visible to everyone. A private item is visible only to the exact hash tagged as its owner: no administrator, partition or operator bypass.

func WithSearchCaller 

func WithSearchCaller(ctx context.Context, caller string) context.Context

WithSearchCaller stamps the calling subsystem's name onto ctx so hybridSearch can attribute the call's count and duration to it in the search stats. A call with no caller records as "internal".

Types

type CallerStat 

type CallerStat struct {
	Caller string
	Count  int64
	AvgMs  float64
}

CallerStat is one caller's search count + mean duration — see WithSearchCaller and SearchStatsSnapshot.ByCaller.

type Config 

type Config struct {
	// Retention gives the days a knowledge item of a partition is kept (0 = no
	// deadline), for the StaleAfter stamped on a write of a knowledge kind. It
	// runs before the store lock is taken.
	Retention func(partition string) int
	// Validate returns why item may not be written, or "": the kind-specific
	// rules the host owns, on top of the store's own. It runs before the store
	// lock is taken.
	Validate func(item Item) string
	// Before amends an item about to be stored, after any merge and before the
	// write, with the store lock held (a host that links an item to what its
	// evidence names does it here).
	Before func(item *Item)
	// Read amends items about to be returned, with the lock held (a host that
	// corrects a status note against live state does it here).
	Read func(items []Item)
	// Dirty and Signal fire with the lock held after every store or delete: the
	// host rebuilds what it derives from the items and wakes its watchers.
	Dirty  func()
	Signal func()
	// Quiet reports an id whose changes are not worth publishing, such as a note
	// rewritten every few seconds. It runs after the lock is released.
	Quiet func(id string) bool
	// Published fires, outside the lock, once per published change.
	Published func()

	// Vocabulary is the host's kinds; the zero value restricts nothing.
	Vocabulary Vocabulary
	// DefaultTTL gives a kind a deadline when an item of it is created without
	// one: decay by default. A kind not listed never expires on its own. An
	// update that carries no deadline keeps the stored one; it never restamps or
	// clears it implicitly, so renewing is the writer's explicit act. With a nil
	// map the store keeps its older rule, where every write of a hypothesis or
	// unknown without a deadline is stamped afresh.
	DefaultTTL map[string]time.Duration
	// RenewOnMerge makes a restatement that merges into an existing item of a
	// DefaultTTL kind re-arm that item's deadline; without it the merged item
	// keeps the deadline it had.
	RenewOnMerge bool
	// Subject is the identity of the instance that holds these items. A private
	// item's Subject is that identity, never its owner (the owner is the user:
	// tag).
	Subject string
	// IDPrefix starts the id of an item written without one. Default "memory-".
	IDPrefix string
	// NewID, when set, supplies the id of an item written without one instead of
	// IDPrefix and a timestamp.
	NewID func() string
	// SkipIDText, when set, names the ids that are left out of an item's search
	// text: an id the host minted says nothing about the item, so a query that
	// happens to resemble it must not find the item. A chosen id is still searched.
	SkipIDText func(id string) bool
	// Logger receives the store's log lines. The zero Logger discards them.
	Logger zerolog.Logger
}

Config is what the Store asks of its host. Every field may be nil or zero.

type ConsolidationGroup 

type ConsolidationGroup struct {
	Kind       string
	Keep       string
	KeepTitle  string
	Superseded []string
	Scores     []float64
}

ConsolidationGroup is one near-duplicate cluster found by a ConsolidateMemory sweep: Keep is the ID a reader should trust going forward, Superseded lists every other member's ID (parallel to Scores, each member's nearDuplicateMemory score against Keep).

type ConsolidationReport 

type ConsolidationReport struct {
	Partition  string
	Scanned    int
	Groups     []ConsolidationGroup
	Superseded int
	Applied    bool
	Error      string `json:"error,omitempty"`
}

ConsolidationReport is the outcome of one Consolidate sweep. Error is set, with every other field zero, when the sweep could not run (empty or unresolved partition); a zero Scanned or Groups then does not mean no duplicates.

type Item 

type Item struct {
	memoryv1.MemoryCellSpec

	// NoEmbed keeps the item's cell out of the vector index: the writer labels
	// the cell so it is never embedded. For records read back by id, never by
	// meaning.
	NoEmbed bool

	// CellOriginated marks an item loaded from an existing cell (a restart or
	// rebuild seed) rather than authored in memory. It lets a writer tell "the
	// cell existed at load time and was deleted since, on purpose" from "a
	// brand-new item that never had a cell", and refuse to resurrect the
	// former. It is a sibling of the spec, so converting the item back to a
	// spec cannot leak it into a cell; json:"-" is a second guard.
	CellOriginated bool `json:"-"`

	// LinksSet marks Links as the caller's authoritative set for this write,
	// even when empty, so Upsert does not fall back to keeping the links the
	// stored item already had. Off by default: an ordinary content edit must
	// never wipe a link added earlier through AddLink. A writer that recomputes
	// its whole link set on every refresh sets it, so a link can drop back to
	// zero just like content and tags. Never persisted.
	LinksSet bool `json:"-"`
	// contains filtered or unexported fields
}

Item is one memory note. It embeds the MemoryCell spec, so a stored item is the cell's own data rather than a copy converted at a boundary; the fields below are in-process only and never reach a cell.

func FilterVisibleToUser 

func FilterVisibleToUser(items []Item, userHash string) []Item

FilterVisibleToUser narrows items to those VisibleToUser allows for userHash.

func FindNearDuplicate 

func FindNearDuplicate(candidates []Item, item Item) (best Item, score float64, ok bool)

FindNearDuplicate returns the highest-scoring active near-duplicate of item among candidates. Candidates that are not Status == "active", or that share item's ID, are ignored — an item never duplicates itself, and a superseded/expired cell is not a live target to update in place.

func (Item) Clone 

func (i Item) Clone() Item

Clone is a copy that shares no slice with i.

func (Item) IsActive 

func (i Item) IsActive() bool

IsActive reports whether the item is active and not yet expired.

func (Item) IsExpired 

func (i Item) IsExpired() bool

IsExpired reports whether the item has a StaleAfter deadline that has passed.

func (Item) TitleWithEventTime 

func (i Item) TitleWithEventTime() string

TitleWithEventTime returns the item's Title with its EventTimeAnnotation appended, if any.

type Lexicon 

type Lexicon struct {
	// contains filtered or unexported fields
}

Lexicon is the store's vocabulary index on its own: a corpus-wide inverted index from the words of each document to the ids that contain them, which answers "which documents could match this query" without scanning them all. A host with its own documents (not memory items) uses it to prefilter a scan before scoring.

A query token matches a document when the token is a substring of one of the document's words, or when it is within one edit of a word (or of two adjacent words joined without a space) and is longer than two characters. The probing is length-banded: a token is compared only with the entries whose length is within one of its own, so the cost does not grow with the whole vocabulary. The set is exact, not a heuristic: it holds every document the store's own lexical match would admit for the token, so scoring the set finds everything a scan of all documents would. It is safe for concurrent use.

func NewLexicon 

func NewLexicon() *Lexicon

NewLexicon returns an empty Lexicon.

func (*Lexicon) Candidates 

func (l *Lexicon) Candidates(query string) map[string]struct{}

Candidates returns the ids of the documents that can match at least one token of query, the query being tokenised as the store tokenises one (punctuation removed, conversational filler dropped unless the query is only filler). A query with no tokens returns nil.

func (*Lexicon) CandidatesForTokens 

func (l *Lexicon) CandidatesForTokens(tokens []string) map[string]struct{}

CandidatesForTokens is Candidates for tokens the caller has already produced with KwTokens.

func (*Lexicon) DocFreq 

func (l *Lexicon) DocFreq(word string) int

DocFreq is the number of documents that contain word as a whole word, lower-cased: the figure a distinctiveness weight is computed from.

func (*Lexicon) Index 

func (l *Lexicon) Index(id, title, content string)

Index (re)indexes the document id from its title and content, replacing whatever id contributed before. Text is split into words the way the store splits an item's: lower-cased, punctuation stripped.

func (*Lexicon) Len 

func (l *Lexicon) Len() int

Len is the number of documents indexed.

func (*Lexicon) Remove 

func (l *Lexicon) Remove(id string)

Remove drops everything id contributed. An unknown id is a no-op.

type Locker 

type Locker interface {
	Lock()
	Unlock()
	RLock()
	RUnlock()
}

Locker is what the store needs of its host's lock. *sync.RWMutex satisfies it.

type PhaseStat 

type PhaseStat struct {
	AvgMs float64
	Count int64
}

PhaseStat is one hybridSearch phase's point-in-time latency readout.

type PhaseStats 

type PhaseStats struct {
	Lexical PhaseStat
	Embed   PhaseStat
	Index   PhaseStat
	Hydrate PhaseStat
	Rerank  PhaseStat
}

PhaseStats is hybridSearch's per-phase latency breakdown. lexical times the keyword scan; embed/index time the two halves of the cell store's semantic round trip (MemoryCellStore.SearchTimed); hydrate times resolving each semantic hit back to a memory.Item; rerank times Rerank. Recorded on whichever branch of hybridSearch actually runs a phase, so counts differ across phases by design (a confident lexical fast path never runs embed/index/hydrate).

type RefutedRecord 

type RefutedRecord struct {
	Reason   string   `json:"reason"`
	Evidence []string `json:"evidence"`
	At       string   `json:"at"`
}

RefutedRecord is the JSON held under RefutedDataKey.

type Removed 

type Removed struct{ ID, Kind string }

Removed names an item taken out of the store.

type ScoredItem 

type ScoredItem struct {
	Item  Item
	Score float32
}

ScoredItem is an item with the activation a traversal gave it.

type SearchConfig 

type SearchConfig struct {
	Store *Store
	// Cells is the vector index behind semantic search, nil when disabled. It is
	// read on every call, so the host may set it late.
	Cells func() *xindex.CellStore
	// Visible reports whether a partition may see item; DocVisible does the same
	// for an index entry before it is hydrated. The host owns what a partition
	// is. A nil predicate means everything is visible.
	Visible    func(item Item, partition string) bool
	DocVisible func(doc xindex.IndexEntry, partition string) bool
	// Hydrate reads a cell the store does not hold (kinds the host keeps out of
	// the working set) from where it lives.
	Hydrate func(ctx context.Context, id string) (Item, bool)
	// Live returns matches from state the host keeps outside the store (live
	// events, say), for a full search. exclude lists ids already in the result.
	Live func(query, kind, partition string, exclude map[string]struct{}) []SearchResult
	// State reports whether the item is known to the host and still open, for
	// kinds that have a lifecycle outside the store: an open one is boosted over
	// a resolved one, an unknown one is left alone. Anchor reports an item the
	// host keeps refreshed as a summary, which outranks lesser matches for a
	// situational query. CollapseKey returns the near-duplicate identity of an
	// item, or "" when it is not eligible: items sharing a key collapse into one.
	State       func(item Item) (open, known bool)
	Anchor      func(item Item) bool
	CollapseKey func(item Item) string
	// LinkExpansion turns on the one-hop ThoughtLink expansion of results. It is
	// read on every call.
	LinkExpansion func() bool
	// Dirty fires after every search, for a stats flush.
	Dirty func()
	// ResolveStrict maps a partition name to its canonical name without falling
	// back to a default; ok is false for an unknown name. Used by Consolidate.
	ResolveStrict func(partition string) (string, bool)
	Logger        zerolog.Logger
	// Stats is where the search counters also go; nil counts only in process.
	Stats StatsSink

	// SemanticGap, when positive, drops semantic matches that score more than
	// this far below the best match the caller can see. SemanticCliff keeps only
	// the matches above the steepest drop in score instead. Both are applied
	// after visibility filtering and before fusion, so what is cut is measured
	// against a hit the caller can read, never against one it cannot.
	SemanticGap   float32
	SemanticCliff bool

	// AlwaysFuse turns the lexical short-circuit off: with a vector index the
	// search always fuses the semantic channel, even when the top lexical item
	// looks like a confident answer. A long question over long cells shares
	// tokens with almost anything, so a lexical answer can look confident
	// without being the answer; a short named-entity query does not need the
	// embedding round trip. Off by default.
	AlwaysFuse bool
	// NoShaping turns the rerank and the near-duplicate collapse off: results
	// come back in fused order. Off by default.
	NoShaping bool
	// SourceRelation, when set, replaces a hit that has an outbound link of this
	// relation by the item it links to (a fact by the episode it was drawn from),
	// provided that item is active and visible; the best-ranked hit to reach a
	// source claims its place, and the source's Via is that hit. A dangling or
	// hidden source keeps the hit itself. Fanout is how many times the limit each
	// channel pulls while this runs, since collapse shrinks the pool again; zero
	// means three, and an explicit value is taken as given.
	SourceRelation string
	Fanout         int
	// SubjectRouting sends the semantic channel through the index of the subject
	// asked for, merged with the global one, instead of the whole index. Off by
	// default, when subject only filters the lexical channel.
	SubjectRouting bool
}

SearchConfig is what the Searcher asks of its host. Store is required; every other field may be nil, and a nil one behaves as the feature being absent.

type SearchResult 

type SearchResult struct {
	Item           Item
	Score          float64
	CollapsedCount int
	Via            string
}

SearchResult is one ranked hit. Score is the rerank score (base rank times the state, anchor, watch and recency multipliers) and orders results within one call only. CollapsedCount is >0 on the survivor of a near-duplicate collapse. Via is the ID of the direct hit whose link surfaced an expanded result, empty otherwise; an expanded result has Score 0.

type SearchStats 

type SearchStats struct {
	// contains filtered or unexported fields
}

SearchStats counts hybrid search: how often the lexical fast path served a confident answer versus how often it fell through to (or failed over from) a semantic round trip, and where the time of a call went. The counters are kept in process for Snapshot and also forwarded to the sink, if any. A nil *SearchStats counts nothing.

func NewSearchStats 

func NewSearchStats(sink StatsSink) *SearchStats

NewSearchStats counts in process and into sink (which may be nil).

func (*SearchStats) RecordHeadroom 

func (s *SearchStats) RecordHeadroom(rows, truncated, charsIn, charsOut int)

RecordHeadroom folds in one headroom.CompressRows call's row/char counts — see Agent.RecordSearchHeadroom, the exported entry point every compression call site uses.

func (*SearchStats) Snapshot 

func (s *SearchStats) Snapshot() SearchStatsSnapshot

Snapshot returns a point-in-time snapshot of hybrid search effectiveness: lexical fast-path hits vs. semantic fallthroughs.

type SearchStatsSnapshot 

type SearchStatsSnapshot struct {
	Total               int64
	KeywordHits         int64
	SemanticCalls       int64
	SemanticFallbacks   int64
	ExhaustiveFallbacks int64
	KWTrigramHits       int64
	KWVoiceHits         int64
	KWVoiceMisses       int64
	// Mean in-process duration of a HybridSearch call, split by the path it
	// took: lexical-only (confident fast path, no embedding round trip) versus
	// fused (lexical + semantic RRF). This is the number that separates search
	// cost from the caller's network and framing.
	AvgLexicalMs float64
	AvgFusedMs   float64
	LexicalCount int64
	FusedCount   int64

	Phases   PhaseStats
	ByCaller []CallerStat

	HeadroomRows      int64
	HeadroomTruncated int64
	HeadroomCharsIn   int64
	HeadroomCharsOut  int64
}

SearchStatsSnapshot is a point-in-time copy of the search counters.

func SnapshotFromOps 

func SnapshotFromOps(ops map[string]int64) SearchStatsSnapshot

SnapshotFromOps builds a snapshot from counters a host keeps itself, keyed by the counter names of this package (StatTotal and the others, with the .count and .ns suffixes a sink's Observe maps to). A host that sums its counters across replicas reads its ledger and calls this, instead of repeating the names.

type Searcher 

type Searcher struct {
	// contains filtered or unexported fields
}

Searcher is hybrid search over a Store: lexical first, semantic fused in when the lexical answer is not confident, then reranked.

func NewSearcher 

func NewSearcher(cfg SearchConfig) *Searcher

NewSearcher builds a searcher with its own search statistics.

func (*Searcher) Consolidate 

func (s *Searcher) Consolidate(partition string, apply bool) ConsolidationReport

Consolidate groups active stable_fact, operational_pattern, watch_item and hypothesis cells visible to partition into near-duplicate clusters and, when apply is true, supersedes every member but each cluster's keeper (highest Confidence, newest on a tie, per BetterRoutingFact). Superseding removes the members' cells from the store; the keeper's Supersedes field holds their IDs. apply=false returns the same groups without writing.

partition is required and resolved through ResolveStrict, never defaulting: an empty or unknown name fails closed with Error set. Cells are filtered by Visible, so in a multi-partition instance an untagged cell is excluded from every sweep.

func (*Searcher) HybridSearch 

func (s *Searcher) HybridSearch(ctx context.Context, query, kind, subject, partition string, limit int) []SearchResult

HybridSearch runs lexical search first and fuses in semantic results by Reciprocal Rank Fusion when the top lexical hit is not a confident (full-coverage) match and a cell store is available. A confident lexical hit returns without an embedding call, so cells outside the working set are only reached on the fused path; callers that need full-store coverage use HybridSearchFull.

partition is the caller's resolved partition (empty resolves to the default): results are limited to items tagged for it or untagged. Cross-partition results are never returned. PartitionAll disables the filter and is for read-only operator surfaces only.

func (*Searcher) HybridSearchFull 

func (s *Searcher) HybridSearchFull(ctx context.Context, query, kind, subject, partition string, limit int) []SearchResult

HybridSearchFull is HybridSearch without the confident-lexical fast path: it always consults the cell store, which holds the cells the working set excludes. A confident lexical match is only confident within the working set. It also adds the host's live matches and promotes an exact id, and costs one extra embedding call per search.

func (*Searcher) Rerank 

func (s *Searcher) Rerank(items []Item, limit int) []SearchResult

Rerank scores items (partition-filtered candidates in relevance order, index 0 best), collapses near-duplicates and truncates to limit. Two collapse passes run: key-based (the host's CollapseKey), then collapseNearDuplicateKnowledge. The output is deterministic: stable sorts with an ID tie-break.

func (*Searcher) Stats 

func (s *Searcher) Stats() *SearchStats

Stats is the search statistics, nil-safe through its methods.

func (*Searcher) Store 

func (s *Searcher) Store() *Store

Store is the store searched.

func (*Searcher) With 

func (s *Searcher) With(mut func(*SearchConfig)) *Searcher

With returns a searcher over the same store that shares this one's statistics and whose configuration is this one's after mut has changed it. The receiver is not modified, so a host swaps the copy in when it flips a switch at run time.

type StatsSink 

type StatsSink interface {
	Add(name string, n int64)
	Observe(name string, d time.Duration)
}

StatsSink receives the search counters as they happen, so a host can keep them where it keeps its other counters. Add adds n to a named counter; Observe records one duration under a name (a host typically keeps a count and a summed nanoseconds beside each name). Either may be called from several goroutines.

type Store 

type Store struct {
	// contains filtered or unexported fields
}

Store is the in-memory set of memory items with its lexical indexes. It shares its host's lock: the host may read the items and its own state together under one mutex, and the hooks in Config run inside it.

func NewStore 

func NewStore(opts ...StoreOption) *Store

NewStore builds an empty store. 

func (s *Store) AddLink(fromID string, link memoryv1.ThoughtLink) bool

AddLink appends a ThoughtLink to the cell fromID. It returns false if the cell does not exist, the link fails ThoughtLinkRejection, or the cell is at MaxModelLinks and the link does not out-weigh the lowest-weight one. A link with the same ToID and Relation replaces the old one and is exempt from the cap.

func (*Store) AllItems 

func (s *Store) AllItems() []Item

AllItems returns every active memory item, private ones included. It is not a general reader: it serves the sync that persists items as cells, which must keep writing a private item's cell back or the sync deletes it. Never wire it to a read surface a caller can reach.

func (*Store) CloneInto 

func (s *Store) CloneInto(l Locker, cfg Config) *Store

CloneInto copies the items into a new store guarded by mu, for a deep copy of the host's state. A cfg with no vocabulary inherits the source's, and the copy's lexical indexes are built from the copied items.

CloneInto takes no lock on the source: the caller holds the source's lock, read or write, for the whole call. That is what lets a host that shares one mutex between the store and its other state clone both under it without deadlocking on a second acquisition. A caller that does not share the lock and clones while writers run must take the store's own lock around the call (see WithLocker); an unguarded concurrent clone is a data race.

func (*Store) Context 

func (s *Store) Context(anchor string, maxDepth int) (anchors []Item, activated []ScoredItem)

Context returns what an anchor activates: the anchor items themselves and everything reachable from them over ThoughtLinks, by spreading activation. Each hop multiplies the score by 0.6 and the link's weight; at most maxDepth hops are followed (default 3, at most 8). Only shared, active items take part.

func (*Store) CountActive 

func (s *Store) CountActive() int

CountActive is the number of active items, private ones included.

func (*Store) Delete 

func (s *Store) Delete(id string) bool

Delete removes a single memory item by ID.

func (*Store) ExactMatches 

func (s *Store) ExactMatches(query, kind, subject string) []Item

ExactMatches returns the active, non-private items whose whole title, or the value of whose service/ tag, equals query once both are normalised: the items a caller who typed a name means. kind and subject narrow the match as they do in a keyword search; empty means any. The newest come first.

func (*Store) Expire 

func (s *Store) Expire(id string) bool

Expire marks a memory item as stale.

func (*Store) Filtered 

func (s *Store) Filtered(status, kind, scope, subject, confidence string, limit int) []Item

Filtered returns memory items filtered by status, kind, scope, subject and confidence; empty filters match all. Items past their stale_after deadline are lazily marked stale.

The subject filter matches the item's literal Subject or a service/<name> tag equal to subject, so get_memory_items(subject="auth") finds items that carry the service in a tag .

func (*Store) ForUser 

func (s *Store) ForUser(userHash, status, kind, scope, subject, confidence string, limit int) []Item

ForUser is Filtered plus userHash's own private items: a private item is included only when its PrivateOwner tag equals userHash. It is the only reader that returns a private item, and only to its owner (no admin bypass); an empty userHash behaves like Filtered.

func (*Store) ForgetLocked 

func (s *Store) ForgetLocked(match func(Item) bool) []Removed

ForgetLocked deletes every item match accepts, with its index entries, and returns what it removed. The caller holds the lock and, once it has released it, hands the result to PublishRemoved.

func (*Store) Get 

func (s *Store) Get(id string) (Item, bool)

Get returns a copy of one memory item by ID, passed through the Read hook.

func (*Store) GetFold 

func (s *Store) GetFold(id string, foldScan bool) (Item, bool)

GetFold is Get plus a case-insensitive fallback, so a query whose casing differs from the stored ID still resolves. The O(1) exact lookup always runs; the O(n) fold scan runs only when foldScan is true, which keeps the classifier's per-event hot path at the plain lookup.

func (*Store) Indexed 

func (s *Store) Indexed(id string) bool

Indexed reports whether id has entries in the lexical index.

func (*Store) Items 

func (s *Store) Items() []Item

Items returns all public memory items sorted by UpdatedAt descending, never a private item (IsPrivateItem) regardless of caller. Items past their stale_after deadline are lazily marked stale. ForUser is the only reader that can return a private item, to its owner; AllItems is the internal exception.

func (*Store) ItemsLocked 

func (s *Store) ItemsLocked() []Item

ItemsLocked is a copy of every item, private ones included. The caller holds the lock (the host derives a summary of the whole set from it).

func (*Store) KeywordSearch 

func (s *Store) KeywordSearch(query, kind, subject string, limit int) []Item

KeywordSearch scans active memory items by lexical match on title+content, gated by trigram-index candidates and ranked by lexScore.

func (*Store) KeywordSearchForUser 

func (s *Store) KeywordSearchForUser(userHash, query, kind, subject string, limit int) []Item

KeywordSearchForUser is KeywordSearch that also considers the private items owned by userHash, which no index holds, so they are scored by the lexical match alone. With no userHash it is KeywordSearch.

func (*Store) KeywordSearchOwn 

func (s *Store) KeywordSearchOwn(userHash, query, kind string, limit int) []Item

KeywordSearchOwn is KeywordSearch over the private items owned by userHash and nothing else, so shared items that score higher cannot push them out of the limit. With no userHash it returns nil.

func (*Store) KeywordSearchPerKind 

func (s *Store) KeywordSearchPerKind(query, kind, subject string, limit, perKind int) []Item

KeywordSearchPerKind is KeywordSearch with per-kind fairness: at most perKind results of any one kind, so a dominant kind cannot fill the limit and starve the smaller categories. Relevance order within a kind is kept and limit still caps the total. It suits a search that spans every kind over a skewed store.

func (*Store) KeywordSearchScored 

func (s *Store) KeywordSearchScored(query, kind, subject string, limit int) (items []Item, confident bool)

KeywordSearchScored runs a keyword search over active memory items and reports whether the top hit is a confident lexical answer: it covers every query token (all but one for 3+ tokens), even if per-token miss penalties lower its raw score. HybridSearch uses it to decide whether fusing in the semantic round trip is worth it.

func (*Store) KeywordSearchScoredWithTrigramHit 

func (s *Store) KeywordSearchScoredWithTrigramHit(query, kind, subject string, limit int) (items []Item, confident bool, trigramHit bool)

KeywordSearchScoredWithTrigramHit is KeywordSearchScored plus whether the trigram index contributed a candidate (see KeywordSearchWithTrigramHit).

func (*Store) KeywordSearchWithTrigramHit 

func (s *Store) KeywordSearchWithTrigramHit(query, kind, subject string, limit int) ([]Item, bool)

KeywordSearchWithTrigramHit is KeywordSearch plus whether the trigram index's candidate set contributed at least one of the final ranked hits (the signal behind a trigram-hit counter).

func (*Store) LenLocked 

func (s *Store) LenLocked() int

LenLocked is the number of items. The caller holds the lock.

func (*Store) MarkSuperseded 

func (s *Store) MarkSuperseded(id, byID string) bool

MarkSuperseded retires id's cell as superseded by byID, in place. Unlike Supersede it inserts no new cell: byID must already name an existing cell (ConsolidateMemory's cluster keeper). The link is recorded on the keeper's Supersedes, so a reader of byID finds every cell it absorbed. id's cell stays in the map with status superseded and leaves the lexical indexes. It returns false, writing nothing, when id or byID is empty, equal, or unknown.

func (*Store) Mirror 

func (s *Store) Mirror(item Item) bool

Mirror reflects a persisted MemoryCell spec into the in-memory map without acting as a mutation: no UpdatedAt stamp, and no notification when nothing changed. Routing cell reconciles through Upsert would feed the mutation-watch sync loop its own writes. A cell older than the in-memory item never clobbers it, since that is a pending local change. It returns true when in-memory state changed (a real external edit).

func (*Store) Mutate 

func (s *Store) Mutate(id string, fn func(item *Item) (changed bool)) (found bool)

Mutate applies fn to the item with id under the lock. When fn reports a change the item's UpdatedAt is stamped and the host told to rebuild what it derives. It reports whether the item exists. For a host-owned amendment of a single field (an item's evidence refs or tags); lexical fields must not be changed through it.

func (*Store) NotifyHit 

func (s *Store) NotifyHit(itemIDs []string, query, kind string, kinds []string)

NotifyHit fires the hit hook with a defensive copy of itemIDs (and kinds, when its length matches itemIDs). An empty hit is not an event; a kinds slice of the wrong length is dropped rather than forwarded misaligned.

func (*Store) PruneExcept 

func (s *Store) PruneExcept(keep map[string]struct{}) int

PruneExcept hard-deletes, without notifying, every item that is cell-originated (loaded from an existing cell) or soft-deleted (stale or superseded) and whose id is not in keep. A host calls it with a fresh list of its own cell ids. It closes two gaps: a cell-originated item whose cell was deleted without this instance seeing the delete, and expired or superseded items that would otherwise pile up for the life of the process. Active items authored in memory are never touched. This is read-model garbage collection, not a state change. It returns the number pruned. 

func (s *Store) PruneInvalidLinks() int

PruneInvalidLinks drops structurally invalid ThoughtLinks from every cell: an empty ToID or a self-link. AddLink rejects both at write time, so this removes only data from before that guard or from another write path.

It does not check that a link's target exists: data cells a host keeps are absent from this map by design, so that would delete legitimate edges. It dirties and signals only when a link was removed, or every cell would be rewritten on each sweep. It returns the number of links removed.

func (*Store) PublishRemoved 

func (s *Store) PublishRemoved(gone []Removed)

PublishRemoved announces items ForgetLocked removed, outside the lock.

func (*Store) Put 

func (s *Store) Put(item Item)

Put stores item exactly as given: no validation, no origin or timestamp stamping and no merge with an existing item, but with its lexical cache and index entries built. For a caller that owns every field of the item (the host's seeding of state from elsewhere).

func (*Store) Recent 

func (s *Store) Recent(kind, scope, subject string, limit int, skip func(Item) bool) []Item

Recent returns the active items matching kind, scope and subject (empty matches all) newest first by creation time, with the update time breaking ties, capped to limit. It is the recall path for memory selected by recency, the opposite of ranking by relevance. skip, when set, drops items the caller treats as expired. Only the 500 most recently updated candidates are sorted.

func (*Store) Refute 

func (s *Store) Refute(id, reason string, evidence []string, now time.Time) (refuted bool, cascaded []string)

Refute refutes a cell: it records data.refuted {reason, evidence, at}, expires the cell through Expire, then expires every active derived cell that cites it, one hop only (an evidenceRef naming the id, with or without a cell: prefix, or a link to it). Observed, human and verified cells are never cascaded. It returns whether the cell existed and the ids expired as a consequence.

func (*Store) ResolveAnchors 

func (s *Store) ResolveAnchors(anchor string) []string

ResolveAnchors maps an anchor string to ids of shared, active items: an exact id first (case-insensitively), then every item whose Subject is the anchor, then every item whose Subject or Title contains it. Private items never resolve.

func (*Store) Restore 

func (s *Store) Restore(items []Item)

Restore replaces the in-memory items with persisted ones from MemoryCell CRs at startup and rebuilds the lexical indexes.

func (*Store) SetChangeHook 

func (s *Store) SetChangeHook(fn func(op string, item *Item, id string, kind string))

SetChangeHook installs (or clears with nil) the callback run after a real memory item mutation, the bridge to the SSE "memory in use" stream. op is "upsert" (item set, id empty) or "delete" (item nil, id set); kind is the item's category, which for a delete is the only way it reaches the hook. It never fires on Mirror's no-op branch, and AddLink's link-only edits are not published.

func (*Store) SetHitHook 

func (s *Store) SetHitHook(fn func(itemIDs []string, query, kind string, kinds []string))

SetHitHook installs (or clears with nil) the callback invoked when a memory search surfaces results that reach a downstream consumer: itemIDs are the ids actually used, query the search string, kind the kind filter applied, and kinds each item's own kind, parallel to itemIDs (nil when the caller has none).

func (*Store) Subject 

func (s *Store) Subject() string

Subject is the instance identity the store was configured with.

func (*Store) Supersede 

func (s *Store) Supersede(oldID string, newItem Item) (Item, string)

Supersede marks the old item as superseded and inserts the new item with its Supersedes field pointing at the old ID.

func (*Store) Traverse 

func (s *Store) Traverse(anchorIDs []string, maxDepth int, decayFactor float32) []Item

Traverse spreads activation over the ThoughtLink graph from the anchor IDs, decaying by decayFactor per hop. It returns items by activation score descending, excluding the anchors and any private item, since the traversal has no caller identity to check an owner tag against. A maxDepth outside 1..8 means 8.

func (*Store) TraverseScored 

func (s *Store) TraverseScored(anchorIDs []string, maxDepth int, decayFactor float32) []ScoredItem

TraverseScored is Traverse that also returns the activation of each item. Each hop multiplies the score by decayFactor and the link's weight, an item keeps the best score any path gave it, and ties order by ID.

func (*Store) Upsert 

func (s *Store) Upsert(item Item) (Item, string)

Upsert adds or updates a single memory item. Returns an error string if validation fails; item is unchanged in that case.

func (*Store) UpsertDedup 

func (s *Store) UpsertDedup(item Item) (result Item, mergedInto string, validationErr string)

UpsertDedup is the write gate for model-authored knowledge cells: a caller restating an existing fact under a fresh ID lands on the same cell instead of minting a copy.

It delegates to Upsert unchanged (mergedInto == "") when item.ID names an existing cell or the kind is not in memoryDedupEligibleKinds. Hydration and machine-state note paths call Upsert directly and never dedup.

Otherwise, under the lock, it scores item against every active cell of the same Kind (FindNearDuplicate) and on a hit updates that cell in place: ID, CreatedAt, Links, Supersedes and CellOriginated are kept, every other field comes from item, Tags is the union (existing first) and UpdatedAt is bumped. No hit inserts item as Upsert would.

func (*Store) ValidateBase 

func (s *Store) ValidateBase(item Item) string

ValidateBase enforces the writing rules that hold for every item and every host; the host's own rules (Config.Validate, Vocabulary) come on top. It returns a message describing the violation, or "" when the item is valid.

func (*Store) Vocabulary 

func (s *Store) Vocabulary() Vocabulary

Vocabulary is the kind vocabulary the store was configured with.

type StoreOption 

type StoreOption func(*Store)

StoreOption configures NewStore.

func WithConfig 

func WithConfig(cfg Config) StoreOption

WithConfig sets the host seams.

func WithLocker 

func WithLocker(l Locker) StoreOption

WithLocker makes the store use the host's lock instead of a private one.

type Vocabulary 

type Vocabulary struct {
	// Rejected lists kinds a model-authored write may not use, such as kinds with
	// their own writers.
	Rejected []string
	// Writable lists the kinds an outside writer (a model, a voice session, an
	// API caller) may write through ValidateWrite. Empty means any. Kinds the host
	// writes itself stay out of it.
	Writable []string
	// Searchable lists the kinds a kind filter may name. Empty means any.
	Searchable []string
	// NonSearchable lists kinds left out of an unfiltered search but valid as an
	// explicit filter or a fetch by id (prompt and identity documents, say).
	NonSearchable []string
	// Coerce rewrites an item's kind before it is validated, for a host that
	// reserves a kind for its own writers and turns a model's claim to it into
	// another kind. It runs with no lock held.
	Coerce func(item *Item)
	// FactConfidenceOptional turns off the store's own rule that a stable fact
	// carries high confidence, for a host that enforces it where outside writers
	// arrive (ValidateWrite) and lets its own code write facts as it likes.
	FactConfidenceOptional bool
	// DataKinds reports kinds the host keeps out of the working set and reads on
	// demand through SearchConfig.Hydrate.
	DataKinds func(kind string) bool
}

Vocabulary is the host's kinds. The zero value restricts nothing: every kind may be written, every kind may be named in a filter, and no kind is hidden from an unfiltered search.

func BaseVocabulary 

func BaseVocabulary() Vocabulary

BaseVocabulary is the kinds X-Ray's own server uses: what an outside writer may write, what a filter may name, and the kinds the server writes itself (episodic, mental_map), which are searchable but not writable. A cell of kind system_configuration is never indexed, so it is left out of the searchable list on purpose: a filter on it could only return nothing.

func (Vocabulary) IsData 

func (v Vocabulary) IsData(kind string) bool

IsData reports whether the host keeps kind out of the working set.

func (Vocabulary) IsNonSearchable 

func (v Vocabulary) IsNonSearchable(kind string) bool

IsNonSearchable reports whether kind is left out of an unfiltered search.

func (Vocabulary) IsRejected 

func (v Vocabulary) IsRejected(kind string) bool

IsRejected reports whether a model-authored write of kind is refused.

func (Vocabulary) IsSearchable 

func (v Vocabulary) IsSearchable(kind string) bool

IsSearchable reports whether kind may be named in a kind filter. The empty string (no filter) is deliberately not searchable, so an unset kind cannot pass a check written the wrong way round; a vocabulary with no list accepts any non-empty kind.

func (Vocabulary) ValidateSearchKind 

func (v Vocabulary) ValidateSearchKind(kind string) string

ValidateSearchKind rejects a kind filter that names no searchable kind, which would otherwise return zero results indistinguishable from "nothing matches". An empty filter means every kind and is valid.

func (Vocabulary) ValidateWrite 

func (v Vocabulary) ValidateWrite(kind, confidence string) string

ValidateWrite checks what an outside writer may put in a cell: a kind from the writable list, and high confidence on a stable fact (a fact that is stable is one the writer is sure of). "" means valid.

Source Files

View all Source files

Directories

Path Synopsis
Package cellsync holds the decisions a controller makes when it reconciles a MemoryCell into a memory store: whether the cell is deleted at its deadline, whether it is embedded, seeded or left out of the vector index, when to look at it again.
 Package cellsync holds the decisions a controller makes when it reconciles a MemoryCell into a memory store: whether the cell is deleted at its deadline, whether it is embedded, seeded or left out of the vector index, when to look at it again.
Package hub carries memory events to connected clients: a search hit, a change to an item, a metrics snapshot.
 Package hub carries memory events to connected clients: a search hit, a change to an item, a metrics snapshot.
Package memtest holds fixtures the tests of memstore and of its host share: the near-duplicate groups that tune the dedup threshold.
 Package memtest holds fixtures the tests of memstore and of its host share: the near-duplicate groups that tune the dedup threshold.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.