Documentation
¶
Overview ¶
Package similarity computes topic similarity between documents. The pairwise scoring engine (edge batching, posting-index pruning, worker fan-out) lives in pairwise.go and posting.go; text tokenization in tokenize.go; the vector/set math primitives in vectormath.go. This file holds the document feature model and the public Compute* entry points.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func ComputeNeuralSimilarityForDoc ¶ added in v0.2.0
func ComputeNeuralSimilarityForDoc(st SimilarityStore, docID, modelID string, threshold float64) error
ComputeNeuralSimilarityForDoc recomputes neural similar_to edges for a single document against all other documents that share the same model_id embedding. Existing neural edges for docID are replaced. Uses the stored similarity threshold from project_metadata (default 0.25).
func ComputeSimilarity ¶
func ComputeSimilarity(st SimilarityStore, threshold float64) error
ComputeSimilarity computes pairwise topic similarity between all documents using a hybrid of TF-IDF text similarity, shared reference targets, and tag overlap. Edges of kind "similar_to" are inserted for pairs whose composite score meets the threshold.
func ComputeSimilarityIncremental ¶ added in v0.2.0
func ComputeSimilarityIncremental(st SimilarityStore, changedDocIDs []string, threshold float64) error
ComputeSimilarityIncremental recomputes similar_to edges for docs in changedDocIDs only, leaving other pairs untouched. Falls back to a full rebuild when: changedDocIDs is empty, k > n/2, or the threshold changed.
IDF drift note: adding a new doc shifts IDF for its terms, which can slightly alter scores for unchanged pairs. These pairs are corrected on their next change or a forced full rebuild (--force on index).
Types ¶
type SimilarityStore ¶ added in v0.3.2
type SimilarityStore interface {
GetAllDocumentNodes() ([]store.Node, error)
GetEdgesBySource(sourceID string) ([]store.Edge, error)
DeleteEdgesByKind(kind string) error
DeleteSimilarityEdgesForDocs(nodeIDs []string) error
DeleteNeuralSimilarityEdgesForDoc(docID string) error
InsertEdges(edges []store.Edge) error
GetEmbeddingsByModel(modelID string) ([]store.Embedding, error)
GetProjectMeta(key string) (string, bool, error)
UpsertProjectMeta(key, value string) error
}
SimilarityStore is the narrow slice of *store.Store that the similarity engine depends on. Decoupling from the 125-method god type keeps this package's contract explicit and testable: callers still pass a concrete *store.Store (which satisfies this interface), so no call site changes. The methods cover document/edge reads (full and incremental rebuilds), similar_to edge cleanup and insertion (TF-IDF and neural paths), neural embedding reads, and the project-metadata threshold cache.
Source Files