rag

package
v0.11.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 17, 2026 License: MIT Imports: 33 Imported by: 0

Documentation

Overview

Package rag implements Retrieval-Augmented Generation for long-term memory.

Data Flow: 

Query → Enricher → Vector Search → Reranker → Context Assembly

1. Enricher (optional): Flash LLM expands query with context
2. Vector Search: Cosine similarity over topics/facts/people/artifacts embeddings
3. Reranker (optional): Flash LLM selects most relevant candidates (agentic)
4. Context Assembly: Loads full content for selected items

Key Invariants:

  • All vectors are user-isolated (map[userID][]vectors)
  • Reranker is optional; fallback to vector top-N if disabled/error
  • Background loop runs every minute for topic/fact extraction
  • Vector store is in-memory; rebuilt from DB on startup

Configuration:

  • RAG.Enabled: Enable/disable RAG system
  • RAG.MinSafetyThreshold: Minimum cosine similarity (default 0.1)
  • RAG.RetrievedTopicsCount: Max topics without reranker (default 10)
  • Agents.Reranker: Flash LLM filtering config
  • Agents.Enricher: Query expansion config

Vector Types:

  • TopicVectorItem: Topic embeddings for conversation search
  • FactVectorItem: Fact embeddings for profile facts
  • PersonVectorItem: Person embeddings for social graph
  • ArtifactVectorItem: File summary embeddings for artifact search

Thread Safety:

  • All vector searches use RLock (concurrent reads OK)
  • Vector updates use Lock (blocks reads during update)
  • Service methods are generally NOT thread-safe except where noted

Main Entry Point: 

Service.Retrieve() - Main RAG pipeline for message processing

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func FilterProfileFacts added in v0.4.8 

func FilterProfileFacts(facts []storage.Fact) []storage.Fact

FilterProfileFacts filters facts to identity and high-importance facts only. Delegates to storage.FilterProfileFacts.

func FormatRecentTopics added in v0.4.8 

func FormatRecentTopics(topics []storage.TopicExtended) string

FormatRecentTopics formats recent topics for inclusion in agent prompts. Delegates to storage.FormatRecentTopics.

func FormatUserProfile added in v0.4.8 

func FormatUserProfile(facts []storage.Fact) string

FormatUserProfile formats user facts for inclusion in agent prompts. Delegates to storage.FormatUserProfile.

func FormatUserProfileCompact added in v0.6.1 

func FormatUserProfileCompact(facts []storage.Fact) string

FormatUserProfileCompact formats user facts without Fact IDs. Delegates to storage.FormatUserProfileCompact.

func RecordArtifactExtraction added in v0.6.0 

func RecordArtifactExtraction(userID storage.ScopeID, durationSeconds float64, success bool)

RecordArtifactExtraction records metrics for artifact extraction job (v0.6.0).

func RecordEmbeddingRequest added in v0.3.0 

func RecordEmbeddingRequest(userID storage.ScopeID, model string, embeddingType string, durationSeconds float64, success bool, tokens int, cost *float64)

RecordEmbeddingRequest records embedding request metrics. embeddingType: searchTypeTopics or searchTypeFacts

func RecordRAGCandidates added in v0.3.3 

func RecordRAGCandidates(userID storage.ScopeID, searchType string, count int)

RecordRAGCandidates records the number of candidates before filtering.

func RecordRAGEnrichment added in v0.3.7 

func RecordRAGEnrichment(userID storage.ScopeID, durationSeconds float64)

RecordRAGEnrichment records the LLM call time for query enrichment.

func RecordRAGLatency added in v0.3.6 

func RecordRAGLatency(userID storage.ScopeID, source string, durationSeconds float64)

RecordRAGLatency records total RAG retrieval time. source: "auto" for buildContext, "tool" for the search_history tool.

func RecordRAGRetrieval added in v0.3.0 

func RecordRAGRetrieval(userID storage.ScopeID, hasContext bool)

RecordRAGRetrieval records the result of a RAG retrieval.

func RecordRerankerCandidatesInput added in v0.4.1 

func RecordRerankerCandidatesInput(userID storage.ScopeID, count int)

RecordRerankerCandidatesInput records the number of input candidates.

func RecordRerankerCandidatesOutput added in v0.4.1 

func RecordRerankerCandidatesOutput(userID storage.ScopeID, count int)

RecordRerankerCandidatesOutput records the number of selected candidates.

func RecordRerankerCost added in v0.4.1 

func RecordRerankerCost(userID storage.ScopeID, cost float64)

RecordRerankerCost records reranker cost.

func RecordRerankerDuration added in v0.4.1 

func RecordRerankerDuration(userID storage.ScopeID, durationSeconds float64)

RecordRerankerDuration records reranker operation duration.

func RecordRerankerFallback added in v0.4.1 

func RecordRerankerFallback(userID storage.ScopeID, reason string)

RecordRerankerFallback records a fallback activation. reason: "timeout", "turn_timeout", "llm_error", "empty_response", "parse_error", "protocol_violation", "model_empty", "all_hallucinated".

func RecordRerankerHallucination added in v0.4.8 

func RecordRerankerHallucination(userID storage.ScopeID, count int)

RecordRerankerHallucination records the number of hallucinated topic IDs.

func RecordRerankerToolCalls added in v0.4.1 

func RecordRerankerToolCalls(userID storage.ScopeID, count int)

RecordRerankerToolCalls records the number of tool calls.

func RecordVectorSearch added in v0.3.0 

func RecordVectorSearch(userID storage.ScopeID, searchType string, durationSeconds float64, vectorsScanned int)

RecordVectorSearch records vector search metrics.

func UpdateArtifactSummaryMetrics added in v0.6.0 

func UpdateArtifactSummaryMetrics(perUserCounts map[storage.ScopeID]int)

UpdateArtifactSummaryMetrics updates the artifact summary index size metric (v0.6.0).

func UpdateVectorIndexMetrics added in v0.3.0 

func UpdateVectorIndexMetrics(topicsCount, factsCount, peopleCount, dim int)

UpdateVectorIndexMetrics updates index size metrics. dim should be the configured embedding dimension; used to derive a ballpark RAM estimate for the vector_index_memory_bytes metric.

Types

type ActiveSessionInfo added in v0.3.0 

type ActiveSessionInfo struct {
	UserID           storage.ScopeID
	MessageCount     int
	FirstMessageTime time.Time
	LastMessageTime  time.Time
	ContextSize      int // Total characters in session messages
}

ActiveSessionInfo contains information about unprocessed messages for a user.

type ArtifactResult added in v0.6.0 

type ArtifactResult struct {
	ArtifactID   int64
	FileType     string
	OriginalName string
	Summary      string
	Keywords     []string
	Entities     []string // Named entities (people, companies, code mentioned)
	RAGHints     []string // Questions this artifact might answer
	Score        float32
}

ArtifactResult represents a matched artifact with its metadata (v0.6.0).

type ContaminationFixStats added in v0.4.6 

type ContaminationFixStats struct {
	MessagesUnlinked      int64 `json:"messages_unlinked"`
	OrphanedTopicsDeleted int   `json:"orphaned_topics_deleted"`
}

ContaminationFixStats contains statistics about contamination repair.

type ContaminationInfo added in v0.4.6 

type ContaminationInfo struct {
	TotalContaminated  int                         `json:"total_contaminated"`
	ContaminatedTopics []storage.ContaminatedTopic `json:"contaminated_topics"`
}

ContaminationInfo contains information about cross-user data contamination.

type DatabaseHealth added in v0.4.6 

type DatabaseHealth struct {
	TotalTopics          int               `json:"total_topics"`
	OrphanedTopics       int               `json:"orphaned_topics"`
	ZeroSizeTopics       int               `json:"zero_size_topics"`
	LargeTopics          int               `json:"large_topics"`
	OverlappingPairs     int               `json:"overlapping_pairs"`
	FactsOnOrphaned      int               `json:"facts_on_orphaned"`
	AvgTopicSize         int               `json:"avg_topic_size"`
	LargeTopicsList      []TopicInfo       `json:"large_topics_list,omitempty"`
	OverlappingPairsList []OverlappingPair `json:"overlapping_pairs_list,omitempty"`
}

DatabaseHealth contains diagnostic information about the database.

type ExtractedTopic 

type ExtractedTopic struct {
	Summary    string `json:"summary"`
	StartMsgID int64  `json:"start_msg_id"`
	EndMsgID   int64  `json:"end_msg_id"`
}

type MaintenanceService added in v0.6.1 

type MaintenanceService interface {
	// Database health and repair
	GetDatabaseHealth(ctx context.Context, userID storage.ScopeID, largeThreshold int) (*DatabaseHealth, error)
	RepairDatabase(ctx context.Context, userID storage.ScopeID, dryRun bool) (*RepairStats, error)

	// Contamination management
	GetContaminationInfo(ctx context.Context, userID storage.ScopeID) (*ContaminationInfo, error)
	FixContamination(ctx context.Context, userID storage.ScopeID, dryRun bool) (*ContaminationFixStats, error)

	// Topic management
	SplitLargeTopics(ctx context.Context, userID storage.ScopeID, thresholdChars int) (*SplitStats, error)
}

MaintenanceService provides database maintenance and diagnostic operations. This interface allows web handlers to be tested without the full RAG service.

type MemoryService added in v0.6.1 

type MemoryService interface {
	ProcessSession(ctx context.Context, userID storage.ScopeID, messages []storage.Message, topicDate time.Time, topicID int64) error
	ProcessSessionWithStats(ctx context.Context, userID storage.ScopeID, messages []storage.Message, topicDate time.Time, topicID int64) (memory.FactStats, error)
}

MemoryService defines the interface for memory service operations used by RAG. This allows for easier testing and decoupling from the concrete implementation.

type OverlappingPair added in v0.4.6 

type OverlappingPair struct {
	Topic1ID      int64  `json:"topic1_id"`
	Topic1Summary string `json:"topic1_summary"`
	Topic2ID      int64  `json:"topic2_id"`
	Topic2Summary string `json:"topic2_summary"`
}

OverlappingPair contains information about two overlapping topics.

type PersonSearchResult added in v0.5.1 

type PersonSearchResult struct {
	Person storage.Person
	Score  float32
}

PersonSearchResult represents a matched person with their score.

type ProcessingStats added in v0.3.0 

type ProcessingStats struct {
	MessagesProcessed int `json:"messages_processed"`
	TopicsExtracted   int `json:"topics_extracted"`
	TopicsMerged      int `json:"topics_merged"`
	FactsCreated      int `json:"facts_created"`
	FactsUpdated      int `json:"facts_updated"`
	FactsDeleted      int `json:"facts_deleted"`
	// People stats (v0.5.1)
	PeopleAdded   int `json:"people_added"`
	PeopleUpdated int `json:"people_updated"`
	PeopleMerged  int `json:"people_merged"`
	// Usage stats from API calls
	PromptTokens     int      `json:"prompt_tokens"`
	CompletionTokens int      `json:"completion_tokens"`
	EmbeddingTokens  int      `json:"embedding_tokens"`
	TotalCost        *float64 `json:"total_cost,omitempty"` // Cost in USD
}

ProcessingStats contains detailed statistics about session processing.

func (*ProcessingStats) AddChatUsage added in v0.3.0 

func (s *ProcessingStats) AddChatUsage(promptTokens, completionTokens int, cost *float64)

AddChatUsage adds usage from a chat completion response.

func (*ProcessingStats) AddEmbeddingUsage added in v0.3.0 

func (s *ProcessingStats) AddEmbeddingUsage(tokens int, cost *float64)

AddEmbeddingUsage adds usage from an embedding response.

type ProgressCallback added in v0.3.0 

type ProgressCallback func(event ProgressEvent)

ProgressCallback is called during processing to report progress.

type ProgressEvent added in v0.3.0 

type ProgressEvent struct {
	Stage    string           `json:"stage"`           // "topics", "consolidation", "facts"
	Current  int              `json:"current"`         // Current item being processed
	Total    int              `json:"total"`           // Total items to process
	Message  string           `json:"message"`         // Human-readable status message
	Complete bool             `json:"complete"`        // True when processing is complete
	Stats    *ProcessingStats `json:"stats,omitempty"` // Final stats when complete
}

ProgressEvent represents a progress update during processing.

type RepairStats added in v0.4.6 

type RepairStats struct {
	OrphanedTopicsDeleted int `json:"orphaned_topics_deleted"`
	FactsRelinked         int `json:"facts_relinked"`
	RangesRecalculated    int `json:"ranges_recalculated"`
	SizesRecalculated     int `json:"sizes_recalculated"`
}

RepairStats contains statistics about repair operations.

type RetrievalDebugInfo 

type RetrievalDebugInfo struct {
	OriginalQuery                string
	EnrichedQuery                string
	EnrichmentPrompt             string
	EnrichmentTokens             int
	Candidates                   []TopicCandidateDebug
	PostReranker                 []TopicSearchResult
	FinalContext                 []TopicSearchResult
	Results                      []TopicSearchResult // Backward-compatible alias for PostReranker.
	UsedReranker                 bool
	RerankerFallback             string
	RerankerInputTokensEstimated int
}

RetrievalDebugInfo contains trace data for RAG debugging

type RetrievalOptions 

type RetrievalOptions struct {
	History        []storage.Message
	SkipEnrichment bool
	Source         string        // "auto" for buildContext, "tool" for search_history
	MediaParts     []interface{} // Multimodal content (images, audio) for enricher and reranker
}

type RetrievalResult added in v0.5.1 

type RetrievalResult struct {
	Topics              []TopicSearchResult
	People              []storage.Person // Selected by reranker (excludes inner circle)
	Artifacts           []ArtifactResult // v0.6.0: Artifact summary matches
	SelectedArtifactIDs []int64          // v0.6.0: Artifact IDs selected by reranker for full content loading
}

RetrievalResult contains both topics and selected people from RAG retrieval.

type Retriever added in v0.6.0 

type Retriever interface {
	// GetRecentTopics returns the N most recent topics for a user with message counts.
	GetRecentTopics(userID storage.ScopeID, limit int) ([]storage.TopicExtended, error)

	// Retrieve performs RAG retrieval for a query.
	Retrieve(ctx context.Context, userID storage.ScopeID, query string, opts *RetrievalOptions) (*RetrievalResult, *RetrievalDebugInfo, error)
}

Retriever is the interface used by agents that need RAG functionality. This allows agents to be tested with mocks instead of the full Service.

type SearchResult 

type SearchResult struct {
	Message storage.Message
	Score   float32
}

SearchResult kept for backward compatibility if needed, but we are moving to TopicSearchResult. We remove it to force type errors in other files to fix them.

type Service 

type Service struct {
	// contains filtered or unexported fields
}

Service provides RAG (Retrieval-Augmented Generation) functionality.

func (*Service) FindSimilarFacts 

func (s *Service) FindSimilarFacts(ctx context.Context, userID storage.ScopeID, embedding []float32, threshold float32) ([]storage.Fact, error)

func (*Service) FixContamination added in v0.4.6 

func (s *Service) FixContamination(ctx context.Context, userID storage.ScopeID, dryRun bool) (*ContaminationFixStats, error)

FixContamination removes foreign messages from contaminated topics. If userID is 0, fixes all users.

func (*Service) ForceProcessUser 

func (s *Service) ForceProcessUser(ctx context.Context, userID storage.ScopeID) (int, error)

func (*Service) ForceProcessUserWithProgress added in v0.3.0 

func (s *Service) ForceProcessUserWithProgress(ctx context.Context, userID storage.ScopeID, onProgress ProgressCallback) (*ProcessingStats, error)

ForceProcessUserWithProgress processes all unprocessed messages for a user with progress reporting. Unlike ForceProcessUser, this runs consolidation and fact extraction synchronously.

func (*Service) GetActiveSessions added in v0.3.0 

func (s *Service) GetActiveSessions() ([]ActiveSessionInfo, error)

GetActiveSessions returns information about unprocessed messages (active sessions) for all users.

func (*Service) GetContaminationInfo added in v0.4.6 

func (s *Service) GetContaminationInfo(ctx context.Context, userID storage.ScopeID) (*ContaminationInfo, error)

GetContaminationInfo returns information about cross-user data contamination. If userID is 0, checks all users.

func (*Service) GetDatabaseHealth added in v0.4.6 

func (s *Service) GetDatabaseHealth(ctx context.Context, userID storage.ScopeID, largeThreshold int) (*DatabaseHealth, error)

GetDatabaseHealth returns diagnostic information about database health. If userID is 0, returns stats for all users.

func (*Service) GetRecentTopics added in v0.4.6 

func (s *Service) GetRecentTopics(userID storage.ScopeID, limit int) ([]storage.TopicExtended, error)

GetRecentTopics returns the N most recent topics for a user with message counts.

func (*Service) LoadNewArtifactSummaries added in v0.6.0 

func (s *Service) LoadNewArtifactSummaries() error

LoadNewArtifactSummaries incrementally loads only new artifact summary embeddings. Called after artifact processing completes.

func (*Service) LoadNewVectors added in v0.2.1 

func (s *Service) LoadNewVectors() error

LoadNewVectors incrementally loads only new topics, facts, and people since last load.

func (*Service) ReembedIfNeeded added in v0.7.0 

func (s *Service) ReembedIfNeeded(ctx context.Context) error

ReembedIfNeeded detects embeddings produced under a different model/dim than the currently configured one and re-generates them in-place. It is intended to run once on startup, synchronously, before vector-index load — see Service.Start.

The operation is resumable: each row is persisted as soon as its new embedding is available, so a crash or context cancellation leaves the DB in a consistent "partially migrated" state. A second run picks up where the first left off.

func (*Service) ReloadVectors 

func (s *Service) ReloadVectors() error

ReloadVectors fully reloads all topic, fact, and people vectors from the database.

func (*Service) RepairDatabase added in v0.4.6 

func (s *Service) RepairDatabase(ctx context.Context, userID storage.ScopeID, dryRun bool) (*RepairStats, error)

RepairDatabase fixes database integrity issues.

func (*Service) Retrieve 

func (s *Service) Retrieve(ctx context.Context, userID storage.ScopeID, query string, opts *RetrievalOptions) (*RetrievalResult, *RetrievalDebugInfo, error)

func (*Service) SearchArtifactsBySummary added in v0.6.0 

func (s *Service) SearchArtifactsBySummary(
	ctx context.Context,
	userID storage.ScopeID,
	embedding []float32,
	threshold float32,
) ([]ArtifactResult, error)

SearchArtifactsBySummary searches for artifacts by summary embedding similarity. Returns top N artifacts.

func (*Service) SearchPeople added in v0.5.1 

func (s *Service) SearchPeople(ctx context.Context, userID storage.ScopeID, embedding []float32, threshold float32, maxResults int, excludeCircles []string) ([]PersonSearchResult, error)

SearchPeople searches for people by vector similarity (v0.5.1). excludeCircles filters out people from specified circles (e.g., "Work_Inner", "Family").

func (*Service) SetAgentLogger added in v0.4.8 

func (s *Service) SetAgentLogger(logger *agentlog.Logger)

SetAgentLogger sets the agent logger for debugging LLM calls.

func (*Service) SetArtifactRepository added in v0.6.0 

func (s *Service) SetArtifactRepository(artifactRepo storage.ArtifactRepository)

SetArtifactRepository sets the artifact repository.

func (*Service) SetContextService added in v0.6.0 

func (s *Service) SetContextService(cs *agent.ContextService)

SetContextService sets the context service for loading user context.

func (*Service) SetEnricherAgent added in v0.5.0 

func (s *Service) SetEnricherAgent(a agent.Agent)

SetEnricherAgent sets the query enrichment agent.

func (*Service) SetExtractorAgent added in v0.6.0 

func (s *Service) SetExtractorAgent(a agent.Agent)

SetExtractorAgent sets the artifact extraction agent.

func (*Service) SetMergerAgent added in v0.5.0 

func (s *Service) SetMergerAgent(a agent.Agent)

SetMergerAgent sets the topic merging agent.

func (*Service) SetPeopleRepository added in v0.5.1 

func (s *Service) SetPeopleRepository(pr storage.PeopleRepository)

SetPeopleRepository sets the people repository for person search (v0.5.1).

func (*Service) SetRerankerAgent added in v0.5.0 

func (s *Service) SetRerankerAgent(a agent.Agent)

SetRerankerAgent sets the topic reranking agent.

func (*Service) SetSplitterAgent added in v0.5.0 

func (s *Service) SetSplitterAgent(a agent.Agent)

SetSplitterAgent sets the topic splitting agent.

func (*Service) SplitLargeTopics added in v0.4.6 

func (s *Service) SplitLargeTopics(ctx context.Context, userID storage.ScopeID, thresholdChars int) (*SplitStats, error)

SplitLargeTopics finds and splits all topics larger than threshold. If userID is 0, processes all users.

func (*Service) Start 

func (s *Service) Start(ctx context.Context) error

Start initializes and starts the RAG service background loops.

func (*Service) Stop 

func (s *Service) Stop()

Stop gracefully shuts down the RAG service. Signals to stop accepting new work and waits for in-flight operations to complete.

func (*Service) TriggerConsolidation 

func (s *Service) TriggerConsolidation()

TriggerConsolidation triggers a consolidation run.

type ServiceBuilder added in v0.6.1 

type ServiceBuilder struct {
	// contains filtered or unexported fields
}

ServiceBuilder provides a fluent API for constructing RAG Service instances. All required dependencies must be set before calling Build(). Optional dependencies (agents, repos) can be set via With* methods.

func NewServiceBuilder added in v0.6.1 

func NewServiceBuilder() *ServiceBuilder

NewServiceBuilder creates a new ServiceBuilder for fluent construction.

func (*ServiceBuilder) Build added in v0.6.1 

func (b *ServiceBuilder) Build() (*Service, error)

Build validates all dependencies and creates the RAG Service. Returns an error if any required dependency is missing.

func (*ServiceBuilder) WithAgentLogger added in v0.6.1 

func (b *ServiceBuilder) WithAgentLogger(l *agentlog.Logger) *ServiceBuilder

WithAgentLogger sets the optional agent logger dependency.

func (*ServiceBuilder) WithArtifactRepository added in v0.6.1 

func (b *ServiceBuilder) WithArtifactRepository(r storage.ArtifactRepository) *ServiceBuilder

WithArtifactRepository sets the optional artifact repository dependency.

func (*ServiceBuilder) WithConfig added in v0.6.1 

func (b *ServiceBuilder) WithConfig(c *config.Config) *ServiceBuilder

WithConfig sets the required config dependency.

func (*ServiceBuilder) WithContextService added in v0.6.1 

func (b *ServiceBuilder) WithContextService(cs *agent.ContextService) *ServiceBuilder

WithContextService sets the optional context service dependency.

func (*ServiceBuilder) WithEnricher added in v0.6.1 

func (b *ServiceBuilder) WithEnricher(a agent.Agent) *ServiceBuilder

WithEnricher sets the optional enricher agent dependency.

func (*ServiceBuilder) WithExtractor added in v0.6.1 

func (b *ServiceBuilder) WithExtractor(a agent.Agent) *ServiceBuilder

WithExtractor sets the optional extractor agent dependency.

func (*ServiceBuilder) WithFactHistoryRepository added in v0.6.1 

func (b *ServiceBuilder) WithFactHistoryRepository(r storage.FactHistoryRepository) *ServiceBuilder

WithFactHistoryRepository sets the required fact history repository dependency.

func (*ServiceBuilder) WithFactRepository added in v0.6.1 

func (b *ServiceBuilder) WithFactRepository(r storage.FactRepository) *ServiceBuilder

WithFactRepository sets the required fact repository dependency.

func (*ServiceBuilder) WithLLMClient added in v0.10.1 

func (b *ServiceBuilder) WithLLMClient(c llm.Client) *ServiceBuilder

WithLLMClient sets the required LLM client dependency.

func (*ServiceBuilder) WithLogger added in v0.6.1 

func (b *ServiceBuilder) WithLogger(l *slog.Logger) *ServiceBuilder

WithLogger sets the required logger dependency.

func (*ServiceBuilder) WithMaintenanceRepository added in v0.6.1 

func (b *ServiceBuilder) WithMaintenanceRepository(r storage.MaintenanceRepository) *ServiceBuilder

WithMaintenanceRepository sets the required maintenance repository dependency.

func (*ServiceBuilder) WithMemoryService added in v0.6.1 

func (b *ServiceBuilder) WithMemoryService(s MemoryService) *ServiceBuilder

WithMemoryService sets the required memory service dependency.

func (*ServiceBuilder) WithMerger added in v0.6.1 

func (b *ServiceBuilder) WithMerger(a agent.Agent) *ServiceBuilder

WithMerger sets the optional merger agent dependency.

func (*ServiceBuilder) WithMessageRepository added in v0.6.1 

func (b *ServiceBuilder) WithMessageRepository(r storage.MessageRepository) *ServiceBuilder

WithMessageRepository sets the required message repository dependency.

func (*ServiceBuilder) WithPeopleRepository added in v0.6.1 

func (b *ServiceBuilder) WithPeopleRepository(r storage.PeopleRepository) *ServiceBuilder

WithPeopleRepository sets the optional people repository dependency.

func (*ServiceBuilder) WithReranker added in v0.6.1 

func (b *ServiceBuilder) WithReranker(a agent.Agent) *ServiceBuilder

WithReranker sets the optional reranker agent dependency.

func (*ServiceBuilder) WithScopeRepository added in v0.10.0 

func (b *ServiceBuilder) WithScopeRepository(r storage.ScopeRepository) *ServiceBuilder

WithScopeRepository sets the optional scope repository, used by background loops to detect channel scopes (Phase 6) for channel-framed prompts. Absent on the Telegram path, where every scope is a DM.

func (*ServiceBuilder) WithSplitter added in v0.6.1 

func (b *ServiceBuilder) WithSplitter(a agent.Agent) *ServiceBuilder

WithSplitter sets the optional splitter agent dependency.

func (*ServiceBuilder) WithTopicRepository added in v0.6.1 

func (b *ServiceBuilder) WithTopicRepository(r storage.TopicRepository) *ServiceBuilder

WithTopicRepository sets the required topic repository dependency.

func (*ServiceBuilder) WithTranslator added in v0.6.1 

func (b *ServiceBuilder) WithTranslator(t *i18n.Translator) *ServiceBuilder

WithTranslator sets the required translator dependency.

func (*ServiceBuilder) WithUserRepository added in v0.10.0 

func (b *ServiceBuilder) WithUserRepository(r storage.UserRepository) *ServiceBuilder

WithUserRepository sets the optional user repository, used by background loops to enumerate the actual memory tenants (transport-agnostic) rather than the Telegram-only config allowlist.

type SplitStats added in v0.4.6 

type SplitStats struct {
	TopicsProcessed  int
	TopicsCreated    int
	FactsRelinked    int
	PromptTokens     int
	CompletionTokens int
	EmbeddingTokens  int
	TotalCost        *float64
}

SplitStats contains statistics about topic splitting.

func (*SplitStats) AddCost added in v0.4.6 

func (s *SplitStats) AddCost(cost *float64)

AddCost adds cost to stats.

type TestMessageResult added in v0.3.0 

type TestMessageResult struct {
	Response                     string
	TimingTotal                  time.Duration
	TimingEmbedding              time.Duration
	TimingSearch                 time.Duration
	TimingLLM                    time.Duration
	PromptTokens                 int
	CompletionTokens             int
	TotalCost                    float64
	TopicsMatched                int
	FactsInjected                int
	ContextPreview               string
	SystemPromptTokensEstimated  int
	MemoryContextTokensEstimated int
	FinalContextTokensEstimated  int
	RAGDebugInfo                 *RetrievalDebugInfo
}

TestMessageResult contains the result of a test message sent through the bot pipeline. Used by the debug chat interface to display detailed metrics.

type TopicCandidateDebug added in v0.11.0 

type TopicCandidateDebug struct {
	TopicID int64
	Score   float32
}

type TopicInfo added in v0.4.6 

type TopicInfo struct {
	ID        int64  `json:"id"`
	SizeChars int    `json:"size_chars"`
	Summary   string `json:"summary"`
}

TopicInfo contains brief topic information for display.

type TopicSearchResult 

type TopicSearchResult struct {
	Topic    storage.Topic
	Score    float32
	Messages []storage.Message
	Reason   string // Why reranker chose this topic (empty if no reason provided)
}

TopicSearchResult represents a matched topic with its messages

type UsageInfo added in v0.3.0 

type UsageInfo struct {
	PromptTokens     int
	CompletionTokens int
	TotalTokens      int
	Cost             *float64
}

UsageInfo holds usage data from a single API call.

type VectorEntry added in v0.10.3 

type VectorEntry struct {
	ID        int64
	Embedding []float32
}

VectorEntry is a single indexed embedding.

type VectorKind added in v0.10.3 

type VectorKind string

VectorKind identifies which vector index an entry belongs to. 

const (
	VectorKindTopics    VectorKind = "topics"
	VectorKindFacts     VectorKind = "facts"
	VectorKindPeople    VectorKind = "people"
	VectorKindArtifacts VectorKind = "artifacts"
)

Vector index kinds. Values double as the metric label for vector search.

type VectorSearchConfig added in v0.6.1 

type VectorSearchConfig struct {
	Threshold float32 // Minimum similarity (0 = default from config)
	Limit     int     // Max results (0 = no limit)
}

VectorSearchConfig controls search behavior for generic vectorSearch.

type VectorSearchResult added in v0.6.1 

type VectorSearchResult struct {
	ID    int64
	Score float32
}

VectorSearchResult holds ID and similarity score from vector search.

type VectorStore added in v0.10.3 

type VectorStore interface {
	// ReplaceAll atomically replaces every entry of the kind (full reload).
	ReplaceAll(kind VectorKind, entries map[storage.ScopeID][]VectorEntry)
	// MaxID returns the highest loaded entry ID for the kind — the watermark
	// incremental loads resume from.
	MaxID(kind VectorKind) int64
	// AppendSince appends entries fetched for IDs above sinceMaxID and
	// advances the watermark. If the watermark no longer equals sinceMaxID —
	// a concurrent incremental load advanced it, or a ReplaceAll reset it —
	// the batch is stale and is discarded (returns 0): appending it would
	// jump the watermark past entries the concurrent reload has yet to fetch,
	// silently dropping them from the index. Returns the number appended.
	AppendSince(kind VectorKind, sinceMaxID int64, entries map[storage.ScopeID][]VectorEntry) int
	// Search returns the user's entries with cosine similarity to query of at
	// least threshold, sorted by score descending, capped at limit
	// (0 = unlimited). scanned is the number of entries compared.
	Search(kind VectorKind, userID storage.ScopeID, query []float32, threshold float32, limit int) (results []VectorSearchResult, scanned int)
	// Count returns the total number of entries of the kind across all users.
	Count(kind VectorKind) int
	// CountByUser returns per-user entry counts for the kind.
	CountByUser(kind VectorKind) map[storage.ScopeID]int
}

VectorStore is the vector index used for RAG similarity search. Implementations must be safe for concurrent use. The only implementation today is in-memory; a backend with native similarity search (e.g. pgvector) can replace it behind this interface.

func NewMemoryVectorStore added in v0.10.3 

func NewMemoryVectorStore() VectorStore

NewMemoryVectorStore creates an empty in-memory vector store.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.