storage

package
v0.6.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 3, 2026 License: MIT Imports: 13 Imported by: 0

Documentation

Overview

Package storage provides SQLite repository interfaces and implementations.

Repository Pattern:

  • All repositories are interfaces for testability
  • SQLiteStore implements all 12+ interfaces
  • Methods return wrapped errors with context

Critical Invariants (MUST READ):

  • Message IDs are GLOBAL auto-increment, NOT per-user
  • ANY query using ID ranges MUST include user_id filter
  • Example: WHERE id >= ? AND id <= ? AND user_id = ?

Thread Safety:

  • SQLiteStore uses a single connection with WAL mode
  • Concurrent reads are safe
  • Writes are serialized by SQLite

Usage Example: 

mockStorage := testutil.NewMockStorage()
mockStorage.On("GetFacts", userID).Return(testutil.TestFacts(), nil)
svc := NewService(mockStorage, ...)

Index

Constants

View Source 
const (
	TagInnerCircle    = "inner_circle"    // Work_Inner + Family, system prompt
	TagRelevantPeople = "relevant_people" // Reranker selected, user prompt
	TagPeople         = "people"          // All people, for Archivist
)

XML tag constants for people formatting.

Variables

This section is empty.

Functions

func FormatPeople added in v0.5.1 

func FormatPeople(people []Person, tag string) string

FormatPeople formats people list with specified XML tag. Format: [Person:ID] Name (@username, aka Alias1, Alias2) [Circle]: Bio If tag is empty, outputs plain list without XML wrapper.

func FormatRecentTopics added in v0.4.8 

func FormatRecentTopics(topics []TopicExtended) string

FormatRecentTopics formats recent topics for inclusion in agent prompts. Returns content wrapped in <recent_topics> tags. Format: - date: "summary" (N msg, ~Xk chars)

func FormatUserProfile added in v0.4.8 

func FormatUserProfile(facts []Fact) string

FormatUserProfile formats user facts for inclusion in agent prompts. Returns content wrapped in <user_profile> tags. Format: - [Fact:X] Category/Type (Updated: date) Content Use this for agents that need Fact IDs (e.g., Archivist for update/delete).

func FormatUserProfileCompact added in v0.6.1 

func FormatUserProfileCompact(facts []Fact) string

FormatUserProfileCompact formats user facts without Fact IDs. Returns content wrapped in <user_profile> tags. Format: - Category/Type (Updated: date) Content Use this for agents that don't need Fact IDs (e.g., Reranker, Laplace). This prevents ID format confusion with [Person:N], [Topic:N], [Artifact:N].

func RecordCleanupDeleted added in v0.3.5 

func RecordCleanupDeleted(table string, count int64)

RecordCleanupDeleted records the number of deleted rows during cleanup.

func RecordCleanupDuration added in v0.3.5 

func RecordCleanupDuration(table string, seconds float64)

RecordCleanupDuration records the duration of a cleanup operation.

func SetStorageSize added in v0.3.5 

func SetStorageSize(bytes int64)

SetStorageSize updates the storage size metric.

func SetTableSize added in v0.3.5 

func SetTableSize(table string, bytes int64)

SetTableSize updates the table size metric.

Types

type AgentLog added in v0.4.8 

type AgentLog struct {
	ID                int64
	UserID            int64
	AgentType         string // laplace, reranker, splitter, merger, enricher, archivist, scout
	InputPrompt       string
	InputContext      string // JSON - full OpenRouter API request
	OutputResponse    string
	OutputParsed      string // JSON - structured output
	OutputContext     string // JSON - full OpenRouter API response
	Model             string
	PromptTokens      int
	CompletionTokens  int
	TotalCost         *float64
	DurationMs        int
	Metadata          string // JSON - agent-specific data
	Success           bool
	ErrorMessage      string
	ConversationTurns string // JSON - all request/response turns for multi-turn agents
	CreatedAt         time.Time
}

AgentLog stores debug traces from LLM agent calls (unified logging for all agents)

type AgentLogFilter added in v0.4.8 

type AgentLogFilter struct {
	UserID    int64
	AgentType string
	Success   *bool
	Search    string
}

AgentLogFilter for filtering agent logs

type AgentLogRepository added in v0.4.8 

type AgentLogRepository interface {
	AddAgentLog(log AgentLog) error
	GetAgentLogs(agentType string, userID int64, limit int) ([]AgentLog, error)
	GetAgentLogsExtended(filter AgentLogFilter, limit, offset int) (AgentLogResult, error)
	GetAgentLogFull(ctx context.Context, id int64, userID int64) (*AgentLog, error)
}

AgentLogRepository handles unified agent debug log operations.

type AgentLogResult added in v0.4.8 

type AgentLogResult struct {
	Data       []AgentLog
	TotalCount int
}

AgentLogResult wraps agent logs with total count for pagination

type Artifact added in v0.6.0 

type Artifact struct {
	ID        int64 `json:"id"`
	UserID    int64 `json:"user_id"`
	MessageID int64 `json:"message_id"`

	// File metadata
	FileType     string `json:"file_type"` // 'image', 'voice', 'pdf', 'video_note', 'document'
	FilePath     string `json:"file_path"` // Relative path from storage dir
	FileSize     int64  `json:"file_size"` // Bytes
	MimeType     string `json:"mime_type"`
	OriginalName string `json:"original_name"` // From Telegram

	// Deduplication
	ContentHash string `json:"content_hash"` // SHA256 of file content

	// Processing status
	State        string  `json:"state"` // 'pending', 'processing', 'ready', 'failed'
	ErrorMessage *string `json:"error_message,omitempty"`

	// Retry tracking (v0.6.0 - CRIT-3)
	RetryCount   int        `json:"retry_count"`
	LastFailedAt *time.Time `json:"last_failed_at,omitempty"`

	// AI-generated metadata (summary-based search, populated in Phase 2)
	Summary   *string   `json:"summary,omitempty"`   // 2-4 sentence description
	Keywords  *string   `json:"keywords,omitempty"`  // JSON array: ["tag1", "tag2"]
	Entities  *string   `json:"entities,omitempty"`  // JSON array: ["person", "company"]
	RAGHints  *string   `json:"rag_hints,omitempty"` // JSON array: ["what questions?"]
	Embedding []float32 `json:"embedding,omitempty"` // Summary embedding for vector search

	// Timestamps
	CreatedAt   time.Time  `json:"created_at"`
	ProcessedAt *time.Time `json:"processed_at"`

	// Usage tracking (v0.6.0)
	ContextLoadCount int        `json:"context_load_count"` // How many times loaded into LLM context
	LastLoadedAt     *time.Time `json:"last_loaded_at"`     // Last time loaded

	// User context (v0.6.0) - text of message(s) when file was sent
	UserContext *string `json:"user_context,omitempty"`
}

Artifact represents a file stored in the artifacts system. v0.6.0: Simplified with summary-based search (no chunks, no full_text).

type ArtifactFilter added in v0.6.0 

type ArtifactFilter struct {
	UserID   int64
	State    string // "pending", "processing", "ready", "failed", or "" for all
	FileType string // "image", "voice", "pdf", "video_note", "document", or "" for all
}

ArtifactFilter defines filtering options for artifact queries.

type ArtifactRepository added in v0.6.0 

type ArtifactRepository interface {
	AddArtifact(artifact Artifact) (int64, error)
	GetArtifact(userID, artifactID int64) (*Artifact, error)
	GetByHash(userID int64, contentHash string) (*Artifact, error)
	// GetPendingArtifacts returns artifacts ready for processing:
	// - state='pending' (new artifacts)
	// - state='failed' with retry_count < maxRetries and sufficient backoff elapsed (v0.6.0)
	GetPendingArtifacts(userID int64, maxRetries int) ([]Artifact, error)
	GetArtifacts(filter ArtifactFilter, limit, offset int) ([]Artifact, int64, error)
	UpdateArtifact(artifact Artifact) error
	RecoverArtifactStates(threshold time.Duration) error
	GetArtifactsByIDs(userID int64, artifactIDs []int64) ([]Artifact, error)
	// IncrementContextLoadCount tracks usage when artifacts are loaded into LLM context (v0.6.0)
	IncrementContextLoadCount(userID int64, artifactIDs []int64) error
	// UpdateMessageID links artifact to history message (called after message is saved)
	// Requires userID for proper data isolation.
	UpdateMessageID(userID, artifactID, messageID int64) error
}

ArtifactRepository handles artifact file metadata operations.

type CheckpointResult added in v0.4.6 

type CheckpointResult struct {
	Busy         int // 0 = success, 1 = blocked by reader
	Log          int // Total frames in WAL file
	Checkpointed int // Frames actually checkpointed
}

CheckpointResult contains the result of a WAL checkpoint operation.

type ContaminatedTopic added in v0.4.6 

type ContaminatedTopic struct {
	TopicID       int64   `json:"topic_id"`
	TopicOwner    int64   `json:"topic_owner"`
	TopicSummary  string  `json:"topic_summary"`
	ForeignUsers  []int64 `json:"foreign_users"`
	ForeignMsgCnt int     `json:"foreign_msg_count"`
	TotalMsgCnt   int     `json:"total_msg_count"`
}

ContaminatedTopic represents a topic containing messages from other users.

type DashboardStats 

type DashboardStats struct {
	TotalTopics         int
	AvgTopicSize        float64
	ProcessedTopicsPct  float64
	ConsolidatedTopics  int
	TotalFacts          int
	FactsByCategory     map[string]int
	FactsByType         map[string]int
	TotalMessages       int
	UnprocessedMessages int
	TotalRAGQueries     int
	AvgRAGCost          float64
	MessagesPerDay      map[string]int
	FactsGrowth         map[string]int
}

type Fact 

type Fact struct {
	ID          int64
	UserID      int64
	Relation    string
	Content     string
	Category    string
	Type        string // identity, context, status
	Importance  int    // 0-100
	Embedding   []float32
	TopicID     *int64 // Nullable
	CreatedAt   time.Time
	LastUpdated time.Time
}

func FilterProfileFacts added in v0.4.8 

func FilterProfileFacts(facts []Fact) []Fact

FilterProfileFacts filters facts to identity and high-importance facts only. This is the standard filter used across all agents.

type FactHistory 

type FactHistory struct {
	ID           int64
	FactID       int64
	UserID       int64
	Action       string // add, update, delete
	OldContent   string
	NewContent   string
	Reason       string
	Category     string
	Relation     string
	Importance   int
	TopicID      *int64
	CreatedAt    time.Time
	RequestInput string
}

type FactHistoryFilter 

type FactHistoryFilter struct {
	UserID   int64
	Action   string
	Category string
	Search   string
}

type FactHistoryRepository 

type FactHistoryRepository interface {
	AddFactHistory(history FactHistory) error
	UpdateFactHistoryTopic(oldTopicID, newTopicID int64) error
	GetFactHistory(userID int64, limit int) ([]FactHistory, error)
	GetFactHistoryExtended(filter FactHistoryFilter, limit, offset int, sortBy, sortDir string) (FactHistoryResult, error)
}

FactHistoryRepository handles fact history operations.

type FactHistoryResult 

type FactHistoryResult struct {
	Data       []FactHistory
	TotalCount int
}

type FactRepository 

type FactRepository interface {
	AddFact(fact Fact) (int64, error)
	GetFacts(userID int64) ([]Fact, error)
	GetFactsByIDs(userID int64, ids []int64) ([]Fact, error)
	GetFactsByTopicID(userID int64, topicID int64) ([]Fact, error)
	GetAllFacts() ([]Fact, error)
	GetFactsAfterID(minID int64) ([]Fact, error)
	GetFactStats() (FactStats, error)
	GetFactStatsByUser(userID int64) (FactStats, error)
	UpdateFact(fact Fact) error
	UpdateFactsTopic(userID int64, oldTopicID, newTopicID int64) error
	DeleteFact(userID, id int64) error
}

FactRepository handles fact operations.

Facts are structured pieces of information extracted from conversations by the Archivist agent. They form the user's long-term profile memory.

Fact Types:

  • identity: Core user information (name, location)
  • importance: User-defined importance score (0-100)
  • Facts with importance ≥ 90 are always included in profile

Each fact has an embedding vector for semantic search and deduplication.

type FactStats 

type FactStats struct {
	CountByType map[string]int
	AvgAgeDays  float64
}

type MaintenanceRepository added in v0.3.5 

type MaintenanceRepository interface {
	GetDBSize() (int64, error)
	GetTableSizes() ([]TableSize, error)
	CleanupFactHistory(keepPerUser int) (int64, error)
	CleanupAgentLogs(keepPerUserPerAgent int) (int64, error)
	CountAgentLogs() (int64, error)
	CountFactHistory() (int64, error)

	// Database health diagnostics
	CountOrphanedTopics(userID int64) (int, error)
	GetOrphanedTopicIDs(userID int64) ([]int64, error)
	CountOverlappingTopics(userID int64) (int, error)
	GetOverlappingTopics(userID int64) ([]OverlappingPair, error)
	CountFactsOnOrphanedTopics(userID int64) (int, error)
	RecalculateTopicRanges(userID int64) (int, error)
	RecalculateTopicSizes(userID int64) (int, error)

	// Cross-user contamination detection and repair
	GetContaminatedTopics(userID int64) ([]ContaminatedTopic, error)
	CountContaminatedTopics(userID int64) (int, error)
	FixContaminatedTopics(userID int64) (int64, error)

	// WAL checkpoint for ensuring data persistence
	Checkpoint() error
}

MaintenanceRepository handles database maintenance operations.

type MemoryBankRepository 

type MemoryBankRepository interface {
	GetMemoryBank(userID int64) (string, error)
	UpdateMemoryBank(userID int64, content string) error
}

MemoryBankRepository handles the legacy memory bank.

type MergeCandidate 

type MergeCandidate struct {
	Topic1 Topic
	Topic2 Topic
}

type Message 

type Message struct {
	ID        int64
	UserID    int64
	Role      string
	Content   string
	TopicID   *int64 // Nullable
	CreatedAt time.Time
}

type MessageRepository 

type MessageRepository interface {
	AddMessageToHistory(userID int64, message Message) error
	ImportMessage(userID int64, message Message) error
	GetRecentHistory(userID int64, limit int) ([]Message, error)
	GetMessagesByIDs(userID int64, ids []int64) ([]Message, error)
	ClearHistory(userID int64) error
	GetMessagesInRange(ctx context.Context, userID int64, startID, endID int64) ([]Message, error)
	GetMessagesByTopicID(ctx context.Context, topicID int64) ([]Message, error)
	UpdateMessageTopic(userID int64, messageID, topicID int64) error
	UpdateMessagesTopicInRange(ctx context.Context, userID, startMsgID, endMsgID, topicID int64) error
	GetUnprocessedMessages(userID int64) ([]Message, error)
	GetRecentSessionMessages(ctx context.Context, userID int64, limit int, excludeIDs []int64) ([]Message, error)
}

MessageRepository handles message history operations.

Messages represent the conversation log between user and assistant. Message IDs are globally auto-incremented across all users.

Critical: Any range query MUST include user_id filter to prevent data leakage.

type OverlappingPair added in v0.4.6 

type OverlappingPair struct {
	Topic1ID      int64
	Topic1Summary string
	Topic2ID      int64
	Topic2Summary string
}

OverlappingPair contains information about two overlapping topics.

type PeopleRepository added in v0.5.1 

type PeopleRepository interface {
	// CRUD operations
	AddPerson(person Person) (int64, error)
	UpdatePerson(person Person) error
	DeletePerson(userID, personID int64) error

	// Retrieval
	GetPerson(userID, personID int64) (*Person, error)
	GetPeople(userID int64) ([]Person, error)
	GetPeopleByIDs(userID int64, ids []int64) ([]Person, error)
	GetAllPeople() ([]Person, error)
	GetPeopleAfterID(minID int64) ([]Person, error)

	// Direct matching (fast path for @username and name lookup)
	FindPersonByTelegramID(userID, telegramID int64) (*Person, error)
	FindPersonByUsername(userID int64, username string) (*Person, error)
	FindPersonByAlias(userID int64, alias string) ([]Person, error)
	FindPersonByName(userID int64, name string) (*Person, error)

	// Merge operations
	MergePeople(userID, targetID, sourceID int64, newBio string, newAliases []string, newUsername *string, newTelegramID *int64) error

	// Extended queries with filtering and pagination
	GetPeopleExtended(filter PersonFilter, limit, offset int, sortBy, sortDir string) (PersonResult, error)

	// Maintenance
	CountPeopleWithoutEmbedding(userID int64) (int, error)
	GetPeopleWithoutEmbedding(userID int64) ([]Person, error)
}

PeopleRepository handles people from the user's social graph.

type Person added in v0.5.1 

type Person struct {
	ID           int64     `json:"id"`
	UserID       int64     `json:"user_id"`
	DisplayName  string    `json:"display_name"`
	Aliases      []string  `json:"aliases"`     // JSON array: ["Гелёй", "@akaGelo"]
	TelegramID   *int64    `json:"telegram_id"` // For direct @mention match
	Username     *string   `json:"username"`    // @username without @
	Circle       string    `json:"circle"`      // Family, Friends, Work_Inner, Work_Outer, Other
	Bio          string    `json:"bio"`         // Aggregated profile (2-3 sentences)
	Embedding    []float32 `json:"embedding"`   // Bio vector (JSON float32 array)
	FirstSeen    time.Time `json:"first_seen"`
	LastSeen     time.Time `json:"last_seen"`
	MentionCount int       `json:"mention_count"`
}

Person represents a person from the user's social graph.

func FilterInnerCircle added in v0.5.1 

func FilterInnerCircle(people []Person) []Person

FilterInnerCircle returns only Work_Inner and Family people.

type PersonFilter added in v0.5.1 

type PersonFilter struct {
	UserID int64
	Circle string
	Search string
}

PersonFilter for filtering people queries.

type PersonResult added in v0.5.1 

type PersonResult struct {
	Data       []Person
	TotalCount int
}

PersonResult wraps people with total count for pagination.

type RerankerCandidate added in v0.4.1 

type RerankerCandidate struct {
	TopicID      int64   `json:"topic_id"`
	Summary      string  `json:"summary"`
	Score        float32 `json:"score"`
	Date         string  `json:"date"`
	MessageCount int     `json:"message_count"`
	SizeChars    int     `json:"size_chars"`
}

RerankerCandidate is a single candidate for JSON serialization

type RerankerToolCall added in v0.4.1 

type RerankerToolCall struct {
	Iteration int                     `json:"iteration"`
	TopicIDs  []int64                 `json:"topic_ids"`
	Topics    []RerankerToolCallTopic `json:"topics"`
}

RerankerToolCall represents one iteration of tool calls

type RerankerToolCallTopic added in v0.4.1 

type RerankerToolCallTopic struct {
	ID      int64  `json:"id"`
	Summary string `json:"summary"`
}

RerankerToolCallTopic contains topic info for tool call display

type SQLiteStore 

type SQLiteStore struct {
	// contains filtered or unexported fields
}

func NewSQLiteStore 

func NewSQLiteStore(logger *slog.Logger, path string) (*SQLiteStore, error)

func (*SQLiteStore) AddAgentLog added in v0.4.8 

func (s *SQLiteStore) AddAgentLog(log AgentLog) error

AddAgentLog inserts a new agent log entry.

func (*SQLiteStore) AddArtifact added in v0.6.0 

func (s *SQLiteStore) AddArtifact(artifact Artifact) (int64, error)

AddArtifact saves a new artifact to the database. Returns the ID of the inserted artifact. If an artifact with the same content_hash exists for the user, returns existing artifact ID.

func (*SQLiteStore) AddFact 

func (s *SQLiteStore) AddFact(fact Fact) (int64, error)

func (*SQLiteStore) AddFactHistory 

func (s *SQLiteStore) AddFactHistory(h FactHistory) error

func (*SQLiteStore) AddMessageToHistory 

func (s *SQLiteStore) AddMessageToHistory(userID int64, message Message) error

func (*SQLiteStore) AddPerson added in v0.5.1 

func (s *SQLiteStore) AddPerson(person Person) (int64, error)

AddPerson creates a new person record. Returns the new person ID.

func (*SQLiteStore) AddStat 

func (s *SQLiteStore) AddStat(stat Stat) error

func (*SQLiteStore) AddTopic 

func (s *SQLiteStore) AddTopic(topic Topic) (int64, error)

func (*SQLiteStore) AddTopicWithoutMessageUpdate added in v0.4.6 

func (s *SQLiteStore) AddTopicWithoutMessageUpdate(topic Topic) (int64, error)

AddTopicWithoutMessageUpdate creates a topic without updating message references. Used when manually managing message-topic relationships (e.g., during topic splitting).

func (*SQLiteStore) Checkpoint added in v0.4.6 

func (s *SQLiteStore) Checkpoint() error

Checkpoint forces a WAL checkpoint to flush all pending writes to the main database file. This is useful before shutdown or after critical writes to ensure data persistence. Returns CheckpointResult with details about what was checkpointed.

func (*SQLiteStore) CleanupAgentLogs added in v0.4.8 

func (s *SQLiteStore) CleanupAgentLogs(keepPerUserPerAgent int) (int64, error)

CleanupAgentLogs removes old agent_logs records, keeping only the N most recent per user per agent type. Returns the number of deleted rows.

func (*SQLiteStore) CleanupFactHistory added in v0.3.5 

func (s *SQLiteStore) CleanupFactHistory(keepPerUser int) (int64, error)

CleanupFactHistory removes old fact_history records, keeping only the N most recent per user. Returns the number of deleted rows.

func (*SQLiteStore) ClearHistory 

func (s *SQLiteStore) ClearHistory(userID int64) error

func (*SQLiteStore) Close 

func (s *SQLiteStore) Close() error

func (*SQLiteStore) CountAgentLogs added in v0.4.8 

func (s *SQLiteStore) CountAgentLogs() (int64, error)

CountAgentLogs returns the total number of agent_logs records.

func (*SQLiteStore) CountContaminatedTopics added in v0.4.6 

func (s *SQLiteStore) CountContaminatedTopics(userID int64) (int, error)

CountContaminatedTopics counts topics with cross-user contamination. If userID is 0, counts all; otherwise only for specified user.

func (*SQLiteStore) CountFactHistory added in v0.4.8 

func (s *SQLiteStore) CountFactHistory() (int64, error)

CountFactHistory returns the total number of fact_history records.

func (*SQLiteStore) CountFactsOnOrphanedTopics added in v0.4.6 

func (s *SQLiteStore) CountFactsOnOrphanedTopics(userID int64) (int, error)

CountFactsOnOrphanedTopics counts facts linked to orphaned topics. If userID is 0, counts for all users.

func (*SQLiteStore) CountOrphanedTopics added in v0.4.6 

func (s *SQLiteStore) CountOrphanedTopics(userID int64) (int, error)

CountOrphanedTopics counts topics with no messages linked to them. If userID is 0, counts for all users.

func (*SQLiteStore) CountOverlappingTopics added in v0.4.6 

func (s *SQLiteStore) CountOverlappingTopics(userID int64) (int, error)

CountOverlappingTopics counts pairs of topics with overlapping message ranges. If userID is 0, counts for all users.

func (*SQLiteStore) CountPeopleWithoutEmbedding added in v0.5.1 

func (s *SQLiteStore) CountPeopleWithoutEmbedding(userID int64) (int, error)

CountPeopleWithoutEmbedding returns count of people missing embeddings.

func (*SQLiteStore) CreateTopic 

func (s *SQLiteStore) CreateTopic(topic Topic) (int64, error)

func (*SQLiteStore) DeleteAllFacts added in v0.5.3 

func (s *SQLiteStore) DeleteAllFacts(userID int64) error

DeleteAllFacts removes all facts for a user in a single query.

func (*SQLiteStore) DeleteAllPeople added in v0.5.3 

func (s *SQLiteStore) DeleteAllPeople(userID int64) error

DeleteAllPeople removes all people for a user in a single query.

func (*SQLiteStore) DeleteAllTopics added in v0.5.3 

func (s *SQLiteStore) DeleteAllTopics(userID int64) error

DeleteAllTopics removes all topics for a user in a single query.

func (*SQLiteStore) DeleteFact 

func (s *SQLiteStore) DeleteFact(userID, id int64) error

func (*SQLiteStore) DeletePerson added in v0.5.1 

func (s *SQLiteStore) DeletePerson(userID, personID int64) error

DeletePerson removes a person record.

func (*SQLiteStore) DeleteTopic 

func (s *SQLiteStore) DeleteTopic(userID int64, id int64) error

func (*SQLiteStore) DeleteTopicCascade 

func (s *SQLiteStore) DeleteTopicCascade(userID int64, id int64) error

func (*SQLiteStore) FindPersonByAlias added in v0.5.1 

func (s *SQLiteStore) FindPersonByAlias(userID int64, alias string) ([]Person, error)

FindPersonByAlias finds people whose aliases contain the given string. Returns multiple matches since aliases might overlap.

func (*SQLiteStore) FindPersonByName added in v0.5.1 

func (s *SQLiteStore) FindPersonByName(userID int64, name string) (*Person, error)

FindPersonByName finds a person by their display name (exact match).

func (*SQLiteStore) FindPersonByTelegramID added in v0.5.1 

func (s *SQLiteStore) FindPersonByTelegramID(userID, telegramID int64) (*Person, error)

FindPersonByTelegramID finds a person by their Telegram ID.

func (*SQLiteStore) FindPersonByUsername added in v0.5.1 

func (s *SQLiteStore) FindPersonByUsername(userID int64, username string) (*Person, error)

FindPersonByUsername finds a person by their @username (without @).

func (*SQLiteStore) FixContaminatedTopics added in v0.4.6 

func (s *SQLiteStore) FixContaminatedTopics(userID int64) (int64, error)

FixContaminatedTopics removes foreign messages from contaminated topics by setting their topic_id to NULL. Returns the number of messages unlinked. If userID is 0, fixes all; otherwise only topics owned by specified user.

func (*SQLiteStore) GetAgentLogFull added in v0.6.0 

func (s *SQLiteStore) GetAgentLogFull(ctx context.Context, id int64, userID int64) (*AgentLog, error)

GetAgentLogFull fetches a single agent log with full context data. Validates userID to prevent cross-user access (security).

func (*SQLiteStore) GetAgentLogs added in v0.4.8 

func (s *SQLiteStore) GetAgentLogs(agentType string, userID int64, limit int) ([]AgentLog, error)

GetAgentLogs returns the most recent agent logs for a specific agent type. If userID is 0, returns logs for all users.

func (*SQLiteStore) GetAgentLogsExtended added in v0.4.8 

func (s *SQLiteStore) GetAgentLogsExtended(filter AgentLogFilter, limit, offset int) (AgentLogResult, error)

GetAgentLogsExtended returns agent logs with filtering and pagination.

func (*SQLiteStore) GetAllFacts 

func (s *SQLiteStore) GetAllFacts() ([]Fact, error)

GetAllFacts retrieves all facts across all users. WARNING: Cross-user access - used for vector index loading only.

func (*SQLiteStore) GetAllPeople added in v0.5.1 

func (s *SQLiteStore) GetAllPeople() ([]Person, error)

GetAllPeople retrieves all people across all users. WARNING: Cross-user access - used for vector index loading only.

func (*SQLiteStore) GetAllTopics 

func (s *SQLiteStore) GetAllTopics() ([]Topic, error)

GetAllTopics retrieves all topics across all users. WARNING: Cross-user access - used for vector index loading and admin operations only. Caller must handle data isolation if needed.

func (*SQLiteStore) GetAllUsers 

func (s *SQLiteStore) GetAllUsers() ([]User, error)

func (*SQLiteStore) GetArtifact added in v0.6.0 

func (s *SQLiteStore) GetArtifact(userID, artifactID int64) (*Artifact, error)

GetArtifact retrieves an artifact by ID and user ID.

func (*SQLiteStore) GetArtifacts added in v0.6.0 

func (s *SQLiteStore) GetArtifacts(filter ArtifactFilter, limit, offset int) ([]Artifact, int64, error)

GetArtifacts retrieves artifacts for a user with optional filters and pagination. UserID is REQUIRED for data isolation.

func (*SQLiteStore) GetArtifactsByIDs added in v0.6.0 

func (s *SQLiteStore) GetArtifactsByIDs(userID int64, artifactIDs []int64) ([]Artifact, error)

GetArtifactsByIDs retrieves artifacts by their IDs (batch load).

func (*SQLiteStore) GetByHash added in v0.6.0 

func (s *SQLiteStore) GetByHash(userID int64, contentHash string) (*Artifact, error)

GetByHash retrieves an artifact by content hash and user ID. Used for deduplication checks.

func (*SQLiteStore) GetContaminatedTopics added in v0.4.6 

func (s *SQLiteStore) GetContaminatedTopics(userID int64) ([]ContaminatedTopic, error)

GetContaminatedTopics finds topics that contain messages from users other than the topic owner. If userID is 0, checks all topics; otherwise only topics owned by userID.

func (*SQLiteStore) GetDBSize added in v0.3.5 

func (s *SQLiteStore) GetDBSize() (int64, error)

GetDBSize returns the size of the database file in bytes.

func (*SQLiteStore) GetDashboardStats 

func (s *SQLiteStore) GetDashboardStats(userID int64) (*DashboardStats, error)

func (*SQLiteStore) GetFactHistory 

func (s *SQLiteStore) GetFactHistory(userID int64, limit int) ([]FactHistory, error)

func (*SQLiteStore) GetFactHistoryExtended 

func (s *SQLiteStore) GetFactHistoryExtended(filter FactHistoryFilter, limit, offset int, sortBy, sortDir string) (FactHistoryResult, error)

func (*SQLiteStore) GetFactStats 

func (s *SQLiteStore) GetFactStats() (FactStats, error)

func (*SQLiteStore) GetFactStatsByUser added in v0.3.5 

func (s *SQLiteStore) GetFactStatsByUser(userID int64) (FactStats, error)

func (*SQLiteStore) GetFacts 

func (s *SQLiteStore) GetFacts(userID int64) ([]Fact, error)

func (*SQLiteStore) GetFactsAfterID added in v0.2.1 

func (s *SQLiteStore) GetFactsAfterID(minID int64) ([]Fact, error)

GetFactsAfterID retrieves facts created after given ID across all users. WARNING: Cross-user access - used for incremental vector index updates.

func (*SQLiteStore) GetFactsByIDs 

func (s *SQLiteStore) GetFactsByIDs(userID int64, ids []int64) ([]Fact, error)

func (*SQLiteStore) GetFactsByTopicID added in v0.4.6 

func (s *SQLiteStore) GetFactsByTopicID(userID int64, topicID int64) ([]Fact, error)

func (*SQLiteStore) GetLastTopicEndMessageID 

func (s *SQLiteStore) GetLastTopicEndMessageID(userID int64) (int64, error)

func (*SQLiteStore) GetMemoryBank 

func (s *SQLiteStore) GetMemoryBank(userID int64) (string, error)

func (*SQLiteStore) GetMergeCandidates 

func (s *SQLiteStore) GetMergeCandidates(userID int64) ([]MergeCandidate, error)

func (*SQLiteStore) GetMessagesByIDs 

func (s *SQLiteStore) GetMessagesByIDs(userID int64, ids []int64) ([]Message, error)

func (*SQLiteStore) GetMessagesByTopicID added in v0.4.6 

func (s *SQLiteStore) GetMessagesByTopicID(ctx context.Context, topicID int64) ([]Message, error)

func (*SQLiteStore) GetMessagesInRange 

func (s *SQLiteStore) GetMessagesInRange(ctx context.Context, userID int64, startID, endID int64) ([]Message, error)

func (*SQLiteStore) GetOrphanedTopicIDs added in v0.4.6 

func (s *SQLiteStore) GetOrphanedTopicIDs(userID int64) ([]int64, error)

GetOrphanedTopicIDs returns IDs of topics with no messages linked. If userID is 0, returns for all users.

func (*SQLiteStore) GetOverlappingTopics added in v0.4.6 

func (s *SQLiteStore) GetOverlappingTopics(userID int64) ([]OverlappingPair, error)

GetOverlappingTopics returns pairs of topics with overlapping message ranges. If userID is 0, returns for all users.

func (*SQLiteStore) GetPendingArtifacts added in v0.6.0 

func (s *SQLiteStore) GetPendingArtifacts(userID int64, maxRetries int) ([]Artifact, error)

GetPendingArtifacts retrieves artifacts ready for processing. Includes: - state='pending' (new artifacts) - state='failed' with retry_count < maxRetries and sufficient backoff elapsed (v0.6.0 - CRIT-3) Backoff schedule: 1 min (retry 0), 5 min (retry 1), 30 min (retry 2+)

func (*SQLiteStore) GetPeople added in v0.5.1 

func (s *SQLiteStore) GetPeople(userID int64) ([]Person, error)

GetPeople retrieves all people for a user.

func (*SQLiteStore) GetPeopleAfterID added in v0.5.1 

func (s *SQLiteStore) GetPeopleAfterID(minID int64) ([]Person, error)

GetPeopleAfterID retrieves people created after a given ID across all users. WARNING: Cross-user access - used for incremental vector index updates.

func (*SQLiteStore) GetPeopleByIDs added in v0.5.1 

func (s *SQLiteStore) GetPeopleByIDs(userID int64, ids []int64) ([]Person, error)

GetPeopleByIDs retrieves people by their IDs.

func (*SQLiteStore) GetPeopleExtended added in v0.5.1 

func (s *SQLiteStore) GetPeopleExtended(filter PersonFilter, limit, offset int, sortBy, sortDir string) (PersonResult, error)

GetPeopleExtended retrieves people with filtering and pagination.

func (*SQLiteStore) GetPeopleWithoutEmbedding added in v0.5.1 

func (s *SQLiteStore) GetPeopleWithoutEmbedding(userID int64) ([]Person, error)

GetPeopleWithoutEmbedding returns people missing embeddings.

func (*SQLiteStore) GetPerson added in v0.5.1 

func (s *SQLiteStore) GetPerson(userID, personID int64) (*Person, error)

GetPerson retrieves a single person by ID.

func (*SQLiteStore) GetRecentHistory 

func (s *SQLiteStore) GetRecentHistory(userID int64, limit int) ([]Message, error)

func (*SQLiteStore) GetRecentSessionMessages added in v0.6.0 

func (s *SQLiteStore) GetRecentSessionMessages(ctx context.Context, userID int64, limit int, excludeIDs []int64) ([]Message, error)

GetRecentSessionMessages returns the last N unprocessed messages (topic_id IS NULL) for artifact context (v0.6.0). Excludes messageIDs to avoid duplicates with MessageGroup messages.

func (*SQLiteStore) GetStats 

func (s *SQLiteStore) GetStats() (map[int64]Stat, error)

func (*SQLiteStore) GetTableSizes added in v0.3.5 

func (s *SQLiteStore) GetTableSizes() ([]TableSize, error)

GetTableSizes returns the size of each table in bytes using SQLite's dbstat virtual table.

func (*SQLiteStore) GetTopics 

func (s *SQLiteStore) GetTopics(userID int64) ([]Topic, error)

func (*SQLiteStore) GetTopicsAfterID added in v0.2.1 

func (s *SQLiteStore) GetTopicsAfterID(minID int64) ([]Topic, error)

GetTopicsAfterID retrieves topics created after given ID across all users. WARNING: Cross-user access - used for incremental vector index updates. Caller must handle data isolation if needed.

func (*SQLiteStore) GetTopicsByIDs added in v0.2.1 

func (s *SQLiteStore) GetTopicsByIDs(userID int64, ids []int64) ([]Topic, error)

func (*SQLiteStore) GetTopicsExtended 

func (s *SQLiteStore) GetTopicsExtended(filter TopicFilter, limit, offset int, sortBy, sortDir string) (TopicResult, error)

func (*SQLiteStore) GetTopicsPendingFacts 

func (s *SQLiteStore) GetTopicsPendingFacts(userID int64) ([]Topic, error)

func (*SQLiteStore) GetUnprocessedMessages 

func (s *SQLiteStore) GetUnprocessedMessages(userID int64) ([]Message, error)

func (*SQLiteStore) ImportMessage 

func (s *SQLiteStore) ImportMessage(userID int64, message Message) error

func (*SQLiteStore) IncrementContextLoadCount added in v0.6.0 

func (s *SQLiteStore) IncrementContextLoadCount(userID int64, artifactIDs []int64) error

IncrementContextLoadCount increments the load counter for artifacts and updates last_loaded_at timestamp. Called asynchronously after artifacts are successfully loaded into LLM context (v0.6.0).

func (*SQLiteStore) Init 

func (s *SQLiteStore) Init() error

func (*SQLiteStore) MergePeople added in v0.5.1 

func (s *SQLiteStore) MergePeople(userID, targetID, sourceID int64, newBio string, newAliases []string, newUsername *string, newTelegramID *int64) error

MergePeople merges source person into target person, then deletes source. newUsername and newTelegramID are the values to set (only if non-nil/non-zero). If target already has username/telegram_id, those are preserved (callers should decide which to keep).

func (*SQLiteStore) RecalculateTopicRanges added in v0.4.6 

func (s *SQLiteStore) RecalculateTopicRanges(userID int64) (int, error)

RecalculateTopicRanges recalculates start_msg_id and end_msg_id for all topics based on actual message assignments. If userID is 0, recalculates for all users.

func (*SQLiteStore) RecalculateTopicSizes added in v0.4.6 

func (s *SQLiteStore) RecalculateTopicSizes(userID int64) (int, error)

RecalculateTopicSizes recalculates size_chars for all topics based on actual message content. If userID is 0, recalculates for all users.

func (*SQLiteStore) RecoverArtifactStates added in v0.6.0 

func (s *SQLiteStore) RecoverArtifactStates(threshold time.Duration) error

RecoverArtifactStates resets zombie 'processing' states to 'pending'. Called on startup to recover from crashes or interruptions. Only recovers artifacts that have been in 'processing' state for longer than threshold to avoid re-processing actively processing artifacts.

func (*SQLiteStore) ResetUserData 

func (s *SQLiteStore) ResetUserData(userID int64) error

func (*SQLiteStore) SetTopicConsolidationChecked 

func (s *SQLiteStore) SetTopicConsolidationChecked(userID int64, topicID int64, checked bool) error

func (*SQLiteStore) SetTopicFactsExtracted 

func (s *SQLiteStore) SetTopicFactsExtracted(userID int64, topicID int64, extracted bool) error

func (*SQLiteStore) UpdateArtifact added in v0.6.0 

func (s *SQLiteStore) UpdateArtifact(artifact Artifact) error

UpdateArtifact updates an artifact's metadata.

func (*SQLiteStore) UpdateFact 

func (s *SQLiteStore) UpdateFact(fact Fact) error

func (*SQLiteStore) UpdateFactHistoryTopic 

func (s *SQLiteStore) UpdateFactHistoryTopic(oldTopicID, newTopicID int64) error

func (*SQLiteStore) UpdateFactsTopic added in v0.5.4 

func (s *SQLiteStore) UpdateFactsTopic(userID int64, oldTopicID, newTopicID int64) error

UpdateFactsTopic updates topic_id for all facts belonging to a user and old topic.

func (*SQLiteStore) UpdateMemoryBank 

func (s *SQLiteStore) UpdateMemoryBank(userID int64, content string) error

func (*SQLiteStore) UpdateMessageID added in v0.6.0 

func (s *SQLiteStore) UpdateMessageID(userID, artifactID, messageID int64) error

UpdateMessageID links an artifact to a history message. Called after message is saved to history (message_id is not known during file processing). Requires userID for proper data isolation (CRIT-2 security fix).

func (*SQLiteStore) UpdateMessageTopic 

func (s *SQLiteStore) UpdateMessageTopic(userID int64, messageID, topicID int64) error

func (*SQLiteStore) UpdateMessagesTopicInRange added in v0.4.6 

func (s *SQLiteStore) UpdateMessagesTopicInRange(ctx context.Context, userID, startMsgID, endMsgID, topicID int64) error

func (*SQLiteStore) UpdatePerson added in v0.5.1 

func (s *SQLiteStore) UpdatePerson(person Person) error

UpdatePerson updates an existing person record.

func (*SQLiteStore) UpsertUser 

func (s *SQLiteStore) UpsertUser(user User) error

type Stat 

type Stat struct {
	UserID     int64
	TokensUsed int
	CostUSD    float64
}

type StatsRepository 

type StatsRepository interface {
	AddStat(stat Stat) error
	GetStats() (map[int64]Stat, error)
	GetDashboardStats(userID int64) (*DashboardStats, error)
}

StatsRepository handles usage statistics.

type Storage 

type Storage interface {
	MessageRepository
	UserRepository
	StatsRepository
	TopicRepository
	MemoryBankRepository
	FactRepository
	FactHistoryRepository
	MaintenanceRepository
	AgentLogRepository
	PeopleRepository
	ArtifactRepository
}

type TableSize added in v0.3.5 

type TableSize struct {
	Name  string
	Bytes int64
}

TableSize represents the size of a database table.

type Topic 

type Topic struct {
	ID                   int64
	UserID               int64
	Summary              string
	StartMsgID           int64
	EndMsgID             int64
	SizeChars            int // Total character count of all messages in topic
	Embedding            []float32
	FactsExtracted       bool
	IsConsolidated       bool
	ConsolidationChecked bool
	CreatedAt            time.Time
}

type TopicExtended 

type TopicExtended struct {
	Topic
	FactsCount   int
	MessageCount int
}

type TopicFilter 

type TopicFilter struct {
	UserID         int64
	Search         string
	HasFacts       *bool // nil = all, true = yes, false = no
	IsConsolidated *bool // nil = all
	TopicID        *int64
}

type TopicRepository 

type TopicRepository interface {
	AddTopic(topic Topic) (int64, error)
	AddTopicWithoutMessageUpdate(topic Topic) (int64, error)
	CreateTopic(topic Topic) (int64, error)
	DeleteTopic(userID int64, id int64) error
	DeleteTopicCascade(userID int64, id int64) error
	GetLastTopicEndMessageID(userID int64) (int64, error)
	GetAllTopics() ([]Topic, error)
	GetTopicsAfterID(minID int64) ([]Topic, error)
	GetTopicsByIDs(userID int64, ids []int64) ([]Topic, error)
	GetTopics(userID int64) ([]Topic, error)
	SetTopicFactsExtracted(userID int64, topicID int64, extracted bool) error
	SetTopicConsolidationChecked(userID int64, topicID int64, checked bool) error
	GetTopicsPendingFacts(userID int64) ([]Topic, error)
	GetTopicsExtended(filter TopicFilter, limit, offset int, sortBy, sortDir string) (TopicResult, error)
	GetMergeCandidates(userID int64) ([]MergeCandidate, error)
}

TopicRepository handles topic operations.

Topics are compressed summaries of conversation chunks created after session archival (inactivity timeout or force-close). Each topic has an embedding vector for RAG retrieval.

Key Fields:

  • StartMsgID/EndMsgID: Message range (inclusive)
  • SizeChars: Total character count for size tracking
  • Embedding: Vector for semantic search

Note: Message IDs within a topic are guaranteed to be from the same user.

type TopicResult 

type TopicResult struct {
	Data       []TopicExtended
	TotalCount int
}

type User 

type User struct {
	ID        int64
	Username  string
	FirstName string
	LastName  string
	LastSeen  time.Time
}

type UserRepository 

type UserRepository interface {
	UpsertUser(user User) error
	GetAllUsers() ([]User, error)
	ResetUserData(userID int64) error
}

UserRepository handles user data operations.

Source Files

View all Source files

Directories

Path Synopsis
Package migrations handles database schema migrations with version tracking.
 Package migrations handles database schema migrations with version tracking.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.