memory

package
v0.11.0-beta.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 23, 2026 License: Apache-2.0, MIT Imports: 18 Imported by: 0

Documentation

Overview

Package memory — semantic + episodic memory orchestration (Go side). Delegates embeddings + vector search to the Python bridge when available. Wired via run.go boot sequence; Postgres-backed.

Package memory — segment-based memory (§8.2 Phase 5 #3, inspired by MemAgent / Yu 2025).

The problem: in a massive codebase, dumping the whole context into a single cold blob is wasteful — most queries only need a slice (the auth module, or just files matching `*_test.go`, or the segment we used last turn). MemAgent's idea: split memory into labeled SEGMENTS with per-segment metadata, then load segments on-demand based on query relevance + recency.

This package provides:

  • Segment: a labeled chunk of context (file globs, a directory, or an arbitrary user-defined group) with metadata for retrieval scoring.
  • SegmentStore: durable per-segment state under ~/.overkill/segments/<id>.json. Touch updates last-access for recency scoring; LoadFiles materializes the contents.
  • Retrieval: top-K selection by a composite score (recency × match × inverse size). Composes with the existing vector path in internal/journal — the journal owns observation similarity, this package owns codebase segmentation.

What this is NOT: a replacement for the agent's context window. Segments are a *retrieval layer* on top of the filesystem. The agent calls segment tools, picks one, and asks Read on the files the segment names. No magic context compression here.

Package memory — bridge-backed semantic recall (master plan §6.1).

The Orchestrator can be augmented with an EmbeddingClient (typically a thin wrapper around bridge.Client) so that Remember computes an embedding and stores a vector, and SemanticRecall searches by query embedding with optional rerank. When no client is wired, the Orchestrator degrades to substring recall via the Postgres store — semantic features are best-effort.

Index

Constants

This section is empty.

Variables

View Source 
var ErrNotFound = errors.New("memory: not found")

Functions

This section is empty.

Types

type BridgeAdapter 

type BridgeAdapter struct {
	// contains filtered or unexported fields
}

BridgeAdapter wraps a *bridge.Client so it satisfies EmbeddingClient.

memory does not import bridge directly to keep the dependency one-way and to allow tests to swap in a fake. Callers that already have a bridge client wrap it via NewBridgeAdapter at the wiring layer.

func NewBridgeAdapter 

func NewBridgeAdapter(c *bridge.Client) *BridgeAdapter

NewBridgeAdapter wraps c. Returns nil when c is nil.

func (*BridgeAdapter) DeleteVector 

func (a *BridgeAdapter) DeleteVector(ctx context.Context, id string) (bool, error)

func (*BridgeAdapter) Embed 

func (a *BridgeAdapter) Embed(ctx context.Context, text, model string) ([]float32, int32, error)

func (*BridgeAdapter) Rerank 

func (a *BridgeAdapter) Rerank(ctx context.Context, query string, documents []string, topN int) ([]RerankHit, error)

func (*BridgeAdapter) SearchVectors 

func (a *BridgeAdapter) SearchVectors(ctx context.Context, query []float32, topK int, threshold float64) ([]VectorHit, error)

func (*BridgeAdapter) StoreVector 

func (a *BridgeAdapter) StoreVector(ctx context.Context, id string, embedding []float32, content string, metadata map[string]string) (string, error)

type EmbeddingClient 

type EmbeddingClient interface {
	Embed(ctx context.Context, text, model string) ([]float32, int32, error)
	StoreVector(ctx context.Context, id string, embedding []float32, content string, metadata map[string]string) (string, error)
	SearchVectors(ctx context.Context, query []float32, topK int, threshold float64) ([]VectorHit, error)
	Rerank(ctx context.Context, query string, documents []string, topN int) ([]RerankHit, error)
	DeleteVector(ctx context.Context, id string) (bool, error)
}

EmbeddingClient is the minimal embedding/vector/rerank surface the Orchestrator needs. bridge.Client satisfies this. Defined locally to keep the dependency direction one-way (memory doesn't import bridge).

type ListOptions 

type ListOptions struct {
	Type      MemoryType
	SessionID string
	Limit     int
	Offset    int
}

type Memory 

type Memory struct {
	ID        string            `json:"id"`
	Type      MemoryType        `json:"type"`
	Content   string            `json:"content"`
	Tags      []string          `json:"tags"`
	SessionID string            `json:"session_id"`
	Timestamp time.Time         `json:"timestamp"`
	Relevance float64           `json:"relevance"`
	Metadata  map[string]string `json:"metadata"`
}

type MemoryType 

type MemoryType string
const (
	MemoryEpisodic   MemoryType = "episodic"
	MemorySemantic   MemoryType = "semantic"
	MemoryProcedural MemoryType = "procedural"
)

type Orchestrator 

type Orchestrator struct {
	// contains filtered or unexported fields
}

func NewOrchestrator 

func NewOrchestrator(store Store, provider providers.Provider, model string) *Orchestrator

func (*Orchestrator) AttachEmbeddings 

func (o *Orchestrator) AttachEmbeddings(client EmbeddingClient, cfg SemanticConfig)

AttachEmbeddings wires an EmbeddingClient into the Orchestrator.

func (*Orchestrator) ExtractMemories 

func (o *Orchestrator) ExtractMemories(ctx context.Context, content string, sessionID string) ([]Memory, error)

func (*Orchestrator) ForgetSemantic 

func (o *Orchestrator) ForgetSemantic(ctx context.Context, id string) error

ForgetSemantic removes the memory from the local store and from the vector store (when wired). Vector deletion errors are swallowed (best-effort).

func (*Orchestrator) HasEmbeddings 

func (o *Orchestrator) HasEmbeddings() bool

HasEmbeddings reports whether semantic features are available.

func (*Orchestrator) Recall 

func (o *Orchestrator) Recall(ctx context.Context, query string, limit int) (*SearchResult, error)

func (*Orchestrator) Remember 

func (o *Orchestrator) Remember(ctx context.Context, content string, memoryType MemoryType, tags []string, sessionID string) error

func (*Orchestrator) RememberSemantic 

func (o *Orchestrator) RememberSemantic(ctx context.Context, content string, memType MemoryType, tags []string, sessionID string) (*Memory, error)

RememberSemantic stores a memory in the local store AND computes/stores its embedding via the bridge. Falls back to plain Remember if no client is wired.

func (*Orchestrator) SemanticRecall 

func (o *Orchestrator) SemanticRecall(ctx context.Context, query string, topK int) (*SearchResult, error)

SemanticRecall searches by embedding similarity, optionally reranks, and returns the corresponding Memory rows from the local store.

When no client is wired this falls back to substring recall via Recall.

func (*Orchestrator) Summarize 

func (o *Orchestrator) Summarize(ctx context.Context, memories []Memory) (string, error)

type PostgresStore 

type PostgresStore struct {
	// contains filtered or unexported fields
}

PostgresStore implements Store backed by PostgreSQL.

func NewPostgresStore 

func NewPostgresStore(db *sql.DB) (*PostgresStore, error)

NewPostgresStore returns a Store backed by the given *sql.DB. Starts a background goroutine that prunes memories older than 90 days every hour. Override with SetRetention().

func (*PostgresStore) Close 

func (s *PostgresStore) Close() error

Close cancels the background prune goroutine. The caller owns the DB lifecycle.

func (*PostgresStore) Delete 

func (s *PostgresStore) Delete(ctx context.Context, id string) error

Delete removes a memory by ID.

func (*PostgresStore) Get 

func (s *PostgresStore) Get(ctx context.Context, id string) (*Memory, error)

Get retrieves a single memory by ID.

func (*PostgresStore) List 

func (s *PostgresStore) List(ctx context.Context, opts ListOptions) ([]Memory, error)

List returns memories with optional filtering.

func (*PostgresStore) Prune 

func (s *PostgresStore) Prune(ctx context.Context, olderThan time.Time) error

Prune deletes memories whose timestamp is before olderThan.

func (*PostgresStore) Retrieve 

func (s *PostgresStore) Retrieve(ctx context.Context, query Query) (*SearchResult, error)

Retrieve searches memories. Pushes type filtering and LIMIT into SQL so we don't pull the entire table into Go. Content scoring still happens in Go on the already-limited result set.

func (*PostgresStore) SetRetention 

func (s *PostgresStore) SetRetention(d time.Duration)

SetRetention configures automatic pruning. Memories older than the configured duration are deleted by the background goroutine. Set to 0 to disable automatic pruning.

func (*PostgresStore) Store 

func (s *PostgresStore) Store(ctx context.Context, memory *Memory) error

Store persists a Memory. Deduplicates by content hash (SHA-256): when identical content is stored again, the existing row's timestamp is bumped to the newer value instead of creating a duplicate row. The Store interface is unchanged — callers still pass a Memory and receive an error.

type Query 

type Query struct {
	Content      string       `json:"content"`
	Types        []MemoryType `json:"types,omitempty"`
	Tags         []string     `json:"tags,omitempty"`
	SessionID    string       `json:"session_id,omitempty"`
	Limit        int          `json:"limit"`
	MinRelevance float64      `json:"min_relevance"`
}

type RankOptions 

type RankOptions struct {
	// RecencyHalfLife is the time after which the recency score
	// drops to 0.5. Default: 24h.
	RecencyHalfLife time.Duration
	// MatchWeight scales the name/desc/tag match contribution.
	MatchWeight float64
	// RecencyWeight scales the recency contribution.
	RecencyWeight float64
	// SizeWeight scales the inverse-size contribution (smaller =
	// faster to load).
	SizeWeight float64
}

RankOptions tunes scoring weights. Zero-value gives sensible defaults: weight recency 1.0, name match 2.0, inverse size 0.5.

type RerankHit 

type RerankHit struct {
	Index int
	Score float64
	Text  string
}

RerankHit mirrors bridge.RerankResult.

type SearchResult 

type SearchResult struct {
	Memories []Memory `json:"memories"`
	Total    int      `json:"total"`
}

type Segment 

type Segment struct {
	ID          string `json:"id"`
	Name        string `json:"name"`
	Description string `json:"description,omitempty"`
	// Globs is the file-pattern list that defines this segment.
	// Relative to the segment's RootDir. Standard glob syntax via
	// filepath.Match plus a `**` recursive wildcard.
	Globs []string `json:"globs"`
	// RootDir is the absolute path the globs resolve against.
	// Empty → store's default root.
	RootDir string `json:"root_dir,omitempty"`
	// Tags is operator-supplied labels for filtering.
	Tags      []string  `json:"tags,omitempty"`
	CreatedAt time.Time `json:"created_at"`
	UpdatedAt time.Time `json:"updated_at"`
	// LastAccessed is bumped by Touch. Recency component of the
	// retrieval score.
	LastAccessed time.Time `json:"last_accessed,omitempty"`
	// AccessCount tracks how often the segment has been loaded —
	// stable signal that this segment is "hot".
	AccessCount int `json:"access_count"`
	// CachedFileCount + CachedTotalBytes are snapshot stats from
	// the last LoadFiles call. Used by the inverse-size component
	// of the retrieval score and surfaced to the user.
	CachedFileCount  int   `json:"cached_file_count,omitempty"`
	CachedTotalBytes int64 `json:"cached_total_bytes,omitempty"`
}

Segment is one labeled slice of the codebase.

type SegmentHit 

type SegmentHit struct {
	Segment *Segment
	Score   float64
}

SegmentHit is a ranked retrieval result.

type SegmentStore 

type SegmentStore struct {
	// contains filtered or unexported fields
}

SegmentStore wraps the on-disk segment directory.

func NewSegmentStore 

func NewSegmentStore(dir, defaultRoot string) *SegmentStore

NewSegmentStore wires the store. defaultRoot is the base path segments use when they don't set their own RootDir — typically the user's project root (passed in by the wiring layer).

func (*SegmentStore) All 

func (s *SegmentStore) All() ([]*Segment, error)

All returns every persisted segment.

func (*SegmentStore) Create 

func (s *SegmentStore) Create(seg *Segment) (*Segment, error)

Create persists a new Segment. ID is auto-assigned; CreatedAt / UpdatedAt are stamped. Empty Globs is rejected (a segment with no files is useless).

func (*SegmentStore) Delete 

func (s *SegmentStore) Delete(id string) error

Delete removes a segment. Idempotent.

func (*SegmentStore) Get 

func (s *SegmentStore) Get(id string) (*Segment, error)

Get returns a segment by ID, or (nil, nil) when not found.

func (*SegmentStore) LoadFiles 

func (s *SegmentStore) LoadFiles(id string) ([]string, error)

LoadFiles resolves the segment's Globs to concrete files and returns the resulting paths. Updates CachedFileCount + CachedTotalBytes so future Rank calls have accurate size info.

Returns paths absolute, sorted, deduplicated. Recursive `**` glob is supported.

func (*SegmentStore) Rank 

func (s *SegmentStore) Rank(query string, topK int, opts RankOptions) ([]SegmentHit, error)

Rank returns the top-K segments for the query, scored by a composite (recency × match × inverse-size). Empty query relies on recency + size alone (closest to "what should I load by default?").

func (*SegmentStore) Search 

func (s *SegmentStore) Search(query string) ([]*Segment, error)

Search returns segments whose Name, Description, or Tags substring-match the query (case-insensitive). Empty query → all.

func (*SegmentStore) Touch 

func (s *SegmentStore) Touch(id string) error

Touch updates LastAccessed and bumps AccessCount. Called by retrieval consumers so recency reflects actual use, not just existence.

type SemanticConfig 

type SemanticConfig struct {
	// EmbedModel is the model name passed to the embedding service. When
	// empty the orchestrator skips embedding entirely.
	EmbedModel string
	// SearchThreshold is the cosine-similarity floor for SearchVectors.
	SearchThreshold float64
	// RerankTopN, when > 0, runs results through Rerank and trims to N.
	RerankTopN int
}

SemanticConfig controls embedding-backed behaviour.

type Store 

type Store interface {
	Store(ctx context.Context, memory *Memory) error
	Retrieve(ctx context.Context, query Query) (*SearchResult, error)
	Delete(ctx context.Context, id string) error
	Get(ctx context.Context, id string) (*Memory, error)
	List(ctx context.Context, opts ListOptions) ([]Memory, error)
}

type VectorHit 

type VectorHit struct {
	ID       string
	Score    float64
	Content  string
	Metadata map[string]string
}

VectorHit mirrors bridge.SearchResult without importing it.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.