memory

package
v0.3.10 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 6, 2026 License: MIT Imports: 19 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

ValidObservationTypes is the set of allowed observation types.

Functions

func BuildAfterToolCallback

func BuildAfterToolCallback(w *Worker, sessionID, project string, excludedTools map[string]bool) func(ctx agent.Context, t toolpkg.Tool, args, result map[string]any, toolErr error) (map[string]any, error)

BuildAfterToolCallback creates an ADK AfterToolCallback that enqueues observations. It wraps the worker's Enqueue in the ADK callback signature for direct use with agent callbacks.

func BuildMemoryCallback

func BuildMemoryCallback(w *Worker, sessionID, project string) func(toolName string, toolInput, toolOutput map[string]any)

BuildMemoryCallback creates an ADK AfterToolCallback that enqueues observations to the worker.

func HasFTS5

func HasFTS5(db *sql.DB) bool

HasFTS5 checks whether the database supports FTS5.

func OpenDB

func OpenDB(dbPath string) (*sql.DB, error)

OpenDB opens (or creates) a SQLite database at the given path with WAL mode and runs pending migrations. Pass ":memory:" for in-memory databases.

The pragmas are set through DSN parameters rather than by executing them on the pool once. database/sql opens connections lazily and without a limit, so `db.Exec("PRAGMA busy_timeout=5000")` configures only whichever single connection happened to serve it; every later connection starts with busy_timeout=0. Under concurrent writers that means SQLITE_BUSY is returned immediately instead of being waited out, and the write is lost — measured at 5 of 12 concurrent CreateSession calls persisted. The driver applies `_pragma` parameters on each new connection, so every one of them is configured.

func StripPrivate

func StripPrivate(s string) string

StripPrivate replaces all <private>...</private> tags with [PRIVATE].

func StripPrivateFromMap

func StripPrivateFromMap(m map[string]any) map[string]any

StripPrivateFromMap deep-walks a map and applies StripPrivate to all string values.

Types

type AfterStoreHook added in v0.0.15

type AfterStoreHook func(ctx context.Context, obs *Observation)

AfterStoreHook is called after an observation is successfully stored. Implementations must not block the pipeline — errors should be logged, not returned.

type Compressor

type Compressor interface {
	CompressObservation(ctx context.Context, raw RawObservation) (*Observation, error)
}

Compressor transforms raw tool observations into structured observations.

func NewCompressor added in v0.1.6

func NewCompressor(name string, orch *subagent.Orchestrator) Compressor

NewCompressor returns the Compressor for a resolved compressor name.

The name is resolved by config.Config.ResolveCompressor; this keeps the mapping in one place so the CLI and the piagent wiring cannot select different compressors for the same configuration.

A nil orchestrator forces the model-free compressor even when the subagent compressor was asked for: spawning subagents needs an orchestrator, and falling back to recording beats failing to record. That is also what makes the disabled-orchestrator paths in tests behave.

type ContextGenerator

type ContextGenerator struct {
	// contains filtered or unexported fields
}

ContextGenerator builds markdown context from recent observations for system instruction injection.

func NewContextGenerator

func NewContextGenerator(store Store, tokenBudget int) *ContextGenerator

NewContextGenerator creates a ContextGenerator with the given store and token budget.

func (*ContextGenerator) Generate

func (g *ContextGenerator) Generate(ctx context.Context, project string) (string, error)

Generate builds a markdown string of recent observations grouped by session. Returns empty string if there are no observations for the project.

type LLMSummarizer added in v0.1.6

type LLMSummarizer struct {
	// contains filtered or unexported fields
}

LLMSummarizer writes session summaries with an in-process model call.

The alternative is SubagentCompressor.SummarizeSession, which spawns the `memory-compressor` subagent as a child `pi` process. That path is measurably unusable for summarization at session end: specs/memory-fixes/research/ findings.md F2 measures 5.6s per spawned observation against a 5s shutdown budget, and records the `sql: database is closed` errors that follow when the parent exits first. A spawned process cannot fit inside a budget the parent has already stopped waiting for.

A single GenerateContent call is the one shape of this work that does fit: it is in-process, so nothing is orphaned when the agent closes, and it is one round trip rather than a process spawn plus a round trip.

func NewLLMSummarizer added in v0.1.6

func NewLLMSummarizer(llm llmmodel.LLM) *LLMSummarizer

NewLLMSummarizer returns a summarizer backed by the given model. A nil model yields a summarizer that reports an error rather than panicking, so callers keep one code path for "summarization unavailable".

func (*LLMSummarizer) SummarizeSession added in v0.1.6

func (s *LLMSummarizer) SummarizeSession(ctx context.Context, sessionID, project string, observations []*Observation) (*SessionSummary, error)

SummarizeSession asks the model to summarize the given observations and parses its reply into a SessionSummary.

Called with no observations it returns an error and does not call the model: there is nothing to summarize, and a model asked to summarize an empty list will invent a plausible session rather than report that it has nothing.

type NoopCompressor added in v0.1.6

type NoopCompressor struct{}

NoopCompressor records an observation without calling a model.

It exists because the per-observation model call is the expensive part of the memory pipeline, not the recording. The default compressor spawns a `pi --mode json` child process per tool call (see SubagentCompressor), which costs a process, a session directory, a database open, a log file and an uncached provider round trip — all before the model has been asked anything. A session of forty tool calls therefore pays forty of those, serially, on a worker whose queue the shutdown path only waits five seconds for.

What the model call actually buys is a title, a type classification and a narrative sentence. None of that is required for an observation to be searchable: the tool name, the file paths and the (truncated) tool output are all readable from the raw event. So this compressor writes exactly the record the failure path already writes, and the session-level summary — one model call at the end, over the whole session — supplies the narrative instead.

The trade is deliberate: per-tool rows stay, so mem-search, mem-timeline, mem-get and the recalled-context block keep working; per-tool inference goes.

func NewNoopCompressor added in v0.1.6

func NewNoopCompressor() *NoopCompressor

NewNoopCompressor returns a Compressor that stores the fallback observation and never calls a model.

func (*NoopCompressor) CompressObservation added in v0.1.6

func (c *NoopCompressor) CompressObservation(_ context.Context, raw RawObservation) (*Observation, error)

CompressObservation builds the fallback observation for raw.

It never returns an error: there is nothing that can fail when no model is involved, and returning one would only make the worker log a compression failure for a path that succeeded.

type Observation

type Observation struct {
	ID              int64
	SessionID       string
	Project         string
	Title           string
	Type            ObservationType
	Text            string
	SourceFiles     []string // stored as JSON array
	ToolName        string
	PromptNumber    int
	DiscoveryTokens int
	CreatedAt       time.Time
}

Observation is a compressed, structured record of a tool usage event.

func FallbackObservation added in v0.1.6

func FallbackObservation(raw RawObservation) *Observation

FallbackObservation builds the model-free observation for a raw tool event.

Shared by the worker's failure path and NoopCompressor so the two cannot drift: a compression failure and a deliberately uncompressed record should produce the same shape, since both are what a reader sees when no model interpretation is available.

type ObservationType

type ObservationType string

ObservationType classifies what an observation represents.

const (
	TypeDecision  ObservationType = "decision"
	TypeBugfix    ObservationType = "bugfix"
	TypeFeature   ObservationType = "feature"
	TypeRefactor  ObservationType = "refactor"
	TypeDiscovery ObservationType = "discovery"
	TypeChange    ObservationType = "change"
)

type RawObservation

type RawObservation struct {
	SessionID  string
	Project    string
	ToolName   string
	ToolInput  map[string]any
	ToolOutput map[string]any
	Timestamp  time.Time
}

RawObservation is the uncompressed tool event captured by AfterToolCallback.

type SQLiteStore

type SQLiteStore struct {
	// contains filtered or unexported fields
}

SQLiteStore implements Store using a SQLite database.

func NewSQLiteStore

func NewSQLiteStore(db *sql.DB) *SQLiteStore

NewSQLiteStore creates a new SQLiteStore wrapping the given database.

func (*SQLiteStore) Close

func (s *SQLiteStore) Close() error

Close closes the underlying database connection.

func (*SQLiteStore) CompleteSession

func (s *SQLiteStore) CompleteSession(ctx context.Context, sessionID string) error

CompleteSession marks a session as completed with a timestamp.

func (*SQLiteStore) CreateSession

func (s *SQLiteStore) CreateSession(ctx context.Context, sess *Session) error

CreateSession inserts a new session record.

func (*SQLiteStore) GetObservations

func (s *SQLiteStore) GetObservations(ctx context.Context, ids []int64) ([]*Observation, error)

GetObservations fetches observations by IDs. Missing IDs are silently skipped.

func (*SQLiteStore) InsertObservation

func (s *SQLiteStore) InsertObservation(ctx context.Context, obs *Observation) error

InsertObservation inserts a new observation, marshaling SourceFiles as JSON.

func (*SQLiteStore) RecentObservations

func (s *SQLiteStore) RecentObservations(ctx context.Context, project string, limit int) ([]*Observation, error)

RecentObservations returns the most recent observations for a project.

func (*SQLiteStore) RecentSummaries

func (s *SQLiteStore) RecentSummaries(ctx context.Context, project string, limit int) ([]*SessionSummary, error)

RecentSummaries returns the most recent session summaries for a project.

func (*SQLiteStore) Search

func (s *SQLiteStore) Search(ctx context.Context, q SearchQuery) (*SearchResult, error)

Search performs FTS5 full-text search over observations, returning compact result rows. Falls back to LIKE matching if FTS5 is unavailable.

func (*SQLiteStore) SearchSummaries

func (s *SQLiteStore) SearchSummaries(ctx context.Context, query string, project string) ([]*SessionSummary, error)

SearchSummaries performs FTS5 search over session summaries. Falls back to LIKE if FTS5 is unavailable.

func (*SQLiteStore) SessionObservations added in v0.1.6

func (s *SQLiteStore) SessionObservations(ctx context.Context, sessionID string) ([]*Observation, error)

SessionObservations returns every observation recorded for one session, in the order they were recorded.

Unlike Store.RecentObservations this is not project-scoped and is not capped: it is what session summarization reads, and a summary that silently dropped the oldest observations would describe a session that never happened.

The sort is on (created_at_epoch, id), not created_at_epoch alone. That column holds whole seconds, and a burst of tool calls routinely lands several observations in the same second; with an equal sort key SQLite's order is undefined, so the same session could come back in different orders on different runs — reordering the summarization prompt and making the summary itself non-reproducible. id is monotonic within a session, which makes the order total.

The consequence worth knowing: within one second, order follows insertion (id), not the sub-second timestamps, which are not persisted. Two observations recorded in the same second therefore come back in insertion order regardless of their CreatedAt values.

func (*SQLiteStore) Timeline

func (s *SQLiteStore) Timeline(ctx context.Context, anchorID int64, before, after int) ([]*Observation, error)

Timeline returns observations around an anchor observation, from the same project. It fetches `before` observations created before the anchor and `after` observations after it.

func (*SQLiteStore) UpsertSummary

func (s *SQLiteStore) UpsertSummary(ctx context.Context, sum *SessionSummary) error

UpsertSummary inserts or replaces a session summary.

type SearchQuery

type SearchQuery struct {
	Query   string
	Project string
	Type    ObservationType
	Limit   int
	Offset  int
}

SearchQuery describes a full-text search request.

type SearchResult

type SearchResult struct {
	Rows  []SearchResultRow `json:"results"`
	Total int               `json:"total"`
}

SearchResult contains paginated search results.

type SearchResultRow

type SearchResultRow struct {
	ID        int64           `json:"id"`
	Title     string          `json:"title"`
	Type      ObservationType `json:"type"`
	CreatedAt time.Time       `json:"created_at"`
	ReadCost  int             `json:"read"`
	WorkCost  int             `json:"work"`
}

SearchResultRow is a compact representation of one search match.

type Session

type Session struct {
	ID          int64
	SessionID   string
	Project     string
	UserPrompt  string
	StartedAt   time.Time
	CompletedAt *time.Time
	Status      string // "active", "completed", "failed"
}

Session represents a single agent interaction session.

type SessionSummarizer added in v0.1.6

type SessionSummarizer struct {
	// contains filtered or unexported fields
}

SessionSummarizer summarizes one session's observations and stores the result.

It is the whole end-of-session step — read the session's observations, ask the model once, write the summary — so the CLI and the piagent façade reach the store through one path and cannot disagree about what a summary is.

A session with no observations is reported as an error rather than silently writing nothing: callers distinguish "nothing to summarize" from "summarized" by their own bookkeeping, but a silent success would hide a store that never recorded anything.

func NewSessionSummarizer added in v0.1.6

func NewSessionSummarizer(store Store, llm llmmodel.LLM) *SessionSummarizer

NewSessionSummarizer returns a summarizer that reads observations from store and writes the summary back to it. A nil model, or a nil store, yields a summarizer that reports an error rather than panicking.

func (*SessionSummarizer) SummarizeSession added in v0.1.6

func (s *SessionSummarizer) SummarizeSession(ctx context.Context, sessionID, project string) error

SummarizeSession summarizes sessionID and stores the summary.

type SessionSummary

type SessionSummary struct {
	ID              int64
	SessionID       string
	Project         string
	Request         string
	Investigated    string
	Learned         string
	Completed       string
	NextSteps       string
	DiscoveryTokens int
	CreatedAt       time.Time
}

SessionSummary is a structured summary generated at session end.

type Store

type Store interface {
	CreateSession(ctx context.Context, s *Session) error
	CompleteSession(ctx context.Context, sessionID string) error
	InsertObservation(ctx context.Context, obs *Observation) error
	GetObservations(ctx context.Context, ids []int64) ([]*Observation, error)
	RecentObservations(ctx context.Context, project string, limit int) ([]*Observation, error)
	SessionObservations(ctx context.Context, sessionID string) ([]*Observation, error)
	UpsertSummary(ctx context.Context, sum *SessionSummary) error
	RecentSummaries(ctx context.Context, project string, limit int) ([]*SessionSummary, error)
	Search(ctx context.Context, q SearchQuery) (*SearchResult, error)
	Timeline(ctx context.Context, anchorID int64, before, after int) ([]*Observation, error)
	Close() error
}

Store defines the interface for memory persistence operations.

type SubagentCompressor

type SubagentCompressor struct {
	// contains filtered or unexported fields
}

SubagentCompressor uses a bundled subagent to compress raw observations.

func NewSubagentCompressor

func NewSubagentCompressor(orch *subagent.Orchestrator) *SubagentCompressor

NewSubagentCompressor creates a compressor that uses the memory-compressor subagent.

func (*SubagentCompressor) CompressObservation

func (c *SubagentCompressor) CompressObservation(ctx context.Context, raw RawObservation) (*Observation, error)

CompressObservation sends raw observation data to the memory-compressor subagent and parses the structured response.

func (*SubagentCompressor) SummarizeSession

func (c *SubagentCompressor) SummarizeSession(ctx context.Context, sessionID, project string, observations []*Observation) (*SessionSummary, error)

SummarizeSession creates a session summary from a list of observations.

type Worker

type Worker struct {
	// contains filtered or unexported fields
}

Worker is a background goroutine that drains raw observations, applies privacy filtering, compresses them, and stores the results.

func NewWorker

func NewWorker(store Store, compressor Compressor, bufSize int) *Worker

NewWorker creates a Worker with the given buffer size for the observation channel.

func (*Worker) Enqueue

func (w *Worker) Enqueue(obs RawObservation)

Enqueue sends a raw observation to the worker. Non-blocking: drops and logs if full.

func (*Worker) OnAfterStore added in v0.0.15

func (w *Worker) OnAfterStore(hook AfterStoreHook)

OnAfterStore registers a hook that runs after each observation is stored. Must be called before Start.

func (*Worker) Shutdown

func (w *Worker) Shutdown(ctx context.Context) error

Shutdown closes the observation channel and waits for all pending observations to be processed, with a timeout.

func (*Worker) Start

func (w *Worker) Start(ctx context.Context)

Start begins processing observations in a background goroutine.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL