Help us shape the future of Go! Take the 2026 Go Developer Survey

memory

package
v0.13.12 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 12, 2026 License: Apache-2.0 Imports: 28 Imported by: 0

Documentation

Overview

Package memory provides optional, bank-scoped long-term agent memory. Run checkpoints remain the runtime store's responsibility.

Index

Constants

View Source
const UpstreamCommit = "017b3f5d888d67341e70f43c102cb8155bb9f51e"

Variables

View Source
var (
	ErrConflict  = errors.New("memory document changed during retain")
	ErrForgotten = errors.New("memory document was forgotten; use a new document ID to reimport")
)

Functions

func BankName

func BankName(scope Scope) string

BankName maps the complete host scope to an upstream bank, preventing two apps with the same human-readable bank ID sharing memory accidentally.

func FactEmbeddingText

func FactEmbeddingText(fact ExtractedFact, mentionedAt time.Time) string

FactEmbeddingText renders the pinned upstream fact embedding input.

func KeywordText

func KeywordText(fact Fact) string

KeywordText mirrors upstream's text/context/entity/date full-text inputs. SQLite's tokenizer and BM25 score remain different from PostgreSQL ranking.

func PruneCheckpoints

func PruneCheckpoints(ctx context.Context, store CheckpointStore, keep int) error

PruneCheckpoints retains the newest keep checkpoints. Call with keep=1 after a successful post-forget upload to remove older snapshots containing forgotten facts. Provider implementations must actually purge object versions.

func RestoreCheckpoint

func RestoreCheckpoint(ctx context.Context, store CheckpointStore, checkpointID, destination string, maxBytes int64, identities ...age.Identity) error

func RestoreEncryptedCheckpoint

func RestoreEncryptedCheckpoint(ctx context.Context, input io.Reader, destination string, maxBytes int64, identities ...age.Identity) (err error)

RestoreEncryptedCheckpoint authenticates and validates a checkpoint before creating a standalone SQLite file. It never overwrites an existing database. maxBytes bounds plaintext output; zero defaults to 1 GiB.

Types

type CausalLink struct {
	TargetIndex  int    `json:"target_index"`
	RelationType string `json:"relation_type"`
}

type Checkpoint

type Checkpoint struct {
	ID              string    `json:"id"`
	CreatedAt       time.Time `json:"created_at"`
	CiphertextBytes int64     `json:"ciphertext_bytes"`
	SHA256          string    `json:"sha256"`
}

type CheckpointAuthority

type CheckpointAuthority interface {
	UploadURL(context.Context, Checkpoint) (CheckpointURL, error)
	DownloadURL(context.Context, string) (CheckpointURL, error)
	List(context.Context) ([]Checkpoint, error)
	Delete(context.Context, string) error
}

CheckpointAuthority authorizes one profile's storage operations. Implement it using the desktop host's authenticated backup API or cloud SDK. Signed URLs work with S3-compatible storage without embedding cloud credentials here. List/Delete include the host's checkpoint index and object-version policy.

type CheckpointStore

type CheckpointStore interface {
	Put(context.Context, Checkpoint, io.Reader) error
	Open(context.Context, string) (io.ReadCloser, error)
	List(context.Context) ([]Checkpoint, error)
	Delete(context.Context, string) error
}

CheckpointStore is selected and authorized by the desktop host for one profile. Implementations upload opaque encrypted bytes. Delete must remove the named checkpoint's historical object versions too when used for forgetting history.

type CheckpointURL

type CheckpointURL struct {
	URL     string
	Headers http.Header
}

CheckpointURL carries an opaque host-issued signed PUT/GET URL. Never persist or log these URLs: they may contain temporary cloud access credentials.

type ConsolidationAction

type ConsolidationAction struct {
	ObservationID string   `json:"observation_id,omitempty"`
	Text          string   `json:"text"`
	SourceFactIDs []string `json:"source_fact_ids"`
	Reason        string   `json:"reason"`
}

type ConsolidationDelete

type ConsolidationDelete struct {
	ObservationID string `json:"observation_id"`
	Reason        string `json:"reason"`
}

type ConsolidationPlan

type ConsolidationPlan struct {
	Creates []ConsolidationAction `json:"creates"`
	Updates []ConsolidationAction `json:"updates"`
	Deletes []ConsolidationDelete `json:"deletes"`
}

type ConsolidationResult

type ConsolidationResult struct {
	Processed int `json:"processed"`
	Created   int `json:"created"`
	Updated   int `json:"updated"`
	Deleted   int `json:"deleted"`
}

type Consolidator

type Consolidator interface {
	Consolidate(context.Context, []Fact, []Observation) (ConsolidationPlan, error)
}

type Embedder

type Embedder interface {
	// ModelID must change when the model, dimensions, or preprocessing changes.
	ModelID() string
	Embed(context.Context, []string) ([][]float32, error)
}

type ExtractedChunk

type ExtractedChunk struct {
	Index int    `json:"index"`
	Text  string `json:"text"`
}

type ExtractedFact

type ExtractedFact struct {
	SourceChunk     *ExtractedChunk `json:"source_chunk,omitempty"`
	Text            string          `json:"text"`
	Where           string          `json:"where,omitempty"`
	Type            string          `json:"type"`
	Entities        []string        `json:"entities"`
	OccurredStart   *time.Time      `json:"occurred_start,omitempty"`
	OccurredEnd     *time.Time      `json:"occurred_end,omitempty"`
	CausalRelations []CausalLink    `json:"causal_relations,omitempty"`
}

type Extractor

type Extractor interface {
	Extract(context.Context, RetainRequest) ([]ExtractedFact, error)
}

type Fact

type Fact struct {
	ID            string            `json:"id"`
	ChunkID       string            `json:"chunk_id,omitempty"`
	DocumentID    string            `json:"document_id"`
	Text          string            `json:"text"`
	Type          string            `json:"type"`
	Entities      []string          `json:"entities"`
	Context       string            `json:"context,omitempty"`
	MentionedAt   time.Time         `json:"mentioned_at"`
	Where         string            `json:"where,omitempty"`
	OccurredStart *time.Time        `json:"occurred_start,omitempty"`
	OccurredEnd   *time.Time        `json:"occurred_end,omitempty"`
	Metadata      map[string]string `json:"metadata,omitempty"`
}

type FactEmbedder

type FactEmbedder interface {
	EmbedFacts(context.Context, []ExtractedFact, time.Time) ([][]float32, error)
}

FactEmbedder optionally supplies upstream fact-specific date/entity augmentation.

type Hindsight

type Hindsight struct {
	// contains filtered or unexported fields
}

Hindsight is the reference backend, using the upstream HTTP API without requiring the Python service in the local implementation's dependency graph.

func NewHindsight

func NewHindsight(baseURL, apiKey string, client *http.Client) (*Hindsight, error)

func (*Hindsight) Forget

func (h *Hindsight) Forget(ctx context.Context, scope Scope, documentID string) error

func (*Hindsight) Recall

func (h *Hindsight) Recall(ctx context.Context, scope Scope, req RecallRequest) (RecallResult, error)

func (*Hindsight) Retain

func (h *Hindsight) Retain(ctx context.Context, scope Scope, req RetainRequest) (RetainResult, error)

type MentalModel

type MentalModel struct {
	ID            string    `json:"id"`
	Name          string    `json:"name"`
	SourceQuery   string    `json:"source_query"`
	Content       string    `json:"content"`
	SourceFactIDs []string  `json:"source_fact_ids"`
	RefreshedAt   time.Time `json:"refreshed_at"`
	IsStale       bool      `json:"is_stale"`
}

type Observation

type Observation struct {
	ID             string     `json:"id"`
	Text           string     `json:"text"`
	SourceFactIDs  []string   `json:"source_fact_ids"`
	ProofCount     int        `json:"proof_count"`
	UpdatedAt      time.Time  `json:"updated_at"`
	IsStale        bool       `json:"is_stale"`
	MentionedAt    time.Time  `json:"mentioned_at"`
	OccurredStart  *time.Time `json:"occurred_start,omitempty"`
	OccurredEnd    *time.Time `json:"occurred_end,omitempty"`
	SourceMemories []Fact     `json:"source_memories,omitempty"`
}

Observation is derived evidence. SourceFactIDs are mandatory provenance, not model-selected permissions. Any changed/forgotten proof invalidates the text.

type ObservationReconciler

type ObservationReconciler interface {
	ReconcileObservations(context.Context, string, string) (ObservationReconciliation, error)
}

type ObservationReconciliation

type ObservationReconciliation struct {
	Action string `json:"action"`
	Text   string `json:"text"`
	Reason string `json:"reason"`
}

type PresignedCheckpointStore

type PresignedCheckpointStore struct {
	// contains filtered or unexported fields
}

func NewPresignedCheckpointStore

func NewPresignedCheckpointStore(authority CheckpointAuthority, client *http.Client) (*PresignedCheckpointStore, error)

func (*PresignedCheckpointStore) Delete

func (*PresignedCheckpointStore) List

func (*PresignedCheckpointStore) Open

func (*PresignedCheckpointStore) Put

func (s *PresignedCheckpointStore) Put(ctx context.Context, checkpoint Checkpoint, body io.Reader) error

type Provider

type Provider struct {
	// contains filtered or unexported fields
}

Provider implements extraction/embeddings using OpenAI-compatible endpoints, and optional cross-encoder reranking using the /rerank endpoint used by vLLM and other local model servers. No inference process is started automatically.

func NewProvider

func NewProvider(cfg ProviderConfig) (*Provider, error)

func (*Provider) Consolidate

func (p *Provider) Consolidate(ctx context.Context, facts []Fact, observations []Observation) (ConsolidationPlan, error)

Consolidate uses the pinned default mission/language and the configured local extraction model. Host-controlled nondefault missions are not supported yet.

func (*Provider) Embed

func (p *Provider) Embed(ctx context.Context, texts []string) ([][]float32, error)

func (*Provider) EmbedFacts

func (p *Provider) EmbedFacts(ctx context.Context, facts []ExtractedFact, mentionedAt time.Time) ([][]float32, error)

EmbedFacts mirrors upstream augment_texts_with_dates. Queries use Embed directly.

func (*Provider) Extract

func (p *Provider) Extract(ctx context.Context, req RetainRequest) ([]ExtractedFact, error)

func (*Provider) ModelID

func (p *Provider) ModelID() string

func (*Provider) ReconcileObservations

func (p *Provider) ReconcileObservations(ctx context.Context, newText, existing string) (ObservationReconciliation, error)

func (*Provider) ReflectionStep

func (p *Provider) ReflectionStep(ctx context.Context, messages []ReflectionMessage, forcedTool string) (ReflectionMessage, error)

func (*Provider) Rerank

func (p *Provider) Rerank(ctx context.Context, query string, documents []string) ([]float64, error)

type ProviderConfig

type ProviderConfig struct {
	BaseURL             string
	APIKey              string
	ExtractionModel     string
	EmbeddingModel      string
	EmbeddingModelID    string
	EmbeddingBaseURL    string
	EmbeddingAPIKey     string
	EmbeddingDimensions int
	RerankModel         string
	Client              *http.Client
	// ExtractionOptions supplies provider-specific sampling/reasoning controls.
	// Core model, messages and response schema cannot be overridden.
	ExtractionOptions map[string]any
}

type RecallRequest

type RecallRequest struct {
	Query          string          `json:"query"`
	IncludeChunks  bool            `json:"include_chunks,omitempty"`
	MaxChunkTokens int             `json:"max_chunk_tokens,omitempty"`
	Limit          int             `json:"limit,omitempty"`
	MaxTokens      int             `json:"max_tokens,omitempty"`
	Types          []string        `json:"types,omitempty"`
	QueryTimestamp time.Time       `json:"query_timestamp,omitempty"`
	TemporalWindow *TemporalWindow `json:"temporal_window,omitempty"`
}

type RecallResult

type RecallResult struct {
	Results              []Result               `json:"results"`
	Chunks               map[string]SourceChunk `json:"chunks,omitempty"`
	EstimatedChunkTokens int                    `json:"estimated_chunk_tokens,omitempty"`
	// EstimatedTokens includes JSON result framing. A caller may supply an exact
	// tokenizer; the default uses Hindsight's o200k_base encoding.
	EstimatedTokens int `json:"estimated_tokens"`
	// Nil means the reference server did not expose its reranking configuration.
	Reranked *bool `json:"reranked,omitempty"`
}

type ReflectBackend

type ReflectBackend interface {
	ReflectMemory(context.Context, Scope, ReflectRequest) (ReflectResult, error)
}

type ReflectRequest

type ReflectRequest struct {
	Query            string   `json:"query"`
	Directives       []string `json:"directives,omitempty"`
	MaxSteps         int      `json:"max_steps,omitempty"`
	MaxContextTokens int      `json:"max_context_tokens,omitempty"`
}

type ReflectResult

type ReflectResult struct {
	Answer         string   `json:"answer"`
	MemoryIDs      []string `json:"memory_ids"`
	ObservationIDs []string `json:"observation_ids"`
	MentalModelIDs []string `json:"mental_model_ids"`
	SourceFactIDs  []string `json:"source_fact_ids"`
	Steps          int      `json:"steps"`
}

type ReflectionMessage

type ReflectionMessage struct {
	Role       string               `json:"role"`
	Content    string               `json:"content,omitempty"`
	ToolCalls  []ReflectionToolCall `json:"tool_calls,omitempty"`
	ToolCallID string               `json:"tool_call_id,omitempty"`
}

type ReflectionModel

type ReflectionModel interface {
	ReflectionStep(context.Context, []ReflectionMessage, string) (ReflectionMessage, error)
}

type ReflectionToolCall

type ReflectionToolCall struct {
	ID       string `json:"id"`
	Type     string `json:"type"`
	Function struct {
		Name      string `json:"name"`
		Arguments string `json:"arguments"`
	} `json:"function"`
}

type Reranker

type Reranker interface {
	Rerank(context.Context, string, []string) ([]float64, error)
}

type Result

type Result struct {
	Fact
	Score       float64        `json:"score"`
	SourceRanks map[string]int `json:"source_ranks,omitempty"`
}

type RetainRequest

type RetainRequest struct {
	DocumentID string            `json:"document_id"`
	Content    string            `json:"content"`
	Context    string            `json:"context,omitempty"`
	Timestamp  time.Time         `json:"timestamp,omitempty"`
	Metadata   map[string]string `json:"metadata,omitempty"`
}

RetainRequest replaces one source document atomically. DocumentID is a stable host/source identifier, not a random ID generated for each retry.

type RetainResult

type RetainResult struct {
	DocumentID string `json:"document_id"`
	FactCount  int    `json:"fact_count"`
	Unchanged  bool   `json:"unchanged"`
}

type SQLite

type SQLite struct {
	// contains filtered or unexported fields
}

func OpenSQLite

func OpenSQLite(ctx context.Context, cfg SQLiteConfig) (*SQLite, error)

OpenSQLite uses the runtime CLI's existing pure-Go SQLite driver. Vector similarity is exact and computed in Go; no extension or embedding daemon is required by storage itself. Inference is supplied through the model interfaces.

func (*SQLite) Close

func (s *SQLite) Close() error

func (*SQLite) Consolidate

func (s *SQLite) Consolidate(ctx context.Context, scope Scope, model Consolidator, limit int) (out ConsolidationResult, err error)

Consolidate processes a bounded batch of previously unprocessed facts. The host schedules this operation; retain never starts background model calls. All inference finishes outside transactions. A bank-state CAS prevents stale inference from restoring information after source replacement or forgetting.

func (*SQLite) ConsolidateMemory

func (s *SQLite) ConsolidateMemory(ctx context.Context, scope Scope, limit int) (ConsolidationResult, error)

ConsolidateMemory uses the configured provider, letting a host schedule work without extracting model credentials from the initialized backend.

func (*SQLite) CreateMentalModel

func (s *SQLite) CreateMentalModel(ctx context.Context, scope Scope, id, name, query string) error

CreateMentalModel registers a host-curated question, then requires an explicit RefreshMentalModel call to derive its content. IDs and scope are host-selected.

func (*SQLite) EncryptedSnapshot

func (s *SQLite) EncryptedSnapshot(ctx context.Context, destination string, recipients ...age.Recipient) (err error)

EncryptedSnapshot creates a consistent age-encrypted checkpoint for upload. Recipients are selected by the host; private identities never leave the device. The destination must not exist. Plaintext temporary files are removed on return.

func (*SQLite) Forget

func (s *SQLite) Forget(ctx context.Context, scope Scope, documentID string) error

Forget removes the document and all its facts, indexes, and graph links. A tombstone prevents an in-flight extraction or a retried import resurrecting it. Older snapshots must also be expired by the application's backup policy.

func (*SQLite) ReadMentalModel

func (s *SQLite) ReadMentalModel(ctx context.Context, scope Scope, id string) (MentalModel, error)

func (*SQLite) Recall

func (s *SQLite) Recall(ctx context.Context, scope Scope, req RecallRequest) (RecallResult, error)

func (*SQLite) Reflect

func (s *SQLite) Reflect(ctx context.Context, scope Scope, req ReflectRequest, model ReflectionModel) (ReflectResult, error)

Reflect runs a bounded native tool-calling loop over derived observations and raw facts. It performs no writes. Directives are supplied by the host.

func (*SQLite) ReflectMemory

func (s *SQLite) ReflectMemory(ctx context.Context, scope Scope, req ReflectRequest) (ReflectResult, error)

func (*SQLite) RefreshMentalModel

func (s *SQLite) RefreshMentalModel(ctx context.Context, scope Scope, id string, model ReflectionModel) (MentalModel, error)

func (*SQLite) Retain

func (s *SQLite) Retain(ctx context.Context, scope Scope, req RetainRequest) (RetainResult, error)

func (*SQLite) SaveCheckpoint

func (s *SQLite) SaveCheckpoint(ctx context.Context, store CheckpointStore, recipients ...age.Recipient) (checkpoint Checkpoint, err error)

SaveCheckpoint encrypts locally before the store receives any bytes. It returns the checkpoint identity even when upload fails, for reconciliation.

func (*SQLite) SearchMentalModels

func (s *SQLite) SearchMentalModels(ctx context.Context, scope Scope, query string, limit int) ([]MentalModel, error)

func (*SQLite) SearchObservations

func (s *SQLite) SearchObservations(ctx context.Context, scope Scope, query string, limit int) ([]Observation, error)

SearchObservations returns derived evidence with its complete proof IDs. This search fuses semantic, keyword, proof-link and temporal evidence, then applies the same combined scoring as raw recall plus the upstream proof-count boost.

func (*SQLite) Snapshot

func (s *SQLite) Snapshot(ctx context.Context, destination string) error

Snapshot creates a consistent standalone SQLite file using VACUUM INTO. The destination must not exist. Encrypt before uploading it to cloud storage.

type SQLiteConfig

type SQLiteConfig struct {
	Path        string
	Extractor   Extractor
	Embedder    Embedder
	Reranker    Reranker
	CountTokens TokenCounter
	// MaxObservations bounds each bank; zero leaves it unlimited.
	MaxObservations int
	// CandidateLimit bounds each retrieval arm before RRF (default 100).
	CandidateLimit int
	// RerankerMaxCandidates caps the fused pool before reranking and combined
	// scoring, matching Hindsight's default of 300. It is independent of the
	// per-arm retrieval limit: changing it also changes rank normalization.
	RerankerMaxCandidates int
}

type Scope

type Scope struct {
	AppID  string `json:"app_id"`
	BankID string `json:"bank_id"`
}

Scope is selected by the host, never by model-supplied tool arguments.

func (Scope) Validate

func (s Scope) Validate() error

type SourceChunk

type SourceChunk struct {
	ID         string `json:"id"`
	DocumentID string `json:"document_id"`
	Index      int    `json:"chunk_index"`
	Text       string `json:"text"`
	Truncated  bool   `json:"truncated"`
}

type TemporalWindow

type TemporalWindow struct {
	Start time.Time `json:"start"`
	End   time.Time `json:"end"`
}

TemporalWindow boosts the temporal retrieval arm. It is not a hard filter on semantic, keyword, or graph results, matching Hindsight's API semantics.

type TokenCounter

type TokenCounter func(string) int

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL