experiment

package
v0.1.11 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 7, 2026 License: MIT Imports: 16 Imported by: 0

Documentation

Overview

Package experiment provides the SQLite-backed database for VaultMind experiment data.

Index

Constants

View Source 
const (
	EventSearch      = "search"
	EventAsk         = "ask"
	EventContextPack = "context_pack"
	EventNoteAccess  = "note_access"
	EventIndexEmbed  = "index_embed"
)

Event type constants.

View Source 
const (
	TelemetryOff       = "off"
	TelemetryAnonymous = "anonymous"
	TelemetryFull      = "full"
)

Telemetry tier constants.

Write-path contract (as of this writing):

  • TelemetryOff → the experiment DB is never opened; no session is started; no events are written. Gated in cmd/root.go.
  • TelemetryAnonymous and TelemetryFull → identical write paths. Both write everything locally. The distinction is future-proofing for an uploader that does not yet exist.

Upload-path contract (for future implementation): 

When a data-return mechanism is built (not yet), the uploader MUST
filter outbound payloads by tier. Under Anonymous, the following fields
MUST be stripped before transmission:
  - event_data.variants.*.results[].note_id
  - event_data.variants.*.results[].path
  - event_data.query.text (if/when query text is added to events)
  - vault_path
Aggregate structure (ranks, scores, counts, timestamps, variant names,
note_type) MAY be sent under Anonymous. Under Full, the uploader MAY
send everything.

If you're adding the uploader: verify this contract still matches the promise in WriteTelemetryPrompt. If the promise changes, update both.

View Source 
const DefaultSpreadingActivationDelta = 0.2

DefaultSpreadingActivationDelta is the Delta weight applied when query similarities are available to the scorer (hybrid retrieval on). The value is ACT-R's standard spreading-activation contribution; use DefaultActivationParamsWithSimilarity to construct a params set with this value set, or DefaultActivationParams when similarities are intentionally absent (Delta stays 0).

View Source 
const MinElapsedHours = 1.0 / 3600.0

MinElapsedHours is the minimum effective-elapsed value substituted when a note access's compressed elapsed time rounds to zero or below. Expressed as 1 second in hours so it's a soft floor that still makes the log(time) term in retrieval strength well-defined. Changing this shifts the retrieval score for very recent accesses.

View Source 
const UserSessionThreshold = 30 * time.Minute

UserSessionThreshold defines how long after the last matching invocation a new one still counts as part of the same user-session. 30 minutes is the default; invocations beyond this gap mint a new user-session id. "Matching" means same caller + same user + same host — companion persona loads and cli queries stay in separate groupings even when close in time.

Variables

This section is empty.

Functions

func BuildAskEventData 

func BuildAskEventData(p AskEventParams) map[string]any

BuildAskEventData composes the event_data payload for an ask event. Retrieval hits are carried as a variant under variants.{RetrievalMode}; shadow variants are merged into the same map. A collision between a shadow variant name and the retrieval mode emits a warn log and the shadow payload wins (deterministic, documented behavior — rename one set if it matters).

func BuildRetrievalEventData 

func BuildRetrievalEventData(variants map[string]any, resultCount int, err error) map[string]any

BuildRetrievalEventData composes the base event_data payload for a retrieval event. resultCount is the total number of hits the retrieval reported (may differ from len(variants.*.results) when limits truncate). When err is non-nil, an "error" field is added so downstream consumers can distinguish genuine zero-hit retrievals (curiosity signal) from crashes (error signal). The error field is omitted on success, including the zero-hit success case.

func BuildShadowVariantResults 

func BuildShadowVariantResults(session *Session, actDef ExperimentDef, noteIDs []string) map[string]any

BuildShadowVariantResults computes activation features for every variant in actDef (primary + shadows) over the given ranked note IDs. Returns a map shaped like the event's `variants` substructure: 

{ variant: { results: [ { note_id, rank, features? } ] } }

Rank is 1-indexed from the position of each note ID in noteIDs. Unknown variant names are skipped (logged at debug). Used by ask and context-pack events to emit shadow-scoring comparisons alongside primary retrieval.

func BuildVariantPayload 

func BuildVariantPayload(variant string, hits []RetrievalHit) map[string]any

BuildVariantPayload produces the event_data.variants substructure for one retrieval variant. The shape matches what LinkOutcomes walks: 

{ variant: { results: [ { note_id, rank, score_final, note_type, path, scores? } ] } }

scores is omitted when the hit has no per-component breakdown.

func CombinedScore 

func CombinedScore(retrieval, storage, similarity, alpha, beta, delta float64) float64

CombinedScore returns score = alpha * retrieval + beta * storage + delta * similarity. This implements the full ACT-R model: base-level activation (retrieval), dual-strength storage (Bjork), and spreading activation (similarity).

func CompressedElapsed 

func CompressedElapsed(active, idle time.Duration, gamma float64) time.Duration

CompressedElapsed returns tj_effective = active + gamma * idle.

func ComputeApproxRetrieval 

func ComputeApproxRetrieval(accessCount int, lastAccessedAt, now time.Time, windows []SessionWindow, gamma, d float64) float64

ComputeApproxRetrieval is the live-retrieval-path approximation of ComputeRetrieval. The notes table stores only (access_count, last_accessed_at) — a scalar count and a single timestamp — not the full per-event access history that ComputeRetrieval consumes. To use the same ACT-R math without a per-event access-times table, this approximator treats every recorded access as if it had happened at last_accessed_at, then defers to ComputeRetrieval. With all N timestamps equal to t, sum(t_k^(-d)) reduces to N*t^(-d), so the score collapses to ln(N) - d*ln(t) — preserving both the count-amplifies and elapsed-decays monotonic properties the ranking layer needs (Track A.4, slice 5b'). Returns 0 for the degenerate cases (no accesses, or no timestamp) so the caller doesn't have to guard against -Inf from log of zero or non-positive elapsed.

func ComputeBatchScores 

func ComputeBatchScores(db *DB, noteIDs []string, params ActivationParams, similarities map[string]float64) (map[string]float64, map[string]map[string]float64, error)

ComputeBatchScores computes activation scores for a batch of notes. Returns noteID->score and noteID->features maps. similarities is optional (nil = no spreading activation). When provided, it maps noteID -> cosine similarity with the current query.

func ComputeBatchScoresAt 

func ComputeBatchScoresAt(db *DB, noteIDs []string, params ActivationParams, similarities map[string]float64, now time.Time) (map[string]float64, map[string]map[string]float64, error)

ComputeBatchScoresAt is ComputeBatchScores with an explicit reference time. Production passes time.Now(); tests pass a fixed instant so activation ranking is deterministic instead of depending on sub-millisecond wall-clock differences between the access events and the scoring call.

func ComputeRetrieval 

func ComputeRetrieval(accessTimes []time.Time, now time.Time, windows []SessionWindow, gamma, d float64) float64

ComputeRetrieval computes Bi = ln(sum(tj_effective^(-d))). Uses session windows to partition each access-to-now interval. Returns 0.0 for empty accessTimes.

func ComputeStorage 

func ComputeStorage(accessCount int) float64

ComputeStorage returns Si = ln(1 + access_count). Never decays. Returns 0.0 for non-positive counts.

func DetectCaller 

func DetectCaller() (string, map[string]any)

DetectCaller resolves the caller identity for a new experiment session. Precedence:

  1. VAULTMIND_CALLER env var — explicit, wins unconditionally. Hooks and scripts set this ("companion-persona-hook", "vaultmind-persona-hook").
  2. CLAUDE_PROJECT_DIR set — we were invoked from a Claude Code session, probably by the agent running bash. Label as "claude-code" so we can distinguish from raw CLI use.
  3. Default to "cli" — a human at a terminal, or an unknown script.

The companion caller_meta captures $USER, $HOSTNAME, and CLAUDE_PROJECT_DIR so we can later answer "whose laptop" and "which project context" without having to encode them into the caller label itself.

func ExportToJSONL 

func ExportToJSONL(db *DB, tier string, w io.Writer) error

ExportToJSONL writes a sanitized snapshot of the experiment DB to w as a JSONL stream: one manifest record on line 1, then one record per session, event, and outcome. Each record is a JSON object with a "kind" discriminator ("manifest" / "session" / "event" / "outcome").

Tier handling:

  • TelemetryOff → returns an error and writes nothing. The user opted out; producing a file implies we collected data we shouldn't have.
  • TelemetryAnonymous → strips vault_path on sessions and events, query_text on events, caller_meta on sessions, and the documented fields inside event_data via SanitizeEventData.
  • TelemetryFull → preserves everything.

func JaccardAtK 

func JaccardAtK(a, b []string, k int) float64

JaccardAtK computes the Jaccard similarity of the top-K items in a and b. Order within each list does not affect the result — only set membership after truncation to K. Two empty lists return 1.0 (conventional). One empty and one non-empty returns 0.0.

func KendallTauShared 

func KendallTauShared(a, b []string) (float64, int)

KendallTauShared computes Kendall's tau-a rank correlation restricted to items appearing in both a and b. Ranks are derived from list position (earlier = smaller rank). Returns (tau, sharedCount). If fewer than 2 items are shared, returns (NaN, sharedCount) — rank correlation is undefined on <2 pairs and callers should treat NaN as "insufficient data."

func ParseExperiments 

func ParseExperiments(raw map[string]any) map[string]ExperimentDef

ParseExperiments parses experiment definitions from Viper's raw map. Reserved keys ("telemetry", "outcome_window_sessions") and non-map values are skipped. For each map entry it extracts enabled (bool), primary (string), and shadows ([]string from []any).

func ParseTelemetryChoice 

func ParseTelemetryChoice(input string) string

ParseTelemetryChoice maps user input to a telemetry tier. Empty or invalid input defaults to anonymous.

func PartitionTime 

func PartitionTime(start, end time.Time, windows []SessionWindow) (active, idle time.Duration)

PartitionTime splits the duration [start, end] into active (overlapping sessions) and idle (gaps between sessions). Windows need not be sorted.

func PromptTelemetry 

func PromptTelemetry(r io.Reader, w io.Writer) string

PromptTelemetry shows the telemetry prompt and reads the user's choice. Returns the selected tier string.

func SanitizeEventData 

func SanitizeEventData(data map[string]any, tier string) map[string]any

SanitizeEventData returns a sanitized copy of event_data for the given telemetry tier. Under TelemetryAnonymous, the contract documented in telemetry.go is enforced: variants[*].results[*].note_id and .path are stripped; aggregate fields (rank, score, scores, type, …) survive. Under TelemetryFull the data is returned as-is.

SanitizeEventData does NOT mutate its input. It walks the structure and rebuilds the parts it touches, so callers can reuse the original map without surprise.

func ScoreFromData 

func ScoreFromData(noteIDs []string, accessMap map[string][]time.Time, windows []SessionWindow, now time.Time, params ActivationParams, similarities map[string]float64) (map[string]float64, map[string]map[string]float64)

ScoreFromData computes activation scores from pre-fetched data. Returns (scores, features). Use this to avoid redundant DB queries when computing multiple variants over the same data. similarities is optional (nil = no similarity component). When provided, it maps noteID -> cosine similarity with the current query, implementing ACT-R spreading activation.

func VariantGamma 

func VariantGamma(variant string) (float64, bool)

VariantGamma returns the gamma for a known variant name.

func VariantPerformance 

func VariantPerformance(db *DB) (map[string]*VariantStats, error)

VariantPerformance computes per-variant stats over the entire experiment DB. Returns one entry per distinct primary_variant observed in events. The map is keyed by variant name; iteration order is not guaranteed — callers that need deterministic output must sort by name themselves.

func WithSession 

func WithSession(ctx context.Context, s *Session) context.Context

WithSession stores s in the context and returns the updated context.

func WriteRollup 

func WriteRollup(r *Rollup, w io.Writer) error

WriteRollup emits a single Rollup record as one indented JSON object. Unlike ExportToJSONL (which emits a stream of records), the rollup IS the unit of analysis — one record per vault — so pretty-printed JSON is the right shape for both machines and humans inspecting the payload before transmission.

func WriteTelemetryPrompt 

func WriteTelemetryPrompt(w io.Writer)

WriteTelemetryPrompt writes the opt-in prompt to w.

Types

type ActivationParams 

type ActivationParams struct {
	Gamma float64 // idle time compression (0.0-1.0)
	D     float64 // decay exponent (ACT-R default: 0.5)
	Alpha float64 // retrieval strength weight
	Beta  float64 // storage strength weight
	Delta float64 // spreading activation / similarity weight
}

ActivationParams holds tunable parameters for activation scoring.

func DefaultActivationParams 

func DefaultActivationParams(gamma float64) ActivationParams

DefaultActivationParams returns params with research-based defaults. Delta defaults to 0.0 (no similarity component) for backward compatibility.

func DefaultActivationParamsWithSimilarity 

func DefaultActivationParamsWithSimilarity(gamma float64) ActivationParams

DefaultActivationParamsWithSimilarity returns a params set for the similarity-available case. Only Delta differs from DefaultActivationParams; everything else is shared so the two constructors are swappable without cascading tuning changes.

type AggregateRow 

type AggregateRow struct {
	PrimaryVariant    string   `json:"primary_variant"`
	ShadowVariant     string   `json:"shadow_variant"`
	EventCount        int      `json:"event_count"`
	MeanJaccardAtK    float64  `json:"mean_jaccard_at_k"`
	MeanKendallTau    *float64 `json:"mean_kendall_tau"`
	KendallEventCount int      `json:"kendall_event_count"`
}

AggregateRow is the per-(primary, shadow) pair summary across many events. MeanKendallTau is a pointer because tau is undefined when no event in the pair had >=2 shared items; nil here means "insufficient data" and serializes to JSON null. Storing NaN here would break encoding/json (NaN is unsupported).

func AggregateComparisons 

func AggregateComparisons(events []ComparableEvent, kCap int) []AggregateRow

AggregateComparisons collapses per-event EventPairs into per-pair aggregates. Jaccard is averaged over every event that contributed a pair. Kendall's tau averages only over events where the pair had >=2 shared items (others would contribute NaN). KendallEventCount reports how many events actually contributed to the tau average. Output rows are sorted by (primary, shadow) for deterministic display.

type AskEventParams 

type AskEventParams struct {
	// RetrievalMode is the retriever label used as the variant key for the
	// actual retrieval hits (e.g. "hybrid", "keyword").
	RetrievalMode string
	// TopHits are the retrieval results, in rank order.
	TopHits []RetrievalHit
	// ShadowVariants are the shadow-scored variant results from
	// BuildShadowVariantResults. Merged into the event's variants map.
	ShadowVariants map[string]any
	// PrimaryVariant is the activation experiment's chosen variant name
	// (recorded only when ActivationOn is true).
	PrimaryVariant string
	// ActivationOn reports whether the activation experiment is enabled for
	// this event. When false, PrimaryVariant is omitted from the payload.
	ActivationOn bool
	// RetrievalErr, when non-nil, causes BuildRetrievalEventData to populate
	// the "error" field so failed retrievals are distinguishable from
	// zero-hit successes.
	RetrievalErr error
}

AskEventParams is the boundary-clean input for composing an ask event payload. The cmd layer converts retrieval/result types into this shape so the experiment package stays agnostic of retrieval implementation details (no import of internal/query).

type CalibrationSnapshot 

type CalibrationSnapshot struct {
	CalibrationID    string  `json:"calibration_id"`
	CreatedAt        string  `json:"created_at"` // RFC3339
	EmbedderLabel    string  `json:"embedder_label"`
	EmbeddingDims    int     `json:"embedding_dims"`
	NoteCount        int     `json:"note_count"`
	NoiseFloor       float64 `json:"noise_floor"`
	NoiseFloorProbes int     `json:"noise_floor_probes"`
	ProbeSetVersion  int     `json:"probe_set_version"`
	NTNCosineMu      float64 `json:"ntn_cosine_mu"`
	NTNCosineSigma   float64 `json:"ntn_cosine_sigma"`
	NTNSampleCount   int     `json:"ntn_sample_count"`
	// VaultPath scopes the snapshot to one vault so the ask path reads THIS
	// vault's floor (LatestCalibrationForVault), not the globally-most-recent
	// one. `json:"-"` — it is a storage discriminator, NOT part of the
	// content-free federated-export surface, and must never be serialized.
	VaultPath string `json:"-"`
}

CalibrationSnapshot is a per-vault retrieval-calibration measurement, stored in the experiment DB. Every field is a content-free scalar — no note ids, no content, no query text — so the snapshot doubles as the vault_features the federated study (Paper #2) aggregates, and can be exported as-is.

  • NoiseFloor (N): the cosine an off-topic probe query gets to any note — the relevance floor. Relevance R = top_cosine - N.
  • NTNCosineMu/Sigma: the vault's note-to-note cosine dispersion (how tight or loose the embedding space is). NTNSampleCount is the number of pairs sampled.

type ComparableEvent 

type ComparableEvent struct {
	EventID string
	Pairs   []EventPair
}

ComparableEvent is one DB event after pair extraction. Events without shadow variants (or without a primary) should be filtered out before this slice is passed to AggregateComparisons.

type ComparableEventFilter 

type ComparableEventFilter struct {
	SessionID    string
	Caller       string
	SinceRFC3339 string
	EventTypes   []string
}

ComparableEventFilter narrows which events LoadComparableEvents returns. Zero-value fields mean "no filter." EventTypes empty → default to ask/search/context_pack.

type DB 

type DB struct {
	// contains filtered or unexported fields
}

DB wraps *sql.DB with schema initialization and experiment-specific helpers.

func Open 

func Open(dbPath string) (*DB, error)

Open opens (or creates) a SQLite database at dbPath, creates the parent directory if needed, applies pragmas (WAL mode, foreign key enforcement), and runs pending migrations using PRAGMA user_version for versioning.

func (*DB) AccessedNoteIDs 

func (d *DB) AccessedNoteIDs() ([]string, error)

AccessedNoteIDs returns all unique note IDs from note_access events.

func (*DB) BatchNoteAccessTimes 

func (d *DB) BatchNoteAccessTimes(noteIDs []string) (map[string][]time.Time, error)

BatchNoteAccessTimes returns access times for multiple notes in one pass.

func (*DB) Begin 

func (d *DB) Begin() (*sql.Tx, error)

Begin starts a transaction.

func (*DB) Close 

func (d *DB) Close() error

Close closes the underlying database connection.

func (*DB) CountEvents 

func (d *DB) CountEvents() (int, error)

CountEvents returns the total number of event rows.

func (*DB) CountOutcomes 

func (d *DB) CountOutcomes() (int, error)

CountOutcomes returns the total number of outcome rows.

func (*DB) CountSessions 

func (d *DB) CountSessions() (int, error)

CountSessions returns the total number of session rows. Used by rollup assembly so the receiver can sanity-check the aggregation horizon (e.g. is this vault active or stale?).

func (*DB) DistinctVariants 

func (d *DB) DistinctVariants() ([]string, error)

DistinctVariants returns the sorted list of distinct variant names in the outcomes table.

func (*DB) EndSession 

func (d *DB) EndSession(sessionID string) error

EndSession sets ended_at for the given session to the current time.

func (*DB) Exec 

func (d *DB) Exec(query string, args ...any) (sql.Result, error)

Exec executes a query that doesn't return rows.

func (*DB) GetSessionCaller 

func (d *DB) GetSessionCaller(sessionID string) (SessionCaller, error)

GetSessionCaller returns the caller attribution for the given session. Returns zero values (empty caller, nil meta, empty UserSessionID) for sessions that predate attribution or were started via the older StartSession.

func (*DB) IsFirstRun 

func (d *DB) IsFirstRun() (bool, error)

IsFirstRun returns true if the experiment DB has no completed sessions.

func (*DB) LatestCalibration 

func (d *DB) LatestCalibration() (*CalibrationSnapshot, error)

LatestCalibration returns the most recently created snapshot across ALL vaults, or (nil, nil) when none exists. Use this only for vault-agnostic checks; the ask path must use LatestCalibrationForVault to avoid reading another vault's floor.

func (*DB) LatestCalibrationForVault 

func (d *DB) LatestCalibrationForVault(vaultPath string) (*CalibrationSnapshot, error)

LatestCalibrationForVault returns the most recent snapshot for one vault, or (nil, nil) when that vault has none. This is the lookup the ask path uses: the experiment DB is global (many vaults), so scoping by vault_path is what keeps a query against vault B from reading vault A's noise floor. Callers should pass a filepath.Clean'd path so "/a/b" and "/a/b/" match the stored key.

func (*DB) LinkOutcomes 

func (d *DB) LinkOutcomes(currentSessionID, noteID string, outcomeWindow int) (int, error)

LinkOutcomes looks back at recent search/ask/context_pack events (within the outcome window) and creates outcome rows for any variant where noteID appears in the results. Returns the number of outcome rows created.

func (*DB) LoadComparableEvents 

func (d *DB) LoadComparableEvents(f ComparableEventFilter) ([]ComparableEvent, error)

LoadComparableEvents returns events that carry at least one shadow variant (i.e., a variants map with primary plus another) with their pairs pre-extracted. Rows whose primary_variant is NULL/empty are skipped.

func (*DB) LogEvent 

func (d *DB) LogEvent(evt Event) (string, error)

LogEvent inserts an event row into the events table and returns the generated event ID. Optional string fields (QueryText, QueryMode, PrimaryVariant) are stored as NULL when empty.

func (*DB) NoteAccessTimes 

func (d *DB) NoteAccessTimes(noteID string) ([]time.Time, error)

NoteAccessTimes returns timestamps of all note_access events for noteID, ordered ascending. Filters by parsing event_data JSON for matching note_id.

func (*DB) NoteRetrievals 

func (d *DB) NoteRetrievals(noteID string) ([]SessionHit, error)

NoteRetrievals returns every (session, event, rank, timestamp) tuple where the given note appeared in a retrieval. Ordered chronologically. Best rank across variants wins when a note appears under multiple variants of one event — so one event contributes one row per distinct note, not per variant.

func (*DB) PerNoteStats 

func (d *DB) PerNoteStats() ([]NoteStat, error)

PerNoteStats returns one row per note that has appeared in a retrieval event, ordered by retrieval_count_total desc (most-often-retrieved first), then by last_retrieved_ts desc to break count ties with recency.

Note: the source-of-truth for which notes exist is the vault index DB, not the experiment DB. This query only knows about notes that have been retrieved at least once; notes that were never retrieved do not appear.

func (*DB) Query 

func (d *DB) Query(query string, args ...any) (*sql.Rows, error)

Query executes a query that returns rows.

func (*DB) QueryRow 

func (d *DB) QueryRow(query string, args ...any) *sql.Row

QueryRow executes a query that returns at most one row.

func (*DB) RecentSessionWindows 

func (d *DB) RecentSessionWindows(limit int) ([]SessionWindow, error)

RecentSessionWindows returns the N most recent completed sessions as SessionWindows. Orphans (NULL ended_at) are excluded.

func (*DB) RecoverOrphans 

func (d *DB) RecoverOrphans() (int, error)

RecoverOrphans finds sessions with NULL ended_at and sets ended_at to the timestamp of their last event, or started_at + 1 minute if no events exist. Returns the number of sessions recovered.

func (*DB) Report 

func (d *DB) Report(variants []string, k int) (*ReportResult, error)

Report computes Hit@K and MRR for the given variants across all linkable events. variants is the list of variant names to report on; k is the rank cutoff for Hit@K.

func (*DB) SessionGaps 

func (d *DB) SessionGaps() ([]SessionGap, error)

SessionGaps returns all sessions with their inter-session gaps, ordered by started_at ascending. Derived from the session_gaps SQL view.

func (*DB) SessionRetrievals 

func (d *DB) SessionRetrievals(sessionID string) ([]RetrievalEventSummary, error)

SessionRetrievals returns every retrieval event in the given session, chronologically ascending. Events of type search / ask / context_pack are included; note_access and index_embed are not (they are not retrievals). An unknown session ID returns an empty slice without error.

func (*DB) SessionsByUserSession 

func (d *DB) SessionsByUserSession(userSessionID string) ([]string, error)

SessionsByUserSession returns all session_ids (invocation-sessions) that belong to the given user_session_id. Useful for reconstructing "everything that happened in this working session" across multiple invocations.

func (*DB) StartSession 

func (d *DB) StartSession(vaultPath string) (string, error)

StartSession inserts a new session row and returns the generated session ID.

func (*DB) StartSessionWithCaller 

func (d *DB) StartSessionWithCaller(vaultPath, caller string, meta map[string]any) (string, error)

StartSessionWithCaller starts a session and records caller attribution. meta is serialized as JSON; nil meta is stored as NULL. A user_session_id is computed from (caller, user, host) against recent sessions — reused when the last matching session is within UserSessionThreshold, otherwise a new id is minted.

func (*DB) StoreCalibration 

func (d *DB) StoreCalibration(snap *CalibrationSnapshot) error

StoreCalibration inserts a calibration snapshot. Snapshots are append-only (history is kept); LatestCalibration reads the most recent.

func (*DB) UpdateSessionVaultPath 

func (d *DB) UpdateSessionVaultPath(sessionID, vaultPath string) error

UpdateSessionVaultPath sets the vault_path for the given session. Called when the vault path becomes known (after command flag resolution).

func (*DB) UsageSummary 

func (d *DB) UsageSummary(topN int) (*UsageSummary, error)

UsageSummary computes the aggregate metrics in one pass across the DB. topN caps the TopNotes slice; 0 means "all recalled notes." topN is applied after the per_note_stats view's own ordering (count desc, recency desc).

type Dispatcher 

type Dispatcher struct {
	// contains filtered or unexported fields
}

Dispatcher routes scoring requests to registered variant scorers.

func NewDispatcher 

func NewDispatcher() *Dispatcher

NewDispatcher creates a dispatcher with the built-in "none" scorer registered.

func (*Dispatcher) Register 

func (d *Dispatcher) Register(variant string, s Scorer)

Register adds a scorer for the given variant name.

func (*Dispatcher) RunAll 

func (d *Dispatcher) RunAll(variants []string, notes []ScoredNote) (map[string][]ScoredNote, error)

RunAll runs all specified variants and returns map[variant][]ScoredNote.

func (*Dispatcher) Score 

func (d *Dispatcher) Score(variant string, notes []ScoredNote) ([]ScoredNote, error)

Score runs the scorer for the given variant. Returns error for unknown variants.

type Event 

type Event struct {
	SessionID      string
	Type           string
	VaultPath      string
	QueryText      string
	QueryMode      string
	PrimaryVariant string
	Data           map[string]any
}

Event holds the data for a single experiment event to be logged.

type EventPair 

type EventPair struct {
	PrimaryVariant string
	ShadowVariant  string
	PrimaryList    []string
	ShadowList     []string
}

EventPair is one (primary, shadow) comparison extracted from a single event's variants map. The two lists are note IDs in rank order (rank 1 first).

func ExtractEventPairs 

func ExtractEventPairs(eventData map[string]any, primaryName string) ([]EventPair, error)

ExtractEventPairs walks an event's decoded event_data JSON and returns one EventPair per shadow variant. Shadow order is deterministic (sorted by name). Returns an error if the primary variant is absent.

type ExperimentDef 

type ExperimentDef struct {
	Enabled bool
	Primary string
	Shadows []string
}

ExperimentDef holds the parsed definition for a single experiment.

func (ExperimentDef) AllVariants 

func (e ExperimentDef) AllVariants() []string

AllVariants returns the primary variant followed by all shadow variants.

type GapStats 

type GapStats struct {
	Count         int
	MedianSeconds int64
	P90Seconds    int64
	MaxSeconds    int64
}

GapStats summarizes inter-session intervals (seconds). Count is the number of gaps — always TotalSessions-1, or 0 when fewer than 2 sessions exist.

type NoteStat 

type NoteStat struct {
	NoteID              string
	RetrievalCountTotal int
	FirstRetrievedTs    string
	LastRetrievedTs     string
}

NoteStat reports aggregate retrieval metrics for a single note, derived from the per_note_stats SQL view over event_data.variants.*.results[]. Counts are per distinct retrieval event, not per variant occurrence — so a note appearing under 3 shadow variants of one event counts once.

type ReportResult 

type ReportResult struct {
	SessionCount int                       `json:"session_count"`
	EventCount   int                       `json:"event_count"`
	OutcomeCount int                       `json:"outcome_count"`
	K            int                       `json:"k"`
	Variants     map[string]VariantMetrics `json:"variants"`
}

ReportResult holds the full experiment report.

type RetrievalEventHit 

type RetrievalEventHit struct {
	NoteID string
	Rank   int
}

RetrievalEventHit is one note surfaced by one event, with its rank in the event's result set. Rank is the minimum across all variants that contained the note (best placement wins).

type RetrievalEventSummary 

type RetrievalEventSummary struct {
	EventID   string
	EventType string
	Timestamp string
	Query     string
	Hits      []RetrievalEventHit
}

RetrievalEventSummary describes one retrieval event — its type, timestamp, query, and the deduplicated list of notes returned. Multiple variants of the same event collapse: a note appearing under "hybrid" and three shadow variants is reported once with its best (lowest) rank across them.

type RetrievalHit 

type RetrievalHit struct {
	NoteID   string
	Rank     int
	Score    float64
	NoteType string
	Path     string
	Scores   map[string]float64
}

RetrievalHit is the input shape for building a variant payload: a single retrieved note with its rank, aggregated score, and optional per-component scores (e.g. fts/dense/sparse/colbert for hybrid retrieval).

Storage/privacy note: payloads built from RetrievalHit are written to the local experiment DB only. The anonymous/full telemetry tiers are identical in the current (local-only) write path — see cmd/root.go. A future uploader MUST filter by tier: anonymous should strip note_id/path; full may send everything.

type Rollup 

type Rollup struct {
	Kind             string                   `json:"kind"`
	SchemaVersion    int                      `json:"schema_version"`
	Tier             string                   `json:"tier"`
	Fingerprint      string                   `json:"vault_fingerprint"`
	NoteCount        int                      `json:"note_count"`
	TypeDistribution map[string]int           `json:"type_distribution"`
	LinkCount        int                      `json:"link_count"`
	AliasCount       int                      `json:"alias_count"`
	EmbeddingCount   int                      `json:"embedding_count"`
	EmbeddingDims    int                      `json:"embedding_dims"`
	VariantStats     map[string]*VariantStats `json:"variant_stats"`
	ExportedAt       string                   `json:"exported_at"`
	SessionCount     int                      `json:"session_count"`
	EventCount       int                      `json:"event_count"`
	OutcomeCount     int                      `json:"outcome_count"`
}

Rollup is the federated-aggregator-shaped payload — one record per vault that captures everything the paper (Paper #2 from the federated-paper design note) needs without exposing content.

Designed to be small, content-free, and stable across schema versions. SchemaVersion tracks payload shape so receivers can dispatch on it.

func (*Rollup) MarshalJSON 

func (r *Rollup) MarshalJSON() ([]byte, error)

MarshalJSON pins the field order via custom marshaling for human-readable preview output. The standard library marshaler would alphabetize; we want kind/version/tier first, fingerprint next, then aggregate features, then variants. Receivers that care about order can rely on the JSON itself; receivers that don't, ignore.

type ScoredNote 

type ScoredNote struct {
	NoteID   string             `json:"note_id"`
	Score    float64            `json:"score"`
	Features map[string]float64 `json:"features,omitempty"`
}

ScoredNote is a note with a computed score, used as input/output for variant scorers.

type Scorer 

type Scorer interface {
	Score(notes []ScoredNote) []ScoredNote
}

Scorer computes variant-specific scores for a list of notes.

type ScorerFunc 

type ScorerFunc func([]ScoredNote) []ScoredNote

ScorerFunc adapts a plain function to the Scorer interface.

func (ScorerFunc) Score 

func (f ScorerFunc) Score(notes []ScoredNote) []ScoredNote

Score implements Scorer for ScorerFunc.

type Session 

type Session struct {
	DB            *DB
	ID            string
	VaultPath     string
	OutcomeWindow int // configurable outcome window; 0 uses default (2)
}

Session holds the active experiment session and its associated database.

func FromContext 

func FromContext(ctx context.Context) *Session

FromContext retrieves the Session from ctx. Returns nil if no Session is stored in the context.

func (*Session) LogAskEvent 

func (s *Session) LogAskEvent(query, primaryVariant string, data map[string]any) (string, error)

LogAskEvent logs an ask event from this session. primaryVariant is the activation experiment's chosen variant name (empty when the activation experiment is disabled or the event has no context pack); it populates the events.primary_variant column so per-variant rollups can group on it without parsing event_data JSON. Without this, the activation experiment's variant choice landed only in the event_data blob and the column stayed NULL — silently breaking VariantPerformance and `vaultmind export --rollup`.

func (*Session) LogContextPackEvent 

func (s *Session) LogContextPackEvent(data map[string]any) (string, error)

LogContextPackEvent logs a context_pack event from this session.

func (*Session) LogNoteAccessEvent 

func (s *Session) LogNoteAccessEvent(noteID, source string) (string, error)

LogNoteAccessEvent logs a note_access event and triggers outcome linkage.

func (*Session) LogSearchEvent 

func (s *Session) LogSearchEvent(query, mode string, data map[string]any) (string, error)

LogSearchEvent logs a search event from this session.

func (*Session) SetVaultPath 

func (s *Session) SetVaultPath(vaultPath string)

SetVaultPath updates the vault path on the session and persists it to the DB.

type SessionCaller 

type SessionCaller struct {
	Caller        string
	Meta          map[string]any
	UserSessionID string
}

SessionCaller describes the invoking agent for a session. Caller is a short label ("companion-persona-hook", "claude-code", "cli", or empty if the session pre-dates attribution). Meta is a flexible JSON blob — project_dir, pid, env-var snapshot — parsed back into a map so callers can read it without knowing the shape. UserSessionID groups multiple invocations into one working session via the UserSessionThreshold time heuristic.

type SessionGap 

type SessionGap struct {
	SessionID     string
	StartedAt     string
	PrevSessionID sql.NullString
	PrevStartedAt sql.NullString
	GapSeconds    sql.NullInt64
}

SessionGap reports one session and its gap from the previous session. Sessions are ordered chronologically ascending. The first session has no predecessor — PrevSessionID, PrevStartedAt, and GapSeconds are NULL.

Primary consumer: compressed-idle-time analysis (gamma parameter fitting).

type SessionHit 

type SessionHit struct {
	SessionID string
	EventID   string
	EventType string
	Timestamp string
	Rank      int
}

SessionHit is one occurrence of a note being retrieved: which session saw it, when, at what rank, under which event. Used by NoteRetrievals to build a cross-session history for a single note.

type SessionWindow 

type SessionWindow struct {
	Start time.Time
	End   time.Time
}

SessionWindow represents a period when VaultMind was actively in use.

type UsageSummary 

type UsageSummary struct {
	TotalSessions       int
	RetrievalEventCount int
	UniqueNotesRecalled int
	TopNotes            []NoteStat
	GapStats            GapStats
}

UsageSummary reports aggregate memory-usage metrics derived from the experiment DB — intended as the "weekly readout" view for someone dogfooding VaultMind: which notes are being recalled, how often, and how sessions are spaced over time.

type VariantMetrics 

type VariantMetrics struct {
	HitAtK     float64 `json:"hit_at_k"`
	MRR        float64 `json:"mrr"`
	EventCount int     `json:"event_count"`
}

VariantMetrics holds computed metrics for a single variant.

type VariantStats 

type VariantStats struct {
	Name         string  `json:"name"`
	EventCount   int     `json:"event_count"`
	OutcomeCount int     `json:"outcome_count"`
	HitAt5       float64 `json:"hit_at_5"`
	HitAt10      float64 `json:"hit_at_10"`
	MRR          float64 `json:"mrr"`
}

VariantStats summarizes one variant's performance across all retrieval events in the experiment DB. The fields are exactly what the federated paper (the federated-paper design note) needs to answer H2.5: which retrieval constants vary meaningfully across vaults vs which are stable.

Reciprocal-rank semantics:

  • For every event whose primary_variant equals this variant AND for which an outcome row was logged (i.e. a note that was retrieved was later actually accessed), we contribute (1 / rank) to the reciprocal-rank sum. MRR = sum / event_count.
  • Hit@K is the fraction of primary-variant events where the accessed-note's rank is ≤ K.

This is the conservative interpretation: shadow variants do not get MRR/Hit@K credit because we don't know what the user "would have" accessed under that variant — only what they actually accessed under the primary. The federated dataset still recovers shadow signal via the side-by-side rank distributions in the variants payload.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.