Documentation
¶
Overview ¶
Package index is the rebuildable cache.
Nothing here is authoritative. Delete .logos/index.db, run `logos index`, and you are back to identical state derived entirely from the markdown.
Index ¶
- Variables
- type Hit
- type Index
- func (ix *Index) Ask(p *provider.Provider, embedModel, chatModel, question string, k, budget int) (string, []Hit, error)
- func (ix *Index) Close() error
- func (ix *Index) EdgeCount() (int, error)
- func (ix *Index) EmbedPending(p *provider.Provider, model string, batch int) (int, error)
- func (ix *Index) Expand(hits []Hit, minConf float64, limit int) ([]Hit, error)
- func (ix *Index) HitBySlug(slug string) (Hit, bool)
- func (ix *Index) HybridSearch(p *provider.Provider, model, query string, k int) ([]Hit, error)
- func (ix *Index) LexicalSearch(query string, k int) ([]Hit, error)
- func (ix *Index) NoteCount() (int, error)
- func (ix *Index) Search(p *provider.Provider, model, query string, k int) ([]Hit, error)
- func (ix *Index) Sync() (SyncReport, error)
- func (ix *Index) SyncCheckpointHomes() (int, error)
- func (ix *Index) SyncInsights() (restored, rescued int, err error)
- func (ix *Index) SyncLog() (int, error)
- func (ix *Index) SyncLoops() (restored, rescued int, err error)
- func (ix *Index) SyncMemories(p *provider.Provider, embedModel string) (imported, rescued int, err error)
- func (ix *Index) SyncMemoryEdits(p *provider.Provider, embedModel string) (imported, rescued int, e memory.Edits, err error)
- func (ix *Index) SyncNotes() (restored, rescued, empty int, err error)
- func (ix *Index) SyncPending() (restored, rescued int, err error)
- func (ix *Index) Synced() (bool, error)
- type SyncReport
Constants ¶
This section is empty.
Variables ¶
var RebuildPaths = map[string]string{
"notes": "every .md in the vault, walked by Sync",
"aliases": "the aliases: frontmatter of each note, read by Sync",
"edges": "the wikilinks and typed links in each note's body, read by Sync",
"notes_fts": "the notes table, refilled by Sync and by migrate's backfill",
"embeddings": "cache: re-embedded from the notes by EmbedPending",
"meta": "cache: last_sync only; the next Sync writes it again",
"memories": "memories/<kind>/ files, read by SyncMemories (quarantined rows from the review queue's files, by SyncPending)",
"memory_log": "memories/log.md, read by SyncLog",
"digest_stamps": "cache: the hand-edit guard's last-written hash per digest",
"memory_file_stamps": "cache: the hand-edit guard's last-written hash per memory file",
"commitments": "loops.md, read by SyncLoops",
"dream_insights": "dream-insights.md, read by SyncInsights",
"sessions": "each open session's uncommitted.md, read by SyncNotes; an ended one lives in its checkpoint under sessions/",
"session_notes": "each open session's uncommitted.md, read by SyncNotes",
"ruling_vectors": "cache: embeddings of failed entries, recomputed on the next before_you_try",
"ruling_files": "cache: the files each failed entry names, re-read from sessions/ checkpoints by its next lookup",
"ruling_shown": "cache: which rulings this session already showed",
"replay_state": "cache: replay's read cursor",
}
RebuildPaths names, for every table index.db can hold, what puts its rows back after .logos is deleted — or why losing them is safe. Invariant 1 broke four times (memories, working notes, checkpoints, the review queue) and every time the same way: a writer added a table, nothing rebuilt it from markdown, and the loss was silent until someone deleted the index. Listing every table here makes the question "what rebuilds this?" one a new table cannot skip: TestEveryIndexTableNamesWhatRebuildsItFromTheVault fails on any CREATE TABLE this map does not name.
"cache:" entries are tables whose loss costs time or one missed detection, never a record. Anything else must name the vault file a Sync pass reads.
Functions ¶
This section is empty.
Types ¶
type Hit ¶
type Hit struct {
Slug, Title, Kind, Body string
Score float64
// FirstSeen is the note's date from its frontmatter, unix seconds, or zero
// when it has none. Carried on the hit rather than looked up later because a
// caller filtering "what did I do last month" needs it for every candidate,
// and a per-hit query to answer that would be the retrieval path's slowest
// step for the sake of one integer already sitting in the row.
FirstSeen int64
// Via is set when the note was pulled in as a graph neighbour rather than
// matched directly, so the UI can show why it is in context.
Via string
}
Hit is one retrieved note.
type Index ¶
func (*Index) Ask ¶
func (ix *Index) Ask(p *provider.Provider, embedModel, chatModel, question string, k, budget int) (string, []Hit, error)
Ask is retrieval plus generation. Context is capped by character budget rather than token count so this stays honest on a 4k-context local model.
func (*Index) EmbedPending ¶
EmbedPending embeds every note that lacks a vector, in batches — a round trip per note makes a 2000-note vault take minutes instead of seconds.
func (*Index) Expand ¶
Expand pulls in notes linked from the top hits. This is the payoff of keeping a graph at all: asking about a project surfaces the people on it even when their notes share no vocabulary with the question.
func (*Index) HybridSearch ¶
HybridSearch fuses vector and lexical rankings with reciprocal rank fusion.
RRF scores each result by 1/(rank+k) summed across both lists, which needs no score normalisation between two incomparable scales (cosine distance vs bm25) — it only trusts the ordering each method is confident about. k=60 is the value from the original RRF paper and is not sensitive.
func (*Index) LexicalSearch ¶
LexicalSearch is the FTS arm on its own, as full Hits. It is the fallback for a machine with no embedding model: worse than hybrid, but it needs no runtime and no vectors, so retrieval still works on a fresh checkout.
func (*Index) Sync ¶
func (ix *Index) Sync() (SyncReport, error)
Sync reads every note in the vault and reconciles it against the cache. Only notes whose content hash changed are re-parsed and marked for re-embedding.
func (*Index) SyncCheckpointHomes ¶ added in v0.5.4
SyncCheckpointHomes re-files any checkpoint whose host_session frontmatter disagrees with the conversation folder it is physically sitting in.
Unlike every other Sync* here, this one does not restore anything the database alone knew — the checkpoint was never lost, only misplaced. It runs on every index rebuild anyway, because the folder layout is derived from the frontmatter (invariant 1) and this is the pass that keeps it honest after an interrupted migration, a sync conflict copy, or a hand move, the same way the rest of indexPass keeps the cache honest.
Returns how many checkpoints it moved.
func (*Index) SyncInsights ¶
SyncInsights restores dreamed insights from the vault.
Must run after SyncMemories: an insight cites the two memory ids it bridges, and `logos dream review` renders those endpoints, so the memories have to be back before the proposal about them can be read.
Returns how many insights it put back.
func (*Index) SyncLog ¶
SyncLog restores the memory timeline from the vault.
Must run after SyncMemories and SyncPending, for the same reason they run in that order: an event is about a memory, and the memory has to be back before the history of it means anything.
Returns how many events it put back, counting the ones it synthesised for memories that predate the file.
func (*Index) SyncLoops ¶
SyncLoops restores open loops from the vault.
The fifth thing that only the database knew, and the one with no excuse left: `logos loop add` wrote a commitment into commitments and nowhere else, so the documented-safe rebuild emptied the list, and the list coming back empty is indistinguishable from having finished everything on it.
Returns how many loops it put back.
func (*Index) SyncMemories ¶
func (ix *Index) SyncMemories(p *provider.Provider, embedModel string) (imported, rescued int, err error)
SyncMemories reconciles the memory store against its markdown in the vault.
The mirror of Sync, for the other half of what the system knows. Notes are rebuilt from files on every index; memories were not, which made "delete the database and reindex" true for one half and quietly destructive for the other. Running both from one command is what makes the guarantee whole.
Returns how many memories the vault held, and how many it rescued out of a cache that was their only copy — see memory.Import.
func (*Index) SyncMemoryEdits ¶ added in v0.5.3
func (ix *Index) SyncMemoryEdits(p *provider.Provider, embedModel string) (imported, rescued int, e memory.Edits, err error)
SyncMemoryEdits is SyncMemories, also saying which memories a hand edit to the files made or removed — see memory.Edits.
func (*Index) SyncNotes ¶
SyncNotes restores working notes the vault holds and the index does not.
The third half of the same guarantee. Checkpoints were always safe because they are read from markdown; memories became safe when Import landed; notes were the last thing that existed only in the database, so `rm -rf .logos` silently threw away exactly the in-flight work note_progress exists to keep.
Returns how many it restored, how many it rescued out of a cache that was their only copy, and how many it left out as saying nothing — see session.ImportNotes.
func (*Index) SyncPending ¶
SyncPending restores the memory review queue from the vault.
The fourth thing that only the database knew. A memory an agent proposed sits in quarantine until a person accepts it, and quarantine's whole point is that it is *not* written into memories/<kind>.md — so the queue lived in .logos/index.db alone, and the documented-safe rebuild threw away proposals nobody had reviewed yet, with `logos doctor` afterwards reporting "nothing pending" as though they had been.
Must run after SyncMemories: a proposal accepted since the last flush is an active memory now, and Import has to have restored it before the queue is allowed to have an opinion about that id.
Returns how many proposals it put back, and how many it rescued out of a cache that was their only copy.
type SyncReport ¶
type SyncReport struct {
Added, Updated, Removed, Unchanged int
// Skipped counts files the walk found but did not index: symlinks,
// derived ledgers, and files that could not be read. They are reported rather than silently
// dropped, because "the index is missing a note" is otherwise invisible.
Skipped int
// ConflictCopies are the sync clients' conflict copies under sessions/
// that the walk left out, vault-relative. Collected here rather than by a
// walk of their own: every index already walks the whole vault once, and
// a second and third walk for two lists doubled the cost of a large one.
ConflictCopies []string
// StaleTemps are WriteAtomic temp files old enough to belong to no live
// write (vault.StaleTempAge), vault-relative, for the caller to remove
// and announce. Sync itself removes nothing: it is also the walk behind
// setup and the MCP server, where a deletion would go unsaid.
StaleTemps []string
// Unreadable is every path the walk could not enter or stat, with why.
// Skipping it is right — one bad directory must not stop the index — but
// silently skipping it made a missing folder of notes look indexed.
Unreadable []string
}
Source Files