Documentation
¶
Overview ¶
Package files is the markdown layer: memory and note files are the source of truth for durable knowledge. It parses/serializes YAML frontmatter, writes atomically, keeps the SQLite index mirrors in sync, and watches the trees for out-of-band edits. Ported from Seam v1 (internal/note + internal/watcher).
Index ¶
- Variables
- func AtomicWrite(path string, data []byte, perm os.FileMode) error
- func ContentHash(content string) string
- func MemoryRelPath(project, name string) string
- func NoteRelPath(project, slug string) string
- func ParseMemory(content, relPath string) (core.Memory, error)
- func ParseNote(content, relPath string) (core.Note, error)
- func RenderMemory(m core.Memory) (string, error)
- func RenderNote(n core.Note) (string, error)
- type Indexer
- func (ix *Indexer) AllFilePaths(ctx context.Context) ([]string, error)
- func (ix *Indexer) ClearContentHash(ctx context.Context, relPath string) error
- func (ix *Indexer) ContentHashByFilePath(ctx context.Context, relPath string) (hash string, found bool, err error)
- func (ix *Indexer) DeleteByFilePath(ctx context.Context, relPath string) error
- func (ix *Indexer) IDByFilePath(ctx context.Context, relPath string) (id string, found bool, err error)
- func (ix *Indexer) IndexMemory(ctx context.Context, m core.Memory) error
- func (ix *Indexer) IndexNote(ctx context.Context, n core.Note) error
- type Manager
- func (m *Manager) Close() error
- func (m *Manager) Indexer() *Indexer
- func (m *Manager) MoveMemory(ctx context.Context, mem core.Memory, toProject string) (core.Memory, error)
- func (m *Manager) Mutate(ctx context.Context, relPath string, fn func(context.Context) error) error
- func (m *Manager) MutateMemory(ctx context.Context, relPath string, ...) (core.Memory, error)
- func (m *Manager) MutateNote(ctx context.Context, relPath string, ...) (core.Note, error)
- func (m *Manager) MutatePaths(ctx context.Context, relPaths []string, fn func(context.Context) error) error
- func (m *Manager) Reconcile(ctx context.Context) error
- func (m *Manager) ReembedStatus() ReembedProgress
- func (m *Manager) Remove(ctx context.Context, relPath string) error
- func (m *Manager) SetEmbedder(e llm.Embedder)
- func (m *Manager) Start(ctx context.Context) error
- func (m *Manager) StartReembed(ctx context.Context) (int, error)
- func (m *Manager) Store() *Store
- func (m *Manager) WriteMemory(ctx context.Context, mem core.Memory) (core.Memory, error)
- func (m *Manager) WriteNote(ctx context.Context, note core.Note) (core.Note, error)
- type ReembedProgress
- type Store
- func (s *Store) DataDir() string
- func (s *Store) Exists(relPath string) bool
- func (s *Store) ReadMemory(relPath string) (core.Memory, error)
- func (s *Store) ReadNote(relPath string) (core.Note, error)
- func (s *Store) Remove(relPath string) (err error)
- func (s *Store) WriteMemory(m core.Memory) (core.Memory, error)
- func (s *Store) WriteNote(n core.Note) (core.Note, error)
Constants ¶
This section is empty.
Variables ¶
var ErrNoChange = errors.New("no change")
ErrNoChange is returned by a MutateMemory/MutateNote callback that decided the item is already in its desired state. The write is skipped and the loaded item is returned with a nil error, so an idempotent flip (favorite_set on an already-starred memory) neither rewrites the file nor re-indexes it. It is a control signal, never surfaced to a caller as a failure.
var ErrNoEmbedder = errors.New("no embedder configured")
ErrNoEmbedder is returned by StartReembed when no embedder is configured: there is nothing to embed with, so the request is refused rather than silently queued.
var ErrNotRegular = errors.New("corpus path is not a regular file")
ErrNotRegular is returned when a corpus file path names a directory, device, socket, or other non-regular entry. Memory and note bodies are regular files; treating another entry type as absent would make a later write ambiguous.
var ErrPathOccupied = errors.New("target path belongs to a different item")
ErrPathOccupied is returned when a write would land on a file owned by a different item (by id). In particular a superseded or archived memory keeps its tombstone file at memory/{project}/{name}.md, so that name stays occupied until the tombstone is deleted; a new memory silently overwriting it would destroy readable supersession history. Callers free the name (memory_delete) or pick another one.
var ErrReembedRunning = errors.New("a re-embed pass is already running")
ErrReembedRunning is returned by StartReembed while a previous pass is still in flight; only one full re-embed runs at a time.
var ErrSymlink = errors.New("symlink not allowed in corpus path")
ErrSymlink is returned when any path component below the configured data directory is a symbolic link. Corpus operations deliberately do not follow even an in-tree link: a synced or agent-created link must never make a file outside the declared tree look like source-of-truth knowledge.
var ErrTreeEscape = errors.New("computed path escapes the item's tree")
ErrTreeEscape is returned when an item's computed file path would land outside its own tree: a memory outside memory/, or a note outside notes/. A hostile or corrupt project value (e.g. "../notes") cleans to a path that stays inside the data dir -- which the traversal guard accepts -- but crosses into the other tree; this containment check is the files-layer backstop behind the MCP layer's project validation.
Functions ¶
func AtomicWrite ¶
AtomicWrite writes data to path atomically: it writes a temp file in the same directory, fsyncs it, then renames over the target. This prevents a crash mid-write from corrupting a source-of-truth markdown file. Ported from Seam v1 (note.AtomicWriteFile).
func ContentHash ¶
ContentHash returns the SHA-256 hex digest of a file's full content. It is the change-detection key the reconciler compares against the index.
func MemoryRelPath ¶
MemoryRelPath returns the data-dir-relative path of a memory file: memory/{project|_global}/{name}.md.
func NoteRelPath ¶
NoteRelPath returns the data-dir-relative path of a note file: notes/{project|_global}/{slug}.md.
func ParseMemory ¶
ParseMemory parses memory file content into a core.Memory. relPath is the data-dir-relative file path recorded on the result.
func RenderMemory ¶
RenderMemory serializes a memory to full markdown file content.
Types ¶
type Indexer ¶
type Indexer struct {
// contains filtered or unexported fields
}
Indexer mirrors memory/note files into the SQLite index tables and the unified self-contained FTS5 table. The files on disk are the source of truth; these mirrors are rebuildable and kept in sync by the watcher + reconciler.
func (*Indexer) AllFilePaths ¶
AllFilePaths returns every data-dir-relative file_path currently in the memory and note indexes. The reconciler uses it to find rows whose file has vanished.
func (*Indexer) ClearContentHash ¶
ClearContentHash blanks the recorded content hash for the row at a path. The empty hash never matches a real digest, so the next reconcile (or watcher event) treats the file as changed and re-indexes it -- the retry mechanism for a failed embed. A missing row is a no-op.
func (*Indexer) ContentHashByFilePath ¶
func (ix *Indexer) ContentHashByFilePath(ctx context.Context, relPath string) (hash string, found bool, err error)
ContentHashByFilePath returns the indexed content hash for a path, and whether a row exists. The reconciler uses it to skip unchanged files.
func (*Indexer) DeleteByFilePath ¶
DeleteByFilePath removes the index (and FTS) row for a data-dir-relative path. It is a no-op if no row references that path. Used by the watcher/reconciler when a file disappears from disk.
func (*Indexer) IDByFilePath ¶
func (ix *Indexer) IDByFilePath(ctx context.Context, relPath string) (id string, found bool, err error)
IDByFilePath returns the id of the index row holding a data-dir-relative path, and whether such a row exists. The write guard uses it to detect a path already owned by a different item (the file_path column is UNIQUE).
func (*Indexer) IndexMemory ¶
IndexMemory upserts a memory into memories_index and refreshes its FTS row.
type Manager ¶
type Manager struct {
// contains filtered or unexported fields
}
Manager is the running files subsystem: it owns the filesystem Store and the SQLite Indexer, reconciles the trees against the index at startup, and watches them for out-of-band edits. Application writes go through it so their own writes are suppressed in the watcher (no re-index loop). An optional embedder keeps the vector index in sync with the file content (best-effort).
func NewManager ¶
NewManager builds a Manager over dataDir backed by db. It does not touch the filesystem or start watching until Start is called.
func (*Manager) Close ¶
Close stops the watcher and drains its work: after Close returns, no debounce handler is running or will run, the event-loop goroutine has exited, and any re-embed pass has been cancelled and drained -- so the caller may close the DB they write to. Safe to call more than once, and without a prior Start.
func (*Manager) MoveMemory ¶
func (m *Manager) MoveMemory(ctx context.Context, mem core.Memory, toProject string) (core.Memory, error)
MoveMemory relocates a memory to another project, keeping its ULID. It mirrors the fixed note-move recipe: refuse when a different memory already owns the target path (WriteMemory's occupancy guard), and write the new file BEFORE removing the old one -- the index row is keyed by id, so the write repoints its file_path and a failed write leaves the memory intact at its old path instead of deleting it outright. The memory keeps its name; inbound [[name]] wiki-links resolve globally by bare name, so a move needs no link rewrite. The caller is responsible for bumping Updated. It is a no-op when toProject already equals the memory's project (idempotent for a retried apply).
func (*Manager) Mutate ¶ added in v0.4.10
Mutate serializes a read-modify-write over relPath: fn runs with the path's lock held, so no other Mutate on the same file can interleave between fn's read and its write.
This is the app-side fix for the lost update. Every mutating handler used to read an item, edit the struct, and write the whole file back; two of them racing on one file both read the same starting content and the second rename won, erasing the first write with no error anywhere -- the index upsert is by id, so even the UNIQUE file_path constraint stayed quiet. seamlessd is a single daemon, which is what makes an in-process lock a complete answer rather than a partial one: every application write to the corpus goes through this Manager. (An out-of-band editor write is a different problem, handled by the expect_hash precondition, which compares against the FILE inside this lock.)
The lock is NOT reentrant. fn must not call Mutate for a path it already holds, and the Manager's own WriteMemory/WriteNote/Remove deliberately take no lock, so calling them from inside fn -- which is the point -- cannot deadlock.
func (*Manager) MutateMemory ¶ added in v0.4.10
func (m *Manager) MutateMemory(ctx context.Context, relPath string, fn func(context.Context, core.Memory) (core.Memory, error)) (core.Memory, error)
MutateMemory reads the memory at relPath, hands it to fn, and writes back what fn returns -- all under the path's lock. It is the typed shorthand for the common shape; reach for Mutate directly when the mutation is not one load-edit-store of a single memory (memory_write, which must handle the file not existing yet, is the notable case).
fn returning ErrNoChange skips the write and reports the loaded memory.
func (*Manager) MutateNote ¶ added in v0.4.10
func (m *Manager) MutateNote(ctx context.Context, relPath string, fn func(context.Context, core.Note) (core.Note, error)) (core.Note, error)
MutateNote is MutateMemory for notes: read, edit, write, all under the lock. fn returning ErrNoChange skips the write and reports the loaded note.
func (*Manager) MutatePaths ¶ added in v0.4.10
func (m *Manager) MutatePaths(ctx context.Context, relPaths []string, fn func(context.Context) error) error
MutatePaths is Mutate over several paths at once, for a mutation that spans two files: a note moving to another project writes the new path and removes the old one, and both halves must be serialized against anything else touching either. Paths are locked in sorted order, which is what keeps two concurrent moves in opposite directions from deadlocking; duplicates collapse so passing the same path twice is safe rather than a self-deadlock.
func (*Manager) Reconcile ¶
Reconcile brings the index into agreement with the trees on disk: it re-indexes changed/new files and drops index rows whose file has been deleted.
func (*Manager) ReembedStatus ¶ added in v0.4.1
func (m *Manager) ReembedStatus() ReembedProgress
ReembedStatus returns a snapshot of the current or most recent re-embed pass.
func (*Manager) Remove ¶
Remove deletes a memory/note file (suppressing the watcher) and its index row.
func (*Manager) SetEmbedder ¶
SetEmbedder enables vector indexing. When set, every (re)indexed item is embedded and its vector upserted; embedding failures are logged, not fatal, so a slow or down embedder never blocks a write or an edit. Nil disables it.
It MUST be called before Start, from the goroutine that owns the Manager. After Start, the watcher's handler goroutines read m.embedder without synchronization, so setting it on a running Manager is a data race. The field is unguarded deliberately: the embedder is fixed at startup (main.go resolves it from config, then starts), so a lock would cost every indexed write to protect against a call that has no legitimate reason to happen.
func (*Manager) Start ¶
Start creates the tree directories, begins watching them, reconciles the index against disk, and launches the event loop in a background goroutine. The loop stops when ctx is cancelled or Close is called.
func (*Manager) StartReembed ¶ added in v0.4.1
StartReembed launches a background pass that re-embeds every indexed memory and note with the current embedder, replacing each item's stored vector (the embeddings row is keyed by item id). This is the migration path after an embedding-model change: vectors from different models are not comparable, so items embedded under the old model are invisible to semantic search until they are re-embedded. It returns how many files were queued. Only one pass runs at a time (ErrReembedRunning), and with no embedder it refuses (ErrNoEmbedder) rather than queueing work that cannot run.
The pass outlives the request that triggered it (context.WithoutCancel) and is cancelled by Close, which also waits for it -- so it never touches the DB after shutdown starts.
func (*Manager) WriteMemory ¶
WriteMemory writes a memory through the Store (suppressing the watcher's view of its own write) and indexes it synchronously. It returns the stored memory with FilePath and ContentHash populated. A path already owned by a different memory -- notably the tombstone file of a superseded memory whose name the write would revive -- is refused with ErrPathOccupied rather than overwritten.
type ReembedProgress ¶ added in v0.4.1
type ReembedProgress struct {
Running bool `json:"running"`
Total int `json:"total"`
Done int `json:"done"`
Failed int `json:"failed"`
StartedAt time.Time `json:"startedAt"`
FinishedAt time.Time `json:"finishedAt"`
}
ReembedProgress reports the state of the current (or most recent) full re-embed pass. A zero value means no pass has run this process.
type Store ¶
type Store struct {
// contains filtered or unexported fields
}
Store reads and writes memory and note files under a data directory. It owns no database; the index mirror and watcher are layered on top.
func (*Store) ReadMemory ¶
ReadMemory reads and parses the memory at a data-dir-relative path.
func (*Store) Remove ¶
Remove deletes the file at a data-dir-relative path. A missing file is not an error (the desired end-state already holds).
func (*Store) WriteMemory ¶
WriteMemory renders m, writes it atomically to its computed path, and returns m updated with FilePath and ContentHash. Name must be a safe filename.
Source Files