Documentation
¶
Overview ¶
Package ingest wires the Source adapters (issue #8), the chunker (issue #7), the optional embedder (issue #7), and the Store (issue #10) together into the `lore add`/`lore refresh` flows (issue #3).
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Pipeline ¶
type Pipeline struct {
Sources source.Registry
Store store.Store
Embedder embed.Embedder // nil disables vector search, per issue #7
Logger *slog.Logger
}
func (*Pipeline) Add ¶
Add implements `lore add <ref>[@version]`, issue #3. ref must be a typed-prefix identity (e.g. "github:owner/repo") or bare adapter-specific reference prefixed the same way; bare-name source inference (issue #3's "or a bare name lore attempts to infer a source for") isn't implemented yet — callers must give the type prefix explicitly for now.
func (*Pipeline) Refresh ¶
Refresh implements `lore refresh [<library_id>[@version]]`, issue #3, using the manual-trigger + content_hash-diff strategy decided in issue #12: re-fetch, and only re-chunk/re-embed pages whose hash changed. (The conditional-request optimization from #12 — ETags/commit SHAs — and the "registry versions are immutable, only discover new ones" nuance are not implemented yet; this always does the content_hash-diff fallback path, which is correct, just not the cheapest case in #12's design.)
Source Files