Documentation
¶
Overview ¶
Package indexing provides the core indexing pipeline for offline data preparation.
This package defines the Indexer interface which serves as the entry point for processing documents through parsing, chunking, embedding, and storage stages.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type FileWatcher ¶
type FileWatcher struct {
// contains filtered or unexported fields
}
FileWatcher 文件监控器
func NewFileWatcher ¶
func NewFileWatcher(indexer Indexer, logger logging.Logger) (*FileWatcher, error)
NewFileWatcher 创建文件监控器
func (*FileWatcher) AddConfigs ¶
func (fw *FileWatcher) AddConfigs(configs ...WatchConfig)
AddConfigs 添加多个监控配置
type Indexer ¶
type Indexer interface {
// IndexFile processes a single file into the Vector/Graph stores.
IndexFile(ctx context.Context, filePath string) (*core.IndexingContext, error)
// IndexDirectory concurrently processes an entire directory.
IndexDirectory(ctx context.Context, dirPath string, recursive bool) error
// IndexText indexes plain text content directly (no file parsing required).
// This is useful for programmatic document management from APIs, databases, etc.
IndexText(ctx context.Context, text string, metadata ...map[string]any) error
// IndexTexts indexes multiple plain text contents in batch.
IndexTexts(ctx context.Context, texts []string, metadata ...map[string]any) error
// IndexDocuments indexes documents directly into Vector/Doc/Graph stores.
IndexDocuments(ctx context.Context, docs ...*core.Document) error
// DeleteDocument removes a document and all its associated chunks and vectors.
DeleteDocument(ctx context.Context, docID string) error
// GetDocument retrieves a document by its ID.
GetDocument(ctx context.Context, docID string) (*core.Document, error)
}
Indexer defines the entry point for the offline data preparation pipeline. It provides methods to process individual files or entire directories into the RAG knowledge base.
Directories
¶
| Path | Synopsis |
|---|---|
|
Package community provides community detection algorithms for GraphRAG.
|
Package community provides community detection algorithms for GraphRAG. |
|
parser
|
|
Click to show internal directories.
Click to hide internal directories.