pipeline

package
v0.0.0-beta.32 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 17, 2026 License: MIT Imports: 23 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type IndexOptions

type IndexOptions struct {
	Force    bool
	Workers  int
	Verbose  bool
	Progress chan<- ProgressEvent
	// Web crawl options (used by IndexURL)
	MaxPages    int
	MaxDepth    int
	SkipSitemap bool
}

IndexOptions controls indexing behavior.

type Pipeline

type Pipeline struct {
	// contains filtered or unexported fields
}

Pipeline orchestrates the 5-phase GraphRAG pipeline.

func New

func New(st *store.Store, prov llm.Provider, cfg *config.Config) *Pipeline

New creates a new Pipeline.

func (*Pipeline) Finalize

func (p *Pipeline) Finalize(ctx context.Context, verbose bool, force ...bool) error

Finalize runs Phases 3-4: community detection + parallel summaries. If force is true, the graph fingerprint cache is ignored and communities are always regenerated.

func (*Pipeline) IndexPath

func (p *Pipeline) IndexPath(ctx context.Context, path string, opts IndexOptions) error

IndexPath indexes a file or directory.

func (*Pipeline) IndexURL

func (p *Pipeline) IndexURL(ctx context.Context, rootURL string, opts IndexOptions) error

IndexURL crawls a documentation website and indexes all discovered pages.

func (*Pipeline) Prune

func (p *Pipeline) Prune(ctx context.Context) (int, error)

versionInfo returns the next version number and canonical ID. Prune removes documents whose source file no longer exists on disk. Returns the number of rows deleted. Only "real" file-backed documents (absolute filesystem paths) are considered — web-crawled rows with http(s):// paths are left alone.

type ProgressEvent

type ProgressEvent struct {
	Phase   string
	Message string
	Done    bool
	Error   error
}

ProgressEvent sent over progress channel.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL