indexer

package

v0.7.0 Latest Latest Go to latest Published: May 30, 2026 License: MIT Imports: 15 Imported by: 0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/prasenjeet-symon/ogcode

Links

Open Source Insights

Documentation ¶

Index ¶

func IsTextFile(ext string) bool
type Indexer
- func New(dir string, docStore *docindex.Store, lr *agent.LoopRunner) *Indexer
type PageCorpus
- func BuildCorpora(pages []PageText) []PageCorpus
type PageText
- func ExtractPages(pdfPath string) ([]PageText, error)
- func ExtractTextFile(filePath string) ([]PageText, error)

Constants ¶

This section is empty.

Variables ¶

This section is empty.

Functions ¶

func IsTextFile ¶ added in v0.7.0

func IsTextFile(ext string) bool

IsTextFile reports whether ext (e.g. ".go") is a supported text/code extension.

Types ¶

type Indexer ¶

type Indexer struct {
	// contains filtered or unexported fields
}

Indexer scans a workspace directory for PDF files and runs the IndexAgent on each one to produce semantic labels per page.

func New ¶

func New(dir string, docStore *docindex.Store, lr *agent.LoopRunner) *Indexer

New creates a new Indexer. Pass an empty model to use the runner's default.

func (*Indexer) IndexDocument ¶

func (idx *Indexer) IndexDocument(ctx context.Context, filePath string) error

IndexDocument extracts text from a PDF or text/code file, builds keyword corpora, then runs the IndexAgent in batches to produce labels. Labels are upserted directly by the agent tool — no pre-registration needed.

func (*Indexer) Run ¶

func (idx *Indexer) Run(ctx context.Context) error

Run scans dir recursively for PDF and text/code files and indexes each one.

func (*Indexer) WithExcludes ¶ added in v0.7.0

func (idx *Indexer) WithExcludes(patterns []string) *Indexer

WithExcludes sets additional patterns to skip during the directory walk. Patterns are matched against directory names and file basenames using filepath.Match.

func (*Indexer) WithModel ¶

func (idx *Indexer) WithModel(model string) *Indexer

WithModel sets the model override for sessions created by the indexer.

type PageCorpus ¶

type PageCorpus struct {
	PageNum  int
	Keywords []string
}

PageCorpus holds the deduplicated keyword corpus for a single PDF page.

func BuildCorpora ¶

func BuildCorpora(pages []PageText) []PageCorpus

BuildCorpora converts page texts to keyword corpora in parallel. Each corpus is deduplicated, lowercased, and stripped of stop words and non-alphabetic tokens.

type PageText ¶

type PageText struct {
	PageNum int
	Text    string
}

PageText holds the raw extracted text for a single PDF page.

func ExtractPages ¶

func ExtractPages(pdfPath string) ([]PageText, error)

ExtractPages reads all pages from the PDF at pdfPath in parallel and returns a slice of PageText sorted by page number.

func ExtractTextFile ¶ added in v0.7.0

func ExtractTextFile(filePath string) ([]PageText, error)

ExtractTextFile reads a text or code file and returns it as a single PageText. HTML/XML/template files have tags stripped before keyword extraction.

Source Files ¶

View all Source files

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL