Documentation
¶
Index ¶
- type TextStreamParser
- func (p *TextStreamParser) GetSupportedTypes() []string
- func (p *TextStreamParser) Parse(ctx context.Context, content []byte, metadata map[string]any) (*core.Document, error)
- func (p *TextStreamParser) ParseStream(ctx context.Context, r io.Reader, metadata map[string]any) (<-chan *core.Document, error)
- func (p *TextStreamParser) Supports(contentType string) bool
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type TextStreamParser ¶
type TextStreamParser struct {
// contains filtered or unexported fields
}
TextStreamParser is a true streaming parser that reads text/markdown files. It complies with the Single Responsibility Principle by only extracting content into Document entities and leaving chunking to the Chunker.
func NewTextStreamParser ¶
func NewTextStreamParser(maxReadBytes int) *TextStreamParser
NewTextStreamParser creates a new parser optimized for raw text.
func (*TextStreamParser) GetSupportedTypes ¶
func (p *TextStreamParser) GetSupportedTypes() []string
GetSupportedTypes returns the MIME types or extensions this parser can handle.
func (*TextStreamParser) Parse ¶
func (p *TextStreamParser) Parse(ctx context.Context, content []byte, metadata map[string]any) (*core.Document, error)
Parse implements core.Parser interface.
func (*TextStreamParser) ParseStream ¶
func (p *TextStreamParser) ParseStream(ctx context.Context, r io.Reader, metadata map[string]any) (<-chan *core.Document, error)
ParseStream reads the incoming io.Reader and yields chunks of the document via a channel.
func (*TextStreamParser) Supports ¶
func (p *TextStreamParser) Supports(contentType string) bool
Source Files
Click to show internal directories.
Click to hide internal directories.