pycode

package

v1.0.4 Latest Latest Go to latest Published: Mar 17, 2026 License: MIT Imports: 8 Imported by: 0

Details

This section is empty.

This section is empty.

This section is empty.

type PycodeStreamParser struct {
	// contains filtered or unexported fields
}

PycodeStreamParser implements the dataprep.Parser interface for Python code files

func NewPycodeStreamParser() *PycodeStreamParser

NewPycodeStreamParser creates a new Python code stream parser

func (p *PycodeStreamParser) GetSupportedTypes() []string

GetSupportedTypes returns the supported formats

func (p *PycodeStreamParser) ParseStream(ctx context.Context, r io.Reader, metadata map[string]any) (<-chan *entity.Document, error)

ParseStream parses Python code from a reader and returns a channel of documents

func (p *PycodeStreamParser) SetChunkOverlap(overlap int)

SetChunkOverlap sets the chunk overlap

func (p *PycodeStreamParser) SetChunkSize(size int)

SetChunkSize sets the chunk size

func (p *PycodeStreamParser) SetExtractClasses(extract bool)

SetExtractClasses sets whether to extract classes

func (p *PycodeStreamParser) SetExtractComments(extract bool)

SetExtractComments sets whether to extract comments

func (p *PycodeStreamParser) SetExtractFunctions(extract bool)

SetExtractFunctions sets whether to extract functions