Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type SlideChunker ¶
type SlideChunker struct {
// contains filtered or unexported fields
}
func New ¶
func New(window uint32, slide uint32) *SlideChunker
Creates new chunker with maximum tokens in one chunk `window` and `slide` overlap between chunks.
func (*SlideChunker) GenerateChunks ¶
func (c *SlideChunker) GenerateChunks(ctx context.Context, streamIterator parser.StreamResultIterator) chunker.ChunkIterator
Click to show internal directories.
Click to hide internal directories.