chunk

package
v1.9.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 27, 2026 License: MIT Imports: 4 Imported by: 0

Documentation

Overview

Package chunk provides exchange-aware chunking for session backfill.

Index

Constants

View Source 
const (
	DefaultMaxChunkChars    = 120000 // ~30K tokens at 4 chars/token
	DefaultOverlapExchanges = 3      // Overlap between chunks for context continuity
)

Default chunking parameters.

Variables

This section is empty.

Functions

This section is empty.

Types

type Chunk 

type Chunk struct {
	StartExchange int    // 1-indexed for display
	EndExchange   int    // inclusive
	Text          string // sanitized, concatenated exchange text
}

Chunk represents a contiguous range of exchanges from a session.

func Exchanges 

func Exchanges(exchanges []sessions.Exchange, maxChunkChars, overlapExchanges int) []Chunk

Exchanges splits session exchanges into overlapping chunks that fit within maxChunkChars. Each chunk includes sanitized user and assistant text with exchange numbering.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.