Documentation
¶
Index ¶
- Constants
- Variables
- type GreedyStrategy
- type ImportanceStrategy
- type MetadataStrategy
- type Option
- type PackedResult
- type Packer
- func (p *Packer) Pack(ctx context.Context, docs []schema.Document) (PackedResult, error)
- func (p *Packer) PackScored(ctx context.Context, docs []schema.ScoredDocument) (PackedResult, error)
- func (p *Packer) PackWithScores(ctx context.Context, docs []schema.Document, scores []float64) (PackedResult, error)
- type PackingStrategy
- type TokenStats
- type UsedDocument
Constants ¶
const CompactTemplate = `{{.Content}}`
CompactTemplate is a minimal format without metadata.
const DefaultTemplate = `{{.Content}}
---
Metadata: {{.Metadata}}
---`
DefaultTemplate is the default format for packing documents.
Variables ¶
var ( // ErrNilTokenizer is returned when a nil tokenizer is provided. ErrNilTokenizer = errors.New("tokenizer cannot be nil") // ErrInvalidMaxTokens is returned when maxTokens is less than or equal to zero. ErrInvalidMaxTokens = errors.New("maxTokens must be greater than zero") // ErrTokenCountFailed is returned when token counting fails. ErrTokenCountFailed = errors.New("failed to count tokens") // ErrTemplateParse is returned when template parsing fails. ErrTemplateParse = errors.New("failed to parse template") // ErrTemplateExecute is returned when template execution fails. ErrTemplateExecute = errors.New("failed to execute template") )
Functions ¶
This section is empty.
Types ¶
type GreedyStrategy ¶
type GreedyStrategy struct{}
GreedyStrategy packs documents in their original order.
func (GreedyStrategy) Name ¶
func (GreedyStrategy) Name() string
type ImportanceStrategy ¶
type ImportanceStrategy struct{}
ImportanceStrategy packs documents ordered by score (highest first).
func (ImportanceStrategy) Name ¶
func (ImportanceStrategy) Name() string
type MetadataStrategy ¶
MetadataStrategy packs documents ordered by a metadata field.
func (MetadataStrategy) Name ¶
func (s MetadataStrategy) Name() string
type Option ¶
type Option func(*options)
Option configures a Packer.
func WithStrategy ¶
func WithStrategy(strategy PackingStrategy) Option
WithStrategy sets the packing strategy.
func WithTemplate ¶
WithTemplate sets the document template. Use {{.content}} for document content and {{.metadata}} for metadata.
type PackedResult ¶
type PackedResult struct {
// Content is the final packed context string.
Content string
// UsedDocuments are the documents that fit in the token budget.
UsedDocuments []UsedDocument
// TokenStats contains token usage statistics.
TokenStats TokenStats
// Truncated indicates whether some documents were dropped.
Truncated bool
}
PackedResult contains the result of packing documents.
type Packer ¶
type Packer struct {
// contains filtered or unexported fields
}
Packer packs documents into a context window within token limits.
func (*Packer) Pack ¶
Pack packs documents into a context string within token limits. Documents are packed atomically - either fully included or dropped.
func (*Packer) PackScored ¶
func (p *Packer) PackScored(ctx context.Context, docs []schema.ScoredDocument) (PackedResult, error)
PackScored packs ScoredDocuments using their scores for importance ordering.
func (*Packer) PackWithScores ¶
func (p *Packer) PackWithScores(ctx context.Context, docs []schema.Document, scores []float64) (PackedResult, error)
PackWithScores packs documents with optional scores for importance ordering. If scores is nil or empty, the default strategy ordering is used.
type PackingStrategy ¶
type PackingStrategy interface {
// Name returns the strategy name for logging and debugging.
Name() string
// Order returns documents in the order they should be packed.
Order(docs []schema.Document, scores []float64) []schema.Document
}
PackingStrategy defines how documents are ordered before packing.
type TokenStats ¶
type TokenStats struct {
// TotalTokens is the total tokens used in the packed content.
TotalTokens int
// MaxTokens is the maximum allowed tokens.
MaxTokens int
// DocumentsConsidered is the number of documents evaluated.
DocumentsConsidered int
// DocumentsPacked is the number of documents that fit.
DocumentsPacked int
}
TokenStats holds token usage statistics.