tokenutil

package
v0.1.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 27, 2026 License: MIT Imports: 3 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func EstimateTokens

func EstimateTokens(text string) int

EstimateTokens returns an approximate token count for the given text.

func Truncate

func Truncate(text string, maxTokens int) string

Truncate shortens text to fit within the given token budget. Returns the original text unchanged if it fits. Appends a truncation marker when content is cut.

func TruncateSourceFiles

func TruncateSourceFiles(files map[string]string, maxTokens int) map[string]string

TruncateSourceFiles truncates a collection of source file contents to fit within a total token budget. Files are included in order until the budget is exhausted.

Types

This section is empty.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL