chunker

package
v0.1.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 27, 2026 License: MIT Imports: 5 Imported by: 0

Documentation

Overview

Package chunker splits source files into syntactic units (functions, types, etc.) so they can be selected whole when a file does not fit within a token budget. Currently only Go has structural support; other languages fall back to a single whole-file chunk.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func SortByPriority

func SortByPriority(chunks []Chunk)

SortByPriority orders chunks for inclusion under a tight budget: exported functions/methods/types first, then unexported, then const/var.

Types

type Chunk

type Chunk struct {
	Kind      string   // "function", "method", "type", "var", "const", "file"
	Name      string   // primary identifier (may be "" for "file" kind)
	Names     []string // all declared identifiers (grouped type/const/var blocks)
	Recv      string   // receiver type name for methods (e.g. "Foo"), else ""
	Refs      []string // type identifiers referenced in a func/method signature
	Exported  bool     // true if the primary identifier is exported (Go convention)
	StartLine int
	EndLine   int
	Body      string // verbatim source for the chunk
}

Chunk represents one syntactic unit of a source file.

func Split

func Split(path string, content []byte) ([]Chunk, error)

Split returns the syntactic chunks of content. Errors only occur for unparseable source in supported languages; callers may treat error as "fall back to whole-file chunking".

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL