Documentation
¶
Overview ¶
Package outputdedupe finds repeated line runs in earlier visible text. An Index belongs to one forward pass over a model request, not to a session.
Index ¶
Constants ¶
const ( Threshold = 192 WindowBytes = 256 << 10 )
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Index ¶
type Index struct {
// contains filtered or unexported fields
}
Index retains at most WindowBytes of referenceable text, plus seed metadata. It is not concurrent-safe. Seedless text advances the window without storage. Call Close after the forward pass to return its buffers to the pool.
func (*Index) Add ¶
func (idx *Index) Add(source int, projection Projection)
Add indexes only the verbatim rows of a projection made by this Index. Source is returned in future spans and should identify this unit to the caller. Marker rows and replaced text cannot become reference targets.
func (*Index) Close ¶
func (idx *Index) Close()
Close clears indexed text before returning buffers. It is safe to call repeatedly. Existing projections own only original text and remain usable.
func (*Index) Project ¶
func (idx *Index) Project(text string) Projection
Project streams line boundaries without per-line scratch storage. Equal-length candidates select the most recent source position, independently of hash seed. Units below Threshold are neither matched nor subsequently indexed by Add.
type Projection ¶
type Projection struct {
Spans []Span
// contains filtered or unexported fields
}
Projection retains original text for Add. Callers must leave Spans unchanged. Render each span as one marker row, preserving its trailing LF when present.
Source Files