Documentation
¶
Index ¶
Constants ¶
View Source
const ( FitInputNote = upstream.FitInputNote RoundingSegment = upstream.RoundingSegment RoundingTotal = upstream.RoundingTotal TokenCharDivisor = upstream.TokenCharDivisor )
Variables ¶
View Source
var ( BigramSet = upstream.BigramSet ChunkLines = upstream.ChunkLines ClampRuneBoundary = upstream.ClampRuneBoundary DefaultCJKRatio = upstream.DefaultCJKRatio EstTokens = upstream.EstTokens EstTokensCJK = upstream.EstTokensCJK EstTokensCJKRunes = upstream.EstTokensCJKRunes EstTokensCJKRunesWithRatio = upstream.EstTokensCJKRunesWithRatio EstTokensOf = upstream.EstTokensOf EstTokensWithRatio = upstream.EstTokensWithRatio EstTokensWithRounding = upstream.EstTokensWithRounding FTS5Query = upstream.FTS5Query FirstNonEmpty = upstream.FirstNonEmpty FitCharsLimit = upstream.FitCharsLimit GlobMatch = upstream.GlobMatch Jaccard = upstream.Jaccard NearDuplicate = upstream.NearDuplicate NumberLines = upstream.NumberLines RuneUnitsCJK = upstream.RuneUnitsCJK SanitizeFileStem = upstream.SanitizeFileStem Similarity = upstream.Similarity SplitRunes = upstream.SplitRunes StripCodeFence = upstream.StripCodeFence Tokens = upstream.Tokens TokensJoined = upstream.TokensJoined TruncBytes = upstream.TruncBytes TruncEllipsis = upstream.TruncEllipsis TruncNote = upstream.TruncNote TruncRunes = upstream.TruncRunes UnitsToTokensCJK = upstream.UnitsToTokensCJK )
Functions ¶
This section is empty.
Types ¶
type TokenRounding ¶ added in v1.8.10
type TokenRounding = upstream.TokenRounding
Click to show internal directories.
Click to hide internal directories.