Documentation
¶
Index ¶
- func ApplyBaseline(clones []domain.Clone, b *BaselineFile) (known, newCount int)
- func ApplyConfigDefaults(opts *ScanOptions, cfg domain.Config)
- func DetectLanguage(path string) *domain.Language
- func FindConfig(startDir string) string
- func FormatReport(report *domain.Report, format string, opts FormatOptions) (string, error)
- func LangForName(name string) *domain.Language
- func LoadConfig(path string) (domain.Config, error)
- func StagedFiles(repoRoot string) ([]string, error)
- func WriteBaseline(path string, clones []domain.Clone, minTokens int) error
- type BaselineFile
- type DetectOptions
- type DetectStats
- type FormatOptions
- type FuncSpan
- type FunctionDef
- type IgnoreRule
- type ScanOptions
- type ScannerService
- type Token
- type TokenKind
- type TokenizedFile
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func ApplyBaseline ¶
func ApplyBaseline(clones []domain.Clone, b *BaselineFile) (known, newCount int)
ApplyBaseline marks clones covered by the baseline (known debt) and returns how many are known vs new. A clone is known when its fingerprint is recorded and its instance count has not grown beyond the recorded count.
func ApplyConfigDefaults ¶
func ApplyConfigDefaults(opts *ScanOptions, cfg domain.Config)
ApplyConfigDefaults fills in ScanOptions fields from cfg where the caller has not already set them (zero value = not set by CLI flag). CLI flags always win; config is only a fallback.
func DetectLanguage ¶
DetectLanguage returns the Language for a file path, or nil if unsupported.
func FindConfig ¶
FindConfig walks up from startDir looking for a .dupehound.yml file. Returns the path to the first one found, or "" if none exists.
func FormatReport ¶
FormatReport formats a Report as text, json, sarif, md, or github.
func LangForName ¶
LangForName returns the Language for a given name, or nil if not found.
func LoadConfig ¶
LoadConfig reads and parses a .dupehound.yml file. Returns a zero-value Config (no error) if path is empty.
func StagedFiles ¶
StagedFiles returns the absolute paths of files staged for commit (added, copied, modified). Returns an empty slice (no error) if not inside a git repository. Only files with supported language extensions are returned.
Types ¶
type BaselineFile ¶
type BaselineFile struct {
Version int `json:"version"`
MinTokens int `json:"min_tokens,omitempty"` // informative: threshold the baseline was built with
Fingerprints map[string]int `json:"fingerprints"` // clone hash → instance count at baseline time
}
BaselineFile is the on-disk format of a duplication baseline: the accepted debt at the moment --write-baseline ran. Fingerprints maps each clone's stable content hash to the number of instances it had. A later scan flags a clone as NEW when its fingerprint is absent — or present with MORE instances, so pasting yet another copy of known-duplicated code still fails.
Fingerprints are content-based (normalized token structure), so the baseline survives line shifts, file renames, and unrelated edits without churn. The file is deterministic (sorted keys) and diff-friendly, meant to be committed next to .dupehound.yml.
func LoadBaseline ¶
func LoadBaseline(path string) (*BaselineFile, error)
LoadBaseline reads and validates a baseline file written by WriteBaseline.
type DetectOptions ¶
type DetectOptions struct {
MinTokens int
MinSimilarity float64
MaxBucket int // max blocks per fuzzy mini-hash bucket (0 = default 500)
// MaxPairs, if > 0, caps the number of fuzzy candidate pairs the detector
// evaluates — a runaway backstop that bounds CPU time on pathological
// inputs. When the cap is reached, type-3 detection stops at the next block
// boundary and returns the matches found so far (a deterministic PARTIAL
// result); type-1/2 clones are unaffected. 0 means no cap.
MaxPairs int
// InScopeFiles, when non-nil, restricts detection to clones where at
// least one instance is in a file marked true. Indexed by the position
// of the file in the `files` slice passed to DetectWithOptions. The
// fuzzy detector also uses this to skip candidate pairs where neither
// block is in scope, which is the main cost win for diff-aware scanning.
// Nil means "all files in scope" (original behavior).
InScopeFiles []bool
}
DetectOptions holds parameters for clone detection.
type DetectStats ¶
type DetectStats struct {
// CappedPairs is true when --max-pairs stopped fuzzy detection early:
// the type-3 results are a deterministic partial set.
CappedPairs bool
// BucketTruncated is true when at least one fuzzy candidate bucket was
// cut to --max-bucket, so some near-miss pairs were never compared.
BucketTruncated bool
// EvaluatedPairs counts the fuzzy candidate pairs actually compared.
EvaluatedPairs int
}
DetectStats reports how complete a detection run was. The scan is normally exhaustive; these flags flip only when a safety guard reduced type-3 coverage. Type-1/2 results are always complete.
func DetectWithOptions ¶
func DetectWithOptions(files []TokenizedFile, opts DetectOptions) ([]domain.Clone, DetectStats)
DetectWithOptions finds all clone groups with full control over detection parameters. The returned DetectStats reports whether any safety guard (--max-pairs, --max-bucket) reduced type-3 coverage.
func (DetectStats) Partial ¶
func (s DetectStats) Partial() bool
Partial reports whether any guard reduced type-3 coverage.
func (DetectStats) Reason ¶
func (s DetectStats) Reason() string
Reason renders a human-readable explanation for a partial result, or "".
type FormatOptions ¶
type FormatOptions struct {
ScanPath string // base path for relative file paths
Verbose bool // show all clones with previews
Top int // max clones shown (0 = use defaultTopClones, -1 = all)
ShowSuppressed bool // include suppressed clones in output
}
FormatOptions controls text output presentation.
type FuncSpan ¶
FuncSpan records the token-index range of one function/method body together with the best-effort name of the function it belongs to. Start and End are inclusive token indices of the body contents. Name is "" when the function is anonymous or its name could not be determined.
Spans are collected by the same per-language markers that build the InFunc mask, so name attribution never disagrees with what the detector considers "inside a function".
type FunctionDef ¶
type FunctionDef struct {
Name string
File string
Line int
Lang string
DefTok int // token index of the function name
}
FunctionDef holds extracted function definition metadata.
type IgnoreRule ¶
type IgnoreRule struct {
Raw string // original text
Hash string // 8+ hex chars → suppress by clone hash
PathGlob string // path glob (may contain **) → suppress all clones touching this path
FilePath string // file:start-end → suppress specific line range
FileStart int
FileEnd int
}
IgnoreRule represents a single line in a .dupehound-ignore file.
type ScanOptions ¶
type ScanOptions struct {
Path string
MinTokens int
MinLines int // deprecated; if MinTokens == 0, converted to MinTokens = MinLines * 10
Exclude []string
Include []string // if non-empty, only files matching at least one pattern are scanned
Language string
MinSimilarity float64 // minimum Jaccard similarity for type-3 detection (0.50–1.00)
MaxBucket int // max blocks per fuzzy bucket (0 = default 500)
Staged bool // only report clones involving git-staged files
Top int // max clones to show in text/md output (0 = all)
Since string // git ref for diff-aware scanning
ShowSuppressed bool // include suppressed clones in output
DeadCode bool // enable dead function detection
GitChurn bool // annotate clones with git churn scores
ChurnDays int // number of days for git churn window (default 90)
IgnoreFile string // path to .dupehound-ignore file (auto-discovered if empty)
MaxFiles int // hard cap on collected files (0 = no cap); fail-fast safety net
MaxPairs int // runaway backstop on fuzzy pairs (0 = no cap); caps type-3 to a partial result past the limit
MaxFileSize int64 // per-file size cap in bytes (0 = no cap); oversized files are skipped before being read
ScanGenerated bool // when true, do NOT skip machine-generated files (default: skip them)
Baseline string // path to a baseline file: known clones become recorded debt, only new ones fail
WriteBaseline string // path to write a new baseline capturing the current clones as accepted debt
}
ScanOptions configures a scan run.
type ScannerService ¶
type ScannerService struct{}
ScannerService performs code duplication detection.
func NewScannerService ¶
func NewScannerService() *ScannerService
NewScannerService creates a new ScannerService.
func (*ScannerService) Scan ¶
func (s *ScannerService) Scan(opts ScanOptions) (*domain.Report, error)
Scan walks the given path, tokenizes all source files, and returns a Report of all detected clones. Uses the token-based detector for type-1 and type-2 clone detection.
type Token ¶
type Token struct {
Kind TokenKind
Text string // empty for Ident/Number/String; verbatim for Keyword/Operator
OrigText string // original text for Ident/Number/String; empty for Keyword/Operator
Line int // 1-indexed original source line
IgnoreMark bool // true if this token is on a dupehound:ignore annotated line marker
}
Token is a normalized lexical unit.
Text is only set for TokKeyword and TokOperator (verbatim value for hashing, so `if` ≠ `for` and `+` ≠ `-`). It is empty for Ident/Number/String.
OrigText is only set for TokIdent, TokNumber, and TokString (original source text, used after detection to classify clones as type-1 vs type-2). It is empty for Keyword/Operator.
func TokenizeFile ¶
TokenizeFile lexes content into a normalized token sequence for the given language. Comments are consumed (no token emitted). Identifiers become TokIdent, keywords become TokKeyword, number literals become TokNumber, string literals become TokString. Operators and punctuation are emitted as TokOperator.
func TokenizeFileWithIgnore ¶
TokenizeFileWithIgnore tokenizes content and also returns which tokens follow a dupehound:ignore marker. Works like TokenizeFile but sets IgnoreMark on the first token on lines immediately after an ignore comment.
type TokenKind ¶
type TokenKind uint8
TokenKind is the semantic category of a token after normalization. Only the Kind is used for hashing — values (identifier names, literal values) are discarded. This means `result := compute()` and `output := compute()` hash identically (type-2 detection).
const ( TokKeyword TokenKind = iota // reserved word, kept verbatim for structure TokIdent // user-defined name → all normalize to same kind TokNumber // numeric literal → normalizes to same kind TokString // string/char literal → normalizes to same kind TokOperator // operator or punctuation, kept verbatim )
type TokenizedFile ¶
type TokenizedFile struct {
Path string
Tokens []Token // normalized token sequence (no newlines)
InFunc []bool // per-token: true if inside a function/method body
Ignored []bool // per-token: true if in a dupehound:ignore annotated block
Funcs []FuncSpan // function body spans with best-effort names, for clone attribution
}
TokenizedFile holds the lexed representation of one source file. RawLines are not stored here to reduce memory; preview lines are loaded lazily from disk only for files that end up in detected clones.
func BuildTokenizedFileWithIgnore ¶
func BuildTokenizedFileWithIgnore(path, content string, lang *domain.Language, rules []IgnoreRule) TokenizedFile
BuildTokenizedFileWithIgnore tokenizes a source file, applying inline suppression markers from both the source (dupehound:ignore comments) and the ignore rules.