README
¶
art-dupl
Professional code clone detection for Go. Analyzes source code at the AST level to find real, fixable duplication — ignoring variable names, literal values, and idiomatic boilerplate so every reported clone is actionable.
A fork of mibk/dupl (via golangci/dupl) with major enhancements: multi-method detection, three matching modes (semantic / exact / structural), 7 output formats, .templ support, generated-code filtering, incremental analysis, baseline CI gating, and a professional CLI.
Quick Start
go install github.com/LarsArtmann/art-dupl/cmd/art-dupl@latest
art-dupl # Scan current directory (threshold: 5 statements)
art-dupl -t 15 # Larger clones only
art-dupl --html > report.html # HTML report with syntax highlighting
art-dupl --json -t 10 # JSON for CI/CD
art-dupl stats # Project health statistics
Full documentation: art-dupl.lars.software
Why art-dupl?
| Capability | Original dupl | art-dupl |
|---|---|---|
| Detection algorithm | Suffix tree only | Suffix tree + hash-based + multi-method |
| Semantic matching | No | Yes — alpha-normalization finds renamed clones (Type 2) |
| Matching modes | 1 | 3 — semantic (default), exact, structural |
| Output formats | Text, HTML | Text, Rich-text, HTML, JSON, Simple-JSON, SARIF, plumbing |
| Stats subcommand | No | Yes — health grades (A–F), CSV, JSON, recommendations |
| Templ support | No | Full .templ AST analysis |
| Generated code filtering | No | Auto-detects sqlc, protobuf, mockgen, stringer, templ, generic |
| Incremental analysis | No | AST caching with SHA-256 content hashing |
| CI baseline gating | No | baseline + check subcommands |
| Diff visualization | No | Side-by-side and inline diffs in HTML |
| Clone classification | No | Type 1 / 2 / 3 labels + extractability scores |
| Actionability filtering | No | Suppresses 15+ boilerplate patterns (test scaffolding, error wrapping, Lock+Defer, etc.) — only in --semantic mode |
| Parallel parsing | No | Worker pool with auto-detect |
| Programmatic SDK | No | pkg/artdupl — Detector interface with streaming |
Detection Methods
Controlled with -m / --detection-methods:
| Method | Flag | Description |
|---|---|---|
| Suffix tree | -m art-dupl (default) |
Ukkonen's algorithm on serialized ASTs — finds structural clones |
| Hash-based | -m hash |
Rolling hash on file content — faster, different tradeoffs |
| Combined | -m "hash,art-dupl" |
Runs both in parallel, deduplicates results |
Three Matching Modes
| Mode | Flag | What it detects | Example |
|---|---|---|---|
| Semantic (default) | --semantic |
Same logic with renamed variables and different literal values. a.String() does NOT match b.Error(). |
Type 2 renamed clones |
| Exact | --exact |
Verbatim name matching. Copy-paste only. | Type 1 exact clones |
| Structural | --structural |
AST shape only. a.String() matches b.Error(). |
Raw structural patterns |
Output Formats
| Format | Flag | Use case |
|---|---|---|
| Text | (default) | Quick terminal review |
| Rich text | --rich-text |
Priority badges and refactoring suggestions |
| HTML | --html |
Detailed report with syntax highlighting and diffs |
| JSON | --json / -j |
CI/CD integration with metadata and summary |
| Simple JSON | --simple-json |
Lightweight JSON with impact scores |
| SARIF | --sarif |
GitHub Advanced Security / CodeQL integration |
| Plumbing | --plumbing / -p |
Machine-readable file:startLine-endLine for scripts |
Generate all formats at once: art-dupl --all --output-dir ./reports
Sort results: --sort size (default), occurrence, hash, total-tokens
CI/CD Integration
# Record accepted clones (commit the baseline file)
art-dupl baseline . -t 5
# CI gate: exits 1 if NEW clones are found
art-dupl check . -t 5
GitHub Actions template included at templates/github-actions-duplicate-check.yml.
Pre-commit hook template at templates/pre-commit-hook.yaml.
SARIF upload to GitHub Security tab:
art-dupl --sarif -t 10 > results.sarif
# Upload via github/codeql-action/upload-sarif
Smart Filtering
Generated code is filtered by default. Override with --include-generated:
art-dupl # Default: filters all generated code
art-dupl --include-generated sqlc # Include sqlc-generated files
art-dupl --include-generated templ,protobuf # Include multiple categories
art-dupl --include-generated all # Include everything
Detected categories: sqlc, templ, protobuf, mockgen, stringer, generic (any "Code generated by" comment).
Additional filtering:
art-dupl --only go # Analyze .go files only
art-dupl --only templ # Analyze .templ files only
art-dupl --exclude-pattern "*_test.go" # Exclude test files
art-dupl --include-pattern "internal/*" # Include specific paths
Performance
art-dupl --workers 8 # Parallel parsing (0 = auto-detect CPU)
art-dupl --incremental # AST caching for faster re-runs
art-dupl --timeout 10m # Execution timeout
Configuration
Create dupl.json:
{
"threshold": 10,
"outputFormat": "json",
"paths": ["./src", "./lib"]
}
art-dupl -c dupl.json # CLI flags override config; config overrides defaults
Programmatic SDK
import "github.com/LarsArtmann/art-dupl/pkg/artdupl"
detector, err := artdupl.New(artdupl.Options{
Threshold: 10,
Paths: []string{"./src"},
})
// ...
result, err := detector.FindClones(ctx)
The SDK has zero imports of config/ or errors/ — fully independent types. See SDK Design and pkg.go.dev.
Supported Languages
| Language | Extension | Notes |
|---|---|---|
| Go | .go |
Full AST analysis, 45 node types |
| Templ | .templ |
Full AST analysis via official templ parser, 28 node types |
.templ source files included by default. Templ-generated *_templ.go files filtered by default.
Architecture
Source Files (.go, .templ)
↓
Parsing (go/parser + templ parser) ← parallel worker pool
↓
AST Serialization → unified syntax.Node[]
↓
Suffix Tree (Ukkonen's algorithm) ← O(1) map-based transitions
↓
Duplicate Search → syntax.Match
↓
Clone Classification → domain.ProcessedCloneGroup
↓
Output Formatting (text, HTML, JSON, SARIF, plumbing, rich-text)
| Package | Purpose |
|---|---|
suffixtree/ |
Ukkonen's suffix tree with O(1) map-based transitions |
syntax/ |
AST handling, Go and Templ parsers, alpha-normalization |
detection/ |
Multi-method detection coordination via goroutines |
hash/ |
XXH3 rolling hash-based detection |
job/ |
File parsing pipeline with parallel workers |
printer/ |
7 output formats with sorting and classification |
domain/ |
Domain types: ProcessedClone, enums, Extractability |
config/ |
Multi-source configuration with typed enums |
baseline/ |
Baseline recording + CI check file format |
cache/ |
File-based AST caching with SHA-256 content hashing |
pkg/artdupl/ |
Public SDK with Detector interface |
Development
go build ./... # Build all packages
go test ./... # Run all tests
go test -race ./... # Run with race detector
golangci-lint run --timeout 5m ./... # Lint
nix build # Reproducible build (Go 1.26)
nix flake check # Full CI check
Tests use standard testing + Ginkgo/Gomega BDD. See CONTRIBUTING.md and TESTING.md.
License
MIT — Based on mibk/dupl (via golangci/dupl).
Directories
¶
| Path | Synopsis |
|---|---|
|
Package baseline records and compares sets of accepted clone hashes so art-dupl can run as a CI gate: a baseline captures the clones a codebase currently has, and a check reports only clones that are new relative to the baseline.
|
Package baseline records and compares sets of accepted clone hashes so art-dupl can run as a CI gate: a baseline captures the clones a codebase currently has, and a check reports only clones that are new relative to the baseline. |
|
Package bdd provides ...
|
Package bdd provides ... |
|
Package cache provides file-based caching for parsed AST nodes.
|
Package cache provides file-based caching for parsed AST nodes. |
|
Package cmd provides ...
|
Package cmd provides ... |
|
art-dupl
command
|
|
|
Package config re-exports domain enum types for backward compatibility.
|
Package config re-exports domain enum types for backward compatibility. |
|
Package detection provides multi-method code duplication detection.
|
Package detection provides multi-method code duplication detection. |
|
Package domain provides core domain types and logic for art-dupl.
|
Package domain provides core domain types and logic for art-dupl. |
|
Package errors provides type-safe error handling for dupl
|
Package errors provides type-safe error handling for dupl |
|
Package hash provides ...
|
Package hash provides ... |
|
internal
|
|
|
filtertest
Package filtertest provides ...
|
Package filtertest provides ... |
|
testhelpers
Package testhelpers provides ...
|
Package testhelpers provides ... |
|
testutil
Package testutil provides ...
|
Package testutil provides ... |
|
utils
Package utils provides utility functions for context management and file processing.
|
Package utils provides utility functions for context management and file processing. |
|
Package job provides ...
|
Package job provides ... |
|
pkg
|
|
|
artdupl
Package artdupl provides a Go SDK for detecting code clones via suffix tree and hash-based detection on Go ASTs.
|
Package artdupl provides a Go SDK for detecting code clones via suffix tree and hash-based detection on Go ASTs. |
|
enum
Package enum provides generic helpers for string-based enum types, eliminating boilerplate IsValid/MarshalJSON/UnmarshalJSON/Parse methods.
|
Package enum provides generic helpers for string-based enum types, eliminating boilerplate IsValid/MarshalJSON/UnmarshalJSON/Parse methods. |
|
format
Package format provides formatting utilities for art-dupl.
|
Package format provides formatting utilities for art-dupl. |
|
logger
Package logger provides ...
|
Package logger provides ... |
|
position
Package position provides utilities for working with source code positions.
|
Package position provides utilities for working with source code positions. |
|
Package suffixtree implements suffix tree for clone detection.
|
Package suffixtree implements suffix tree for clone detection. |
|
Package syntax provides unified AST representation for code duplication detection.
|
Package syntax provides unified AST representation for code duplication detection. |
|
golang
Package golang provides Go AST parsing and transformation for clone detection.
|
Package golang provides Go AST parsing and transformation for clone detection. |
|
templ
Package templ provides AST parsing for templ files using the official templ parser.
|
Package templ provides AST parsing for templ files using the official templ parser. |
Click to show internal directories.
Click to hide internal directories.