art-dupl

module
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 12, 2026 License: MIT

README

art-dupl

Go Report Card codecov

art-dupl is a code duplication detection tool for Go projects. It analyzes source code at the AST level to find structural clones — ignoring literal values so it focuses on patterns, not string contents.

A fork of mibk/dupl (via golangci/dupl) with major enhancements: multi-method detection, semantic-aware matching, 7 output formats, templ support, incremental analysis, and a professional CLI.

Why art-dupl?

Capability Original dupl art-dupl
Detection algorithm Suffix tree only Suffix tree + hash-based + multi-method
Semantic matching No Yes — distinguishes identifiers by name
Output formats Text, HTML Text, HTML, JSON, Simple-JSON, SARIF, plumbing, rich-text
Stats subcommand No Yes — with health grades, CSV, JSON
Templ support No Full .templ AST analysis
Generated code filtering No Auto-detects sqlc, protobuf, mockgen, stringer
Incremental analysis No AST caching with git-aware mode
Diff visualization No Side-by-side and inline diffs in HTML
Parallel parsing No Worker pool with auto-detect

Installation

go install github.com/LarsArtmann/art-dupl/cmd/art-dupl@latest

Or build from source:

git clone https://github.com/LarsArtmann/art-dupl.git
cd art-dupl
just build    # Creates ./dist/art-dupl

Or with Nix:

nix build     # Builds with Go 1.26, static binary
nix run .#art-dupl -- ./src

Quick Start

# Scan current directory (default: threshold 15 tokens)
art-dupl

# Larger clones only (50+ tokens)
art-dupl -t 50

# HTML report with diff visualization
art-dupl --html --diff side-by-side > report.html

# JSON for CI/CD pipelines
art-dupl --json -t 20

# Structural-only matching (shows more results, including noise)
art-dupl --structural ./src

# Hash-based detection (faster)
art-dupl -m hash ./src

# Both detection methods combined
art-dupl -m "hash,art-dupl" ./src

# All output formats to a directory
art-dupl --all --output-dir ./reports ./src

# Project statistics
art-dupl stats
art-dupl stats --format json

Supported Languages

Language Extension Notes
Go .go Full AST analysis
Templ .templ Full AST analysis via official templ parser

.templ source files are included by default. Templ-generated *_templ.go files are filtered by default (use --include-templ to include them). Use --only go or --only templ to restrict analysis to a single file type.

Detection Methods

Controlled with -m / --detection-methods:

Method Flag Description
Suffix tree -m art-dupl (default) Ukkonen's algorithm on serialized ASTs — finds structural clones
Hash-based -m hash Rolling hash on file content — faster, different tradeoffs
Combined -m "hash,art-dupl" Runs both in parallel, deduplicates results
Semantic vs. Structural Matching
  • Default (semantic): Matches by structure AND identifier names. a.String() does NOT match b.Error(). Distinguishes methods by receiver type.
  • --structural: Matches by AST shape only. a.String() matches b.Error(). Use for raw structural analysis.

Output Formats

Format Flag Use case
Text (default) Quick terminal review
Rich text --rich-text Enhanced text with priority badges and suggestions
HTML --html Detailed report with syntax highlighting and diff visualization
JSON --json / -j CI/CD integration with metadata and summary
Simple JSON --simple-json Lightweight JSON with impact scores
SARIF --sarif GitHub Advanced Security / CodeQL integration
Plumbing --plumbing / -p Machine-readable file:startLine-endLine for scripts

Sort results with -s / --sort: size (default), occurrence, hash, total-tokens.

CLI Reference

art-dupl [flags] [paths...]
art-dupl stats [flags] [paths...]
art-dupl completion [bash|zsh|fish|powershell]
Core Flags
Flag Short Default Description
--threshold -t 15 Minimum token sequence size
--detection-methods -m art-dupl Detection method(s)
--sort -s size Sort: size, occurrence, hash, total-tokens
--semantic true Match by structure AND identifier names (default)
--config -c Path to JSON config file
--verbose -v Verbose logging (repeat for more)
--version Show version
Output Flags
Flag Short Description
--html HTML report with syntax highlighting
--json -j Structured JSON with statistics
--simple-json Lightweight JSON with impact scores
--sarif SARIF 2.1.0 format
--plumbing -p Machine-readable format
--rich-text Enhanced text with badges
--all -a Generate all formats
--output-dir -o Output directory (with --all)
--diff Diff visualization: side-by-side, inline, or true
Filtering Flags
Flag Default Description
--vendor false Include vendor directory
--include-node-modules false Include node_modules (hash detection)
--include-sqlc false Include sqlc-generated files (filtered by default)
--include-templ false Include *_templ.go generated files (filtered by default)
--include-protobuf false Include .pb.go files (filtered by default)
--include-mockgen false Include mockgen-generated files (filtered by default)
--include-stringer false Include stringer-generated files (filtered by default)
--include-pattern [] Glob patterns to include (overrides filters)
--exclude-pattern [] Glob patterns to exclude
--only "" Restrict to go or templ file type
--files -f Read file paths from stdin
Performance Flags
Flag Default Description
--workers 0 Concurrent workers (0 = auto-detect CPU cores)
--incremental false Enable AST caching for incremental analysis
--since "" Git ref for incremental mode (e.g., HEAD~1, main)
--cache-dir "" Cache directory (default: .cache/art-dupl)
--clear-cache false Clear cache before running
Stats Subcommand
art-dupl stats [flags] [paths...]

Supports all shared flags plus:

Flag Default Description
--format text Output format: text, json, csv

Statistics include files scanned, clone groups, duplicate lines/tokens, complexity score, impact score, clone size distribution, health grade (A–F), and top files by duplication.

Configuration

Create dupl.json:

{
  "threshold": 30,
  "outputFormat": "json",
  "paths": ["./src", "./lib"]
}
art-dupl -c dupl.json

CLI flags override config file values. Config file overrides defaults.

Examples

# CI/CD: fail if too many clones
TOTAL=$(art-dupl --json . | jq '.summary.total_clones')
[ "$TOTAL" -gt 100 ] && echo "Too many clones: $TOTAL" && exit 1

# Find test file duplicates
find . -name '*_test.go' | art-dupl -files

# Progressive refactoring: baseline → refactor → measure
art-dupl --json -t 15 > baseline.json
# ... refactor ...
art-dupl --json -t 15 > after.json

# Most widespread clones first
art-dupl --plumbing --sort occurrence ./src

# Filter all generated code (default behavior)
art-dupl ./src

# Override: include sqlc-generated files
art-dupl --include-sqlc ./src

Architecture

Source Files (.go, .templ)
    ↓
Parsing (go/parser + templ parser)
    ↓
AST Serialization → unified syntax.Node[]
    ↓
Suffix Tree (Ukkonen's algorithm, O(1) transitions)
    ↓
Duplicate Search → syntax.Match
    ↓
Clone Classification → domain.ProcessedCloneGroup
    ↓
Output Formatting (text, HTML, JSON, SARIF, plumbing)
Key Packages
Package Purpose
suffixtree/ Ukkonen's suffix tree with O(1) map-based transitions
syntax/ AST handling, Go and Templ parsers, serialization
detection/ Multi-method detection coordination via goroutines
hash/ XXH3 rolling hash-based detection
job/ File parsing pipeline with parallel workers
printer/ 7 output formats with sorting and classification
domain/ Domain types: Filepath, LineNumber, CloneSeverity, ProcessedClone
config/ Multi-source configuration with typed enums
errors/ 11 typed error categories with stack traces
pkg/artdupl/ Public SDK with Detector interface for programmatic use

Development

just build          # Build binary
just test           # Run all tests
just check          # Run linting
just ci             # Format + lint + test
just coverage       # Generate coverage report
just bench          # Run benchmarks
just test-race      # Run with race detector
just check-coverage # Verify ≥80% coverage

Tests use standard testing + Ginkgo/Gomega BDD. See CONTRIBUTING.md for guidelines.

License

MIT — Based on mibk/dupl (via golangci/dupl).

Directories

Path Synopsis
Package bdd provides ...
Package bdd provides ...
Package cache provides file-based caching for parsed AST nodes.
Package cache provides file-based caching for parsed AST nodes.
cmd
Package cmd provides ...
Package cmd provides ...
art-dupl command
Package config provides ...
Package config provides ...
Package detection provides multi-method code duplication detection.
Package detection provides multi-method code duplication detection.
Package domain provides core domain types and logic for art-dupl.
Package domain provides core domain types and logic for art-dupl.
Package errors provides type-safe error handling for dupl
Package errors provides type-safe error handling for dupl
Package hash provides ...
Package hash provides ...
internal
filtertest
Package filtertest provides ...
Package filtertest provides ...
testhelpers
Package testhelpers provides ...
Package testhelpers provides ...
testutil
Package testutil provides ...
Package testutil provides ...
utils
Package utils provides utility functions for context management and file processing.
Package utils provides utility functions for context management and file processing.
Package job provides ...
Package job provides ...
pkg
artdupl
Package artdupl provides a Go SDK for detecting code clones via suffix tree and hash-based detection on Go ASTs.
Package artdupl provides a Go SDK for detecting code clones via suffix tree and hash-based detection on Go ASTs.
format
Package format provides formatting utilities for art-dupl.
Package format provides formatting utilities for art-dupl.
logger
Package logger provides ...
Package logger provides ...
position
Package position provides utilities for working with source code positions.
Package position provides utilities for working with source code positions.
Package suffixtree implements suffix tree for clone detection.
Package suffixtree implements suffix tree for clone detection.
Package syntax provides unified AST representation for code duplication detection.
Package syntax provides unified AST representation for code duplication detection.
golang
Package golang provides ...
Package golang provides ...
templ
Package templ provides AST parsing for templ files using the official templ parser.
Package templ provides AST parsing for templ files using the official templ parser.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL