dj

dj - Dynamic JS File Extractor

中文 | English

dj intelligently detects dynamically loaded JavaScript files by statically analyzing website HTML and JS code, including webpack chunks, import() lazy loading, and more.

Features

• Deep analysis of website HTML and JS to extract dynamically loaded JavaScript files
• Smart detection of dynamic loading patterns: import(), require(), webpack chunks, vite preload, etc.
• Support for multiple frontend framework chunk mappings: Next.js, Nuxt.js, Vite, SvelteKit, Webpack, and more
• Automatic Source Map discovery and original source code restoration (from sourcesContent, with mappings VLQ fallback)
• Cache reuse: second run on the same site restores results from local cache with zero network requests
• TLS fingerprint spoofing (uTLS Chrome fingerprint) to bypass Cloudflare and other WAFs
• HTTP/2 and HTTP/1.1 protocol auto-negotiation
• SOCKS5/HTTP/HTTPS proxy support with authentication
• Environment variable proxy configuration (HTTPS_PROXY, ALL_PROXY, NO_PROXY, etc.)
• Custom User-Agent and browser-like request headers
• Multiple output formats: text, JSON, markdown

Installation

go install (recommended)

go install github.com/ejfkdev/dj@latest

Build from source

git clone https://github.com/ejfkdev/dj.git
cd dj
go build -ldflags="-X main.version=1.0.0" -o dj .

Download prebuilt binaries

Visit the Releases page to download binaries for your platform.

Usage

dj [options] <URL>

Basic usage

# Extract JS URLs (real-time output)
dj https://example.com

# Output in JSON format
dj -f json https://example.com

# Output in Markdown format
dj -f md https://example.com

Command line options

Examples

# Custom User-Agent
dj --useragent="Mozilla/5.0 (iPhone; CPU iPhone OS 14_0 like Mac OS X) ..." https://example.com

# Use HTTP proxy
dj --proxy="http://127.0.0.1:7890" https://example.com

# Use SOCKS5 proxy
dj --proxy="socks5://127.0.0.1:1080" https://example.com

# Use HTTPS proxy
dj --proxy="https://proxy.example.com:443" https://example.com

# Proxy with authentication
dj --proxy="socks5://user:pass@127.0.0.1:1080" https://example.com

# Use environment variable proxy (HTTPS_PROXY, HTTP_PROXY, ALL_PROXY)
HTTPS_PROXY=http://127.0.0.1:7890 dj https://example.com

# Bypass proxy for specific hosts
ALL_PROXY=socks5://127.0.0.1:1080 NO_PROXY=localhost,example.com dj https://example.com

# Inject cookies for Cloudflare bypass
dj --cookie="cf_clearance=xxx" https://example.com

# Enable debug mode
dj --debug https://example.com

Tested websites

dj https://docs.qq.com
dj https://vue.ruoyi.vip
dj https://gitee.com
dj https://nuxt.com.cn
dj https://chat.z.ai
dj https://show.cool-admin.com/login
dj https://demo.1panel.cn

Supported patterns and frameworks (16 plugins)

How it works

1. Download the target webpage HTML
2. Launch plugin analysis - each URL is processed concurrently by a goroutine:
   • Download JS content
   • Detect Content-Type (skip static resources returning HTML)
   • Dispatch to all plugins for pattern matching
3. Plugins discover new JS URLs or path fragments, add to processing queue
4. Probe for Source Map files (via sourceMappingURL or HTTP header)
5. Restore original source code from Source Maps:
   • Priority: extract from sourcesContent field (complete original source)
   • Fallback: reconstruct from mappings (VLQ decoding) when sourcesContent is missing
   • Restored files preserve original directory structure under sources/
6. Collect all discovered JS URLs and output

Cache reuse

When caching is enabled (default), the second run on the same site skips network requests entirely:

• Loads previously discovered JS URLs from meta.json
• Restores source maps and source code from local cache
• Use --cache=false to force a full re-scan

Output formats

Text (default)

https://example.com/js/main.js
https://example.com/js/chunk-abc123.js
https://example.com/js/async-def456.js

JSON

{
  "summary": {
    "jsCount": 3,
    "sourceMapCount": 1,
    "sourceCount": 15
  },
  "jsURLs": [
    "https://example.com/js/main.js",
    "https://example.com/js/chunk-abc123.js"
  ],
  "cacheBase": "/tmp/ejfkdev/dj/example.com",
  "cacheDirs": {
    "js": "/tmp/ejfkdev/dj/example.com/js",
    "sourceMap": "/tmp/ejfkdev/dj/example.com/source_map",
    "source": "/tmp/ejfkdev/dj/example.com/sources",
    "html": "/tmp/ejfkdev/dj/example.com/html/web.html"
  }
}

Caching

Caching is enabled by default. Cache is stored in the system temp directory:

Cache structure:

<temp_dir>/ejfkdev/dj/<origin>/
├── js/                    # Downloaded JS files
├── source_map/            # Source Map files
├── sources/               # Restored original source code (preserves directory structure)
├── html/                  # Original HTML
└── meta.json             # Site metadata (JS URLs, source map paths, restored sources, cache dirs)

FAQ

Why aren't some dynamically loaded JS files being extracted?

This tool uses static analysis of JS code to detect dynamic loading patterns. If a website uses special loading methods, they may not be covered. If you find a website whose dynamic JS cannot be extracted, feel free to submit an Issue with the site URL and any relevant code clues.

License

MPL 2.0 (Mozilla Public License)