README
¶
acurl
See your website the way AI agents see it.
acurl is a free, open-source tool by tragent.io.
acurl fetches a URL several times, once per agent profile: a real Chrome browser, AI agents
such as ChatGPT-User and Claude-Web, search crawlers, and plain curl. It then shows how each one was
treated side by side.
It is built for store owners and developers who want to know whether AI shopping and search agents can actually read their pages.
Install
go install github.com/tragent-io/acurl/cmd/acurl@latest
Usage
acurl https://example.com
acurl --agents ChatGPT-User,Googlebot https://yourstore.com/products/1
acurl --category ai-assistant,ai-search https://yourstore.com
acurl --diff GPTBot https://yourstore.com # what text did GPTBot not see?
acurl robots https://yourstore.com/products/1 # robots.txt, content signals, llms.txt
acurl profiles list
Each result is classified as allowed, blocked, challenged (WAF or captcha page),
rate-limited, cloaked (visible text differs a lot from what Chrome got),
robots-disallowed or error.
Every run ends with an AI readiness score (0-100) and concrete fixes for anything that fails.
More: profiles · robots.txt and llms.txt · scoring · CI
Output formats: table (default), json, markdown, csv, junit and sarif. Use -o file to write to a file.
CI
acurl batch urls.txt -f sarif -o acurl.sarif --fail-on blocked,challenged,score<80
Exit code 0 means nothing matched --fail-on, 1 means something did, and 2 means a request failed.
See the CI guide for GitHub Actions, JUnit, batch runs and .acurl.yaml.
Status
Early development. Expect breaking changes until v1.0.
License
Apache 2.0. See LICENSE.
About
acurl is built and maintained by tragent.io, which helps online stores get found, understood and bought from by AI shopping agents. acurl is the free, open-source diagnostic we use ourselves, and contributions are welcome.
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
acurl
command
Command acurl shows how AI agents see a URL compared to a real browser.
|
Command acurl shows how AI agents see a URL compared to a real browser. |
|
internal
|
|
|
batch
Package batch runs acurl checks over many URLs.
|
Package batch runs acurl checks over many URLs. |
|
classifier
Package classifier labels each fetch result with how the site treated it.
|
Package classifier labels each fetch result with how the site treated it. |
|
cli
Package cli implements the acurl command line interface.
|
Package cli implements the acurl command line interface. |
|
config
Package config applies settings from .acurl.yaml files and ACURL_* environment variables to command flags.
|
Package config applies settings from .acurl.yaml files and ACURL_* environment variables to command flags. |
|
detect
Package detect recognises bot-challenge and WAF block pages.
|
Package detect recognises bot-challenge and WAF block pages. |
|
extract
Package extract pulls schema.org product data out of HTML pages, so acurl can show whether an agent was given the price and availability a browser sees.
|
Package extract pulls schema.org product data out of HTML pages, so acurl can show whether an agent was given the price and availability a browser sees. |
|
fetcher
Package fetcher retrieves a URL once per agent profile.
|
Package fetcher retrieves a URL once per agent profile. |
|
llmstxt
Package llmstxt finds and validates /llms.txt files (https://llmstxt.org).
|
Package llmstxt finds and validates /llms.txt files (https://llmstxt.org). |
|
output
Package output renders acurl reports in human and machine formats.
|
Package output renders acurl reports in human and machine formats. |
|
profiles
Package profiles loads and filters the agent profiles acurl fetches with.
|
Package profiles loads and filters the agent profiles acurl fetches with. |
|
render
Package render estimates how much of a page depends on JavaScript, which most AI agents do not execute.
|
Package render estimates how much of a page depends on JavaScript, which most AI agents do not execute. |
|
robots
Package robots fetches robots.txt and answers RFC 9309 access questions.
|
Package robots fetches robots.txt and answers RFC 9309 access questions. |
|
score
Package score turns an acurl report into an AI-readiness score from 0 to 100, with the checks behind it.
|
Package score turns an acurl report into an AI-readiness score from 0 to 100, with the checks behind it. |
|
textdiff
Package textdiff compares what different agents were shown on a page.
|
Package textdiff compares what different agents were shown on a page. |
|
pkg
|
|
|
acurl
Package acurl is the Go API behind the acurl command: it fetches a URL once per agent profile and reports how each agent was treated.
|
Package acurl is the Go API behind the acurl command: it fetches a URL once per agent profile and reports how each agent was treated. |
|
types
Package types holds the data structures shared across acurl packages.
|
Package types holds the data structures shared across acurl packages. |