Documentation
¶
Overview ¶
Package urlscan is the library behind the urlscan command line: the HTTP client, request shaping, and the typed data models for urlscan.io.
The Client here is the spine every command shares. It sets a real User-Agent, paces requests so a busy session stays polite, and retries the transient failures (429 and 5xx) that any public site throws under load. Build your endpoint calls and JSON decoding on top of it.
Index ¶
Constants ¶
const BaseURL = "https://" + Host
BaseURL is the root every request is built from.
const DefaultUserAgent = "urlscan/dev (+https://github.com/tamnd/urlscan-cli)"
DefaultUserAgent identifies the client to urlscan.io. A real, honest User-Agent is both polite and the thing most likely to keep you unblocked.
const Host = "urlscan.io"
Host is the site this client talks to.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Client ¶
type Client struct {
HTTP *http.Client
UserAgent string
// Rate is the minimum gap between requests. Zero means no pacing.
Rate time.Duration
Retries int
// contains filtered or unexported fields
}
Client talks to urlscan.io over HTTP.
func NewClient ¶
func NewClient() *Client
NewClient returns a Client with sensible defaults: a 30s timeout, a 1s minimum gap between requests, and five retries on transient errors.
func (*Client) Get ¶
Get fetches url and returns the response body. It paces and retries according to the client's settings. The caller owns nothing extra; the body is read fully and closed here.
type Domain ¶
type Domain struct{}
Domain is the urlscan.io driver. It carries no state; the per-run client is built by the factory Register hands kit.
func (Domain) Classify ¶
Classify turns any accepted input into the canonical (type, id). A 36-char UUID-like string (with hyphens) maps to "uuid"; a URL maps to "url"; everything else is treated as a search query.
func (Domain) Info ¶
func (Domain) Info() kit.DomainInfo
Info describes the scheme, the hostnames a pasted link is matched against, and the identity reused for the binary's help and version.
type ScanDetail ¶
type ScanDetail struct {
UUID string `kit:"id" json:"uuid"`
URL string `json:"url"`
Domain string `json:"domain"`
Title string `json:"title"`
IP string `json:"ip"`
Country string `json:"country"`
City string `json:"city"`
Server string `json:"server"`
ASN string `json:"asn"`
ASNName string `json:"asn_name"`
IPs []string `json:"ips"`
Domains []string `json:"domains"`
Requests int `json:"requests"`
}
ScanDetail is the full detail for a single scan result.
type ScanResult ¶
type ScanResult struct {
UUID string `kit:"id" json:"uuid"`
URL string `json:"url"`
Domain string `json:"domain"`
Time string `json:"time"`
Method string `json:"method"`
Title string `json:"title"`
IP string `json:"ip"`
Country string `json:"country"`
Server string `json:"server"`
}
ScanResult is a single entry from a search response.
Source Files