Documentation
¶
Overview ¶
Package batch runs acurl checks over many URLs.
Index ¶
- Constants
- func Dedupe(urls []string) []string
- func ReadURLs(r io.Reader, csvInput bool) ([]string, error)
- func Run(ctx context.Context, c Checker, urls []string, opts Options) *types.BatchReport
- func SitemapURLs(ctx context.Context, get Getter, root string, limit int) ([]string, error)
- func Summarize(items []types.BatchItem) *types.BatchReport
- type Checker
- type Getter
- type Options
Constants ¶
View Source
const MaxSitemapSize = 50 << 20
MaxSitemapSize is the largest sitemap file read (the protocol limit is 50 MB uncompressed).
Variables ¶
This section is empty.
Functions ¶
func ReadURLs ¶
ReadURLs reads one URL per line, skipping blank lines and # comments. When csvInput is true the input is CSV: the "url" column is used if the header has one, otherwise the first column.
func SitemapURLs ¶
SitemapURLs returns up to limit page URLs from the sitemap at root, following sitemap indexes up to two levels deep. Gzipped sitemaps are supported. A limit <= 0 means no limit.
Types ¶
type Options ¶
type Options struct {
// Parallel is how many URLs are checked at once. Default 2.
Parallel int
// Delay is the minimum time between starting checks on the same host.
Delay time.Duration
// Progress, if set, is called after each URL completes.
Progress func(done, total int, url string)
}
Options configures a batch run.
Source Files
Click to show internal directories.
Click to hide internal directories.