robotscheck

package
v0.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 20, 2026 License: MIT Imports: 4 Imported by: 0

Documentation

Overview

Package robotscheck reports on robots.txt: presence, declared sitemaps, and crawled URLs that violate disallow rules.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Analyzer 

type Analyzer struct{}

Analyzer inspects the robots.txt collected during the crawl.

func New 

func New() *Analyzer

New returns a robots.txt analyzer.

func (Analyzer) Analyze 

func (a Analyzer) Analyze(_ context.Context, result *crawler.Result) []analyze.Issue

func (Analyzer) Description 

func (Analyzer) Description() string

func (Analyzer) Name 

func (Analyzer) Name() string

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.