scraper

package
v0.0.0-...-eccf244 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 3, 2017 License: Apache-2.0 Imports: 9 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func JSONMarshalUnescaped 

func JSONMarshalUnescaped(v interface{}) ([]byte, error)

JSONMarshalUnescaped does what json.Marshal does without escaping &, <, >.

Types

type Config 

type Config struct {
	URLs    []string  `json:"urls"`
	Targets []*target `json:"targets"`
	Output  *output   `json:"output"`
	// contains filtered or unexported fields
}

Config holds information from config file.

type Scraper 

type Scraper struct {
	Config  *Config
	Results results
}

Scraper struct used for scraping activity.

func New 

func New(configPath string) (*Scraper, error)

New creates new Scraper and returns pointer to it, with error (if occurred).

func (*Scraper) Start 

func (s *Scraper) Start()

Start will start scraping in separate goroutine and save results when it is done in file that is defined in Config.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.