tinyspider

package module

v0.0.0-...-929266b Latest Latest Go to latest Published: Apr 18, 2025 License: MIT Imports: 3 Imported by: 0

Details

TinySpider是一款轻量级的爬虫框架, 使用Go语言编写. 提供最小化的callback function, 简单易用. DOM操作使用goquery库.

该框架旨在轻量化, 不过度封装, 让使用者能够更灵活地使用. 适合小型爬虫项目, 或者作为爬虫框架的入门学习.

参考example目录下的例子, 简单易懂.

This section is empty.

This section is empty.

This section is empty.

type Spider struct {
	URL string
	// contains filtered or unexported fields
}

func NewSpider() *Spider

func (s *Spider) AddTask(task func(doc *goquery.Document))

func (s *Spider) AfterReq(cb func(res *http.Response) error)

func (s *Spider) BeforeReq(cb func(req *http.Request) error)

func (s *Spider) Get(url string) error

Path	Synopsis
example
extension