Documentation
¶
Overview ¶
Package extensions implements various helper addons for Colly
Index ¶
- func RandomMobileUserAgent(c *colly.Collector)
- func RandomUserAgent(c *colly.Collector)
- func Referer(c *colly.Collector)
- func URLLengthFilter(c *colly.Collector, URLLengthLimit int)
- type CDPDriver
- func (c *CDPDriver) Cache(request *http.Request, bodySize int, checkHeadersFunc colly.CheckHeadersFunc, ...) (*colly.Response, error)
- func (c *CDPDriver) CheckRedirect(f func(req *http.Request, via []*http.Request) error)
- func (c *CDPDriver) Cookies(url *url.URL) []*http.Cookie
- func (c *CDPDriver) Do(request *http.Request, bodySize int, checkHeadersFunc colly.CheckHeadersFunc) (*colly.Response, error)
- func (c *CDPDriver) GetJar() http.CookieJar
- func (c *CDPDriver) GetMatchingRule(domain string) *colly.LimitRule
- func (c *CDPDriver) GetTimeout() time.Duration
- func (c *CDPDriver) Init(jar http.CookieJar)
- func (c *CDPDriver) Jar(j http.CookieJar)
- func (c *CDPDriver) Limit(rule *colly.LimitRule) error
- func (c *CDPDriver) Limits(rules []*colly.LimitRule) error
- func (c *CDPDriver) Proxy(pf colly.ProxyFunc)
- func (c *CDPDriver) SetClient(client *http.Client)
- func (c *CDPDriver) SetCookies(url *url.URL, cookies []*http.Cookie) error
- func (c *CDPDriver) Timeout(t time.Duration)
- func (c *CDPDriver) Transport(t http.RoundTripper)
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func RandomMobileUserAgent ¶ added in v1.1.4
RandomMobileUserAgent generates a random MOBILE browser user-agent on every requests
func RandomUserAgent ¶
RandomUserAgent generates a random DESKTOP browser user-agent on every requests
func Referer ¶ added in v1.1.4
Referer sets valid Referer HTTP header to requests. Warning: this extension works only if you use Request.Visit from callbacks instead of Collector.Visit.
func URLLengthFilter ¶
URLLengthFilter filters out requests with URLs longer than URLLengthLimit
Types ¶
type CDPDriver ¶ added in v1.1.7
type CDPDriver struct {
// contains filtered or unexported fields
}
func ChromeDriver ¶ added in v1.1.7
func ChromeDriver() *CDPDriver
func (*CDPDriver) CheckRedirect ¶ added in v1.1.7
func (*CDPDriver) GetMatchingRule ¶ added in v1.1.7
func (*CDPDriver) GetTimeout ¶ added in v1.1.7
func (*CDPDriver) SetCookies ¶ added in v1.1.7
func (*CDPDriver) Transport ¶ added in v1.1.7
func (c *CDPDriver) Transport(t http.RoundTripper)
Source Files
Click to show internal directories.
Click to hide internal directories.