Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Headers ¶ added in v1.0.0
func (*Headers) MarshalJSON ¶ added in v1.0.0
type Request ¶
type Request struct {
Method string `json:"method,omitempty"`
URL string `json:"endpoint,omitempty"`
Body string `json:"body,omitempty"`
Depth int `json:"-"`
SkipValidation bool `json:"-"`
Headers map[string]string `json:"headers,omitempty"`
Tag string `json:"tag,omitempty"`
Attribute string `json:"attribute,omitempty"`
RootHostname string `json:"-"`
Source string `json:"source,omitempty"`
CustomFields map[string][]string `json:"custom_fields,omitempty"`
Raw string `json:"raw,omitempty"`
}
Request is a navigation request for the crawler
func NewNavigationRequestURLFromResponse ¶
func NewNavigationRequestURLFromResponse(path, source, tag, attribute string, resp *Response) *Request
newNavigationRequestURL generates a navigation request from a relative URL
func (*Request) RequestURL ¶
RequestURL returns the request URL for the navigation
type Response ¶
type Response struct {
Resp *http.Response `json:"-"`
Depth int `json:"-"`
Reader *goquery.Document `json:"-"`
StatusCode int `json:"status_code,omitempty"`
Headers Headers `json:"headers,omitempty"`
Body string `json:"body,omitempty"`
ContentLength int64 `json:"content_length,omitempty"`
RootHostname string `json:"-"`
Technologies []string `json:"technologies,omitempty"`
Raw string `json:"raw,omitempty"`
Forms []Form `json:"forms,omitempty"`
XhrRequests []Request `json:"xhr_requests,omitempty"`
StoredResponsePath string `json:"stored_response_path,omitempty"`
}
Response is a response generated from crawler navigation
func (Response) AbsoluteURL ¶
func (Response) IsRedirect ¶ added in v1.0.5
Click to show internal directories.
Click to hide internal directories.