README
¶
formats
Formats implements support for passing a number of request source as input providers to nuclei to be tested for fuzzing related issues.
Currently the following formats are implemented -
- Burp Suite XML Request/Response file
- Proxify JSONL output file
- OpenAPI Specification file
- Postman Collection file
- Swagger Specification file
Each implementation implements either the entire or a subset of the features of the specifications. These can be increased further to add support as new things or requirements are identified.
Refer to the specific code for each implementation to understand supported features of the specs.
OpenAPI Specification File
It is designed to generate HTTP requests based on an OpenAPI 3.0 Schema. Here is how these schema components are processed:
Servers
The module supports multiple server URLs defined in the Servers section of the OpenAPI document. It will send requests to all the server URLs defined in the schema.
Paths and Operations
The module supports all HTTP methods defined under each path in the Paths section. For each operation on a path, HTTP requests are generated and sent to the defined server URL. If the operation cannot generate a valid request, a warning will be logged.
Parameters
The module recognizes parameters defined in the query, header, path, and cookie categories. When generating requests, if the requiredOnly flag is true, only the required parameters are included. Otherwise, all parameters, regardless of their required status, are used.
The generateExampleFromSchema function is used to generate suitable example data for each parameter from their respective schema definitions.
RequestBody
The module also comprehends request bodies and supports various media types defined in the Content field. Currently, the following content-types are supported:
-
application/json: The module creates application-specific JSON from the defined example schema. -
application/xml: The example schema is converted into xml format usingmxjlibrary. -
application/x-www-form-urlencoded: The example schema is converted into URL-encoded form data. -
multipart/form-data: The module supports multipart form-data and differentiates between fields and files using thebinaryformat under the property schema. -
text/plain: Converts the example schema into string format and send as plain text.
For unsupported media types, no appropriate content type is found for the body. After setting up the body of the request, the module dumps the request and sends it to the defined server URL.
Example Request Generation
This module converts each operation into one or more example HTTP requests. Each request is dumped into a string format, accompanied by its method, URL, headers, and body. These are send as a callback for further processing.
Please note: This document does not cover other features of OpenAPI specification like responses, security schemes, links, callbacks, etc. as these are not currently handled by the module.
Postman Collection file
This module parser Postman Collection JSON files.
1. Request Parsing:
Able to parse requests detailed in the Postman package. The parser is capable of interpreting the HTTP method, URL, and Body of each request present in the collection.
2. Header Parsing:
All HTTP headers set in the collection's request are parsed and set in the request.
3. Auth Type Parsing:
Able to parse and set the Authentication options provided in the postman collection in the request headers.
Supported types of authentication:
- API Key: In header
- Basic: Setting basic auth through username, password.
- Bearer Token: Involves setting bearer auth using tokens.
- No Auth: No authentication is set.
Note: Not all parts of the Postman Collection specification are supported. This parser does not currently support Postman variables or collection level variables and items. It also does not support more authentication types than detailed above.
Limitations:
- Does not support Postman variables
- Does not support Collection level variables and items
- Limited Authentication types supported
Swagger Specification file
Swagger specification file is converted from OpenAPI 2.0 format to OpenAPI 3.0 format. After this, the OpenAPI parser from above is used.
Burp XML / Proxify JSONL
These modules are generic and parse raw requests from these respective tools.
Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ( DefaultVarDumpFileName = "required_openapi_params.yaml" ErrNoVarsDumpFile = errors.New("no required params file found") )
Functions ¶
func WriteOpenAPIVarDumpFile ¶
func WriteOpenAPIVarDumpFile(vars *OpenAPIParamsCfgFile) error
WriteOpenAPIVarDumpFile writes the required vars dump file
Types ¶
type Format ¶
type Format interface {
// Name returns the name of the format
Name() string
// Parse parses the input and calls the provided callback
// function for each RawRequest it discovers.
Parse(input io.Reader, resultsCb ParseReqRespCallback, filePath string) error
// SetOptions sets the options for the input format
SetOptions(options InputFormatOptions)
}
Format is an interface implemented by all input formats
type InputFormatOptions ¶
type InputFormatOptions struct {
// Variables is list of variables that can be used
// while generating requests in given format
Variables map[string]interface{}
// SkipFormatValidation is used to skip format validation
// while debugging or testing if format is invalid then
// requests are skipped instead of creating invalid requests
SkipFormatValidation bool
// RequiredOnly only uses required fields when generating requests
// instead of all fields
RequiredOnly bool
// VarsTextTemplating uses Variables and inject it into the input
// this is used for text templating of variables based on carvel ytt
// Only available for Yaml formats
VarsTextTemplating bool
// VarsFilePaths is the path to the file containing variables
VarsFilePaths []string
}
InputFormatOptions contains options for the input this can be variables that can be passed or overrides or some other options
type OpenAPIParamsCfgFile ¶
type OpenAPIParamsCfgFile struct {
Var []string `yaml:"var"`
OptionalVars []string `yaml:"-"` // this will be written to the file as comments
}
OpenAPIParamsCfgFile is the structure of the required vars dump file
func ReadOpenAPIVarDumpFile ¶
func ReadOpenAPIVarDumpFile() (*OpenAPIParamsCfgFile, error)
ReadOpenAPIVarDumpFile reads the required vars dump file
type ParseReqRespCallback ¶
type ParseReqRespCallback func(rr *types.RequestResponse) bool
ParseReqRespCallback is a callback function for discovered raw requests
type SpecDownloader ¶ added in v3.6.0
type SpecDownloader interface {
// Download downloads the spec from the given URL and saves it to tmpDir
// Returns the path to the downloaded file
// httpClient is a retryablehttp.Client instance (can be nil for fallback)
Download(url, tmpDir string, httpClient *retryablehttp.Client) (string, error)
// SupportedExtensions returns the list of supported file extensions
SupportedExtensions() []string
}
SpecDownloader is an interface for downloading API specifications from URLs
Source Files
Directories