chaff

package module
v0.3.5 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 30, 2026 License: MIT Imports: 21 Imported by: 3

README

go-chaff

A Json Schema Faker for Go 🙈.

It will genreate random data that should validate against a given schema.

[CC0 by @Iroshi_]

Documentation

Documentation for the library functions can be found here.

Installation

go get github.com/ryanolee/go-chaff@1

Usage

package main

import (
	"encoding/json"
	"fmt"

	"github.com/ryanolee/go-chaff"
)

const schema = `{"type": "number"}`

func main() {
	generator, err := chaff.ParseSchemaStringWithDefaults(schema)
	if err != nil {
		panic(err)
	}
	
	fmt.Println(generator.Metadata.Errors)
	result := generator.GenerateWithDefaults()
	if err != nil {
		panic(err)
	}

	res, err := json.Marshal(result)
	if err != nil {
		panic(err)
	}
	fmt.Println(string(res))
}

CLI

There is also a cli tool available for this package that can be installed from the releases section.

CLI too for generating random JSON data matching given JSON schema
Usage: go-chaff [flags]
  -allow-insecure
        Allow fetching remote $ref documents over insecure HTTP connections.
  -allow-outside-cwd
        Allow fetching $ref documents from file system paths outside the current working directory.
  -allowed-hosts string
        Comma separated list of allowed hosts to fetch remote $ref documents from over HTTP(S). If empty http and https resolution will fail.
  -allowed-paths string
        Comma separated list of allowed file system paths to fetch $ref documents from.
  -bypass-cyclic-reference-check
        Bypass cyclic reference check when generating schemas with cyclic $ref references.
  -cutoff-generation-steps int
        Maximum number of generation steps to perform before aborting generation entirely and returning what was generated. (default 2000)
  -file string
        Specify a file path to read the JSON Schema from
  -format
        Format JSON output.
  -help
        Print out help.
  -maximum-generation-steps int
        Maximum number of generation steps to perform before reducing the effort put into the generation process to a bare minimum. (default 1000)
  -maximum-if-attempts int
        Maximum number of attempts to satisfy 'if' conditions when generating data. (default 100)
  -maximum-oneof-attempts int
        Maximum number of attempts to satisfy 'oneOf' conditions when generating data. (default 100)
  -maximum-reference-depth int
        Maximum depth of $ref references to resolve at once when generating data. (default 10)
  -output string
        Specify file path to write generated output to.
  -verbose
        Print out detailed error information.
  -version
        Print out cli version information.

You can also pipe into STDIN for the cli

echo '{"type": "string", "format": "ipv4"}' | go-chaff
"217.2.244.95"

Current support:

  • Strings: (Including pattern through regen and formats through go faker), minLength, maxLength
  • Number / Integer: multipleOf, min, max, exclusiveMin, exclusiveMax
  • Constant types: enum, const, null
  • References: $ref, $defs, definitions, $id
  • Object: properties, patternProperties, additionalProperties, minProperties, maxProperties, required
  • Array: items, minItems, maxItems, contains, minContains, maxContains, prefixItems, additionalItems, unevaluatedItems, uniqueItems (Limited support)
  • Combination types anyOf / oneOf / allOf
  • Support for if / then / else
  • Support for not combinator (excluding anyOf, oneOf / allOf and if/then/else)
  • Multi document resolution for $ref over http(s) or file schemes.

Credits / Dependencies

What is left to do?

  • Better test coverage (Property based and unit of various generators)
  • Handle many edge cases where this package might not generate schema compliant results
  • Overcome the limitations of the current oneOf, anyOf and allOf keywords implementation.

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func NewFileSystemDocumentFetcher added in v0.2.0

func NewFileSystemDocumentFetcher(config FileSystemFetchOptions) (documentFetcherInterface, error)

func NewHttpDocumentFetcher added in v0.2.0

func NewHttpDocumentFetcher(parserConfig HTTPFetchOptions) (documentFetcherInterface, error)

Types

type DocumentFetchOptions added in v0.2.0

type DocumentFetchOptions struct {
	// HTTP Fetch Options
	HTTPFetchOptions HTTPFetchOptions `json:"httpFetchOptions,omitempty" jsonschema:"title=HTTP Document Resolver Options"`

	// File System Fetch Options
	FileSystemFetchOptions FileSystemFetchOptions `json:"fileSystemFetchOptions,omitempty" jsonschema:"title=File System Document Resolver Options"`
}

Options for fetching external documents during parsing. Should be used with cautions Especially if schemas passed to the schema faker can be from untrusted sources.

type FileSystemFetchOptions added in v0.2.0

type FileSystemFetchOptions struct {
	// If go-chaff is allowed to access the file system to resolve schema references
	//    THIS SHOULD BE DISABLED UNLESS YOU REALLY REALLY NEED IT (does not work in contexts without filesystems such as in browser)
	//    Please if doing so, limit the AllowedPaths field as much as possible )
	Enabled bool `json:"enabled,omitempty" jsonschema:"title=Fetch documents From File System Enabled"`

	// Overrides allowOutsideCwd to specifically allow for access to a list of paths schemas might reference
	// relative paths will be resolved against the current working directory at the time the parser is initialized
	// Symlinks that resolve to outside of the allowed paths will still be blocked
	AllowedPaths []string `json:"allowedPaths,omitempty" jsonschema:"title=Allowed Paths"`

	// Failsafe to prevent directory traversal attacks
	AllowOutsideCwd bool `json:"allowOutsideCwd,omitempty" jsonschema:"title=Allow Access To Paths Outside Of Current Working Directory"`
}

Options for fetching external documents from the file system

type Generator

type Generator interface {
	fmt.Stringer
	Generate(*GeneratorOptions) interface{}
}

type GeneratorOptions

type GeneratorOptions struct {
	// The source of randomness to use for the given generation.
	// Please note that some parts of the generators use different sources of randomness.
	// ("regex" generation and "format" strings)
	Rand *rand.RandUtil `json:"-"`

	// The default minimum number value
	DefaultNumberMinimum int `json:"defaultNumberMinimum,omitempty" jsonschema:"title=Default Number Minimum"`

	// The default maximum number value
	DefaultNumberMaximum int `json:"defaultNumberMaximum,omitempty" jsonschema:"title=Default Number Maximum"`

	// The default minimum String length
	DefaultStringMinLength int `json:"defaultStringMinLength,omitempty" jsonschema:"title=Default String Minimum Length"`

	// The default maximum String length
	DefaultStringMaxLength int `json:"defaultStringMaxLength,omitempty" jsonschema:"title=Default String Maximum Length"`

	// The default minimum array length
	DefaultArrayMinItems int `json:"defaultArrayMinItems,omitempty" jsonschema:"title=Default Array Minimum Items"`

	// The default maximum array length
	// This will be set min + this inf the event a minimum value is set
	DefaultArrayMaxItems int `json:"defaultArrayMaxItems,omitempty" jsonschema:"title=Default Array Maximum Items"`

	// The default minimum object properties (Will be ignored if there are fewer properties available)
	DefaultObjectMinProperties int `json:"defaultObjectMinProperties,omitempty" jsonschema:"title=Default Object Minimum Properties"`

	// The default maximum object properties (Will be ignored if there are fewer properties available)
	DefaultObjectMaxProperties int `json:"defaultObjectMaxProperties,omitempty" jsonschema:"title=Default Object Maximum Properties"`

	// The maximum dnumber of references to resolve at once in terms of depth (Default: 10)
	MaximumReferenceDepth int `json:"maximumReferenceDepth,omitempty" jsonschema:"title=Maximum Reference Depth"`

	// In the event that schemas are recursive there is a good chance the generator
	// can run forever. This option will bypass the check for cyclic references
	// Please defer to the MaximumReferenceDepth option if possible when using this
	BypassCyclicReferenceCheck bool `json:"bypassCyclicReferenceCheck,omitempty" jsonschema:"title=Bypass Cyclic Reference Check"`

	// Used to keep track of references during a resolution cycle (Used internally and can be ignored)
	ReferenceResolver referenceResolver `json:"-"`

	// Though technically in some cases a schema may allow for additional
	// values it might not always be desireable. this option suppresses fallback_n values
	// so that they will only appear to make up a "minimum value" forces them to
	SuppressFallbackValues bool `json:"suppressFallbackValues,omitempty" jsonschema:"title=Suppress Fallback Values"`

	// The maximum number of times to attempt to generate a unique value
	// when using unique* constraints in schemas
	MaximumUniqueGeneratorAttempts int `json:"maximumUniqueGeneratorAttempts,omitempty" jsonschema:"title=Maximum Unique Generator Attempts"`

	// The maximum number of times to attempt to satisfy "if" statements
	// before giving up
	MaximumIfAttempts int `json:"maximumIfAttempts,omitempty" jsonschema:"title=Maximum If Attempts"`

	// The maximum number of times to attempt to satisfy "oneOf" statements
	// before giving up
	MaximumOneOfAttempts int `json:"maximumOneOfAttempts,omitempty" jsonschema:"title=Maximum OneOf Attempts"`

	// The maximum number of steps to take when generating a value
	// after which the the generator will begin to do the "bare minimum" to generate a value
	MaximumGenerationSteps int `json:"maximumGenerationSteps,omitempty" jsonschema:"title=Maximum Generation Steps"`

	// The maximum number of steps to take before giving up entirely and aborting generation
	// This is a hard cap on generation steps to prevent extremely long generation times
	CutoffGenerationSteps int `json:"cutoffGenerationSteps,omitempty" jsonschema:"title=Cutoff Generation Steps"`
	// contains filtered or unexported fields
}

func (*GeneratorOptions) ScaledRetryBudget added in v0.3.0

func (g *GeneratorOptions) ScaledRetryBudget(baseAttempts int) int

ScaledRetryBudget returns a reduced number of retry attempts proportional to how close the overall complexity is to the cutoff. When complexity is below the "minimize" threshold the full budget is returned. As complexity grows towards CutoffGenerationSteps the budget is linearly reduced down to 1. This prevents premature reduction on moderately complex schemas while still capping runaway retries on deeply cyclic ones.

func (*GeneratorOptions) ShouldCutoff added in v0.2.0

func (g *GeneratorOptions) ShouldCutoff() bool

func (*GeneratorOptions) ShouldMinimize added in v0.2.0

func (g *GeneratorOptions) ShouldMinimize() bool

type HTTPFetchOptions added in v0.2.0

type HTTPFetchOptions struct {
	// If go-chaff is allowed to make HTTP requests to resolve schema references
	Enabled bool `json:"enabled,omitempty" jsonschema:"title=Fetch documents Over HTTP(s) Enabled"`

	// Allowed hosts to fetch from (If empty, all hosts are allowed)
	AllowedHosts []string `json:"allowedHosts,omitempty" jsonschema:"title=Allowed Hosts"`

	// Allow insecure connections (http)
	AllowInsecure bool `json:"allowInsecure,omitempty" jsonschema:"title=Allow Insecure Connections"`
}

Options for fetching external documents over HTTP

type ParserOptions

type ParserOptions struct {
	// Options for the regex generator used for generating strings with the "pattern property"
	RegexStringOptions *regen.GeneratorArgs `json:"regexStringOptions,omitempty" jsonschema:"title=Regex String Options"`

	// Options for the regex generator used for pattern properties
	RegexPatternPropertyOptions *regen.GeneratorArgs `json:"regexPatternPropertyOptions,omitempty" jsonschema:"title=Regex Pattern Property Options"`

	// Parser fetch configurations for external document fetching
	DocumentFetchOptions DocumentFetchOptions `json:"documentFetchOptions,omitempty" jsonschema:"title=Document Fetch Options"`

	// Base path to resolve relative document references against for $ref resolution when fetching external documents
	RelativeTo string `json:"relativeTo,omitempty" jsonschema:"title=Relative To"`

	// Maximum recursion depth during parsing to prevent stack overflow from circular schemas.
	// If zero, defaults to 100.
	MaxParseDepth int `json:"maxParseDepth,omitempty" jsonschema:"title=Max Parse Depth"`
}

Options to take into account when parsing a json schema

type RootGenerator

type RootGenerator struct {
	Generator Generator
	// For any "$defs"
	Defs map[string]Generator
	// For any "definitions"
	Definitions map[string]Generator
	// Metadata related to parser operations
	Metadata *parserMetadata
}

Root generator a given schema. Call the Generate method on this to generate a value

func ParseSchema

func ParseSchema(schema []byte, opts *ParserOptions) (RootGenerator, error)

Parses a Json Schema byte array. If there is an error parsing the schema, an error will be returned.

func ParseSchemaFile

func ParseSchemaFile(path string, opts *ParserOptions) (RootGenerator, error)

Parses a Json Schema file at the given path. If there is an error reading the file or parsing the schema, an error will be returned

func ParseSchemaFileWithDefaults

func ParseSchemaFileWithDefaults(path string) (RootGenerator, error)

Parses a Json Schema file at the given path with default options. If there is an error reading the file or parsing the schema, an error will be returned

func ParseSchemaString

func ParseSchemaString(schema string, opts *ParserOptions) (RootGenerator, error)

Parses a Json Schema string. If there is an error parsing the schema, an error will be returned.

func ParseSchemaStringWithDefaults

func ParseSchemaStringWithDefaults(schema string) (RootGenerator, error)

func ParseSchemaWithDefaults

func ParseSchemaWithDefaults(schema []byte) (RootGenerator, error)

Parses a Json Schema byte array with default options. If there is an error parsing the schema, an error will be returned.

func (RootGenerator) Generate

func (g RootGenerator) Generate(opts *GeneratorOptions) interface{}

Generates values based on the passed options

func (RootGenerator) GenerateWithDefaults

func (g RootGenerator) GenerateWithDefaults() interface{}

func (RootGenerator) String

func (g RootGenerator) String() string

Directories

Path Synopsis
internal
jsonschema
Credit to https://github.com/santhosh-tekuri/jsonschema vendored from https://github.com/kaptinlin/jsonschema/blob/main/formats.go to keep size smaller
Credit to https://github.com/santhosh-tekuri/jsonschema vendored from https://github.com/kaptinlin/jsonschema/blob/main/formats.go to keep size smaller
regen
Package regen is a library for generating random strings from regular expressions.
Package regen is a library for generating random strings from regular expressions.
test_data

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL