schemaexec

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 4, 2025 License: MIT Imports: 20 Imported by: 0

README

schemaexec — JSON Schema Symbolic Execution for jq

Production-ready symbolic executor for jq over OpenAPI/JSON Schema. Infer the output schema from an input schema and a jq program. KLEE-style execution with allocation-site abstraction. Used in production as part of Speakeasy's Transformation Extensions.

Install

go get github.com/speakeasy-api/jq

Quick start (symbolic)

import (
  "context"
  "github.com/speakeasy-api/jq"
  "github.com/speakeasy-api/jq/schemaexec"
)

input := schemaexec.BuildObject(map[string]*oas3.Schema{
  "items": schemaexec.ArrayType(schemaexec.ObjectType()),
}, []string{"items"})
q, _ := jq.Parse(".items | map({id, total: (.price * .qty)})")
out, _ := schemaexec.RunSchema(context.Background(), q, input)
// out.Schema is the inferred output schema

Quick examples

  • Map: .items | map({id, total: (.price * .qty)}) → array<{id: string, total: number}>
  • Select with narrowing: .items | map(select(.qty > 0) | {id}) → array<{id: string}>
  • Conditionals to enum: if .score >= 90 then "gold" else "silver" → enum["gold","silver"]

Symbolic capabilities

  • Property access and navigation: .foo, .[], .[0]
  • Object build/merge: {a: .x}, {a} + {b: 1}
  • Array ops: map, select, any, all, reduce, foreach
  • Conditionals and error handling: if/then/else, try/catch, alternative //
  • Arithmetic and comparisons with type propagation
  • Variables and closures: as $x | ...
  • Schema inference: required tracking, anyOf unions, enum synthesis
  • OpenAPI aware: designed for x-speakeasy-transform-from-api workflows

Why this exists

Use jq to describe JSON transformations while preserving type information. schemaexec symbolically evaluates jq to infer precise output schemas, keeping downstream codegen and tooling type-safe.

Architecture (brief)

  • KLEE-style multi-state execution with path joining
  • Allocation-site abstraction for arrays/objects; lattice-based union
  • Termination guards and memoization for fast, safe execution

Testing

Run: go test ./schemaexec/...

Thanks

  • Cristian Cadar / KLEE authors. Thanks for the research paving the golden path towards symbolic executors and teaching it to your undergrad students!
  • itchyny. Thanks for the base project!

Documentation

Overview

Package schemaexec provides symbolic execution of jq queries over JSON schemas.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ArrayType 

func ArrayType(items *oas3.Schema) *oas3.Schema

ArrayType creates a basic array schema with the given items schema.

func BoolType 

func BoolType() *oas3.Schema

BoolType creates a basic boolean schema.

func Bottom 

func Bottom() *oas3.Schema

Bottom returns a schema that matches nothing (the "never" type). By convention, we use nil to represent Bottom/Never.

func BuildArray 

func BuildArray(items *oas3.Schema, elements []*oas3.Schema) *oas3.Schema

BuildArray creates an array schema from element schemas.

func BuildObject 

func BuildObject(props map[string]*oas3.Schema, required []string) *oas3.Schema

BuildObject creates an object schema from property map. Simplified version for Phase 1.

func ConstBool 

func ConstBool(b bool) *oas3.Schema

ConstBool creates a schema for true or false.

func ConstInteger 

func ConstInteger(n int64) *oas3.Schema

ConstInteger creates a schema for a specific integer.

func ConstNull 

func ConstNull() *oas3.Schema

ConstNull creates a null schema.

func ConstNumber 

func ConstNumber(n float64) *oas3.Schema

ConstNumber creates a schema for a specific number.

func ConstString 

func ConstString(s string) *oas3.Schema

ConstString creates a schema for a specific string literal.

func GetProperty 

func GetProperty(obj *oas3.Schema, key string, opts SchemaExecOptions) *oas3.Schema

GetProperty extracts the schema for a property from an object schema. Simplified version for Phase 1.

func HasProperty 

func HasProperty(obj *oas3.Schema, key string, opts SchemaExecOptions) *oas3.Schema

HasProperty refines an object schema to require a property exists. Used for guards like: select(has("foo"))

func IntegerType 

func IntegerType() *oas3.Schema

IntegerType creates a basic integer schema (unconstrained).

func Intersect 

func Intersect(a, b *oas3.Schema, opts SchemaExecOptions) *oas3.Schema

Intersect creates a schema that matches all input schemas (allOf). This is used for narrowing and constraint combination.

func MergeObjects 

func MergeObjects(a, b *oas3.Schema, opts SchemaExecOptions) *oas3.Schema

MergeObjects combines two object schemas (for the + operator on objects).

func MightBeArray 

func MightBeArray(s *oas3.Schema) bool

MightBeArray checks if schema could be an array.

func MightBeNumber 

func MightBeNumber(s *oas3.Schema) bool

MightBeNumber checks if schema could be a number.

func MightBeObject 

func MightBeObject(s *oas3.Schema) bool

MightBeObject checks if schema could be an object.

func MightBeString 

func MightBeString(s *oas3.Schema) bool

MightBeString checks if schema could be a string.

func NullType 

func NullType() *oas3.Schema

NullType creates a null schema.

func NumberType 

func NumberType() *oas3.Schema

NumberType creates a basic number schema (unconstrained).

func ObjectType 

func ObjectType() *oas3.Schema

ObjectType creates a basic object schema (unconstrained).

func RequireType 

func RequireType(s *oas3.Schema, typ oas3.SchemaType, opts SchemaExecOptions) *oas3.Schema

RequireType narrows a schema to a specific type. Used for type guards like: select(type == "array")

func StringType 

func StringType() *oas3.Schema

StringType creates a basic string schema (unconstrained).

func TestExecuteShorthand 

func TestExecuteShorthand(t *testing.T)

func Top 

func Top() *oas3.Schema

Top returns a schema that matches any value (union of all types).

func Union 

func Union(schemas []*oas3.Schema, opts SchemaExecOptions) *oas3.Schema

Union creates a schema that matches any of the input schemas (anyOf). Implements proper flattening, deduplication, and widening when limits exceeded.

Types

type AValue 

type AValue struct {
	Kind    ValueKind
	Schema  *oas3.Schema
	Closure *Closure
}

AValue is the abstract value stored on the symbolic VM stack. Phase 1a: only VSchema is used by the existing code. VClosure will be used in Phase 1b+.

func NewClosureValue 

func NewClosureValue(pc, scopeIdx int) AValue

func NewSchemaValue 

func NewSchemaValue(s *oas3.Schema) AValue

NewSchemaValue constructs an AValue containing a Schema.

func (AValue) AsClosure 

func (v AValue) AsClosure() (*Closure, bool)

func (AValue) AsSchema 

func (v AValue) AsSchema() (*oas3.Schema, bool)

type Closure 

type Closure struct {
	PC         int
	ScopeIndex int
}

Closure abstracts a function value captured by pushpc. PC is the entry address, ScopeIndex is the captured lexical scope index.

type LogLevel 

type LogLevel int

LogLevel represents the severity level for logs. 

const (
	LevelError LogLevel = iota
	LevelWarn
	LevelInfo
	LevelDebug
)

func ParseLogLevel 

func ParseLogLevel(s string) LogLevel

ParseLogLevel parses a string into a LogLevel.

func (LogLevel) String 

func (l LogLevel) String() string

type Logger 

type Logger interface {
	// Debugf, Infof, Warnf, Errorf log formatted messages at respective levels.
	Debugf(format string, args ...any)
	Infof(format string, args ...any)
	Warnf(format string, args ...any)
	Errorf(format string, args ...any)

	// With returns a child logger augmented with the provided fields.
	With(fields map[string]any) Logger
}

Logger is the interface used by the executor for logging.

func NewLogger 

func NewLogger(level LogLevel, w io.Writer) Logger

NewLogger creates a default logger with the given level. If w is nil, os.Stderr is used.

type PathSegment 

type PathSegment struct {
	Key        interface{} // string (property), int (array index), or PathWildcard (symbolic)
	IsSymbolic bool        // True if this is a wildcard from .[] iteration
}

PathSegment represents one segment of a path expression

type PathWildcard 

type PathWildcard struct{}

PathWildcard represents a symbolic array index (from .[])

type SValue 

type SValue struct {
	Schema *oas3.Schema
}

SValue wraps a schema for the schema VM stack. This is the value type that flows through the schema virtual machine.

type SchemaExecOptions 

type SchemaExecOptions struct {
	// Limits to prevent combinatorial explosion
	AnyOfLimit int // Max branches in anyOf before widening (default: 10)
	EnumLimit  int // Max enum values before widening to plain type (default: 50)
	MaxDepth   int // Max recursion depth (default: 100)

	// Behavior flags
	StrictMode     bool // If true, fail on unsupported ops; if false, widen to Top (default: false)
	EnableWarnings bool // If true, collect precision-loss warnings (default: true)
	EnableMemo     bool // If true, enable memoization for performance (default: true)

	// Widening level controls how aggressively we simplify schemas
	// 0 = none (keep all precision)
	// 1 = conservative (keep types, drop facets when limits exceeded)
	// 2 = aggressive (collapse to Top when limits exceeded)
	WideningLevel int // default: 1

	// Logging configuration
	LogLevel             string // Log level: "error", "warn", "info", "debug" (default: "warn")
	LogMaxEnumValues     int    // Max enum values to show in logs (default: 5)
	LogMaxProps          int    // Max object properties to show in logs (default: 5)
	LogStackPreviewDepth int    // Max stack depth to preview in logs (default: 3)
	LogSchemaDeltas      bool   // If true, include schema deltas in debug logs (default: true)
}

SchemaExecOptions configures symbolic execution behavior.

func DefaultOptions 

func DefaultOptions() SchemaExecOptions

DefaultOptions returns the default configuration for schema execution.

type SchemaExecResult 

type SchemaExecResult struct {
	Schema   *oas3.Schema // The resulting schema after transformation
	Warnings []string     // Warnings about precision loss or unsupported operations
}

SchemaExecResult contains the output schema and diagnostic information.

func ExecSchema 

func ExecSchema(ctx context.Context, code *gojq.Code, input *oas3.Schema, opts SchemaExecOptions) (*SchemaExecResult, error)

ExecSchema executes compiled jq bytecode symbolically on an input schema. This is the core execution function - Phase 2 implementation.

func RunSchema 

func RunSchema(ctx context.Context, query *gojq.Query, input *oas3.Schema, opts ...SchemaExecOptions) (*SchemaExecResult, error)

RunSchema executes a jq query symbolically on an input JSON Schema. It parses and compiles the query, then performs symbolic execution to compute the output schema.

Example: 

query, _ := gojq.Parse(".foo.bar")
inputSchema := &oas3.Schema{...}
result, err := schemaexec.RunSchema(context.Background(), query, inputSchema)
if err != nil {
    log.Fatal(err)
}
fmt.Printf("Output schema: %+v\n", result.Schema)

func (*SchemaExecResult) String 

func (r *SchemaExecResult) String() string

String returns a string representation of the result for debugging.

type SchemaLogOptions 

type SchemaLogOptions struct {
	LogMaxEnumValues     int // default 5
	LogMaxProps          int // default 5
	LogMaxAnyOfBranches  int // default 5
	LogStackPreviewDepth int // default 3 (not used here, but kept for parity)
}

SchemaLogOptions control verbosity for schema summaries/deltas.

type ValueKind 

type ValueKind uint8

ValueKind classifies entries on the abstract stack. 

const (
	VSchema ValueKind = iota
	VClosure
)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.