gentest

gentesting

Tools to test generated code.

This uses the go plugin feature and is therefore not portable to Windows.

Goals

This tool is used to run tests against generated go code.

It allows to verify the behavior of that code, not the code itself.

Code generated by the go-openapi tools should meet the following requirements:

1. It builds
2. It passes go vet analysis
3. It passes common linting rules
4. It exports the desired types and functions
5. The exposed functionality works as expected (e.g. marshaling, validation, etc)

Notice that this tool carries out black box testing capabilities: it is unable to introspect the private details of a package.

How it works

When testing a new generated package the following things happen when calling Builder.Build():

1. Parse the source package and extract exported identifiers
2. Generate a bootstrap main.go to host a go plugin.

The bootstrap code imports the target package and re-export all public symbols from the package:

• constants are exported as variables initialized with the constant value
• variables are exported as variables - if some initialization has been performed in the package, the re-exported symbols should be shallow clones
• types are exported as variables that instantiate this type
• functions are exported as variable of the function type.

1. Build a shared library on top of the plugin bootstrap using go -buildmode=plugin
2. Expose all known exported symbols in an index, by category (constants, variables, types, functions)
3. Expose a driver to run test

Example

func TestBuilder(t *testing.T) {
	const testPackage = "generated"
	testPackagePath := filepath.Join("fixtures", testPackage)

	g := New(testPackagePath)
	t.Cleanup(g.Cleanup)

	require.NoError(t, g.Build())

	require.Equal(t, testPackage, g.Name())
	require.NotEmpty(t, g.Symbols(SymbolConst))
	require.NotEmpty(t, g.Symbols(SymbolVar))
	require.NotEmpty(t, g.Symbols(SymbolType))
	require.NotEmpty(t, g.Symbols(SymbolFunc))
	require.NotEmpty(t, g.Plugin())
	require.NotEmpty(t, g.Driver())
}

Test driver helper

When the build is done, Builder.Driver() yields a test driver.

With this type, you may iterate over the exported symbols by kind, or look up for a given symbol.

The return Entry type is essentially a wrapper around reflect.Value and reflect.Type of the symbol pointed to by the plugin. It provides methods to assert the properties of this symbol (is this a struct, a slice, etc) like you would do with reflect.

Example:

import (
    "testing"

    "github.com/fredbi/core/codegen/gentesting"
	"github.com/stretchr/testify/assert"
	"github.com/stretchr/testify/require"
)

func TestGenerated(t *testing.T) {
    builder := gentesting.New("./generated")
    require.NoError(t, builder.Build())

    driver := builder.Driver()

    model, modelIsExported := driver.Type("Model")
    require.True(t, modelIsExported)

    require.True(t, model.IsStruct())
    require.True(t, model.HasField("A"))

    ...
}

Test asserter

The Asserter is a helper to write nice-looking test code when using the test Driver.

The Asserter is constructed like so: << TODO >>

It wraps assertions like:

    ...

    t.Run(`package "generated" should export type "Model"`, func(t *testing.T) {
        model, modelIsExported := driver.Type("Model")
        require.True(t, modelIsExported)

        t.Run(`type "Model" should be a struct`, func(t *testing.T) {
            require.True(t, model.IsStruct())

            t.Run(`struct "Model" should contain field "A"`, func(t *testing.T) {
                require.True(t, model.HasField("A"))

                t.Run(`field "A" of struct "Model" should be int64`, func(t *testing.T) {
                    modelA, _ := model.Field("A")
                    assert.True(t, modelA.IsOfType("int64")
                })
            })
        })
    })

into:

func TestGenerated(t *testing.T) {
    builder := gentesting.New("./generated")
    require.NoError(t, builder.Build())
    tp := gentesting.AsserterFor(t, builder)

    model := tp.MustExportType("Model") // required
    model.MustBeStruct()
    modelA := model.MustHaveField("A")
    modelA.ShouldBeOfType("int64")
/bin/bash: line 1: :b1: command not found

Testing methods

The Driver allows to call package functions and type methods.

Example with a function:

    ...

    driver := builder.Driver()
    model, modelIsExported := driver.MakeWithType("Model")
    require.True(t, modelIsExported)

    modelConstructor, hasConstructor := driver.Function("NewModel")
    require.True(t, hasConstructor)

    require.True(t, modelConstructor.HasNoParameter())
    require.True(t, modelConstructor.Returns(model.Pointer())

    t.Run(`"NewModel" should initialize with defaults`, func(t *testing.T) {
        result, err := driver.CallFunction(modelConstructor)
        require.NoError(t, err)

        require.NotNil(t, result.Type)
        newModel := result.IsSameTypeAs(model.Pointer())

        assert.Equal(t, int64(1), int64(result.Field("A").Value()))
        assert.Equal(t, string("test"), int64(result.Field("B").Value()))
    })

Example with a method:

    ...

    driver := builder.Driver()
    model, modelIsExported := driver.MakeWithType("Model")

    // we need a pointer receiver
    _, hasUnmarshalJSON := model.Pointer().Method("UnmarshalJSON")
    require.True(t, hasUnmarshalJSON)

    var ( 
        data []byte
        err error
    )
    require.True(t, modelConstructor.HasParametersOfType(data))
    require.True(t, modelConstructor.ReturnsValueOfType(err)

	testJSON := []byte(`{"a": 4, "b": "fred"}`)
    result, err := model.CallMethod("UnmarshalJSON", testJSON)
    require.NoError(t, err)
    require.Nil(t, result.Type)

    // verify the unmarshaled values
    assert.Equal(t, int64(4), int64(model.Field("A").Value()))
    assert.Equal(t, string("fred"), string(model.Field("B").Value()))

Checking that a type implements an interface:

The same as above may be simpler when considering interfaces.

import (
    "encoding/json"
    ...
)
    ...

    driver := builder.Driver()
    model, modelIsExported := driver.MakeWithType("Model")

    // we need a pointer receiver
    iface := model.Pointer().Iface()
    jsonUnmarshaler, implementsUnmarshalJSON := iface.(json.Unmarshaler)
    require.True(t, implementsUnmarshalJSON)

	testJSON := []byte(`{"a": 4, "b": "fred"}`)
    require.NoError(t, jsonUnmarshaler.UnmarshalJSON(testJSON))

    assert.Equal(t, int64(4), int64(model.Field("A").Value()))
    assert.Equal(t, string("fred"), string(model.Field("B").Value()))

    jsonMarshaler, implementsMarshalJSON := iface.(json.Marshaler)
    require.True(t, implementMarshalJSON)

    data, err := jsonMarshaler.MarshalJSON()
    require.NoError(t, err)
    assert.JSONEq(t, string(testJSON), string(data))

The asserter knows a few common methods to achieve the same result:

    tp := builder.Asserter(t)

    model := tp.MustExportType("Model") // required
	testJSON := []byte(`{"a": 4, "b": "fred"}`)
    model.Pointer().ShouldUnmarshalJSON(testJSON)
    gentest.ShouldReturnEquivalentJSON(t, testJSON, model.ShouldMarshalJSON())

For validations:

    tp := builder.Asserter(t)

    model := tp.MustExportType("Model") // required
	testJSON := []byte(`{"a": 4, "b": "fred"}`)
    ctx := context.Background()
    model.Pointer().ShouldUnmarshalJSON(testJSON)
    require.NoError(t, model.ShoulValidateContext(ctx))

TODO

• call go vet
• call linter
• register test coverage
• multi-packages analysis?