e2e

End-to-End Testing

This directory contains our end-to-end tests which exercise smartcontract, controller, activator, client, and the agent on device. We run docker containers for all of the components in the system, using testcontainers-go. The tests are completely isolated and run in parallel.

The docker images depend on images exposed via our container registry. To access, you need to login with your github username and a personal access token with read access. Go to your Personal Access Tokens page and create a token with read:packages access. See this github doc for more details.

The docker images will automatically build whenever you run make test. You do not need to explicitly build them, but if you'd like to you can run make build

Run the tests with:

make test

Or run specific tests:

# Using `go test`:
go test -tags e2e -v -run=TestE2E_IBRL$

# Using `make test`:
# NOTE: If you are using the special character $, you need to wrap it in single
# quotes and escape with an extra $, as shown below:
make test run='TestE2E_IBRL$$'

If you're running tests with go test directly, and you're making changes to the components, you'll need to run make build before your go test command for the changes to be included.

⚠️ Note If you are running the full test suite in parallel on Mac with Docker for Mac, you'll likely need to configure it with sufficient resources. We recommend at least 38GB of memory and 8 CPUs, or more.

To run the tests with lower parallelism or sequentially, use the parallel argument:

# Using `go test`:
# NOTE: If you are running all the tests sequentially:
go test -tags e2e -v -parallel=1 -timeout=20m

# Using `make test`:
make test parallel=1

To run the tests without building the docker images first use the nobuild makefile argument:

make test nobuild

# Or, any combination of the previous args with it:
make test run=TestE2E_IBRL nobuild

If you want the docker containers to keep running after the tests finish, set the TESTCONTAINERS_RYUK_DISABLED env var to true. You will need to manually clean up the containers when you're done with them.

Topology

Each test spins up a local devnet with all components running in containers, and internal CYOA networks for devices and clients.

graph LR
  subgraph Default_Net["Default Network"]
    Ledger["Ledger/SmartContract"]
    Activator["Activator"]
    Controller["Controller"]
  end

  subgraph CYOA_Net["CYOA Network (10.X.Y.0/24)"]
    Device["Device/Agent @ 10.X.Y.8"]
    Client1["Client @ 10.X.Y.100"]
    Client2["Client @ 10.X.Y.110"]
  end

  Controller <--> Ledger
  Activator <--> Ledger
  Client1 <--> Device
  Client1 <--> Ledger
  Client2 <--> Device
  Client2 <--> Ledger
  Controller --> Device

  style Default_Net fill:#f0f0f0,stroke:#666,stroke-width:2px
  style CYOA_Net    fill:#d6eaf8,stroke:#2980b9,stroke-width:2px

Test Structure

The test framework is designed to be modular and reusable. Here's how the components fit together:

1. TestDevnet: The main test infrastructure that sets up:

   • A local devnet with ledger, manager, controller, and activator
   • A CYOA network for devices and clients
   • Helper methods for common operations (connecting tunnels, checking state, etc.)
   • TestDevnet is mostly just a wrapper around Devnet, which is responsible for provisioning and managing the component containers

2. Test Cases: Each of the current tests follow a common pattern:

func TestE2E_IBRL(t *testing.T) {
  t.Parallel()

  // 1. Set up test environment
  dn := NewSingleDeviceSingleClientTestDevnet(t)
  client := dn.Clients[0]
  device := dn.Devices[0]

  if !t.Run("connect", func(t *testing.T) {
    // Setup steps
    // Connect steps
    // Verify post-connect state
  }) {
    t.Fail()
    return
  }

  if !t.Run("disconnect", func(t *testing.T) {
    // Disconnect steps
    // Verify post-disconnect state
  }) {
    t.Fail()
  }
}

1. State Verification: Tests use helper methods to verify state:
   • WaitForClientTunnelUp: Ensures tunnel is established
   • WaitForAgentConfigMatchViaController: Verifies agent configuration
   • Custom verification functions for specific test cases

Adding a New Test

To add a new test:

1. Create a new test file (e.g., ibrl_test.go) in the e2e directory
2. Use NewSingleDeviceSingleClientTestDevnet to set up the test environment
3. Implement connect/disconnect test cases following the pattern above, if applicable
4. Add state verification functions specific to your test case (see checkIBRLPostConnect in ibrl_test.go for example)
5. Use fixtures for expected output verification if appropriate

Example test structure:

func TestE2E_IBRL(t *testing.T) {
	t.Parallel()

	dn := NewSingleDeviceSingleClientTestDevnet(t)
	client := dn.Clients[0]
	device := dn.Devices[0]

	if !t.Run("connect", func(t *testing.T) {
		dn.ConnectIBRLUserTunnel(t, client)

		dn.WaitForClientTunnelUp(t, client)

		checkIBRLPostConnect(t, dn, device, client)
	}) {
		t.Fail()
		return
	}

	if !t.Run("disconnect", func(t *testing.T) {
		dn.DisconnectUserTunnel(t, client)

		checkIBRLPostDisconnect(t, dn, device, client)
	}) {
		t.Fail()
	}
}