virtio-go

virtio-go

This repository provides a collection of Go packages related to TUN/TAP, VirtIO, Virtqueues and the kernel-level vhost devices. All fully implemented in native Go.

These packages serve the purpose to make it possible to write Go code that can send and receive packets over the vhost-net device within the Linux kernel without having to actually start a real virtual machine.

This implementation is not designed to be used in production data paths and is not optimized for this use case. It is rather a useful tool to experiment with the vhost-net implementation, to test offloads, or to build automated integration tests.

Packages

tuntap

vhost-net is often used in conjunction with Linux TAP devices, therefore this package provides simple utilities to create such devices from Go and write and read packets into/from them.

You can find an example for this in tuntap/device_test.go.

virtio

This package is a collection of virtio-related definitions which are used by some of the other packages.

virtqueue

The communication between virtio devices and drivers is based on queues called "virtqueues". This package implements the driver-side of a split virtqueue which is the queue variant with the widest support. This queue implementation is designed to work with any virtio device, not just with the vhost implementation within the Linux kernel. But please note that it was only tested with the latter one.

vhost

While the kernel-level virtio server within the Linux kernel provides multiple accelerated devices for networking, storage and more, they all share some of the same ioctls. This package contains the device-type independent ones.

vhost-net

The Linux kernel can act as an accelerated virtio networking device using the vhost-net implementation. This package contains simple to use helper methods to configure an instance of it, register the virtqueues and transmit and receive packets.

Strictly speaking, this Go package implements neither a virtio driver, nor a device, at least not in the proper sense. It is also not a replacement for qemu or even uses virtualization in any way. It rather allows to communicate with a vhost device as if it was a virtio driver within a VM guest. The handy part about this is, that this allows direct access to the vhost-net device and especially the virtio_net_hdr struct to be able to experiment with offloads.

For a comprehensive example for initializing the vhost-net device, transmitting and receiving packets, as well as using the virtio_net_hdr to configure offloads, please check out vhostnet/device_test.go.

Testing

This project uses virtrun to run tests inside a virtual environment. This way the interaction with the kernel can be tested without requiring root privileges and messing with the local system.

The tests can be run like this:

go test -exec \
"go tool virtrun -verbose \
-kernel /path/to/kernel/6.16/vmlinuz \
-addModule /path/to/kernel/6.16/drivers/net/tap.ko.zst \
-addModule /path/to/kernel/6.16/drivers/vhost/vhost_iotlb.ko.zst \
-addModule /path/to/kernel/6.16/drivers/vhost/vhost.ko.zst \
-addModule /path/to/kernel/6.16/drivers/vhost/vhost_net.ko.zst \
-smp 2" \
./...

The quickest way to obtain the needed kernel image and modules is to extract them from packages in the Ubuntu mainline PPA.

Useful links

• Cloudflare: Virtual networking 101: bridging the gap to understanding TAP
• Red Hat: Virtqueues and virtio ring: How the data travels
• Red Hat: Introduction to virtio-networking and vhost-net
• OASIS OPEN: virtio Specification Version 1.2
• rust-vmm: Explanations about virtqueues (device perspective)
• Stephen Marz: Virtual I/O Devices (Videos: VirtIO Overview, VirtIO Descriptors)
• Linux Documentation: Segmentation Offloads
• Linux source code: vhost, vhost-net, vhost-net-test
• WireGuard tun implementation

Disclaimer

This library is provided as an open source repository in the hope that it is helpful. Hetzner does not provide official support for the contents of this repository.