goProbe

goProbe

High-performance network packet metadata capture and storage of flows

The tool goProbe is responsible for capturing packet metadata off the wire. Under the hood, it makes use of slimcap to extract the following attributes which are used to classify the packet in a flow-like data structure:

• sip: source IP
• dip: destination IP
• dport: destination port (if available)
• proto: IP protocol

Available flow counters are

• bytes_sent: bytes sent
• bytes_rcvd: bytes received
• pkts_sent: packets sent
• pkts_rcvd: packets received

Note: a goProbe-flow is hence not a NetFlow-flow. Nonetheless, the limited metadata collected in a goProbe-flow has helped resolved numerous network incidents and mis-configurations for almost a decade at Open Systems AG and half a decade at nect.

Invocation

To start capturing, run

./goProbe -config goprobe.yaml

The tool is meant to run as a service/daemon by means of init scripts or systems such as systemctl. Examples for such intergrations can be found inside the examples/config folder.

Configuration

Refer to goprobe-example-config.yaml for configuration options.

The configuration can be provided as YAML or as JSON.

Live Config

The interfaces section of the configuration file is watched by goProbe and reloaded periodically. This is in order to reflect changes to individual interfaces without having to restart capturing. This ensures that only the affected interfaces have a short downtime while capturing resumes for all other interfaces.

All other changes to the configuration require a restart of goProbe.

API

By default, goProbe spawns a command-and-control HTTP API server, to provide access to its internal state as well as a query API to to query data from the goDB database to which it writes.

The API is able to bind on UNIX sockets.

Documentation

The goProbe API is laid out in the OpenAPI 3.0 Specification.

Note: some tools only accept a single OpenAPI file. To merge the specification into one output file, use swagger-cli:

swagger-cli bundle ../../pkg/api/goprobe/spec/openapi.yaml --outfile _build/openapi.yaml --type yaml

Using gpctl

The tool gpctl was specifically designed to cover the more common control API calls to inspect goProbe's internal state.

Example:

gpctl --server.addr unix:/var/run/goprobe status eth0 eth1

Client

There is a client package available that allows to make calls to the API programmatically and retrieve data structures used by goProbe.

Both gpctl and global-query use it internally.