decompose

decompose

Reverse-engineering tool for docker environments.

Takes all network connections from your docker containers and exports them as:

• graphviz dot
• structurizr dsl
• pseudographical tree
• json stream of items:

type Item struct {
    Name       string              `json:"name"`              // container name
    IsExternal bool                `json:"is_external"`       // this host is external
    Image      *string             `json:"image,omitempty"`   // docker image (if any)
    Meta       *Meta               `json:"meta,omitempty"`    // metadata, see 'metadata'
    Process    *Process            `json:"process,omitempty"` // process info
    Listen     []string            `json:"listen"`            // ports description i.e. '443/tcp'
    Networks   []string            `json:"networks"`          // network names
    Volumes    []*Volume           `json:"volumes"`           // volumes
    Connected  map[string][]string `json:"connected"`         // name -> ports slice
}

type Meta struct {
    Info string   `json:"info"`
    Tags []string `json:"tags"`
}

type Volume struct {
    Type string `json:"type"`
    Src  string `json:"src"`
    Dst  string `json:"dst"`
}

type Process struct {
    Cmd []string `json:"cmd"`
    Env []string `json:"env"`
}

example with full info and metadata filled:

{
    "name": "foo-1",
    "is_external": false,
    "image": "repo/foo:latest",
    "process": {
        "cmd": [
            "foo",
            "-foo-arg"
        ],
        "env": [
            "FOO=1"
        ]
    },
    "meta": {
        "info": "info for foo",
        "tags": ["some"]
    },
    "listen": ["80/tcp"],
    "networks": ["test-net"],
    "volumes": [
        {
            "type": "volume",
            "src": "/var/lib/docker/volumes/foo_1/_data",
            "dst": "/data"
        },
        {
            "type": "bind",
            "src": "/path/to/foo.conf",
            "dst": "/etc/foo.conf"
        }
    ],
    "connected": {
        "bar-1": ["443/tcp"]
    }
}

See stream.json for simple stream example.

metadata format

To enrich output with detailed descriptions, you can provide additional json file, with metadata i.e.:

{
    "foo": {
        "info": "info for foo",
        "tags": ["some"]
    },
    "bar": {
        "info": "info for bar",
        "tags": ["other", "not-foo"]
    }
}

Using this file decompose can enrich output with info and additional tags, for every container that match by name with one of provided keys, like foo-1 or bar1 for this example.

See csv2meta.py for example how to create such json fom csv, and meta.json for metadata sample.

clusterization rules

You can join your services into clusters by exposed ports, with clusterization rules, for example:

[
    {"name": "ingress", "ports": ["80/tcp", "443/tcp"]},
    {"name": "backend", "ports": ["8080-8090/tcp"]},
    {"name": "store", "ports": ["3306/tcp", "5432/tcp"]},
    {"name": "redis", "ports": ["6379/tcp"]},
    {"name": "queue", "ports": ["9092/tcp", "4222/tcp"]}
]

Source: cluster.json

features

• os-independent, it uses different strategies to get container connections:
  • running on linux as root is the fastest way and it will work with all types of containers (even scratch-based)
  • running as non-root or on non-linux OS will attempt to run netsat inside container, if this fails (i.e. for missing netstat binary), no connections for such container will be gathered
• produces detailed connections graph with ports
• fast, scans ~400 containers in around 5 seconds
• 100% test-coverage

known limitations

• only established and listen connections are listed

installation

• binaries / deb / rpm for Linux, FreeBSD, macOS and Windows.

usage

decompose [flags]

possible flags with default values:

  -cluster string
        json file with clusterization rules
  -follow string
        follow only this container by name
  -format string
        output format: dot, json, tree or sdsl for structurizr dsl (default "dot")
  -full
        extract full process info: (cmd, args, env) and volumes info
  -help
        show this help
  -load value
        load json stream, can be used multiple times
  -local
        skip external hosts
  -meta string
        json file with metadata for enrichment
  -no-loops
        remove connection loops (node to itself) from output
  -out string
        output: filename or "-" for stdout (default "-")
  -proto string
        protocol to scan: tcp, udp or all (default "all")
  -silent
        suppress progress messages in stderr
  -skip-env string
        environment variables name(s) to skip from output, case-independent, comma-separated
  -version
        show version

environment variables:

• DOCKER_HOST - connection uri
• DOCKER_CERT_PATH - directory path containing key.pem, cert.pm and ca.pem
• DOCKER_TLS_VERIFY - enable client TLS verification

examples

Get dot file:

decompose > connections.dot

Process json stream:

decompose -format json | jq '{name}'

Get only tcp connections as dot:

decompose -proto tcp > tcp.dot

Save full json stream:

decompose -full -format json > nodes-1.json

Display tree:

decompose -format tree

Merge graphs from json streams, filter by protocol, skip remote hosts and save as dot:

decompose -local -proto tcp -load nodes-1.json -load nodes-2.json > graph-merged.dot

Load json stream, enrich and save as structurizr dsl:

decompose -load nodes-1.json -meta metadata.json -format sdsl > workspace.dsl

example result

Scheme taken from redis-cluster:

it may be too heavy to display it with browser, use save image as and open it locally

Steps to reproduce:

git clone https://github.com/s0rg/redis-cluster-compose.git
cd redis-cluster-compose
docker compose up

in other terminal:

decompose | dot -Tsvg > redis-cluster.svg

license