coredns-incus-plugin

coredns-incus-plugin

The incus plugin for CoreDNS: DNS records for Incus instances, sourced from the Incus API and kept current by its event stream.

One zone per Incus project. No zone files, no reload polling. Answers are filtered per querier: an instance resolves from networks it shares with the client, and the answer holds its addresses on those networks alone. Every client gets an address it can reach. Reverse answers under the same rule, so an address is reversed by the clients that can reach it and by nobody else.

Status

PoC. It reaches the Incus API through incus-compose's iclient, which is not in a release yet, so go.mod and the Dockerfile both point at a checkout at /vendor/incus-compose. A stock incusd is fine - nothing here needs a patched daemon.

Answers Depend On Who Is Asking

Two instances asking for the same name get different answers, and that is the point rather than a quirk.

An Incus fleet is not one flat network. A project has its own bridges, an instance sits on some of them, and two instances that share none cannot reach each other at all. A single answer for a name is therefore wrong for somebody: it either hands out an address the client cannot route to, or it tells a client about a host it has no business seeing.

So a querier is placed on the set of networks it sits on, and sees only the names reachable there, with only their addresses on those networks. A multi-homed host is answered on the network it shares with the client. Every answer holds an address the querier can actually reach, and a name it cannot reach is NXDOMAIN rather than an address that times out.

It fails closed. A querier that lands on no known network is refused, and a name that exists but is invisible answers NXDOMAIN rather than NODATA, so response codes leak nothing about what else is out there.

How the querier is identified

Two ways, in this order:

1. EDNS0 Client Subnet - an optional field a resolver may attach to a query, carrying the address of the client it is forwarding for. Without it, a forwarding resolver looks like the only client in the world, because the packet reaching us carries the resolver's address. Incus's own dnsmasq fills it in with add-subnet=32,128.
2. The query's source address, when no client subnet is present - which is the case when an instance's resolv.conf points straight at CoreDNS and nothing relays.

The order matters: when a client subnet is there, it is the only truth, because whatever forwarded the query has already replaced the source address with its own.

Worth knowing: a client subnet is asserted by whoever sends it. Anything that can reach the server can claim to be any client. Reaching the server is the real boundary, which is why this belongs on a network only its clients are on.

How It Works

incusd event stream ──> one router ──> one goroutine per project ──> one folder ──> snapshot
                                                                                      │
                                        query ──────────────────────────────────> atomic load

Four ideas, and the rest follows from them:

Work happens when things change, not when queries arrive. The Incus event stream drives everything. A whole-fleet snapshot is built when something moves, records rendered and filtered ahead of time, and answering is then three map lookups - no filtering, no intersection, no I/O on the query path. A query never touches Incus.

One writer, no locks. Every piece of state has exactly one owning goroutine and they meet only on channels. Readers meet the writer at a single atomic.Pointer load, so a query reads a snapshot that can no longer change. Neither plugin holds a mutex.

Two plugins, one-way. ecs_view is the engine: it holds the records and answers, and knows nothing about Incus. incus is a source: it builds finished snapshots and hands them over. The engine derives nothing, which is what keeps it reusable for a source that is not Incus.

Restarts and outages are planned for. With data_dir set, what was last served is on disk with its zone serials, so a restart answers before it has reached Incus - and a secondary polling the SOA does not see the serial go backwards. Restored records are served as stale until every project has been re-read, and they are retired only when Incus says the project is gone, never on a timer.

It measures at ~120k queries/second with per-querier filtering, zero loss and 33 MiB for a 480-instance fleet - within ~12% of CoreDNS's built-in hosts plugin, which does one map lookup and has no idea who is asking. See docs/benchmark.md, including the parts that are not explained.

Using It

External plugins are compiled into CoreDNS. Clone CoreDNS, add the plugin to its plugin.cfg, and rebuild:

git clone https://github.com/coredns/coredns.git
cd coredns
# add to plugin.cfg:
#   incus:github.com/jochumdev/coredns-incus-plugin/plugin/incus
go generate && go build

plugin.cfg here is a ready-made version of that list, and Dockerfile builds a container image from it.

A minimal Corefile - both plugins are needed, since incus writes no reply of its own:

.:53 {
    ecs_view
    incus https://10.0.0.1:8443 {
        client_cert /etc/coredns/client.crt
        client_key  /etc/coredns/client.key
        data_dir    /var/lib/coredns-incus
    }
    forward . 10.0.0.1
    errors
    log
}

See docs/README.md for the full deployment walkthrough.

Building This Repository

main.go is a CoreDNS build with the plugin registered, for trying it out without a separate CoreDNS checkout:

just build        # build the coredns binary
just test-local   # the tests that need no Incus
just test         # those, plus the ones that talk to Incus
just test-e2e     # all of it, including the long end-to-end tests
just --list       # all commands

Documentation

• docs/README.md - deploying and operating the plugins
• docs/architecture.md - the two plugins and the boundary
• docs/ecs_view.md - the engine, inside
• docs/incus.md - the Incus source, inside
• docs/incus-components.md - what runs inside the source
• docs/incus-event-flow.md - one event, end to end
• docs/benchmark.md - what it costs, and how to measure it
• plugin/ecs_view/README.md - engine syntax
• plugin/incus/README.md - source syntax

Contributing

See CONTRIBUTING.md. Contributions are Apache 2.0, and every commit needs a Signed-off-by line (git commit -s).

License

Apache 2.0, see LICENSE.