Documentation
¶
Index ¶
- Constants
- func IsDHCPPlugin(driver string) bool
- type CapabilitiesResponse
- type CreateEndpointRequest
- type CreateEndpointResponse
- type CreateNetworkRequest
- type DHCPNetworkOptions
- type DeleteEndpointRequest
- type DeleteNetworkRequest
- type EndpointHealth
- type EndpointInterface
- type GwAllocCheckRequest
- type GwAllocCheckResponse
- type HealthCheck
- type HealthResponse
- type IPAMData
- type InfoRequest
- type InfoResponse
- type InterfaceName
- type IpamAddressSpacesResponse
- type IpamCapabilitiesResponse
- type JoinRequest
- type JoinResponse
- type LeaveRequest
- type Options
- type Plugin
- func (p *Plugin) Close() error
- func (p *Plugin) CreateEndpoint(ctx context.Context, r CreateEndpointRequest) (CreateEndpointResponse, error)
- func (p *Plugin) CreateNetwork(r CreateNetworkRequest) error
- func (p *Plugin) DeleteEndpoint(ctx context.Context, r DeleteEndpointRequest) error
- func (p *Plugin) DeleteNetwork(r DeleteNetworkRequest) error
- func (p *Plugin) EndpointOperInfo(ctx context.Context, r InfoRequest) (InfoResponse, error)
- func (p *Plugin) Join(ctx context.Context, r JoinRequest) (JoinResponse, error)
- func (p *Plugin) Leave(ctx context.Context, r LeaveRequest) error
- func (p *Plugin) Listen(bindSock string) error
- func (p *Plugin) ListenMetrics(addr string) error
- func (p *Plugin) ReleaseAddress(req ReleaseAddressRequest) error
- func (p *Plugin) RequestAddress(ctx context.Context, req RequestAddressRequest) (RequestAddressResponse, error)
- func (p *Plugin) RequestPool(req RequestPoolRequest) (RequestPoolResponse, error)
- type ReleaseAddressRequest
- type ReleasePoolRequest
- type RequestAddressRequest
- type RequestAddressResponse
- type RequestPoolRequest
- type RequestPoolResponse
- type StaticRoute
Constants ¶
const ( RouteTypeNextHop = 0 RouteTypeOnLink = 1 )
libnetwork's route-type encoding for StaticRoute.RouteType. See https://github.com/moby/libnetwork/blob/master/docs/remote.md — 0 ("via gateway") expects a NextHop; 1 ("on-link / connected") has no next hop.
const ( ModeBridge = "bridge" ModeMacvlan = "macvlan" ModeIPvlan = "ipvlan" )
Network attachment modes selected by the `mode` driver option.
const ( // ReleaseNever is the default: no path sends a DHCPRELEASE or a // DHCPv6 Release, which is exactly the v1.9.0 behaviour (#800). ReleaseNever = "never" // ReleaseOnStop hands the lease back when the endpoint leaves its // sandbox, which is every `docker stop`, every `docker rm` of a // running container and every `docker network disconnect`. A // container that restarts asks for a fresh lease. ReleaseOnStop = "on_stop" // ReleaseOnRemove is #962's third value. It is refused until it // lands, which is #984 on the v2.2.0 milestone: #962 itself ships // in v2.1.1 and carries only the two values above. See // releaseOnRemoveRefusal. ReleaseOnRemove = "on_remove" )
The values of the `release_lease` network option (#962).
RELEASE IS OFF BY DEFAULT AND THAT IS #800's RULE, NOT AN OVERSIGHT. A container is a host on this segment and a host does not hand its address back when it is switched off; the lease expires on the server's clock and a container that comes back before then re-claims it. Networks that would rather have the address back early say so.
const CLIOptionsKey string = "com.docker.network.generic"
CLIOptionsKey is the key used in create network options by the CLI for custom options
const DriverName string = "net-dhcp"
DriverName is the name of the Docker Network Driver
const MinEngineAPIVersion = "1.41"
MinEngineAPIVersion is the API version the floor row reported, and it is a MEASUREMENT, not a threshold: engine-matrix run 34594749584 saw 20.10.24 answer with API 1.41 and 19.03.15 with 1.40. Nothing refuses on it; see the block above.
const MinEngineIfnameVersion = "29.8"
MinEngineIfnameVersion is the lowest engine that applies a remote driver's requested container-side interface name (#125, #670).
MEASURED, not read off a changelog: the same nested-daemon rig the engine matrix uses was pointed at each line with an endpoint carrying `com.docker.network.endpoint.ifname=lan0`, and the container's own `ip link` was read. 28.5.2 and 29.7.2 name the interface by the driver's prefix and index; 29.8.0 names it `lan0`. That boundary is moby/moby#52866, which taught libnetwork's remote proxy to pass DstName through instead of dropping it, and which shipped in 29.8.0.
A VERSION IS A PROXY FOR THE BEHAVIOUR, and this one can be wrong in both directions. #670 makes the point about this exact field: an engine that reports a number may be built without the change, and a vendor may backport it into a line below this one. Nothing here reads what the engine actually does, so a backported 29.7.x is warned about and counted, and a 29.8 build with the change patched out is called supported.
THE ALTERNATIVE WAS PRICED AND NOT TAKEN. Probing the behaviour means creating an endpoint with a requested name and reading the link back, which at CreateEndpoint is somebody's live container on somebody's production host, at the moment they asked for something else. The version comparison is what is left, and it is wrong only where an engine's build departs from its version. The behaviour itself IS probed where a throwaway container is free: the integration suite asks the engine under test what it did and fails if the two part (test/integration/interface_name_test.go: engineAppliesIfname there reads the container's own `ip link`, and engineVersionAppliesIfname there is the version expectation it is checked against). That covers the engine a run happens to be on, not the population this constant covers. The naming is the same on both sides: a name carrying `Version` compares numbers, one without it reads behaviour.
BELOW IT NOTHING FAILS, and that is the whole problem. The plugin returns DstName on every engine, the engine below this one ignores the field, and the container comes up on a working network with a name the operator did not ask for. Nothing in Docker reports that, so the plugin says it: a line at CreateEndpoint naming the engine, and the ifname_unsupported counter for an operator who is not reading logs.
const MinEngineVersion = "20.10"
MinEngineVersion is the lowest Docker Engine release this plugin is measured to run on (#670).
IT IS A MEASUREMENT, NOT A JUDGEMENT. Every engine from this one up to the current release was driven through the whole baseline — plugin create, enable, a network in each of bridge, macvlan and ipvlan, a lease confirmed in the DHCP server's own log, and an endpoint that survives `docker restart` on the same address — by .github/workflows/engine-matrix.yml, one nested daemon per row.
WHAT THE ROW BELOW IT DID, exactly, because the difference decides what this number may be called. On the cgroup v2 host the lane runs on, 19.03 could not start an ordinary container at all: the cell's control step, which runs no code of ours, failed with "cgroups: cgroup mountpoint does not exist". Nothing of the plugin ran on that row, so it is UNMEASURED and neither this file nor the documentation says the plugin fails on 19.03. What is measured is that 20.10 is the lowest engine the plugin was shown to work on.
THE FORMAT IS LOAD-BEARING. major.minor, quoted, on one line, with this exact constant name: scripts/engine-floor.sh reads the floor out of this declaration rather than carrying a second copy of the number, and a reformatting it cannot match is a refusal there, not an empty answer. Patch versions are deliberately absent — the matrix drives tags, which follow their line, and a floor naming a patch would claim evidence about one build of it.
CHANGING IT IS A MEASUREMENT, NOT AN EDIT. The engine matrix reconciles this constant against the lowest row that actually passed and goes red when they disagree, and this file is in that lane's `paths:` so the reconciliation runs on the commit that moves it. Raising it drops support for installs that worked before, which #674 asks the maintainer to treat as a major-version question.
Variables ¶
This section is empty.
Functions ¶
func IsDHCPPlugin ¶
IsDHCPPlugin checks if a Docker network driver is an instance of this plugin
Types ¶
type CapabilitiesResponse ¶
type CapabilitiesResponse struct {
Scope string
ConnectivityScope string
// GwAllocChecker tells libnetwork it may ask this driver whether a
// network needs a gateway address allocated. It is what stops every
// `docker network create` in IPAM mode from running a DHCP exchange
// for an address no container uses.
//
// DECLARING IT MAKES /NetworkDriver.GwAllocCheck REQUIRED. libnetwork
// calls that RPC only when this field is true, and a 404 from it is a
// real error rather than the tolerated one the unrouted RPCs get. The
// route and this field are one change; see routes().
GwAllocChecker bool
}
CapabilitiesResponse returns whether or not this network is global or local
type CreateEndpointRequest ¶
type CreateEndpointRequest struct {
NetworkID string
EndpointID string
Interface *EndpointInterface
Options map[string]interface{}
}
CreateEndpointRequest is sent by the daemon when an endpoint should be created
type CreateEndpointResponse ¶
type CreateEndpointResponse struct {
Interface *EndpointInterface
}
CreateEndpointResponse is sent as a response to a CreateEndpointRequest
type CreateNetworkRequest ¶
type CreateNetworkRequest struct {
NetworkID string
Options map[string]interface{}
IPv4Data []*IPAMData
IPv6Data []*IPAMData
}
CreateNetworkRequest is sent by the daemon when a network needs to be created
type DHCPNetworkOptions ¶
type DHCPNetworkOptions struct {
// Mode selects the attachment strategy: "bridge" (default, requires
// `bridge`) or "macvlan" (requires `parent`).
Mode string `mapstructure:"mode"`
Bridge string
Parent string `mapstructure:"parent"`
// Gateway, if set, overrides the default gateway returned by the
// upstream DHCP server. Useful for split-horizon LANs where
// containers should egress via a different router than the one
// the DHCP server advertises (e.g. VPN gateway).
Gateway string
IPv6 bool
LeaseTimeout time.Duration `mapstructure:"lease_timeout"`
// IgnoreConflicts skips the BRIDGE OVERLAP check at CreateNetwork:
// whether some other Docker network already has this bridge, or an
// address range covering it. It is a question about this host's own
// configuration, asked once, before any container exists.
//
// IT IS NOT conflict_check AND THE TWO ARE NOT ALTERNATIVES.
// conflict_check is RFC 5227 address conflict detection: whether
// some OTHER DEVICE ON THE SEGMENT already holds the address the
// DHCP server just leased to a container, asked on the wire, once
// per acquisition and then continuously for the life of the lease.
// One knob is about Docker's bookkeeping and the other is about the
// LAN; a network legitimately sets either, both or neither.
IgnoreConflicts bool `mapstructure:"ignore_conflicts"`
// ConflictCheck selects RFC 5227 address conflict detection for
// every endpoint on this network (D23). Empty is
// dhcp.DefaultConflictCheck, which is the library's own default
// mode by name.
//
// wait probe before the address is used. The container's
// address is configured only after RFC 5227 section 2.1
// has cleared it, which costs 4-7s on every acquisition
// (section 2.1.1's schedule; see dhcp.ConflictWindow) and
// is why lease_timeout's default covers it.
// async use the address at once and probe beside it. `docker
// run` is as fast as it was; a conflict found afterwards
// is a DHCPDECLINE and an address CHANGE on a running
// container.
// off no probing and no listener. RFC 2131 section 4.4.1's
// check is a SHOULD, so this is conformant; section
// 3.1(5)'s DECLINE remains a MUST for a conflict detected
// by other means, and the plugin can still report one.
//
// The value is validated at CreateNetwork against the library's own
// list of modes, so a typo fails the create rather than silently
// selecting the default.
ConflictCheck string `mapstructure:"conflict_check"`
SkipRoutes bool `mapstructure:"skip_routes"`
// PropagateDNS, when true, makes the plugin write DHCP option 6
// (v4 DNS server list) or option 23 (v6) into the container's
// /etc/resolv.conf on every bind/renew with a non-empty list.
// Default false to preserve historical behaviour where Docker's
// embedded resolver handled DNS — flipping this on means LAN-DNS
// names suddenly resolve from inside containers.
PropagateDNS bool `mapstructure:"propagate_dns"`
// PropagateMTU, when true, makes the plugin set the container link's
// MTU to DHCP option 26 on every bind/renew with a non-zero value.
// Default false because some networks advertise non-standard MTUs
// for reasons unrelated to host capability (e.g. hand-rolled tunnel
// fragments) and silently re-MTU'ing a container could surprise an
// operator. Opt-in keeps the behaviour change visible.
PropagateMTU bool `mapstructure:"propagate_mtu"`
// ClientID, when non-empty, overrides the derived DHCP option 61
// (Client Identifier) for every endpoint on this network. Bytes go
// on the wire prefixed with type byte 0x00 (RFC 2132 opaque).
//
// Default empty = derive per endpoint: from the MAC in bridge and
// macvlan (unique, and preserved across a restart, so the lease
// survives), from the Docker endpoint ID in ipvlan (whose slaves
// share the parent MAC). See resolveClientID.
//
// Operator caveat: a static ClientID across containers means the
// upstream DHCP server can't differentiate them — each new
// container will appear to be the same logical client and may
// receive the same lease. Typically only useful when paired with
// VendorClass to drive class-based policy that doesn't depend on
// per-client identity.
ClientID string `mapstructure:"client_id"`
// VendorClass, when non-empty, overrides the default DHCP option
// 60 (Vendor Class Identifier) value of "docker-net-dhcp" for
// every endpoint on this network. Lets DHCP servers using
// class-based policy (Cisco / Aruba / etc.) differentiate
// net-dhcp containers from other clients on the same LAN —
// for example to issue a different gateway or option set to
// containers tagged with a known vendor string.
VendorClass string `mapstructure:"vendor_class"`
// ValidateDHCP, when true, makes CreateNetwork run a one-shot
// DHCP probe on the parent NIC before the network is created,
// failing fast with a clear error if no DHCP server answers
// within the budget (see preflightProbeBudget). Catches
// misconfigurations (parent isolated from any DHCP server,
// firewall blocking UDP/67-68, broken VLAN tag) at create time
// rather than the first `docker run` attempt.
//
// macvlan / ipvlan modes only — bridge mode's "parent" is an
// existing Linux bridge, where the probe semantics are different
// and not yet implemented.
//
// The probe runs a full DHCPDISCOVER → REQUEST → ACK cycle -- an
// OFFER alone does not prove the server will commit -- so the upstream
// pool briefly sees one extra lease per `docker network create`
// with this opt-in. The probe MAC is random (locally-administered
// bit set) so it doesn't collide with anything stable upstream;
// the lease times out naturally rather than dragging CreateNetwork
// on a slow release path.
ValidateDHCP bool `mapstructure:"validate_dhcp"`
// RegisterDNS, when true, makes every endpoint on this network send
// the DHCP FQDN option (81 v4 / 39 v6) built from
// its resolved hostname, asking the DHCP server to register that name
// in DNS (forward + reverse). Default false: dynamic-DNS registration
// is a network-policy decision, never silent. Best-effort and advisory
// — many consumer routers ignore option 81, so this requests
// registration, it does not guarantee resolution. Reuses the same
// hostname already sent as the option-12 hint (#261).
RegisterDNS bool `mapstructure:"register_dns"`
// AuditLog, when true, appends every lease-lifecycle event on
// this network (bound / renew / stopped, plus stop_failed when the
// client's shutdown didn't complete) to STATE_DIR/leases.jsonl —
// an append-only JSONL audit trail answering "which IP did this
// container hold last Tuesday?" without dnsmasq-log archaeology
// (#109). Rotated at 16 MB or 30 days, whichever first; one
// rotated generation is kept. Default false: the ledger costs a
// disk write per lease event, and container-ID/IP correlation on
// disk is privacy-relevant in some environments — operators opt
// in deliberately. Append failures bump ledger_write_failures on
// /Plugin.Health and never affect lease handling.
AuditLog bool `mapstructure:"audit_log"`
// DHCPServers is an ordered preference list of DHCPv4 server
// addresses, e.g. "1.1.1.1,2.2.2.2": the first that answers within
// its slice of the acquisition budget wins, and the list is
// exhaustive — if none answers, acquisition fails rather than
// falling back to whichever server happened to reply. Naming your
// servers is what makes the list complete (#111).
//
// Empty (the default) accepts whichever OFFER arrives first, the
// historical behaviour.
DHCPServers string `mapstructure:"dhcp_servers"`
// DenyServers is an unordered list of DHCPv4 server addresses this
// network must never take a lease from, e.g. "3.3.3.3" — a rogue
// appliance or a second router on the segment (#669).
//
// This is a permission, not a preference: it composes with
// DHCPServers rather than competing with it. See serverPolicy for
// how the two are composed before either reaches the client.
DenyServers string `mapstructure:"dhcp_deny_servers"`
// ReleaseLease decides whether this network hands a lease back
// instead of letting it expire (#962). `never` (the default) is
// v1.9.0's rule: no path sends a DHCPRELEASE, and a stopped
// container's address stays leased until it expires, so a restart
// re-claims it. `on_stop` sends the release when the endpoint
// leaves its sandbox, which frees the address at once and costs the
// restarted container its address.
//
// The value is validated at CreateNetwork against the list in
// parseReleaseLease, so a typo fails the create rather than
// silently selecting the default.
ReleaseLease string `mapstructure:"release_lease"`
}
DHCPNetworkOptions contains options for the DHCP network driver
type DeleteEndpointRequest ¶
DeleteEndpointRequest is sent by the daemon when an endpoint needs to be removed
type DeleteNetworkRequest ¶
type DeleteNetworkRequest struct {
NetworkID string
}
DeleteNetworkRequest is sent by the daemon when a network needs to be removed
type EndpointHealth ¶
type EndpointHealth struct {
Endpoint string `json:"endpoint"`
Network string `json:"network"`
Mode string `json:"mode"`
// Address is the lease the renewal client currently holds, in CIDR
// form. Empty means it holds none: the client is still acquiring,
// or its lease was lost.
Address string `json:"address,omitempty"`
// LeaseState is `bound` when the client holds a lease and
// `acquiring` when it does not. Deliberately two values and not the
// protocol's state machine: RENEWING and REBINDING are visible in
// the times below, and a state name the plugin cannot read from the
// client would be a guess.
LeaseState string `json:"lease_state"`
// RenewAt / RebindAt / ExpiresAt are T1, T2 and the lease's end as
// ABSOLUTE times (RFC 3339), not as remaining seconds. A remaining
// second is only meaningful together with the instant it was read,
// and this document is polled, cached and pasted into issues.
//
// An empty ExpiresAt on a bound endpoint is the protocol's infinite
// lease (0xFFFFFFFF), which the library represents as a zero time
// so that "no expiry" is a value rather than a threshold to guess.
RenewAt string `json:"renew_at,omitempty"`
RebindAt string `json:"rebind_at,omitempty"`
ExpiresAt string `json:"expires_at,omitempty"`
// Server is the DHCP server that granted the lease (option 54).
Server string `json:"server,omitempty"`
// LastEvent is the most recent lifecycle event the manager saw --
// `bound`, `renew`, `nak`, and so on -- with the time it saw it.
// Empty on an endpoint whose client has not reported anything yet.
LastEvent string `json:"last_event,omitempty"`
LastEventAt string `json:"last_event_at,omitempty"`
// ConflictCheck is the RFC 5227 mode this client runs in (D23) and
// ACDPhase is where that check has got to. Read the phase against
// the mode and never alone: in `off` the phase is `idle` because
// nothing runs, which is not the same fact as `idle` in `wait`.
ConflictCheck string `json:"conflict_check"`
ACDPhase string `json:"acd_phase"`
}
EndpointHealth is one entry of /Plugin.Health's `endpoints` array: what the plugin-wide counters cannot say, which is WHICH container is in what state.
Every field is READ, none is accumulated. The address, the lease times and the server come from the lease the renewal client holds; the RFC 5227 phase and the conflict mode come from the same client; the ids, the mode and the last event come from the manager that owns it. Nothing here is a counter, so nothing here needs a reset rule.
The array is bounded by active_endpoints, which is the same map length that field already reports, so a host with no containers has an empty array rather than an absent one.
type EndpointInterface ¶
EndpointInterface contains endpoint interface information
type GwAllocCheckRequest ¶
type GwAllocCheckRequest struct {
Options map[string]interface{}
}
GwAllocCheckRequest asks the NETWORK driver whether this network needs a gateway address allocated for it. It carries the driver options and nothing else -- no network id and no address space.
type GwAllocCheckResponse ¶
GwAllocCheckResponse answers it.
type HealthCheck ¶
type HealthCheck struct {
Status string `json:"status"`
ObservedValue int64 `json:"observedValue"`
ObservedUnit string `json:"observedUnit"`
// Time is when the counter behind this check LAST MOVED, in
// RFC3339 with nanoseconds -- not when this response was built.
// The flags latch, so without it `fail` cannot be read as anything
// but "at some point during this process", and "faulted an hour
// ago" and "faulting right now" are the same document. A counter
// that has never moved carries the time of this reading, which is
// the honest statement for a zero: nothing has been observed as of
// now.
Time string `json:"time"`
// Output is omitted for a passing check, per section 4.8.
Output string `json:"output,omitempty"`
}
HealthCheck is one element of the `checks` object.
Field names are the draft's, camelCase and all, rather than this repo's snake_case: the point of the shape is that a reader who knows the draft can read this document, and a renamed field is a shape that only looks like one.
type HealthResponse ¶
type HealthResponse struct {
// Status is draft-inadarei-api-health-check-06 section 3.1's
// pass/warn/fail, and Checks is its section 3.6 object. They are a
// REFINEMENT of Healthy, never a replacement: `fail` is exactly
// `healthy: false`, because both are read from the one declaration
// (metricDef.healthy) rather than derived twice. What they add is
// WHICH counter, WHEN it last moved, and a middle value for the
// counters the reference tells an operator to watch without
// calling them a fault.
//
// Healthy stays exactly as it was. 1.x dashboards read it, and a
// field whose meaning is narrowed by a new sibling is a field that
// silently changed.
Status string `json:"status"`
// Version, Commit and Library are what this binary was built from
// (pkg/buildinfo). They are also the labels of net_dhcp_build_info.
Version string `json:"version"`
Commit string `json:"commit"`
Library string `json:"library"`
// EngineVersion and APIVersion are what the DAEMON said when this
// process started, not what this process assumed (#670).
// EngineVersion is the engine's own version string and is the value
// the minimum is measured and compared on; APIVersion is what the
// client library NEGOTIATED with it, which is min(our maximum, the
// daemon's maximum) and so can be lower than either side supports.
//
// Both read `unknown` when the daemon did not answer at startup.
// Docker restarts this plugin during its own startup and the socket
// is routinely not serving yet at that moment (#383), so `unknown`
// is a state an operator can actually see, and it means "this
// process never found out" rather than "there is no engine".
EngineVersion string `json:"engine_version"`
APIVersion string `json:"api_version"`
Healthy bool `json:"healthy"`
// InstanceID identifies the plugin process that served this
// response. Every counter below is in-memory and returns to zero
// when the process does, so two reads are only comparable as a
// delta when their InstanceID matches (#405).
//
// uptime_seconds is a weaker version of the same signal: it does
// reset, but a plugin that restarts early in a long window and then
// runs longer than the first reading shows uptime going *up* across
// the pair, and the reset goes unnoticed. Comparing ids has no such
// blind spot.
InstanceID string `json:"instance_id"`
UptimeSeconds float64 `json:"uptime_seconds"`
ActiveEndpoints int `json:"active_endpoints"`
PendingHints int `json:"pending_hints"`
RecoveredOK int32 `json:"recovered_ok"`
// RecoveryFailed counts post-restart recoveries that failed for a
// container that was still running: it has no renewal client and
// will lose its lease at expiry. Healthy-affecting.
//
// Two conditions were folded into this counter historically and are
// now split out, because neither leaves a running container without
// a renewal client and both are routine after a daemon restart:
// RecoveryDeferred (#383) and RecoveryAbortedContainerGone (#376).
RecoveryFailed int32 `json:"recovery_failed"`
// RecoveryDeferred counts the times recovery met a daemon that was
// not serving yet and was retried once the socket came up (#383).
// Docker respawns the plugin during its own startup, so this is the
// expected state at that moment, not a fault — NOT Healthy-affecting.
// A rise paired with recovery_failed means the retry ran out too:
// that pair is the signal that endpoints really are unrecovered.
RecoveryDeferred int32 `json:"recovery_deferred"`
// RecoveryAbortedContainerGone counts recoveries abandoned because
// the container had already exited or been removed (#376). Not
// Healthy-affecting: nothing is running without a renewal client.
// The recovery-side twin of JoinAbortedContainerGone, and normal
// after a daemon restart that outlived some containers.
RecoveryAbortedContainerGone int32 `json:"recovery_aborted_container_gone"`
// RecoveryNetworkGone counts networks skipped during post-restart
// recovery because they had been removed between the NetworkList
// that found them and the NetworkInspect that reads their detail
// (#648). Not Healthy-affecting: a network that is gone leaves no
// running container without a renewal client. Counted rather than
// silent so a host churning networks under a restarting daemon is
// still visible. It landed in recovery_failed until #648, where it
// was fatal.
RecoveryNetworkGone int32 `json:"recovery_network_gone"`
// RecoveryFingerprintsSkipped counts endpoints recovery adopted but
// could not describe: the ContainerInspect that would have supplied
// the hostname did not answer, or answered with no hostname (#721).
// Not Healthy-affecting: the endpoint has a renewal client, so no
// running container is without one — what it has lost is the
// tombstone that would have carried its MAC and address across its
// next `docker restart`.
//
// It exists because #721's fix would otherwise have inherited the
// invisibility of the bug it closes. A skipped fingerprint means no
// tombstone, and the only outward sign of that was
// tombstones_consumed staying flat — indistinguishable from a quiet
// host. A hostname REFUSED by safeHostname is not counted here; it
// moves unsafe_hostnames_rejected instead, so "the daemon would not
// answer me" stays distinguishable from "a container sent a hostname
// nobody should send".
RecoveryFingerprintsSkipped int32 `json:"recovery_fingerprints_skipped"`
// RecoveryAlreadyManaged counts endpoints a recovery walk found
// already registered to another manager and therefore left alone —
// a Join reached them first. Not Healthy-affecting: the endpoint has
// a renewal client, it just is not the one this walk would have
// built. Counted because it is the only outward evidence of recovery
// racing a Join, and because the completion log used to report those
// endpoints as recovered (#480).
RecoveryAlreadyManaged int32 `json:"recovery_already_managed"`
// JoinStartFailures counts persistent-client Start failures at
// Join time (#317): a running container with no renewal client.
// Healthy-affecting — same operator action as recovery_failed
// (find the cause in the plugin log, restart the container).
JoinStartFailures int32 `json:"join_start_failures"`
// JoinAbortedContainerGone counts attaches abandoned because the
// container exited before the persistent client was up (#373). Not
// Healthy-affecting: there is no running container without a
// renewal client. Worth watching anyway — a rise means containers
// are dying seconds after start.
JoinAbortedContainerGone int32 `json:"join_aborted_container_gone"`
// JoinAbortedNoContainer counts attaches abandoned because no
// container ever claimed the endpoint on the network (#566). The
// address is LEFT TO EXPIRE. It was released here until #800
// removed that path, and `release_lease=on_stop` does not restore
// it: a release happens at Leave, and an endpoint no container
// claimed reaches no Leave at all, whatever the release is built
// from (#962).
// Not Healthy-affecting: nothing is running without a renewal
// client, because nothing is running. A rise means endpoints are
// being created for containers that never attach.
JoinAbortedNoContainer int32 `json:"join_aborted_no_container"`
// JoinAttachSlow counts attaches that succeeded only after
// outlasting AwaitTimeout, waiting on a daemon that was busy with
// the container being attached. Not healthy-affecting — these are
// successes — but a rising count is the visible form of #406.
JoinAttachSlow int32 `json:"join_attach_slow"`
// JoinAttachCompleted counts successful attaches. It is the
// population the three buckets below partition, and without it a
// bucket of zero cannot be told from a lane that attached nothing.
JoinAttachCompleted int32 `json:"join_attach_completed"`
// JoinAttachUnder1s and JoinAttach1sToBudget are the body of the
// distribution JoinAttachSlow is the tail of. Under a second, then
// a second up to and including AwaitTimeout; above it is
// JoinAttachSlow, so the three sum to JoinAttachCompleted.
//
// They exist because the per-attach timing line is Debug and the
// shipped LOG_LEVEL is info: on a host nobody has reconfigured,
// these are the only per-attach durations there are (#403).
JoinAttachUnder1s int32 `json:"join_attach_under_1s"`
JoinAttach1sToBudget int32 `json:"join_attach_1s_to_budget"`
// JoinAttachMsMax is the longest successful attach in
// milliseconds, saturating at MaxInt32. Not an average: #403 asks
// how close a loaded host comes to AwaitTimeout, and an average
// over a quiet host hides exactly the attach that answers it.
JoinAttachMsMax int32 `json:"join_attach_ms_max"`
// RestartLinkUpWaited counts child links brought up only after
// waiting out the departing link's hold on the address (#408). Not
// healthy-affecting: this is the fix working, and it is counted so
// the window is visible rather than inferred — the same reason
// JoinAttachSlow exists.
RestartLinkUpWaited int32 `json:"restart_link_up_waited"`
// RestartLinkUpTimeouts counts that wait outlasting its budget. The
// restart then fails with `address already in use`. Not
// healthy-affecting despite being a real failure: it surfaces
// through CreateEndpoint to the operator directly, and `healthy`
// is for faults nothing else reports (#422).
RestartLinkUpTimeouts int32 `json:"restart_link_up_timeouts"`
// JoinAbortedEndpointLeft counts attaches cancelled because the
// endpoint left while the attach was still running. Not
// healthy-affecting: there is no running container missing a
// renewal client.
JoinAbortedEndpointLeft int32 `json:"join_aborted_endpoint_left"`
// TombstoneWriteFailures counts tombstone persistence failures.
// Healthy-affecting: an endpoint will not keep its address across a
// restart.
//
// It moves on a failed READ as well as a failed write. Since #724,
// a transient read error (EIO, EMFILE, a read racing a writer) makes
// the write path refuse rather than rewrite the file from nothing,
// and that refusal is counted here — the consequence is identical to
// a failed write, and the name being narrower than the meaning is
// worth one sentence rather than a fourth counter.
TombstoneWriteFailures int32 `json:"tombstone_write_failures"`
// TombstoneQuarantines counts times the tombstone file was found
// unparseable and moved aside as tombstones.json.corrupt-<ts>
// (#724). Healthy-affecting, and the counter that costs the most
// when it moves: a write failure loses ONE container's MAC and
// address, a quarantine loses every live tombstone on the host, so
// every container restarting for the rest of the TTL window comes
// back with a new identity.
//
// Separate from TombstoneWriteFailures on purpose. The two have
// different remedies — a write failure means the disk is full or
// read-only, a quarantine leaves a file to read — and merging them
// would leave an operator unable to tell which one they are being
// paged for.
//
// WHY IT LATCHES `healthy`, WHICH IS NOT OBVIOUS. The argument
// against is real: the condition is self-healing by construction —
// the file is renamed away, the plugin continues correctly from an
// empty set, and the cost is bounded at one TTL window of address
// instability for containers that happen to restart in it. Against
// that, the remedy for a latched `healthy` is to restart the
// plugin, which tears down every managed endpoint's renewal client:
// strictly more damaging than the fault. On those terms alone it
// would not latch.
//
// It latches anyway, for two reasons. Consistency first:
// TombstoneWriteFailures is already healthy-affecting, and a
// quarantine is the same family — tombstones did not work. Splitting
// them would mean an I/O error latches and actual file corruption
// does not. And the one that decides it: a quarantine does not mean
// tombstones had a bad minute, it means SOMETHING WROTE GARBAGE
// into stateDir — a host bind mount that survives `docker plugin rm`
// and upgrade, and that now also holds the versioned options file.
// The self-healing is about the tombstones. The signal is about the
// disk, and that is worth an operator's attention even though this
// particular symptom cleared itself.
TombstoneQuarantines int32 `json:"tombstone_quarantines"`
// UnsafeHostnamesRejected counts container hostnames dropped before
// reaching the DHCP request because they carried a control
// character (#692). NOT healthy-affecting: the drop is the
// safe outcome and the lease proceeds. It is reported because a
// legitimate hostname never contains one, so a rising value is
// somebody probing rather than background noise.
UnsafeHostnamesRejected int32 `json:"unsafe_hostnames_rejected"`
// UnsafeOptionValuesDropped counts server-chosen DHCP string
// values refused before use because they carried a control
// character, plus option-15 domains truncated at their first space.
// NOT healthy-affecting: dropping is the safe outcome and the lease
// proceeds. Its sibling above covers the value the CONTAINER
// chooses; this one covers the values the SERVER chooses, which is
// the larger set and the one nothing filtered before (#703, #704).
UnsafeOptionValuesDropped int32 `json:"unsafe_option_values_dropped"`
// NetworkOptionsRejected counts endpoint operations that met a
// network's stored options and would not act on them as written:
// an interface name the kernel would not accept, or a mode this
// plugin does not implement (#727). DeleteEndpoint counts without
// refusing, so a rise does not mean nothing was torn down. NOT
// healthy-affecting: refusing is the safe outcome and the
// operation already fails visibly to Docker; one network's record
// is broken, not the plugin. A non-zero value means options
// written before name validation existed (#705), or a hand-edited
// state directory.
NetworkOptionsRejected int32 `json:"network_options_rejected"`
// IPAMReplayHits counts stored endpoint addresses this plugin
// confirmed at a daemon restart from its own lease record (#110).
// Only moves on networks created with this plugin as their IPAM
// driver. NOT healthy-affecting: it is the mechanism working. It is
// the denominator for the counter below.
IPAMReplayHits int32 `json:"ipam_replay_hits"`
// IPAMReplayMiss counts stored endpoint addresses this plugin
// refused to confirm because no lease record in that network holds
// them. NOT healthy-affecting: the refusal is the safe outcome and
// the network driver's own recovery adopts the endpoint from
// Docker's view. Worth investigating: the lease record and Docker's
// store have drifted apart.
IPAMReplayMiss int32 `json:"ipam_replay_miss"`
// IPAMRebindAmbiguous counts address requests that met more than one
// recently-removed endpoint on the network, so nothing said which
// address to ask for and the DHCP server decided. NOT
// healthy-affecting: every container still gets an address. Watch
// it: it is the one signal that addresses moved for a reason the
// operator can act on.
IPAMRebindAmbiguous int32 `json:"ipam_rebind_ambiguous"`
// IPAMReserveDuplicateMAC counts address requests refused because the
// network was already leasing an address for that hardware address.
// NOT healthy-affecting for the host: refusing is the safe outcome,
// and the alternative is two endpoints holding one address. Worth
// investigating, because every move is a container that did not
// start: two endpoints on one network were pinned to one
// --mac-address.
IPAMReserveDuplicateMAC int32 `json:"ipam_reserve_duplicate_mac"`
// IPAMReleaseUnknown counts addresses libnetwork released that no
// lease record of ours holds. NOT healthy-affecting and not a
// fault: a release for an address whose record is already retained
// or closed is the normal ordering.
IPAMReleaseUnknown int32 `json:"ipam_release_unknown"`
// DNSPropagationPIDMismatches counts DNS propagations refused
// because the container PID resolved through Docker no longer
// belonged to that container by the time the plugin acted on it
// (#688). NOT healthy-affecting: refusing is the safe outcome and
// the container keeps the resolv.conf it had. It is reported
// because the plugin shares the host PID namespace, so each one is
// a write that would otherwise have gone to an unrelated host
// process.
DNSPropagationPIDMismatches int32 `json:"dns_propagation_pid_mismatches"`
// NetnsPIDMismatches counts sandbox network-namespace opens refused
// because the container PID resolved through Docker no longer named
// that container. The attach fails, so this is not silent -- but the
// failure looks like a slow start; only this counter distinguishes a
// recycled PID from one.
NetnsPIDMismatches int32 `json:"netns_pid_mismatches"`
// SandboxKeyEntries, SandboxKeyEntryFailures and SandboxPIDFallbacks
// say which route the plugin took into each container's network
// namespace. SandboxKeyEntries counts opens carried by the sandbox
// key the daemon publishes; SandboxKeyEntryFailures counts refusals
// of that route; SandboxPIDFallbacks counts the endpoints that ended
// up on the /proc/<pid>/ns/net route instead.
//
// READ FALLBACKS AGAINST ENTRIES. Zero fallbacks with zero entries
// is not "the key route works" -- it is "nothing was opened". The
// pair is what makes the claim have a domain, and it is the evidence
// for whether the host PID namespace and CAP_SYS_PTRACE are still
// needed for the netns route on this host. Neither is
// healthy-affecting: a fallback that succeeds is a working endpoint.
SandboxKeyEntries int32 `json:"sandbox_key_entries"`
SandboxKeyEntryFailures int32 `json:"sandbox_key_entry_failures"`
SandboxPIDFallbacks int32 `json:"sandbox_pid_fallbacks"`
// The four arms SandboxKeyEntryFailures folds together, published
// separately because the aggregate cannot say WHICH refusal
// happened and the two most likely ones want opposite remedies.
//
// SandboxKeyNotANamespace is the expected one where the sandbox
// netns mount is private (sandbox_netns_propagation=0):
// the entry is the placeholder file libnetwork creates before it
// bind-mounts the namespace over it, and the plugin's own
// /var/run/docker bind was taken before that mount existed. Nothing
// to do about it; the PID route carries the attach.
//
// SandboxKeyNotPermitted is the one that looks identical in the
// aggregate and is NOT expected: the daemon is publishing keys
// somewhere this plugin does not accept, which is what a
// non-default `dockerd --exec-root` produces. The remedy there is a
// change to this plugin, not to the host.
//
// SandboxKeyWrongNSType has never been observed and is published
// anyway, because "never observed" is a claim that needs a counter
// to stay true. SandboxKeyUnavailable is the residual: the entry
// never became openable inside the attach budget.
//
// SandboxKeyAbsent is the endpoint no key was published for at
// all, by either source. It used to land in
// SandboxKeyNotPermitted, whose documented cause and remedy are
// about a key that exists.
//
// They sum to SandboxKeyEntryFailures exactly. None is
// healthy-affecting.
SandboxKeyAbsent int32 `json:"sandbox_key_absent"`
SandboxKeyNotPermitted int32 `json:"sandbox_key_not_permitted"`
SandboxKeyNotANamespace int32 `json:"sandbox_key_not_a_namespace"`
SandboxKeyWrongNSType int32 `json:"sandbox_key_wrong_ns_type"`
// DockerAPINonGETRefusals counts requests to the Docker API the
// plugin refused to send because their method was not GET. The
// plugin's whole Docker surface is four read calls, so this is
// expected to stay zero for the life of an installation; a non-zero
// value means code in this process tried to write to the daemon
// (#691). NOT healthy-affecting: the refusal is the safe outcome.
DockerAPINonGETRefusals int32 `json:"docker_api_non_get_refusals"`
// DHCPRoutesApplied counts DHCP option-121 classless static routes
// handed to Docker. DHCPDefaultRouteSuperseded counts the Joins
// where those routes cover 0.0.0.0/0 by union rather than by a
// literal default entry -- i.e. the container's egress goes to the
// option-121 next hop even though the reported gateway, and
// `docker inspect`, still name the router from option 3. Neither is
// healthy-affecting: this is legitimate split-tunnel behaviour as
// often as it is not. They are the evidence trail (#700).
DHCPRoutesApplied int32 `json:"dhcp_routes_applied"`
DHCPDefaultRouteSuperseded int32 `json:"dhcp_default_route_superseded"`
// MTURefused counts option-26 MTUs outside the range the plugin
// will apply; the link keeps the MTU it had. NOT healthy-affecting.
// Read it because the alternative was silent: a link clamped near
// the RFC floor black-holes path MTU discovery and looks like a
// slow network, not a misconfiguration (#702).
MTURefused int32 `json:"mtu_refused"`
// TombstonesConsumed counts CreateEndpoints that replayed a fresh
// tombstone and so handed a recreated container its previous
// MAC/IP. Not Healthy-affecting: this is the address-stability
// mechanism working.
//
// It is the counterpart to RecoveredOK. Between them they say which
// of the two paths preserved an address across a restart, which is
// what makes "the address survived, but via neither path" a
// detectable state rather than a silent pass (#386).
TombstonesConsumed int32 `json:"tombstones_consumed"`
// LeaseChanged counts renewals where the server returned a different
// IP than the manager last recorded. Not Healthy-affecting (it
// doesn't break Docker's view fatally — see plugin.go for the
// truthfulness-gap discussion), but worth alerting on for
// long-running containers.
LeaseChanged int32 `json:"lease_changed"`
// AddressConflicts counts leased addresses found already in use on
// the segment (#524, D12), in both families. Healthy-affecting: the
// endpoint is up and reporting an address that does not work, and
// no other counter moves for it.
//
// Since 2.0 it covers the whole life of the lease, not just the
// moment after acquisition: RFC 5227 section 2.1's probes before
// the address is used AND section 2.4's listener afterwards. A
// conflict that appears an hour into a container's life moves it.
//
// IT IS THE SUM OF AddressConflictsV4 AND AddressConflictsV6, and
// only the v4 half is the population ACDConflictsDetected counts.
// See those two fields.
AddressConflicts int32 `json:"address_conflicts"`
// ACDProbesSent, ACDAnnouncementsSent, ACDConflictsDetected and
// ACDARPSendFailures are the library's own RFC 5227 counters.
//
// READ ACDProbesSent BEFORE BELIEVING AddressConflicts IS ZERO.
// That is the whole reason these are here: a zero conflict count
// over a plugin that never sent a probe is not a clean segment, and
// the two readings were indistinguishable in #524.
// ACDProbesSent and ACDAnnouncementsSent move on every acquisition
// in conflict_check=wait and =async, and never in =off.
//
// ACDConflictsDetected is the library's count of the same conflicts
// AddressConflicts counts from the chassis side. They must agree;
// a divergence is a defect in this seam, not a property of the
// segment.
//
// ACDARPSendFailures is probes and announcements the ARP socket
// refused. NOT Healthy-affecting on its own — but a probe that was
// never sent proves nothing about the address, so a rise here is
// what turns "no conflict" into "no question asked".
ACDProbesSent int32 `json:"acd_probes_sent"`
ACDAnnouncementsSent int32 `json:"acd_announcements_sent"`
ACDConflictsDetected int32 `json:"acd_conflicts_detected"`
ACDARPSendFailures int32 `json:"acd_arp_send_failures"`
// ACDResumedUnchecked counts endpoints resumed from a record whose
// section 2.1 check had not completed when the previous plugin
// process stopped (D23). NOT healthy-affecting: the resumed client
// re-runs the check on its INIT-REBOOT acknowledgement, so the
// window closes on its own. It is a `warn` check because during
// that window a container holds an address nothing finished
// checking.
ACDResumedUnchecked int32 `json:"acd_resumed_unchecked"`
// SandboxNetnsVisible is how many sandbox netns entries the plugin
// can currently see, or -1 when it cannot read the directory at all
// (#567). Sampled at request time rather than accumulated — it
// describes the plugin's view of the host right now, not something
// that happened.
//
// It exists because the evidence sandboxGone depends on was
// unreachable for the entire life of this plugin and nothing said
// so. The directory is not part of the image; it is bind-mounted by
// config.json, and before #567 it was not mounted at all, so
// os.ReadDir failed on every call and sandboxGone answered "no
// usable evidence" forever. A dead branch is invisible precisely
// because it never does anything.
//
// READ IT AGAINST ACTIVE_ENDPOINTS, NOT ON ITS OWN. The two
// failure modes are opposite and only the comparison separates
// them:
//
// -1 the directory is unreadable — the mount is missing. Every
// sandboxGone answer is "no evidence", which is safe but
// useless: the API 404 becomes the only source of truth.
// 0 with endpoints attached, the directory is readable but
// WRONG — mounted from somewhere with no sandboxes in it.
// This is the dangerous one. sandboxGone finds no entry
// matching any key and concludes every container has
// vanished, which is worse than never answering.
//
// A plain zero with no endpoints attached is neither: there is
// genuinely nothing to see.
SandboxNetnsVisible int32 `json:"sandbox_netns_visible"`
// SandboxNetnsPropagation says whether a mount the daemon makes
// under the sandbox netns directory AFTER this process started can
// reach this process at all.
//
// 1 the mount carries a propagation link, so it can. An attach
// can then enter the sandbox by its key.
// 0 the mount is private. Every attach is for a sandbox younger
// than this process, so every attach is refused with
// sandbox_key_not_a_namespace and carried by the container
// PID, which is what pidhost and CAP_SYS_PTRACE are for.
// -1 mountinfo is unreadable, or no mount covers any permitted
// directory. The directory not existing yet is NOT this
// reading: the daemon creates it inside the mount that
// already covers its parent, so the answer is that mount's.
//
// It exists because the zero reading is the whole of SECURITY.md's
// causal sentence, and until now that sentence was an inference
// from a refusal count. A refusal count is equally consistent with
// a key shape this plugin declines, which wants the opposite
// remedy. See sandboxNetnsPropagationIn for the bound on the 1.
SandboxNetnsPropagation int32 `json:"sandbox_netns_propagation"`
// SandboxNetnsInitMounts is how many sandbox netns mounts exist in
// PID 1's mount table.
//
// -2 PID 1 shares this process's mount namespace, so reaching
// the sandbox key through /proc/1/root reaches the table
// this process already has.
// -1 PID 1's mount table could not be read.
// 0 a different mount namespace that carries none of them.
// N a different mount namespace that carries N. Read it
// against sandbox_netns_visible.
//
// Under a nested engine PID 1 is that engine's init and not the
// outer host's, so this reads differently on the integration lane
// and on a systemd host, and a route judged on one of them alone
// is judged on the wrong number.
SandboxNetnsInitMounts int32 `json:"sandbox_netns_init_mounts"`
// DHCP-wire counters (T2-4). Naming intentionally drops the
// Prometheus `_total` suffix to stay consistent with the
// existing fields above; the issue's proposal listed them with
// `_total` for documentation clarity but the wire field is the
// shorter form.
//
// Each of these is the SUM of its *_v4 and *_v6 halves below, added
// in healthSnapshot (#730). It is not a counter in its own right,
// and nothing increments it. The meaning operators alert on is
// unchanged — it was a v4+v6 total before and it is a v4+v6 total
// now — but it is now derived from the halves rather than the
// halves being derived from it.
LeasesObtained int32 `json:"leases_obtained"`
LeasesRenewed int32 `json:"leases_renewed"`
// RenewalsUnanswered counts renewal requests that got no answer,
// one per request, while the client kept running (#940). Read it
// beside LeasesRenewed and ahead of DHCPTimeouts: renewals
// completing with this flat is a healthy lease; this climbing with
// LeasesRenewed flat is a DHCP server that has gone quiet.
//
// HOW EARLY IT MOVES IS A PROPERTY OF THE LEASE, not a constant.
// It moves at the first retransmission, and RFC 2131 section 4.4.5
// has the client "wait one-half of the remaining time until T2 (in
// RENEWING state) and one-half of the remaining lease time (in
// REBINDING state), down to a minimum of 60 seconds". The 60
// seconds is a FLOOR under that wait, which proto.renewalDelay
// implements as max(RenewRetransmitFloor, half), so the wait is a
// minute only when T2 is about two minutes off and is hours on a
// long lease. On the 24 hour lease #940 was reported from, T1 is at
// 12h and T2 at 21h, so the first retransmission is ~4h30m after
// the client's first renewal request at T1: the MEASURED four
// requests across 7h52m are that halving schedule, not a
// one-minute one. DHCPTimeouts
// first moves for a held lease when the lease ends, at 24h, so what
// this buys on that lease is about 7.5 hours of warning.
//
// NOT Healthy-affecting, and not a `warn` check either. A single
// lost datagram moves it on a segment that is working, so non-zero
// is not by itself the abnormal state a check can fire on; what is
// actionable is a rise with no renewals completing beside it, which
// is a relationship between two counters and not a threshold on
// one.
//
// The request currently in flight is not counted: one is proven
// unanswered only by the retransmission that follows it. A client
// that has sent N requests into silence reports N-1.
RenewalsUnanswered int32 `json:"renewals_unanswered"`
// DHCPServerTierFallbacks counts STEPS DOWN the dhcp_servers
// ladder: one per preferred entry that did not answer inside its
// slice of the budget and handed on to the next (#111). One
// acquisition against three silent preferred servers adds 2, not
// 1 — the counter measures how far down the list acquisition had
// to walk, which is the number worth having and is what the code
// has always produced. Three of the four places this was described
// said "acquisitions" instead, and #731 is that drift.
//
// Not healthy-affecting — the endpoint still got an address; a
// steady rise is how a silently-dead primary shows up.
DHCPServerTierFallbacks int32 `json:"dhcp_server_tier_fallbacks"`
// DHCPServerPolicyExhausted counts acquisitions abandoned because no
// server listed in dhcp_servers answered (#111). Not Healthy-
// affecting on its own: the acquisition failure it accompanies is
// already counted and already fails the operation.
DHCPServerPolicyExhausted int32 `json:"dhcp_server_policy_exhausted"`
// DHCPServerPolicyTimeouts counts dhcp_timeouts on endpoints whose
// renewal client is restricted to dhcp_servers (#731). A strict
// subset of DHCPTimeouts and NOT Healthy-affecting: every tick it
// counts is already counted there, and weighting one outage twice
// would make a policy-restricted endpoint look worse than an
// unrestricted one failing identically.
DHCPServerPolicyTimeouts int32 `json:"dhcp_server_policy_timeouts"`
DHCPTimeouts int32 `json:"dhcp_timeouts"`
// ClientStopFailures counts renewal clients that did not shut down
// cleanly when the plugin signalled them at teardown. Not
// Healthy-affecting: the endpoint is going away either way.
//
// It does NOT mean a lease was not handed back. Whether a lease
// goes back at all is `release_lease`'s question (#962), and on the
// default `never` no path sends a DHCPRELEASE: a stopped
// container's lease expires on the server's clock, like any other
// host's (#800). The counter that answers the release question is
// ReleaseFailures. This one was called lease_release_failures until
// v1.9.0, when it stopped describing a release at all.
ClientStopFailures int32 `json:"client_stop_failures"`
// ReleasesSent and ReleaseFailures are the sum of their per-family
// halves below (#962), stored the way every other pair is: the
// halves are the counters and this is their sum, computed in
// healthSnapshot rather than incremented anywhere.
//
// ReleaseFailures is warn-classified because it names addresses
// that are still leased upstream against the operator's stated
// intent. It is not Healthy-affecting: nothing on this host is
// broken by it, and the outcome is the one a `never` network has
// on every teardown.
ReleasesSent int32 `json:"releases_sent"`
ReleaseFailures int32 `json:"release_failures"`
// NAKsReceived counts server NAKs on renewal/rebind. Not
// Healthy-affecting on its own — the client recovers by
// re-DISCOVERing — but each NAK-triggered re-bind widens the
// docker-inspect divergence tracked by lease_changed (#128).
NAKsReceived int32 `json:"naks_received"`
// DisplacedStops counts managers displaced at Join — a Join that
// found a recovery-registered manager still in the registry for
// the same endpoint (plugin restart racing a container restart).
// Not Healthy-affecting: the displaced client is stopped and
// released, and the new one takes over. A climbing value means
// containers are restarting into a plugin that had recovered them,
// so pair it with recovered_ok when diagnosing a restart loop.
DisplacedStops int32 `json:"displaced_stops"`
// ParentLinkWaits / ParentLinkWaitTimeouts cover contention on a
// shared parent NIC. A parent is a macvlan port or an ipvlan port,
// never both, so the validate_dhcp probe holding one across a DHCP
// round trip can collide with an endpoint asking for the other
// (#486/#549). The plugin queues them per parent instead.
//
// Waits counts the operations that had to queue; timeouts counts
// those that gave up after parentGateBudget and went to the kernel
// anyway. Neither is Healthy-affecting: queuing is the mechanism
// working, and a timeout only restores the behaviour that existed
// before the queue did. Timeouts climbing is the actionable one —
// it means a reclaim is holding a parent far longer than its DORA
// should take, and container starts on that NIC are failing with
// "device or resource busy".
ParentLinkWaits int32 `json:"parent_link_waits"`
ParentLinkWaitTimeouts int32 `json:"parent_link_wait_timeouts"`
// LedgerWriteFailures counts failed appends to the audit_log
// lease ledger (#109). Not Healthy-affecting — a lost audit line
// degrades forensics, not networking; operators using audit_log
// alert on this directly.
LedgerWriteFailures int32 `json:"ledger_write_failures"`
// IfnameUnsupported counts endpoints created with a custom
// interface name on an engine that does not apply one (#125, #670).
// The request is accepted and the network works; the interface
// carries the driver's prefix and index instead of the requested
// name. Nothing else reports that, which is why it is counted.
IfnameUnsupported int32 `json:"ifname_unsupported"`
// StateFileChmodFailures counts files the startup sweep could not
// tighten, plus one for a STATE_DIR it could not read at all
// (#804). Not Healthy-affecting: nothing the plugin does is
// degraded by a loose mode on a state file. It is a `warn` check
// because the remedy is an operator's to apply, one `chmod` on the
// path the plugin log names, and because a sweep that failed and a
// sweep that found nothing to do are otherwise the same reading.
StateFileChmodFailures int32 `json:"state_file_chmod_failures"`
// Per-family breakdown of the wire counters (#212, #730). Both
// halves are STORED; the un-suffixed field above is their sum,
// computed in healthSnapshot from the same two values rendered
// here. It is not a third counter, and neither half is a subset of
// it. On a dual-stack host this isolates the v6-specific failure
// signal (NAK/timeout) the aggregate hides.
//
// Until #730 the v4 share was not stored at all: the un-suffixed
// field was the counter and the v4 number was recovered by
// subtracting *_v6 from it at render time. Two independently
// updated atomics combined by subtraction can produce a value lower
// than the previous read, and a counter that decreases is a reset
// to Prometheus. Storing both and adding for the total is
// monotonic under every interleaving; subtracting is not.
LeaseChangedV4 int32 `json:"lease_changed_v4"`
LeasesObtainedV4 int32 `json:"leases_obtained_v4"`
LeasesRenewedV4 int32 `json:"leases_renewed_v4"`
// RenewalsUnansweredV4 is the IPv4 half of RenewalsUnanswered.
RenewalsUnansweredV4 int32 `json:"renewals_unanswered_v4"`
DHCPTimeoutsV4 int32 `json:"dhcp_timeouts_v4"`
NAKsReceivedV4 int32 `json:"naks_received_v4"`
// ClientStopFailuresV4 is the v4 half of ClientStopFailures.
ClientStopFailuresV4 int32 `json:"client_stop_failures_v4"`
// ReleasesSentV4 and ReleaseFailuresV4 are the `release_lease`
// pair for IPv4 (#962): DHCPRELEASE messages that left the host,
// and attempts that produced none. Both stay at zero on a network
// that does not set the option, which is every network by default.
ReleasesSentV4 int32 `json:"releases_sent_v4"`
ReleaseFailuresV4 int32 `json:"release_failures_v4"`
// AddressConflictsV4 is the RFC 5227 half of AddressConflicts, and
// it is the ONLY half that may be compared against ACDProbesSent
// and ACDConflictsDetected: those two count ARP, which no DHCPv6
// conflict can produce.
AddressConflictsV4 int32 `json:"address_conflicts_v4"`
// THE v6 FIELDS BELOW HAVE WRITERS AGAIN (#911). Each one is
// incremented by a DHCPv6 client running beside the v4 one, and a
// zero means the thing did not happen rather than "this build
// cannot report it" -- which is what it meant while 2.0 was IPv4-
// only, and is the reason that statement was written here at all.
//
// A zero is still not evidence of health on its own. Two of them --
// dhcpv6_not_offered and dhcpv6_no_router_advert -- are absences
// the plugin TOLERATES, and their integration proofs assert
// dnsmasq's log beside the counter for exactly that reason: a
// counter is the plugin's belief, and the exchange is what
// happened.
LeaseChangedV6 int32 `json:"lease_changed_v6"`
LeasesObtainedV6 int32 `json:"leases_obtained_v6"`
LeasesRenewedV6 int32 `json:"leases_renewed_v6"`
// RenewalsUnansweredV6 is the DHCPv6 half: Renew and Rebind
// messages (RFC 9915 sections 18.2.4 and 18.2.5) the server did not
// answer. A v6-only silence is invisible in the sum.
RenewalsUnansweredV6 int32 `json:"renewals_unanswered_v6"`
DHCPTimeoutsV6 int32 `json:"dhcp_timeouts_v6"`
NAKsReceivedV6 int32 `json:"naks_received_v6"`
// AddressConflictsV6 is the DHCPv6 half of AddressConflicts: an
// address the kernel's Duplicate Address Detection (RFC 4862
// section 5.4) found on the link, declined to the server under RFC
// 9915 section 18.2.8. NOTHING ARP-SHAPED COUNTS IT -- not
// ACDProbesSent, not ACDConflictsDetected -- so a non-zero here
// beside a zero ACDConflictsDetected is the two protocols, not a
// seam defect.
//
// The replacement address the library then wins arrives as an
// ordinary bind and is applied to the container's interface.
// Docker's record of the endpoint is NOT updated, exactly as for a
// v4 lease change (#104); read LeaseChangedV6 beside this.
AddressConflictsV6 int32 `json:"address_conflicts_v6"`
// ClientStopFailuresV6 is the v6 share of ClientStopFailures
// (#608): the persistent DHCPv6 client held a binding and did not
// shut down cleanly when the plugin signalled it. No release is
// involved in THIS counter: on a `release_lease=on_stop` network
// the release is attempted at Leave, before the client is
// signalled, and it is counted in ReleasesSentV6 and
// ReleaseFailuresV6 below (#962). On every other network nothing
// this plugin runs sends one (#800).
ClientStopFailuresV6 int32 `json:"client_stop_failures_v6"`
// ReleasesSentV6 and ReleaseFailuresV6 are the same pair for
// DHCPv6 Release messages (RFC 9915 section 18.2.7). Read them per
// family and never as a sum: a dual-stack endpoint that handed its
// v4 address back and could not hand its v6 one back is the case
// the split exists to make visible.
ReleasesSentV6 int32 `json:"releases_sent_v6"`
ReleaseFailuresV6 int32 `json:"release_failures_v6"`
// DHCPv6ConfigOnly counts DHCPv6 information replies -- address-less
// configuration from a network advertising the RA "other config"
// flag (#815). NOT healthy-affecting: it is a normal exchange on a
// stateless network. Before #815 these were dropped unread, so such
// a network was indistinguishable from one that answered nothing.
// It has no v4 half; see the atom for why.
DHCPv6ConfigOnly int32 `json:"dhcpv6_config_only"`
// DHCPv6NotOffered counts endpoints created without a DHCPv6
// address because the segment advertised no managed DHCPv6 --
// stateless or SLAAC (#868). NOT healthy-affecting: on those
// networks it is the correct outcome, there being no DHCPv6
// address on them to be had. The endpoint has no global IPv6
// address FROM THIS PLUGIN; whether the kernel forms one from the
// advertised prefix is the segment's decision since #875, which
// leaves accept_ra=2/autoconf=1 on the interface. See v6_absence.go
// and docs/reference.md.
DHCPv6NotOffered int32 `json:"dhcpv6_not_offered"`
// DHCPv6NoRouterAdvert counts endpoints created without a DHCPv6
// address because no router advertisement arrived at all (#868).
// Kept apart from DHCPv6NotOffered because "no DHCPv6 here" and
// "nothing said anything" call for different operator action.
DHCPv6NoRouterAdvert int32 `json:"dhcpv6_no_router_advert"`
// IPv6LinkEnableFailures counts container links IPv6 could not be
// enabled on before a DHCPv6 client was started. Distinguishes a
// quiet segment from one the plugin could never have heard.
IPv6LinkEnableFailures int32 `json:"ipv6_link_enable_failures"`
// RouterAdvertGuardFailures counts steps of the Router-Advertisement
// guard that did not take on a container link (#875): a sysctl
// write that failed, or a read-back holding something other than
// what was written. Three knobs, two steps each. Non-zero means
// some container's kernel may not be processing advertisements, and
// DHCPv6 supplies no route of its own -- so the endpoint looks
// healthy now and loses its route when the advertisement it has
// expires. It does not count a privileged process inside the
// container undoing the settings; see docs/reference.md.
RouterAdvertGuardFailures int32 `json:"router_advert_guard_failures"`
// Checks is one entry per named check, keyed by the counter behind
// it. Each value is a SINGLE-ELEMENT ARRAY because section 4 says
// so: the draft's keys point to arrays so that a sub-component
// backed by several nodes can report each of them, and it asks for
// a one-element array where that is not relevant, "for
// consistency".
Checks map[string][]HealthCheck `json:"checks"`
// Endpoints is one entry per registered manager, bounded by
// ActiveEndpoints. Not in /metrics: a series per container is a
// cardinality decision, and it is taken separately.
Endpoints []EndpointHealth `json:"endpoints"`
}
HealthResponse is the payload returned by /Plugin.Health.
WHAT `Healthy` MEANS ¶
False when any of FIVE counters is non-zero: recovery_failed, join_start_failures, tombstone_write_failures, address_conflicts and tombstone_quarantines. Each is marked Healthy-affecting on its field below, and docs/reference.md states the same set in four more places; scripts/check-health-contract.sh keeps those in step.
This comment said "at least one plugin-restart recovery failed" — ONE counter — from before v1.6.0 until #724. The expression 350 lines below had four by then. It is the comment a developer reads first when adding a counter, which is exactly how it stayed wrong for two releases: the gate reads reference.md and the expression, not this. Corrected here rather than only in the docs, because the next person to add a Healthy-affecting counter reads this file (#638, #724).
IT LATCHES, AND THE OBVIOUS REMEDY DOES NOT CLEAR IT ¶
Every counter behind the flag is a monotonic atomic; nothing decrements them. So `healthy: false` means "a fault occurred at some point during THIS plugin process", not "something is wrong right now". An operator who restarts the affected containers fixes the condition — and the flag stays false. The only thing that clears it is restarting the plugin, which tears down the renewal client of every managed endpoint on the host, so it is not a free action and must not be taken as routine hygiene. Pair a reading with InstanceID to tell "still the same process, still latched" from "a new process that has already gone bad".
That is deliberate. An alert that goes quiet on its own is worse than one that never clears, because the operator learns nothing from the silence. If "unhealthy right now" is ever wanted, it is a new field, not a change to this one.
type IPAMData ¶
type IPAMData struct {
AddressSpace string
Pool string
Gateway string
AuxAddresses map[string]interface{}
}
IPAMData contains IPv4 or IPv6 addressing information
type InfoRequest ¶
InfoRequest is sent by the daemon when querying endpoint information
type InfoResponse ¶
InfoResponse is endpoint information sent in response to an InfoRequest
type InterfaceName ¶
InterfaceName consists of the name of the interface in the global netns and the desired prefix to be appended to the interface inside the container netns.
DstName, when non-empty, asks libnetwork for that exact name inside the container instead of DstPrefix+index. The remote-driver API has carried the field for years, but the remote proxy dropped it (drivers/remote/driver.go called `iface.SetNames(SrcName, DstPrefix, "")`) until moby/moby#52866, merged 2026-08-26 and milestoned for engine 29.8.0, where it shipped. Built-in drivers got per-driver interface_name in engine 28; remote drivers were left out until that fix. MEASURED with a nested daemon per line (#670): 28.5.2 and 29.7.2 name the interface by DstPrefix and index, 29.8.0 names it as asked. We return it either way: it is the documented response shape, costs nothing on engines that ignore it, and activates by itself on the first engine that honours it (#125). What an engine below 29.8.0 costs is said at CreateEndpoint and counted as ifname_unsupported, because a silently different interface name is otherwise invisible.
type IpamAddressSpacesResponse ¶
type IpamAddressSpacesResponse struct {
LocalDefaultAddressSpace string
GlobalDefaultAddressSpace string
}
IpamAddressSpacesResponse answers /IpamDriver.GetDefaultAddressSpaces.
type IpamCapabilitiesResponse ¶
IpamCapabilitiesResponse answers /IpamDriver.GetCapabilities.
type JoinRequest ¶
type JoinRequest struct {
NetworkID string
EndpointID string
SandboxKey string
Options map[string]interface{}
}
JoinRequest is sent by the Daemon when an endpoint needs be joined to a network
type JoinResponse ¶
type JoinResponse struct {
InterfaceName InterfaceName
Gateway string
GatewayIPv6 string
StaticRoutes []*StaticRoute
DisableGatewayService bool
}
JoinResponse is sent in response to a JoinRequest
type LeaveRequest ¶
LeaveRequest is sent by the daemon when a endpoint is leaving a network
type Options ¶
type Options struct {
// AwaitTimeout caps the polling helpers (sandbox readiness, link
// rename, netns appearance). AWAIT_TIMEOUT, default 10s.
AwaitTimeout time.Duration
// RequestCaptureDir, when non-empty, tees every libnetwork request
// body into that directory so an integration run can be turned into
// the replay fixtures under pkg/plugin/testdata/requests (#644).
// REQUEST_CAPTURE_DIR, default empty (disabled).
//
// Test instrumentation: it is declared in config-cover.json only,
// alongside GOCOVERDIR, and empty here costs the shipped plugin
// nothing — captureHandler returns the mux unwrapped.
RequestCaptureDir string
}
Options carries the plugin's runtime knobs. Every field is sourced from an environment variable declared in config.json and parsed in cmd/net-dhcp; a zero field means "unset", and NewPlugin substitutes the documented default. Grouping them beats growing NewPlugin's parameter list one knob at a time.
type Plugin ¶
type Plugin struct {
// contains filtered or unexported fields
}
Plugin is the DHCP network plugin
func NewPlugin ¶
NewPlugin creates a new Plugin. Zero-valued Options fields take the documented defaults, so NewPlugin(Options{}) is a valid production configuration.
func (*Plugin) CreateEndpoint ¶
func (p *Plugin) CreateEndpoint(ctx context.Context, r CreateEndpointRequest) (CreateEndpointResponse, error)
CreateEndpoint creates the per-endpoint host-side network plumbing (veth pair in bridge mode, macvlan child in macvlan mode), runs a one-shot DHCP client to acquire an initial lease, and stashes the result for Join. Docker moves the link into the container's netns when it acts on our Join response.
func (*Plugin) CreateNetwork ¶
func (p *Plugin) CreateNetwork(r CreateNetworkRequest) error
CreateNetwork validates network creation: option shape (pure), then existence of the parent interface (bridge or NIC depending on mode), the null IPAM driver requirement, and — for bridge mode — that no other Docker network already owns this bridge's address space.
func (*Plugin) DeleteEndpoint ¶
func (p *Plugin) DeleteEndpoint(ctx context.Context, r DeleteEndpointRequest) error
DeleteEndpoint deletes the host-side network plumbing for an endpoint. In bridge mode that's the veth pair (deleting one side removes the peer). In macvlan mode the link has typically already been moved into the container netns and reaped with it, so cleanup is best-effort.
func (*Plugin) DeleteNetwork ¶
func (p *Plugin) DeleteNetwork(r DeleteNetworkRequest) error
DeleteNetwork "deletes" a DHCP network (the bridge is managed by the user). We also evict any persistent DHCP managers attached to this network: libnetwork doesn't issue Leave for endpoints in stopped containers when the network is removed, so without this prune they linger as ghost entries in /Plugin.Health.active_endpoints. Stop is safe to call against a manager whose underlying netns is gone — it just unblocks the event loop and returns; the client itself may have already stopped because its netns vanished.
func (*Plugin) EndpointOperInfo ¶
func (p *Plugin) EndpointOperInfo(ctx context.Context, r InfoRequest) (InfoResponse, error)
EndpointOperInfo retrieves some info about an existing endpoint
func (*Plugin) Join ¶
func (p *Plugin) Join(ctx context.Context, r JoinRequest) (JoinResponse, error)
func (*Plugin) Leave ¶
func (p *Plugin) Leave(ctx context.Context, r LeaveRequest) error
Leave stops the persistent DHCP client for an endpoint
func (*Plugin) ListenMetrics ¶
Close stops the plugin. The HTTP server is shut down FIRST so no new Join can register a manager while (or after) we stop the existing ones — with the old ordering a Join dispatched during the stop fan-out installed a manager into the fresh registry that nobody ever stopped, leaking its DHCP client. Persistent DHCP clients are then stopped before process exit, so that a plugin upgrade or `docker plugin disable` does not leave clients renewing leases for endpoints this plugin no longer manages.
Since #800 this is NOT about releasing anything. Close arrives through Stop and not StopForLeave, so it releases nothing even on a `release_lease=on_stop` network (#962) — the containers are still running, and telling the server their addresses are free is the duplicate assignment #524 detects. A stopped client's address stays leased until it expires, which is the intended behaviour. What must not survive the shutdown is the CLIENT — a stray renewer keeps an address alive that nothing is using, and collides with the client a restarted plugin builds for the same endpoint. ListenMetrics starts the optional TCP listener for /metrics.
Off unless METRICS_ADDR is set, and that default is deliberate. The plugin holds CAP_NET_ADMIN, CAP_SYS_ADMIN and CAP_SYS_PTRACE with "network": {"type": "host"} in config.json, so any port it opens is on the host's own network namespace. Opening one has to be a decision an operator made, not something they inherited by upgrading (#651).
The mux here carries /metrics ALONE. See the metricsServer field for why that is load-bearing rather than tidy.
Returns once the listener is bound, so a bad METRICS_ADDR fails at startup where an operator will see it, rather than in a goroutine that logs and leaves the plugin running without the endpoint they asked for.
func (*Plugin) ReleaseAddress ¶
func (p *Plugin) ReleaseAddress(req ReleaseAddressRequest) error
ReleaseAddress gives one address back.
For a live endpoint this arrives AFTER DeleteEndpoint, whose record is already RETAINED, so there is nothing left to do. The case that needs work is the other one: CreateEndpoint failed, so an address was reserved and no endpoint was ever created, and libnetwork releases it. Retaining the reservation with the tombstone deadline is what lets a restart policy's next attempt claim the same address back instead of burning a second lease on the server. No DHCPRELEASE goes on the wire (D-7): the address is left to expire exactly as any other host on the segment leaves one.
THAT HOLDS ON EVERY VALUE OF `release_lease`, INCLUDING `on_stop`, and it is the one place the option does not reach (#962). A release happens at Leave, built from the endpoint's lease record; a reservation whose CreateEndpoint failed has no endpoint and reaches no Leave, so nothing on that path can see it. What it leaves behind on an `on_stop` network is a real lease the server granted that nothing will ever hand back, held only by the retention above until it expires. Giving it back here instead is a decision about which of the two wins, retention or release, and it is the follow-on this issue names and not a fold.
func (*Plugin) RequestAddress ¶
func (p *Plugin) RequestAddress(ctx context.Context, req RequestAddressRequest) (RequestAddressResponse, error)
RequestAddress is the whole IPAM dispatch, and the dispatch is a question about the RECORD STORE rather than about the request.
It has to be, because three of the shapes are wire-identical. An aux address at create, an aux address at the daemon-start replay, and a stored endpoint's address replayed at that same restart all arrive as an address with no options: libnetwork does not persist the endpoint's IPAM options, and it injects the MAC only when it is creating an endpoint. So what tells them apart is what this plugin knows about the address: the network's own saved gateway and aux set, and whether a record in an answering phase holds it.
func (*Plugin) RequestPool ¶
func (p *Plugin) RequestPool(req RequestPoolRequest) (RequestPoolResponse, error)
RequestPool answers with the pool identity and writes nothing durable.
The identity is the canonical request, so the SAME call at the daemon's start-up replay -- where libnetwork sends back the pool this driver returned, together with the option map it persisted -- derives the PoolID the daemon already stored. A driver that minted a fresh id, or that wrote state here, would unbind every network at every restart.
type ReleaseAddressRequest ¶
ReleaseAddressRequest gives one address back.
type ReleasePoolRequest ¶
type ReleasePoolRequest struct {
PoolID string
}
ReleasePoolRequest gives a pool back.
type RequestAddressRequest ¶
RequestAddressRequest asks for one address out of a pool.
type RequestAddressResponse ¶
RequestAddressResponse answers it, in CIDR form.
type RequestPoolRequest ¶
type RequestPoolRequest struct {
AddressSpace string
Pool string
SubPool string
Options map[string]string
V6 bool
}
RequestPoolRequest asks for an address pool.
type RequestPoolResponse ¶
RequestPoolResponse answers it.
type StaticRoute ¶
StaticRoute contains static route information
Source Files
¶
- capture.go
- conflict.go
- container_netns.go
- dhcp_manager.go
- dhcp_probe.go
- dhcp_values.go
- docker_client.go
- docker_transport.go
- endpoints.go
- engine_floor.go
- engine_probe.go
- health_checks.go
- health_endpoints.go
- http_limits.go
- ipam.go
- ipam_endpoint.go
- ipam_mode.go
- ipam_pool.go
- ipam_reserve.go
- ledger.go
- metrics.go
- mount_propagation.go
- netlink_seam.go
- network.go
- parent_attached.go
- parent_gate.go
- plugin.go
- records.go
- release_lease.go
- renewal.go
- resolvconf.go
- routes.go
- sandbox_netns.go
- servers.go
- state.go
- tombstone_store.go
- v6_absence.go
- v6_budget.go
- v6_link.go