Documentation
¶
Overview ¶
Package activity shows what each earner container does on the network: how many connections it opens and how many fail, which ports and networks it talks to, and whether it reaches into the owner's own network or sends mail. It is the Desktop port of the web repository's Activity page (app/activity.py, app/activity_ui.py, docs/activity.md in GeiserX/CashPilot), with one machine instead of a fleet.
It is off by default. When the owner switches it on, every 5 minutes one short-lived helper container per network namespace reads the kernel's socket tables and TCP/UDP counters from /proc/net (or, for an earner on the host network, the sockets of the earner's own cgroup through sock_diag). It never reads traffic contents. The helper script is the web repository's app/procnet.py, vendored byte for byte, so there is one parser and one set of classification rules.
Index ¶
- Constants
- Variables
- func Fit(w *Window)
- func JoinableRuntimes(runtimes map[string]RuntimeSpec) map[string]bool
- func Key(ip string) string
- func NormalIP(s string) string
- func ProbeConfig(spec ProbeSpec) (*container.Config, *container.HostConfig, error)
- func ProbeName(worker, ownerID string) string
- func ShouldSweep(p ProbeRef, worker string) bool
- func Spike(history []*float64, latest *float64, floor float64) bool
- func WindowEnd(now time.Time) int64
- type Container
- type Deltas
- type Dest
- type DockerEngine
- func (e *DockerEngine) Earners(ctx context.Context) ([]Container, error)
- func (e *DockerEngine) EnsureImage(ctx context.Context, image string) error
- func (e *DockerEngine) Inspect(ctx context.Context, idOrName string) (Container, bool)
- func (e *DockerEngine) Network(ctx context.Context, id string, earners map[string]bool) NetworkInfo
- func (e *DockerEngine) Probes(ctx context.Context) ([]ProbeRef, error)
- func (e *DockerEngine) Remove(ctx context.Context, id string) error
- func (e *DockerEngine) RunProbe(ctx context.Context, spec ProbeSpec) (out []byte, leftover bool, err error)
- func (e *DockerEngine) Runtimes(ctx context.Context) map[string]RuntimeSpec
- type Engine
- type Finding
- type Geo
- type Group
- type IPGeo
- type IPGeoStatus
- type Lookup
- type Network
- type NetworkContext
- type NetworkInfo
- type ProbeError
- type ProbeRef
- type ProbeResult
- type ProbeSpec
- type Runner
- type RuntimeSpec
- type Sample
- type Window
Constants ¶
const ( // WindowSeconds is the length of one window; rounds start on these wall-clock boundaries. WindowSeconds = 300 // ProbeSnapshots and ProbeGap are what the helper is asked for: 5 snapshots 2 s apart. ProbeSnapshots = 5 ProbeGap = 2 // ProbeTimeout bounds one helper's whole life. ProbeTimeout = 30 * time.Second // MaxParallel helpers run at once, and one round has RoundBudget in all. MaxParallel = 4 RoundBudget = 200 * time.Second // MaxDeltaSeconds: a counter change over a longer gap is not one window's worth. MaxDeltaSeconds = 2 * 300.0 // ProbeLabel marks every helper, so a sweep can find and remove leftovers. ProbeLabel = "cashpilot.probe" ProbeKind = "activity" WorkerLabel = "cashpilot.probe.worker" ProbePrefix = "cashpilot-activity-probe-" )
const ( IPGeoURL = "https://iptoasn.com/data/ip2asn-combined.tsv.gz" IPGeoMaxAge = 7 * 24 * time.Hour )
The network table: iptoasn.com's ip2asn-combined, "Licensed under Public Domain (PDDL v1.0)" in its own words, looked up locally. No address is ever sent anywhere. The country is where the network is registered, not where the address is. A port of app/ipgeo.py.
const ( MaxOut = 40 MaxIn = 20 MaxDestinations = 400 MaxSamples = 64 )
Caps on one sample, as the web UI applies them (app/activity_ui.py).
const ( MailCooldownHours = 24 LanCooldownHours = 6 SpikeCooldownHours = 12 SpikeMinHours = 72 SpikeRatio = 10 SpikeBytesFloor = 1 << 30 SpikeOpensFloor = 2000 )
Cooldowns per alert kind, in hours.
const HelperImage = "python:3.15-alpine@sha256:20a82dbc2783112ddb91d1534603f802d222a3e63e593cd002eb2ac534e70211"
HelperImage is the helper's image, pinned by its multi-arch index digest. It is pulled once, on the first round after the owner switches Activity on.
Variables ¶
var ( PlainRuntimes = []string{"", "runc", "io.containerd.runc.v2", "crun"} PlainBinaries = []string{"runc", "crun", "nvidia-container-runtime", "youki"} )
PlainRuntimes are the runtime names whose containers keep their sockets in the kernel's network namespace. PlainBinaries are the runtime binaries that do so under any registered name.
var ( LanAlertClasses = []string{"lan", "tailnet", "docker_host", "docker_peer", "metadata", "link_local"} MailPorts = []int{25, 465, 587, 2525} )
Alert rules, as in app/activity_ui.py.
var CgroupReasons = map[string]string{
"cgroup_v1": "this machine uses cgroup v1; a host on cgroup v2 is needed",
"cgroup_missing": "its cgroup was not found",
"sock_diag_refused": "the kernel refused the socket list",
"kernel_no_cgroup_id": "this kernel does not report a socket's cgroup",
}
CgroupReasons are why the helper could not tell a host-network earner's sockets apart, each with the plain words the page shows.
var Classes = []string{"infra", "metadata", "docker_host", "peer", "docker_peer", "lan", "tailnet", "link_local", "multicast", "public"}
Classes are the kinds of destination procnet.classify names.
var DeltaKeys = []string{"opens", "passive_opens", "attempt_fails", "resets", "udp_in", "udp_out"}
DeltaKeys are the counters a window reports as changes since the last one.
var ErrBadOutput = errors.New("bad_output")
ErrBadOutput is a helper that printed something we do not read.
var ErrBusy = errors.New("a round is already running")
ErrBusy is a round asked for while another is still running.
var ErrDownloadBusy = errors.New("a download is already running")
ErrDownloadBusy is a download asked for while another runs.
var ProcnetScript string
ProcnetScript is app/procnet.py of the web repository, byte for byte (`make catalog-sync` copies it; a test compares it when ../CashPilot exists).
var RoundTimeout = 270 * time.Second
RoundTimeout bounds a whole round, Docker calls and the helper image pull included, so a stalled daemon cannot hold the sampler loop (and its hourly purge) past the next window. A variable so tests can shorten it.
var Statuses = []string{"ok", "host_network", "no_network", "sandboxed", "probe_failed", "skipped_budget", "native"}
Statuses a sample can have. "native" is Desktop's own: a program run natively, not in a container, which no helper can join.
Functions ¶
func Fit ¶
func Fit(w *Window)
Fit enforces the destinations cap across a window, keeping each sample's first rows and marking the ones that lost rows.
func JoinableRuntimes ¶
func JoinableRuntimes(runtimes map[string]RuntimeSpec) map[string]bool
JoinableRuntimes is every runtime whose containers keep their sockets in the kernel's network namespace: runc and crun under any name, and gVisor only with --network=host. runtimes maps a registered name to (path, runtimeArgs).
func ProbeConfig ¶
ProbeConfig is the helper's container config and host config: user 65534, no capabilities, read-only, no new privileges, 32 processes and 48 MiB at most, a 64 KiB tmpfs at /data, no mounts, a json-file log. A host-network earner's helper joins the host network and the host cgroup namespace (to find the earner's cgroup in the read-only /sys/fs/cgroup); every other helper joins the earner's network namespace.
func ProbeName ¶
ProbeName is the helper's container name: one per app and earner, so two apps on one Docker host never remove each other's helper.
func ShouldSweep ¶
ShouldSweep says whether a sweep removes this helper: any of ours, any finished (exited or dead) helper of another app, and a helper from an app that labelled no worker (an older release) once it has stopped. A finished helper is never mid-round and its log keeps that round's addresses, so it must not outlive the app that started it. Another app's running or merely created helper is never touched: it may be mid-round.
Types ¶
type Container ¶
type Container struct {
ID string
Name string
Labels map[string]string
NetworkMode string
Runtime string
DNS []string
Networks []Network
}
Container is the part of docker inspect a round needs.
type Deltas ¶
type Deltas struct {
// contains filtered or unexported fields
}
Deltas holds the counter state between rounds, one entry per namespace.
func (*Deltas) Apply ¶
Apply sets the delta fields of s from this reading and stores it for the next. A delta exists only for the same namespace, with no counter going backwards, and a last reading at most MaxDeltaSeconds old. counters is nil when the helper read none (ActiveOpens, PassiveOpens, AttemptFails, EstabResets, UDP in, UDP out; -1 for a UDP counter it could not read).
type Dest ¶
type Dest struct {
Proto string `json:"proto,omitempty"`
IP string `json:"ip"`
Port int `json:"port"`
Class string `json:"class"`
Seen int `json:"seen"`
States map[string]int `json:"states,omitempty"`
}
Dest is one sampled destination. Outbound rows carry Proto and States.
type DockerEngine ¶
DockerEngine is Engine over a Docker (or Podman) socket.
func (*DockerEngine) Earners ¶
func (e *DockerEngine) Earners(ctx context.Context) ([]Container, error)
Earners is every running container this app manages.
func (*DockerEngine) EnsureImage ¶
func (e *DockerEngine) EnsureImage(ctx context.Context, image string) error
func (*DockerEngine) Network ¶
func (e *DockerEngine) Network(ctx context.Context, id string, earners map[string]bool) NetworkInfo
func (*DockerEngine) Probes ¶
func (e *DockerEngine) Probes(ctx context.Context) ([]ProbeRef, error)
func (*DockerEngine) Runtimes ¶
func (e *DockerEngine) Runtimes(ctx context.Context) map[string]RuntimeSpec
type Engine ¶
type Engine interface {
Earners(ctx context.Context) ([]Container, error)
Inspect(ctx context.Context, idOrName string) (Container, bool)
Runtimes(ctx context.Context) map[string]RuntimeSpec
Network(ctx context.Context, id string, earners map[string]bool) NetworkInfo
Probes(ctx context.Context) ([]ProbeRef, error)
Remove(ctx context.Context, id string) error
EnsureImage(ctx context.Context, image string) error
// RunProbe runs one helper and returns what it printed. It removes the
// helper on every path; leftover is true when Docker would not.
RunProbe(ctx context.Context, spec ProbeSpec) (out []byte, leftover bool, err error)
}
Engine is the Docker side of a round, so the round itself can be tested.
type Finding ¶
Finding is one alert a window raises. It never carries an address.
func AlertFindings ¶
AlertFindings is the mail and LAN alerts of one window. Mail: outbound TCP to a public address on a mail port, in any state. LAN: any outbound destination in LanAlertClasses.
type Group ¶
Group is one network namespace to probe and the earners that live in it. Kind "netns" joins the namespace; kind "cgroup" is an earner on the host network, whose sockets are matched by its cgroup instead.
type IPGeo ¶
type IPGeo struct {
Dir string
URL string
Client *http.Client
// contains filtered or unexported fields
}
IPGeo is the local table under the app's data directory.
func (*IPGeo) Build ¶
Build writes the table from TSV to a temp file, then moves it into place. A file too small, too large or too broken to be the real table leaves the existing one untouched.
func (*IPGeo) Lookup ¶
Lookup is the network and country of each address the table knows. A missing or unreadable table gives an empty map: unknown, never a wrong answer.
func (*IPGeo) NeedsRefresh ¶
NeedsRefresh is true when the table is missing or older than IPGeoMaxAge.
func (*IPGeo) Status ¶
func (g *IPGeo) Status() IPGeoStatus
Status says whether the table is there, how old it is and how many ranges it holds.
type IPGeoStatus ¶
type IPGeoStatus struct {
Present bool `json:"present"`
AgeSeconds *int64 `json:"ageSeconds"`
Rows *int64 `json:"rows"`
}
IPGeoStatus is what the page shows about the table.
type Network ¶
type Network struct {
ID string
Gateway string
IPAddress string
IPPrefixLen int
IPv6Gateway string
GlobalIPv6 string
IPv6PrefixLen int
}
Network is one network a container is attached to, as docker inspect shows it.
type NetworkContext ¶
type NetworkContext struct {
Gateways []string `json:"gateways"`
DNS []string `json:"dns"`
Subnets [][2]string `json:"subnets"`
Peers []string `json:"peers"`
}
NetworkContext is what the helper's classify needs: gateways, resolvers, subnets with their drivers, and the addresses of the earners beside it.
func BuildContext ¶
func BuildContext(owner Container, kind string, networks map[string]NetworkInfo, inspect func(id string) NetworkInfo) NetworkContext
BuildContext is the network context of a namespace owner. networks caches one inspect per network for the round. A host-network earner has no bridge: no gateways, subnets or peers.
type NetworkInfo ¶
NetworkInfo is a network's driver and the addresses of the earners on it.
type ProbeError ¶
type ProbeError struct{ Reason string }
ProbeError is a helper that produced no reading. Reason is a fixed code.
func (*ProbeError) Error ¶
func (e *ProbeError) Error() string
type ProbeResult ¶
type ProbeResult struct {
Netns string
Counters *[6]int64
EstMax *int64
SynSentMax *int64
UDPBound *int64
Remotes *int64
Truncated bool
ListenPorts []int
Out []Dest
In []Dest
Attribution string
Reason string
}
ProbeResult is what one helper printed, checked.
func ParseProbe ¶
func ParseProbe(raw []byte) (ProbeResult, error)
ParseProbe checks a helper's output. Every row that is not exactly what procnet prints is dropped, and the caps are applied again: the helper runs beside an earner, so its output is checked like anything else from outside.
type ProbeSpec ¶
type ProbeSpec struct {
Name string
OwnerID string
Kind string // "netns" or "cgroup"
Worker string
Context NetworkContext
}
ProbeSpec is one helper: which earner, how to reach its sockets, and what to tell the script.
type Runner ¶
type Runner struct {
Engine Engine
Worker string
Deltas *Deltas
// Now and Monotonic are swappable for tests.
Now func() time.Time
// contains filtered or unexported fields
}
Runner runs rounds. One round at a time; a round asked for while another runs is skipped.
type RuntimeSpec ¶
RuntimeSpec is one runtime docker info lists.
type Sample ¶
type Sample struct {
Container string `json:"container"`
Members []string `json:"members"`
NetworkMode string `json:"network_mode"`
Status string `json:"status"`
Reason string `json:"reason"`
Opens *int64 `json:"opens"`
PassiveOpens *int64 `json:"passive_opens"`
AttemptFails *int64 `json:"attempt_fails"`
Resets *int64 `json:"resets"`
UDPIn *int64 `json:"udp_in"`
UDPOut *int64 `json:"udp_out"`
DeltaSeconds *float64 `json:"delta_seconds"`
EstMax *int64 `json:"est_max"`
SynSentMax *int64 `json:"syn_sent_max"`
UDPBound *int64 `json:"udp_bound"`
ListenPorts []int `json:"listen_ports"`
RemotesTotal *int64 `json:"remotes_total"`
Truncated bool `json:"truncated"`
Out []Dest `json:"out"`
In []Dest `json:"in"`
}
Sample is one container's row for one window. Nil numbers are unknown, never zero.
func BuildSample ¶
BuildSample turns one group's helper reading into its sample.