Documentation
¶
Index ¶
- Constants
- Variables
- func MaskIP(ip string) string
- func MaskIPs(ips []string) []string
- func SumDiskIOPS(disks []DiskRate) (readIOPS, writeIOPS float64)
- func SumDiskThroughput(disks []DiskRate) (readMBs, writeMBs float64)
- type Action
- type ActiveSession
- type AnalysisResult
- type AppBehaviorAnomaly
- type AppDockerContainer
- type AppIdentity
- type AppInstance
- type AppMetrics
- type AppRCAFinding
- type AppResourceShare
- type AuditRule
- type BaselineAnomaly
- type BeaconIndicator
- type BigDir
- type BigFile
- type BlameEntry
- type CPUMetrics
- type CPUTimes
- type Capacity
- type CausalDAG
- type CausalEdge
- type CausalNode
- type CausalNodeType
- type CgThrottleEntry
- type CgroupMetrics
- type CgroupRate
- type CloseWaitLeaker
- type CloseWaitTrend
- type CollectionHealth
- type ConntrackDissection
- type ConntrackIPCount
- type ConntrackStats
- type ConntrackTimeouts
- type CoverageInfo
- type CrossCorrelation
- type DNSAnomalyEntry
- type DNSTunnelIndicator
- type DatabaseInfo
- type DegradationWarning
- type DeletedOpenFile
- type DetectedService
- type DiagFinding
- type DiagMetrics
- type DiagSeverity
- type DirectReclaimEntry
- type DiskRate
- type DiskStats
- type DiskType
- type DockerContainer
- type DockerContainerNet
- type DockerMount
- type DockerPort
- type DockerStack
- type DockerStackNetwork
- type Domain
- type DomainScore
- type DotNetProcessMetrics
- type Entity
- type EntityGraph
- type EntityKind
- type EphemeralPorts
- type Event
- type Evidence
- type EvidenceCheck
- type ExecEventEntry
- type ExhaustionPrediction
- type FDStats
- type Fact
- type FactConfidence
- type FactKind
- type FactSeverity
- type FailedAuthSource
- type FilelessProcess
- type FleetAgentConfig
- type FleetHeartbeat
- type FleetHost
- type FleetHubConfig
- type FleetIncident
- type FleetProcess
- type FlowRateEntry
- type GPUDevice
- type GPUProcess
- type GPUSnapshot
- type GateResult
- type GlobalMetrics
- type GoldenSignalSummary
- type GuardStatus
- type HAProxyInfo
- type HealthCheckMetrics
- type HealthLevel
- type HealthProbeResult
- type HostIncident
- type HostStatus
- type ImpactScore
- type IncidentDiff
- type IncidentFrame
- type IncidentUpdateType
- type JA3Entry
- type JournalFinding
- type K8sNodeInfo
- type KeepalivedInfo
- type LabelKind
- type LoadAvg
- type LogExcerpt
- type LogMetrics
- type MemoryMetrics
- type MetricChange
- type MetricCorrelation
- type ModLoadEntry
- type MountRate
- type MountStats
- type Narrative
- type NetRate
- type NetworkStats
- type NewListeningPort
- type OOMKillEntry
- type OptDomain
- type OutboundEntry
- type Owner
- type OwnerAttribution
- type PHPFPMApp
- type PHPFPMFSFinding
- type PHPFPMFunctionHit
- type PHPFPMIPHit
- type PHPFPMIPURIHit
- type PHPFPMIssue
- type PHPFPMMaster
- type PHPFPMMetrics
- type PHPFPMScriptHit
- type PHPFPMURIHit
- type PHPFPMWebShellSuspect
- type PHPFPMWorker
- type PSILine
- type PSIMetrics
- type PSIResource
- type PktDropEntry
- type PktDropLocation
- type PktDropProto
- type PortScanEntry
- type PortUser
- type ProbeResult
- type ProcessAnomaly
- type ProcessMetrics
- type ProcessRate
- type ProxmoxDiskConf
- type ProxmoxFirewall
- type ProxmoxHAResource
- type ProxmoxMetrics
- type ProxmoxNetConf
- type ProxmoxReplication
- type ProxmoxStorage
- type ProxmoxVM
- type PtraceEventEntry
- type RCAEntry
- type RateSnapshot
- type RemoteIPStats
- type ReverseShellProc
- type RoleScore
- type RuleStatus
- type RunbookMatch
- type RuntimeEntry
- type RuntimeMetrics
- type RuntimeProcessMetrics
- type SMARTDisk
- type SUIDBinary
- type SaturationDetail
- type SecurityMetrics
- type SentinelConnLatEntry
- type SentinelData
- type SentinelRetransEntry
- type ServerIdentity
- type ServerProfile
- type ServerRole
- type ServiceCensus
- type ServiceDiag
- type ServiceLogStats
- type Severity
- type Snapshot
- type SockStateEntry
- type SocketStats
- type SoftIRQStats
- type SynFloodEntry
- type SysInfo
- type SystemChange
- type TCPConnState
- type TCPFlagAnomaly
- type TCPMetrics
- type TCPResetEntry
- type TemporalChain
- type TemporalEvent
- type TimelineEntry
- type TraceSample
- type UDPMetrics
- type USECheck
- type VMStatMetrics
- type VPNInfo
- type VerificationTier
- type VerifiedCause
- type Warning
- type WatchdogState
- type WebsiteInfo
- type WebsiteMetrics
- type ZScoreAnomaly
Constants ¶
const ( // POST: agent → hub, per tick FleetEndpointHeartbeat = "/v1/heartbeat" // POST: agent → hub, on incident start / update / resolve FleetEndpointIncident = "/v1/incident" // GET: UI / agent → hub, list all known hosts FleetEndpointHosts = "/v1/hosts" // GET: UI → hub, get one host's latest state FleetEndpointHost = "/v1/host/" // + hostname // GET: UI → hub, list recent incidents across fleet FleetEndpointIncidents = "/v1/incidents" // GET: UI → hub, stream events (SSE) FleetEndpointStream = "/v1/stream" // Auth header — agent sends its token here FleetAuthHeader = "X-XTop-Token" // Default hub listen port. Chosen to avoid collisions with common services // (Prometheus 9100, Elasticsearch 9200, Grafana 3000). Override at runtime // with `xtop hub --listen=:NNNN` or `XTOP_HUB_LISTEN=:NNNN`. FleetDefaultPort = 9898 )
const ( NVMeWarnSpare = 1 << 0 // available spare below threshold NVMeWarnTemperature = 1 << 1 // temperature above critical threshold NVMeWarnReliability = 1 << 2 // reliability degraded (media/internal errors) NVMeWarnReadOnly = 1 << 3 // media placed in read-only mode NVMeWarnBackup = 1 << 4 // volatile memory backup device failed )
NVMeCriticalWarning bitmap constants.
const CurrentSchemaVersion = 1
CurrentSchemaVersion is the on-disk schema version this engine writes. Frames with a higher version are rejected by the replay harness (operator must upgrade); lower versions can be migrated.
Variables ¶
var MaskIPsEnabled bool
MaskIPsEnabled controls whether IP addresses are masked in output.
Functions ¶
func SumDiskIOPS ¶ added in v0.50.0
SumDiskIOPS returns host-total read/write IOPS without double-counting LVM/RAID layers.
func SumDiskThroughput ¶ added in v0.50.0
SumDiskThroughput returns host-total read/write MB/s without double-counting LVM/RAID layers.
Types ¶
type ActiveSession ¶
type ActiveSession struct {
User string
TTY string
From string
LoginAt string
Idle string
Command string
}
ActiveSession represents a currently logged-in user.
type AnalysisResult ¶
type AnalysisResult struct {
Health HealthLevel
Confidence int // 0-100
// Coverage reports which collector mode produced this analysis and which
// signal classes were not observable (e.g. lean/fleet mode omits several
// collectors). Lets consumers avoid over-reading a reduced-signal verdict.
Coverage CoverageInfo `json:"coverage,omitempty"`
// PrimaryVerified is true when the formal verifier confirmed the primary
// bottleneck's cause (Tier A/B). The verifier is additive — Health is still
// score-driven — so when this is false on a degraded/critical result, the
// UI/API should label the verdict "unverified".
PrimaryVerified bool `json:"primary_verified"`
// Facts is the typed-evidence layer (NEXTGEN Phase 2). Each entry
// is a single observation with full provenance — see Fact in
// model/fact.go. Phase 2 emits these in parallel to the legacy
// Evidence/EvidenceV2 fields; Phase 4 makes them the canonical
// input to the verifier gates.
Facts []Fact `json:"facts,omitempty"`
// Entities is the host-local entity graph (NEXTGEN Phase 3) — a
// snapshot of the ownership tree at the time of this analysis.
// Verifier gates consult it for ownership-consistency + blast-
// radius checks. Built once per tick from snap.Processes + cgroups.
Entities *EntityGraph `json:"entities,omitempty"`
// VerifiedCauses is the verifier's output (NEXTGEN Phase 4). One
// entry per candidate cause that went through the multi-level
// gate system. Tier A entries are the only ones counted toward
// the engine's precision target — everything weaker is abstention.
VerifiedCauses []VerifiedCause `json:"verified_causes,omitempty"`
// Primary diagnosis
PrimaryBottleneck string
PrimaryScore int
PrimaryEvidence []string
PrimaryChain []string
PrimaryCulprit string
PrimaryPID int
PrimaryProcess string
PrimaryAppName string // resolved app name for primary culprit
// Sustained pressure tracking
Sustained bool // true if pressure persisted >10 ticks
SustainedTicks int // number of recent ticks with elevated pressure
// Next risk (early warning)
NextRisk string
// All RCA results ranked
RCA []RCAEntry
// Capacity headroom
Capacities []Capacity
// Top owners per subsystem
CPUOwners []Owner
MemOwners []Owner
IOOwners []Owner
NetOwners []Owner
// Warnings
Warnings []Warning
// Suggested actions
Actions []Action
// Causal chain
CausalChain string
CausalDAG *CausalDAG // structured causal chain (nil = not computed)
// Anomaly tracking
AnomalyStartedAgo int // seconds since primary bottleneck first appeared (0=not active)
AnomalyTrigger string // which signal first crossed threshold
CulpritSinceAgo int // seconds since culprit became top consumer
// Deployment correlation: process that started near anomaly onset
RecentDeploy string // e.g. "node server.js"
RecentDeployPID int // PID of recently deployed process
RecentDeployAge int // seconds since the process started
// Hidden latency detection (metrics look fine but threads are waiting)
HiddenLatency bool // true if hidden latency detected
HiddenLatencyDesc string // human-readable explanation
HiddenLatencyPct float64 // estimated off-CPU wait percentage
HiddenLatencyComm string // top waiting process
// Stability tracking
StableSince int // seconds system has been continuously OK (0=not stable)
BiggestChange string // description of biggest metric change in last 30s
BiggestChangePct float64 // magnitude of the biggest change
TopChanges []MetricChange // top N biggest changes for "what changed?" display
// Predictive exhaustion
Exhaustions []ExhaustionPrediction
// Slow degradation warnings
Degradations []DegradationWarning
// CLOSE_WAIT leaker data (for actions access)
CloseWaitLeakers []CloseWaitLeaker
// DiskGuard
DiskGuardMounts []MountRate
DiskGuardWorst string // worst state across all mounts: "OK", "WARN", "CRIT"
DiskGuardMode string // "Monitor", "Contain", "Action"
// Watchdog auto-trigger state
Watchdog WatchdogState
// System identity
SysInfo *SysInfo
// Narrative engine output
Narrative *Narrative
// ConfigDriftSuggestions holds "SUGGESTED: ..." remediation lines produced
// by config-drift correlation, which runs before the narrative is built.
// They are appended to Narrative.Evidence after BuildNarrative. Transient.
ConfigDriftSuggestions []string `json:"-"`
// Temporal causality chain
TemporalChain *TemporalChain
// Cross-signal correlation
CrossCorrelations []CrossCorrelation
// Blame attribution
Blame []BlameEntry
// Statistical intelligence (v0.31.0)
BaselineAnomalies []BaselineAnomaly // Evidence deviating from learned baseline
Correlations []MetricCorrelation // Discovered metric correlations
ZScoreAnomalies []ZScoreAnomaly // Statistically unusual values vs recent window
ProcessAnomalies []ProcessAnomaly // Processes deviating from learned profile
AppAnomalies []AppBehaviorAnomaly `json:"app_anomalies,omitempty"` // Phase 4: per-app baseline deviations
AppRCA []AppRCAFinding `json:"app_rca,omitempty"` // per-app rule-engine findings
ProbeResults []ProbeResult `json:"probe_results,omitempty"` // Phase 6: active investigation captures
// Lifecycle echo from the incident recorder. Populated each tick by the
// engine after IncidentRecorder.Record so downstream consumers (fleet
// client, trace dump) don't need to plumb the recorder through.
// IncidentState is one of "" (no incident), "suspected", "confirmed", "resolved".
IncidentState string `json:"incident_state,omitempty"`
IncidentConfirmedAt time.Time `json:"incident_confirmed_at,omitempty"`
GoldenSignals *GoldenSignalSummary // Approximated Golden Signal metrics
// USE Method checklist (v0.36.6)
USEChecks []USECheck `json:"use_checks,omitempty"`
// Change detection (v0.36.6)
Changes []SystemChange `json:"changes,omitempty"`
// Impact quantification (v0.36.6)
ImpactSummary string `json:"impact_summary,omitempty"`
// Historical context — short human-readable summary from past similar incidents.
HistoryContext string `json:"history_context,omitempty"`
// IncidentDiff is a structured comparison of the current incident against
// the last N similar ones (same signature). Nil if no prior matches exist.
IncidentDiff *IncidentDiff `json:"incident_diff,omitempty"`
// Runbook is the best-matching operator runbook for this incident (if any
// were loaded from ~/.xtop/runbooks/). The engine populates this field;
// the UI reads it to show a "see runbook: <name>" hint and can load the
// full content via the engine's RunbookLibrary.Lookup(path).
Runbook *RunbookMatch `json:"runbook,omitempty"`
// Guard reports what the resource guard decided this tick (level, skip
// flags, reason). Populated only when XTOP_GUARD=1. Nil otherwise so
// status lines cleanly hide the indicator when it's off.
Guard *GuardStatus `json:"guard,omitempty"`
// TraceSamples are OpenTelemetry trace summaries that overlap the current
// incident window, loaded by the engine's TraceCorrelator from a simple
// JSONL feed. xtop never speaks OTLP directly — any existing OTel pipeline
// can be pointed at ~/.xtop/otel-samples.jsonl to enable correlation.
TraceSamples []TraceSample `json:"trace_samples,omitempty"`
// LogExcerpts are notable lines pulled from app log files during an
// incident. The engine's log tailer scans a small set of well-known paths
// (nginx/apache error logs, mysql/postgres logs, systemd journal of the
// culprit unit), filters for severity keywords, and attaches the top
// matches here so the UI can show "the RCA says mysql, and here's what
// mysql's error.log said at the same moment."
LogExcerpts []LogExcerpt `json:"log_excerpts,omitempty"`
// Set when UI is showing a pinned result after recovery (sticky RCA).
// Value is seconds since health returned to OK. 0 = live incident.
PinnedResolvedSec int `json:"pinned_resolved_sec,omitempty"`
// Baseline readiness: 0.0 = all metrics warming up, 1.0 = fully ready
BaselineReadiness float64 `json:"baseline_readiness,omitempty"`
// Forecast warning from Holt-Winters trend prediction
ForecastWarning string `json:"forecast_warning,omitempty"` // e.g. "Memory will hit 95% in ~45s at current rate"
// Cross-host correlation: related incidents on other hosts
CrossHostCorrelation string `json:"cross_host_correlation,omitempty"` // e.g. "Host db-server also reports IO bottleneck (score 78)"
// JournalFindings are structured log findings for the top suspect services
// under active RCA investigation (Tier-1 journal evidence, P2.4).
// Each entry is a single finding extracted from systemd journald.
JournalFindings []DiagFinding `json:"journal_findings,omitempty"`
}
type AppBehaviorAnomaly ¶
type AppBehaviorAnomaly struct {
AppName string `json:"app_name"`
CgroupPath string `json:"cgroup_path,omitempty"`
Metric string `json:"metric"` // "cpu_pct", "rss_mb"
Current float64 `json:"current"`
HourBaseline float64 `json:"hour_baseline"` // mean for this hour-of-week
HourStdDev float64 `json:"hour_stddev"`
Sigma float64 `json:"sigma"`
HourOfWeek int `json:"hour_of_week"` // 0..167 (mon 00:00 = 0)
Note string `json:"note,omitempty"` // e.g. "frozen-during-incident", "cold-start"
}
AppBehaviorAnomaly is a per-app baseline deviation, anchored on cgroup + hour-of-week so "Postgres is busy at 09:00 Monday" is normal but "Postgres is busy at 03:00 Sunday" is flagged. Phase 4: per-app baselines.
type AppDockerContainer ¶
type AppDockerContainer struct {
ID string `json:"id"`
Name string `json:"name"`
Image string `json:"image"`
State string `json:"state"` // running, exited, paused, etc.
Status string `json:"status"` // "Up 7 weeks", "Exited (0) 12 months ago"
Health string `json:"health"` // healthy, unhealthy, none
CPUPct float64 `json:"cpu_pct"`
MemUsedBytes float64 `json:"mem_used_bytes"`
MemLimitBytes float64 `json:"mem_limit_bytes"`
MemPct float64 `json:"mem_pct"`
NetRxBytes float64 `json:"net_rx_bytes"`
NetTxBytes float64 `json:"net_tx_bytes"`
BlockRead float64 `json:"block_read"`
BlockWrite float64 `json:"block_write"`
PIDs int `json:"pids"`
RestartCount int `json:"restart_count"`
ExitCode int `json:"exit_code"`
// From container inspect
Ports []DockerPort `json:"ports,omitempty"`
Mounts []DockerMount `json:"mounts,omitempty"`
Networks []DockerContainerNet `json:"networks,omitempty"`
RestartPolicy string `json:"restart_policy"` // no/always/unless-stopped/on-failure
User string `json:"user"`
Privileged bool `json:"privileged"`
Entrypoint string `json:"entrypoint"`
Command string `json:"command"`
MemLimit uint64 `json:"mem_limit"` // bytes, 0 = unlimited
CPUQuota float64 `json:"cpu_quota"` // cores, 0 = unlimited
CreatedAt string `json:"created_at"`
HasHealthChk bool `json:"has_health_check"`
RWLayerSize int64 `json:"rw_layer_size"`
StackName string `json:"stack_name"` // compose project or "standalone"
StackType string `json:"stack_type"` // compose/swarm/k8s/standalone
ImageSize int64 `json:"image_size"`
}
AppDockerContainer holds per-container stats.
type AppIdentity ¶
type AppIdentity struct {
PID int
Comm string // raw comm from /proc/PID/stat
AppName string // resolved application name ("Elasticsearch")
AppVersion string // version if detectable
BinaryPath string // /proc/PID/exe target
Cmdline string // full cmdline (truncated to 256 chars)
ServiceUnit string // systemd unit name
ContainerID string // container ID prefix (12 chars)
ParentComm string // parent process comm
ParentPID int
CgroupPath string
DisplayName string // pre-formatted: "Elasticsearch [java, elasticsearch.service]"
}
AppIdentity holds the resolved application identity for a process.
type AppInstance ¶
type AppInstance struct {
ID string `json:"id"` // "mysql-1", "nginx-0"
AppType string `json:"app_type"` // "mysql", "nginx", etc.
DisplayName string `json:"display_name"` // "MySQL (1)", "Nginx"
PID int `json:"pid"`
Port int `json:"port"`
Status string `json:"status"` // "active"
Version string `json:"version"`
UptimeSec int64 `json:"uptime_sec"`
// Tier 1: process-level (always available)
CPUPct float64 `json:"cpu_pct"`
RSSMB float64 `json:"rss_mb"`
Threads int `json:"threads"`
FDs int `json:"fds"`
Connections int `json:"connections"`
// apps are collected. See docs/USAGE.md §6 for the SRE framing.
Share AppResourceShare `json:"share,omitempty"`
// Tier 2: deep metrics (needs credentials)
HasDeepMetrics bool `json:"has_deep_metrics"`
DeepMetrics map[string]string `json:"deep_metrics,omitempty"`
// Health
HealthScore int `json:"health_score"`
HealthIssues []string `json:"health_issues,omitempty"`
// Runtime is where the app runs: "docker", "containerd", "podman", "k8s",
// "lxc", or "native" (bare host process). Derived from the PID's cgroup.
Runtime string `json:"runtime,omitempty"`
// DeepPending is true for a tier-1-lite instance (first tick or guardian
// throttling deep probes): version and connection count were NOT measured
// yet, so consumers must render them as "—", not a misleading 0.
DeepPending bool `json:"deep_pending,omitempty"`
// Config
ConfigPath string `json:"config_path,omitempty"`
NeedsCreds bool `json:"needs_creds"`
// Docker containers (only for Docker app type)
Containers []AppDockerContainer `json:"containers,omitempty"`
// Docker stacks (grouped containers)
Stacks []DockerStack `json:"stacks,omitempty"`
// Docker orchestration type: "standalone", "compose", "swarm", "k8s", "mixed"
OrchestrationType string `json:"orchestration_type,omitempty"`
// Websites (for hosting panels, nginx, apache, php-fpm)
Websites []WebsiteMetrics `json:"websites,omitempty"`
}
AppInstance represents a detected application instance.
type AppMetrics ¶
type AppMetrics struct {
Instances []AppInstance `json:"instances,omitempty"`
}
AppMetrics holds all detected application instances.
type AppRCAFinding ¶
type AppRCAFinding struct {
App string `json:"app"` // app instance ID, e.g. "mongodb-0"
AppType string `json:"app_type"` // mongodb / mysql / redis / etc.
Rule string `json:"rule"` // rule ID, stable for filtering / dedup
Severity string `json:"severity"` // info | warn | crit
Title string `json:"title"` // one-line headline
Detail string `json:"detail"` // measured value + threshold context
Action string `json:"action,omitempty"` // recommended next step
Metric string `json:"metric,omitempty"`
Value float64 `json:"value,omitempty"`
Threshold float64 `json:"threshold,omitempty"`
}
AppRCAFinding is one diagnostic conclusion produced by the per-app rule engine. The engine runs purely against already-collected DeepMetrics — no new probes, no subprocesses. Findings are produced cheaply (sub- millisecond) so they're safe to surface every tick.
Severity: "info" (worth noting), "warn" (action recommended), "crit" (likely cause of an active incident).
type AppResourceShare ¶
type AppResourceShare struct {
// Absolute per-dimension usage
// Share-of-capacity (0..100) per dimension
// Headroom — what's still available on THIS host after this app
// Rank across all apps (1 = highest consumer on that dimension).
// Zero = not ranked (fewer than N apps on that dimension).
// Not a sum of dimensions — it's the engine's ImpactScore, already a
// well-calibrated "how much is this contributing to pain" measure.
Impact float64 `json:"impact"`
// BottleneckShare is the % contribution this app makes to the CURRENT
// primary bottleneck dimension. Populated only when an incident is
// active. Lets the UI answer "who's causing 72% of the IO pressure?"
}
AppResourceShare is the SRE-actionable per-app resource view.
Design principle: never collapse dimensions into a single composite score. Each dimension is reported independently in capacity terms (cores, GB, MB/s, #active conns) with a rank across apps and, when an incident is firing, a contribution-share of the bottleneck dimension.
All fields are derived from per-tick data — no historical state required beyond what the rates snapshot already carries. Per-app baselines (7d Δ) are a planned follow-up and will land on this struct later.
type AuditRule ¶
type AuditRule struct {
Domain OptDomain `json:"domain"`
Name string `json:"name"`
Description string `json:"description"`
Current string `json:"current"`
Recommended string `json:"recommended"`
Impact string `json:"impact"`
Fix string `json:"fix,omitempty"` // one-liner remediation command
Status RuleStatus `json:"status"`
Weight int `json:"weight"` // 1=minor, 5=important, 10=critical
}
AuditRule is a single optimization check result.
type BaselineAnomaly ¶
type BaselineAnomaly struct {
EvidenceID string // e.g. "cpu.busy"
Value float64 // current value
Baseline float64 // EWMA mean
StdDev float64 // sqrt(EWMA variance)
ZScore float64 // (value - mean) / stddev
Sigma float64 // how many sigma above baseline
}
BaselineAnomaly represents an evidence value that deviates from its learned EWMA baseline.
type BeaconIndicator ¶
type BeaconIndicator struct {
PID int `json:"pid"`
Comm string `json:"comm"`
DstIP string `json:"dst_ip"`
DstPort uint16 `json:"dst_port"`
AvgIntervalSec float64 `json:"avg_interval_sec"`
Jitter float64 `json:"jitter"`
SampleCount int `json:"sample_count"`
}
BeaconIndicator holds BPF-detected C2 beacon-like periodic connection indicators.
type BigDir ¶ added in v0.50.0
type BigDir struct {
Path string
SizeBytes uint64
FileCount int // number of regular files counted in the subtree
}
BigDir represents a directory subtree consuming significant disk space. SizeBytes is the recursive total of files walked beneath Path (du-style), bounded by the scanner's stat budget — i.e. a lower bound, not exact du(1).
type BlameEntry ¶
type BlameEntry struct {
Comm string
AppName string // resolved app name from identity
PID int
CgroupPath string
Metrics map[string]string // "cpu" → "45.2%", "io" → "12 MB/s"
ImpactPct float64
}
BlameEntry identifies a top offending process or cgroup for the current bottleneck.
type CPUMetrics ¶
type CPUMetrics struct {
Total CPUTimes
PerCPU []CPUTimes
LoadAvg LoadAvg
NumCPUs int
// CtxSwitches is the cumulative total system context switches
// from /proc/stat's "ctxt N" line — the canonical kernel counter.
// The prior implementation estimated this from per-process
// VoluntaryCtxSwitches+NonVoluntaryCtxSwitches sums, which
// chronically undercounts on hosts with many kernel threads
// (the per-pid scan can miss short-lived workers and PSI-stalled
// ksoftirqd/k* tasks). Use this for the rate computation; fall
// back to the per-process sum only if this is zero.
CtxSwitches uint64
}
CPUMetrics holds all CPU-related metrics.
type CPUTimes ¶
type CPUTimes struct {
User uint64
Nice uint64
System uint64
Idle uint64
IOWait uint64
IRQ uint64
SoftIRQ uint64
Steal uint64
Guest uint64
GuestNice uint64
}
CPUTimes holds CPU time counters from /proc/stat (in jiffies/ticks).
type Capacity ¶
type Capacity struct {
Label string
Pct float64 // % remaining (0-100)
Current string // current value string
Limit string // limit/max string
}
Capacity represents headroom for one resource.
type CausalDAG ¶
type CausalDAG struct {
Nodes []CausalNode
Edges []CausalEdge
LinearChain string // human-readable "→"-joined string
}
CausalDAG represents a directed acyclic graph of causal relationships.
type CausalEdge ¶
CausalEdge is a directed edge in the causal DAG.
type CausalNode ¶
type CausalNode struct {
ID string
Label string
Type CausalNodeType
Domain Domain
EvidenceIDs []string
}
CausalNode is a node in the causal DAG.
type CausalNodeType ¶
type CausalNodeType string
CausalNodeType identifies a node's role in the causal DAG.
const ( CausalRootCause CausalNodeType = "root_cause" CausalIntermediate CausalNodeType = "intermediate" CausalSymptom CausalNodeType = "symptom" )
type CgThrottleEntry ¶
CgThrottleEntry holds a BPF-traced cgroup CPU throttle event.
type CgroupMetrics ¶
type CgroupMetrics struct {
Path string
Name string // leaf name for display
// CPU
UsageUsec uint64
UserUsec uint64
SystemUsec uint64
ThrottledUsec uint64
NrThrottled uint64
NrPeriods uint64
// Memory
MemCurrent uint64
MemLimit uint64 // max or high, whichever is set
MemSwap uint64
OOMKills uint64
PgFault uint64
PgMajFault uint64
// IO (aggregated across devices)
IORBytes uint64
IOWBytes uint64
IORIOs uint64
IOWIOs uint64
// PIDs
PIDCount uint64
PIDLimit uint64
// Kubernetes attribution — populated when the cgroup is under
// kubepods.slice (or the cgroup-v1 equivalent). Empty on non-k8s hosts.
PodName string
PodNamespace string
ContainerName string
PodQoS string // "Guaranteed", "Burstable", "BestEffort"
}
CgroupMetrics holds metrics for a single cgroup.
type CgroupRate ¶
type CgroupRate struct {
Path string
Name string
CPUPct float64
ThrottlePct float64
MemPct float64
IORateMBs float64
IOWRateMBs float64
OOMKillDelta uint64 // OOM kills since last tick (delta, not cumulative)
}
CgroupRate holds computed per-cgroup rates.
type CloseWaitLeaker ¶
type CloseWaitLeaker struct {
PID int
Comm string
Count int // CW sockets held
OldestAge int // seconds
NewestAge int // seconds
TopRemotes []string // up to 3 remote IPs
}
CloseWaitLeaker holds per-PID CLOSE_WAIT socket attribution.
type CloseWaitTrend ¶
CloseWaitTrend holds CLOSE_WAIT growth trend data.
type CollectionHealth ¶
CollectionHealth tracks the health of individual collectors in a collection cycle.
type ConntrackDissection ¶
type ConntrackDissection struct {
Available bool
TCPCount int
UDPCount int
ICMPCount int
OtherCount int
AgeLt10s int // TTL remaining < 10s
Age10s60s int // 10s-60s
Age1m5m int // 1m-5m
AgeGt5m int // >= 5m
TopSrcIPs []ConntrackIPCount
TopDstIPs []ConntrackIPCount
CTStates map[string]int // "ESTABLISHED" -> count
TotalParsed int
}
ConntrackDissection holds parsed /proc/net/nf_conntrack data.
type ConntrackIPCount ¶
ConntrackIPCount holds an IP address and its connection count.
type ConntrackStats ¶
type ConntrackStats struct {
Count uint64
Max uint64
Buckets uint64 // hash table buckets (ideal max ~= buckets * 4)
Found uint64
Invalid uint64
Insert uint64
InsertFailed uint64 // failed inserts (table full)
Delete uint64
Drop uint64
EarlyDrop uint64 // evicted before timeout
SearchRestart uint64 // hash contention / CPU pressure
}
ConntrackStats holds conntrack data.
type ConntrackTimeouts ¶
type ConntrackTimeouts struct {
Available bool
Established int // default 432000 (5 days!)
TimeWait int // default 120
Close int
CloseWait int
SynSent int
SynRecv int
FinWait int
LastAck int
}
ConntrackTimeouts holds TCP timeout values from sysctl.
type CoverageInfo ¶ added in v0.50.0
type CoverageInfo struct {
Mode string `json:"mode"` // "full" | "lean"
OmittedSignals []string `json:"omitted_signals,omitempty"` // collectors not run in this mode
}
AnalysisResult is the full output of one analysis cycle. CoverageInfo describes the signal coverage behind an AnalysisResult.
type CrossCorrelation ¶
type CrossCorrelation struct {
Cause string `json:"cause"` // evidence ID of the leading signal
Effect string `json:"effect"` // evidence ID of the lagging signal
LeadTimeSec float64 `json:"lead_time_sec"` // seconds the cause preceded the effect
Confidence float64 `json:"confidence"` // 0-1 confidence in the correlation
Explanation string `json:"explanation"` // human-readable description
LeadSamples int `json:"lead_samples,omitempty"` // lag in samples where cross-correlation peaks (positive = cause leads)
LaggedR float64 `json:"lagged_r,omitempty"` // Pearson R at the best lag
}
CrossCorrelation describes a detected cause-effect relationship between two signals across different domains (e.g., memory reclaim causing IO latency).
type DNSAnomalyEntry ¶
type DNSAnomalyEntry struct {
PID int `json:"pid"`
Comm string `json:"comm"`
QueryCount uint64 `json:"query_count"`
AvgQueryLen int `json:"avg_query_len"`
TotalRespBytes uint64 `json:"total_resp_bytes"`
QueriesPerSec float64 `json:"queries_per_sec"`
}
DNSAnomalyEntry holds BPF-detected DNS anomaly indicators per process.
type DNSTunnelIndicator ¶
type DNSTunnelIndicator struct {
PID int `json:"pid,omitempty"`
Comm string `json:"comm,omitempty"`
SrcIP string `json:"src_ip,omitempty"`
DomainHash string `json:"domain_hash,omitempty"`
TXTRatio float64 `json:"txt_ratio"`
AvgQueryLen int `json:"avg_query_len"`
QueryRate float64 `json:"query_rate,omitempty"`
}
DNSTunnelIndicator holds BPF-detected DNS tunneling indicators per process.
type DatabaseInfo ¶
type DatabaseInfo struct {
Engine string `json:"engine"`
Name string `json:"name"`
SizeMB float64 `json:"size_mb,omitempty"`
Connections int `json:"connections,omitempty"`
ReplicaRole string `json:"replica_role,omitempty"`
}
DatabaseInfo holds discovered database information.
type DegradationWarning ¶
type DegradationWarning struct {
Metric string // e.g. "IO latency", "Memory reclaim"
Direction string // "rising", "falling"
Duration int // seconds the trend has persisted
Rate float64 // change per minute
Unit string // e.g. "ms/min", "%/min"
}
DegradationWarning describes a slow, sustained trend.
type DeletedOpenFile ¶
DeletedOpenFile represents a file that was deleted but is still held open.
type DetectedService ¶
type DetectedService struct {
Name string `json:"name"`
Version string `json:"version,omitempty"`
Ports []int `json:"ports,omitempty"`
Running bool `json:"running"`
Healthy bool `json:"healthy"`
Unit string `json:"unit,omitempty"`
BinaryPath string `json:"binary_path,omitempty"`
Extra map[string]string `json:"extra,omitempty"`
}
DetectedService represents a service discovered on the system.
type DiagFinding ¶
type DiagFinding struct {
Severity DiagSeverity
Category string // "config", "performance", "replication", "memory", "connections"
Summary string
Detail string
Advice string
}
DiagFinding holds a single diagnostic finding for a service.
type DiagMetrics ¶
type DiagMetrics struct {
Services []ServiceDiag
}
DiagMetrics holds diagnostics for all detected services.
type DiagSeverity ¶
type DiagSeverity string
DiagSeverity represents the severity of a diagnostic finding.
const ( DiagOK DiagSeverity = "ok" DiagInfo DiagSeverity = "info" DiagWarn DiagSeverity = "warn" DiagCrit DiagSeverity = "crit" )
type DirectReclaimEntry ¶
DirectReclaimEntry holds a BPF-traced direct reclaim stall per PID.
type DiskRate ¶
type DiskRate struct {
Name string
ReadMBs float64
WriteMBs float64
ReadIOPS float64
WriteIOPS float64
AvgAwaitMs float64
UtilPct float64
QueueDepth uint64
}
DiskRate holds computed per-device rates.
type DiskStats ¶
type DiskStats struct {
Name string
ReadsCompleted uint64
ReadsMerged uint64
SectorsRead uint64
ReadTimeMs uint64
WritesCompleted uint64
WritesMerged uint64
SectorsWritten uint64
WriteTimeMs uint64
IOsInProgress uint64
IOTimeMs uint64
WeightedIOMs uint64
// Extended (kernel 4.18+)
DiscardsCompleted uint64
DiscardTimeMs uint64
FlushesCompleted uint64
FlushTimeMs uint64
}
DiskStats holds per-device IO counters from /proc/diskstats.
type DockerContainer ¶
type DockerContainer struct {
ID string `json:"id"`
Name string `json:"name"`
Image string `json:"image"`
Status string `json:"status"`
Ports string `json:"ports,omitempty"`
Purpose string `json:"purpose,omitempty"` // inferred from image: "vpn", "web", "database", etc.
}
DockerContainer represents a discovered Docker container.
type DockerContainerNet ¶
type DockerContainerNet struct {
Name string `json:"name"`
IP string `json:"ip"`
Gateway string `json:"gateway"`
}
DockerContainerNet holds per-network info for a container.
type DockerMount ¶
type DockerMount struct {
Type string `json:"type"` // bind, volume, tmpfs
Source string `json:"source"` // host path or volume name
Target string `json:"target"` // container path
ReadOnly bool `json:"read_only"`
}
DockerMount holds a volume/bind mount.
type DockerPort ¶
type DockerPort struct {
ContainerPort int `json:"container_port"`
HostPort int `json:"host_port"`
HostIP string `json:"host_ip"`
Protocol string `json:"protocol"` // tcp/udp
}
DockerPort holds a published port mapping.
type DockerStack ¶
type DockerStack struct {
Name string `json:"name"` // compose project name or container name
Type string `json:"type"` // "compose", "swarm", "k8s", "standalone"
WorkingDir string `json:"working_dir"` // compose file directory
ComposeFile string `json:"compose_file"` // compose file path
Networks []DockerStackNetwork `json:"networks,omitempty"`
Containers []AppDockerContainer `json:"containers,omitempty"`
HealthScore int `json:"health_score"`
Issues []string `json:"issues,omitempty"`
}
DockerStack represents a group of containers from the same compose project or standalone.
type DockerStackNetwork ¶
type DockerStackNetwork struct {
Name string `json:"name"`
Driver string `json:"driver"`
Subnet string `json:"subnet"`
}
DockerStackNetwork holds network info for a stack.
type DomainScore ¶
type DomainScore struct {
Domain OptDomain `json:"domain"`
Score int `json:"score"` // 0-100
Issues int `json:"issues"`
Rules []AuditRule `json:"rules"`
}
DomainScore is the optimization score for one domain.
type DotNetProcessMetrics ¶
type DotNetProcessMetrics struct {
PID int `json:"pid"`
Comm string `json:"comm"`
GCHeapSizeMB float64 `json:"gc_heap_size_mb"`
Gen0GCCount uint64 `json:"gen0_gc_count"`
Gen1GCCount uint64 `json:"gen1_gc_count"`
Gen2GCCount uint64 `json:"gen2_gc_count"`
TimeInGCPct float64 `json:"time_in_gc_pct"`
AllocRateMBs float64 `json:"alloc_rate_mbs"`
ThreadPoolCount int `json:"threadpool_count"`
ThreadPoolQueue int `json:"threadpool_queue"`
ExceptionCount uint64 `json:"exception_count"`
MonitorLockCount uint64 `json:"monitor_lock_count"`
WorkingSetMB float64 `json:"working_set_mb"`
RequestsPerSec float64 `json:"requests_per_sec"`
CurrentRequests int `json:"current_requests"`
}
DotNetProcessMetrics holds .NET Core runtime metrics for a single process.
type Entity ¶
type Entity struct {
// ID is the canonical entity identifier. Stable across ticks for
// the lifetime of the entity. Required.
ID string `json:"id"`
// Kind buckets the entity for verifier dispatch.
Kind EntityKind `json:"kind"`
// Name is the human-readable label (process comm, service name,
// cgroup leaf path, etc.). May be ambiguous; ID is the unique key.
Name string `json:"name,omitempty"`
// OwnerID is this entity's parent in the ownership hierarchy.
// Empty for root entities (the host). Conventions:
// process → cgroup or runtime
// cgroup → parent cgroup, eventually "host"
// service → cgroup
// container → cgroup or pod
// pod → host
// socket → process (owning pid)
// mount → host
//
// Resolving an OwnerID via EntityGraph.Lookup yields the parent
// Entity, which has its own OwnerID, and so on up to "host".
OwnerID string `json:"owner_id,omitempty"`
// Tags carries free-form metadata: PID (when ID==cgroup), CPU%
// rollup, RSS bytes, port number, mount type, etc. Verifier gates
// read these for the ownership-consistency check.
Tags map[string]string `json:"tags,omitempty"`
}
Entity is one node in the host-local entity graph. Kept flat + JSON- serializable so it can ship over the fleet wire and survive replay.
ID conventions (matches Fact.EntityID format):
"host" "pid:1234" "cgroup:/sys/fs/cgroup/system.slice/mongod.service" "service:mongod" "container:docker/abc123def" "mount:/var/lib/mysql" "socket:tcp/127.0.0.1:27017"
type EntityGraph ¶
type EntityGraph struct {
// Entities is the flat list, ordered for deterministic serialization.
Entities []Entity `json:"entities"`
// contains filtered or unexported fields
}
EntityGraph is the snapshot-scoped collection of Entities and a fast lookup index. Built once per RCA tick by BuildEntityGraph(snap), read many times by detectors + verifier gates.
Not thread-safe — graphs are owned by a single AnalysisResult and not shared across goroutines.
func (*EntityGraph) Add ¶
func (g *EntityGraph) Add(e Entity)
Add inserts an entity into the graph. If an entity with the same ID already exists, the new one REPLACES it — callers must dedupe upstream if that's not desired.
func (*EntityGraph) AncestorChain ¶
func (g *EntityGraph) AncestorChain(id string) []*Entity
AncestorChain returns the chain of ancestors from `id` (exclusive) up to the root. Empty slice if entity is absent or has no owner. Stops at the first cycle (defensive — graphs should be acyclic but pathological cgroup setups have produced cycles in the wild).
func (*EntityGraph) Len ¶
func (g *EntityGraph) Len() int
Len returns the number of entities in the graph.
func (*EntityGraph) Lookup ¶
func (g *EntityGraph) Lookup(id string) *Entity
Lookup returns the entity with the given ID, or nil if absent. O(1).
func (*EntityGraph) Owner ¶
func (g *EntityGraph) Owner(id string) *Entity
Owner walks one step up the ownership chain. Returns nil if entity is absent, has no owner, or owner has been pruned.
func (*EntityGraph) Reindex ¶
func (g *EntityGraph) Reindex()
Reindex rebuilds the byID map. Call after deserializing an EntityGraph from JSON or after bulk-appending to Entities without using Add().
Note: The index rebuild is O(n) by design. This is acceptable since Reindex() is called only on deserialization and bulk-add paths, not on hot lookup paths. Only optimize this if profiling shows it to be a bottleneck (YAGNI).
type EntityKind ¶
type EntityKind string
EntityKind classifies an Entity by what it represents. Verifier gates dispatch on Kind — e.g. "ownership-consistency" only applies if the claimed root Entity's Kind is one that can own a resource (process, cgroup, container).
const ( EntityKindProcess EntityKind = "process" EntityKindCgroup EntityKind = "cgroup" EntityKindService EntityKind = "service" // systemd unit / aaPanel app / docker container — populated later EntityKindContainer EntityKind = "container" // docker / podman / lxc EntityKindPod EntityKind = "pod" // k8s pod (parent cgroup with multiple containers) EntityKindRuntime EntityKind = "runtime" // go/jvm/dotnet/node/python process group EntityKindMount EntityKind = "mount" // /var, /home, etc. EntityKindSocket EntityKind = "socket" // TCP/UDP endpoint EntityKindDependency EntityKind = "dependency" // remote service this host depends on EntityKindHost EntityKind = "host" // the host itself, root of the graph )
type EphemeralPorts ¶
type EphemeralPorts struct {
RangeLo int // from /proc/sys/net/ipv4/ip_local_port_range
RangeHi int
InUse int // count of connections with local port in ephemeral range
TimeWaitIn int // TIME_WAIT specifically in ephemeral range
EstablishedIn int // ESTABLISHED in ephemeral range
CloseWaitIn int // CLOSE_WAIT in ephemeral range
SynSentIn int // SYN_SENT in ephemeral range
TopUsers []PortUser
}
EphemeralPorts holds ephemeral port usage data.
type Event ¶
type Event struct {
ID string `json:"id"`
StartTime time.Time `json:"start_time"`
EndTime time.Time `json:"end_time,omitempty"`
Duration int `json:"duration_sec,omitempty"`
PeakHealth HealthLevel `json:"peak_health"`
Bottleneck string `json:"bottleneck"`
PeakScore int `json:"peak_score"`
Evidence []string `json:"evidence,omitempty"`
CausalChain string `json:"causal_chain,omitempty"`
CulpritCgroup string `json:"culprit_cgroup,omitempty"`
CulpritProcess string `json:"culprit_process,omitempty"`
CulpritPID int `json:"culprit_pid,omitempty"`
PeakCPUBusy float64 `json:"peak_cpu_busy,omitempty"`
PeakMemUsedPct float64 `json:"peak_mem_used_pct,omitempty"`
PeakIOPSI float64 `json:"peak_io_psi,omitempty"`
Active bool `json:"active"`
Timeline []TimelineEntry `json:"timeline,omitempty"`
}
Event represents a detected performance incident.
type Evidence ¶
type Evidence struct {
ID string // e.g. "io.psi.some", "mem.available.low"
Message string // human-readable description
Window string // time window e.g. "avg10", "1s"
Domain Domain // resource domain
Severity Severity // severity level
Strength float64 // 0..1 normalized signal strength
Confidence float64 // 0..1 measurement confidence
Value float64 // raw measured value
Threshold float64 // critical threshold
Measured bool // true if from direct measurement (BPF/counter)
Owners []OwnerAttribution
Tags map[string]string // e.g. "weight": "psi", "device": "sda"
// Sustained-duration tracking (Phase 1: verdict discipline).
// FirstSeenAt is the wall-clock time this evidence ID first fired in the
// current incident; zero value means "first-tick onset".
// SustainedForSec is now() - FirstSeenAt at the moment the verdict is built.
// Stamped by stampSustainedDurations() in engine, using History.signalOnsets.
FirstSeenAt time.Time `json:"first_seen_at,omitempty"`
SustainedForSec float64 `json:"sustained_for_sec,omitempty"`
}
Evidence is a v2 structured evidence object with smooth scoring.
type EvidenceCheck ¶
type EvidenceCheck struct {
Group string // evidence group name (e.g. "PSI", "D-state", "Disk latency")
Label string // human-readable check (e.g. "IO PSI full avg10=0.12")
Passed bool // did this signal fire?
Value string // current value for display
Confidence string // "H" = BPF tracepoint, "M" = /proc counter, "L" = heuristic/derived
Source string // "procfs", "sysfs", "bpf", "derived"
Strength float64 // 0.0-1.0 signal strength
}
EvidenceCheck is a single signal check with pass/fail.
type ExecEventEntry ¶
type ExecEventEntry struct {
PID uint32
PPID uint32
UID uint32
Comm string
Filename string
Count uint64
Timestamp int64
}
ExecEventEntry holds a BPF-traced process execution event.
type ExhaustionPrediction ¶
type ExhaustionPrediction struct {
Resource string // "FD", "Memory", "Swap", "Conntrack"
CurrentPct float64 // current usage percent
TrendPerS float64 // percentage-point change per second (positive = growing)
EstMinutes float64 // estimated minutes to exhaustion (-1 = not trending)
Confidence float64 // 0.0–1.0 confidence in the prediction (based on trend quality)
}
ExhaustionPrediction estimates when a resource will be exhausted.
type Fact ¶
type Fact struct {
// ID is a stable, unique identifier. Convention:
// "<domain>.<metric>" for raw signals, "<domain>.<metric>.<entity>"
// when scoped. Example: "cpu.psi.avg10", "cpu.runqueue.host".
ID string `json:"id"`
// Kind buckets this fact for verifier dispatch.
Kind FactKind `json:"kind"`
// Source identifies the producer (collector name, eBPF program,
// adapter). Used by the verifier to weight conflicting facts.
// Example: "procfs", "eBPF/biolatency", "mongo-adapter".
Source string `json:"source"`
// EntityID is the thing this fact is ABOUT. Empty when host-scope.
// Conventions:
// "host" — global host fact
// "pid:1234" — about a specific process
// "cgroup:/sys/fs/cgroup/system.slice/mongod.service"
// "service:mongod"
// "mount:/var/lib/mysql"
EntityID string `json:"entity_id,omitempty"`
// OwnerID, when set, identifies the owner of EntityID — the upward
// pointer in the entity graph. Example: a process's cgroup, a
// mount's owning service.
OwnerID string `json:"owner_id,omitempty"`
// Domain routes this fact to the right detector / verifier gates.
Domain Domain `json:"domain"`
// Metric is the human-readable name of what is being measured.
// Example: "psi.avg10", "rss_bytes", "queue_depth".
Metric string `json:"metric"`
// Value is the measurement. Float so we can represent rates,
// percentages, counts, and durations uniformly. NaN is reserved
// for "explicitly unknown" — prefer omitting the fact.
Value float64 `json:"value"`
// Unit names the unit of Value. "%", "bytes", "ms", "count".
// Empty when dimensionless or when carried in Tags.
Unit string `json:"unit,omitempty"`
// MeasuredAt is the wall-clock time the measurement was taken.
// Required; zero value indicates a malformed Fact.
MeasuredAt time.Time `json:"measured_at"`
// FirstSeenAt is when this signal first crossed its threshold.
// Tracked by the engine across ticks; for a brand-new fact it
// equals MeasuredAt. Pointer so omitempty works (zero time.Time
// isn't treated as empty by encoding/json).
FirstSeenAt *time.Time `json:"first_seen_at,omitempty"`
// LastSeenAt is when this signal was last observed above threshold.
// Distinct from MeasuredAt so the verifier can detect "still active"
// vs "transient spike". Pointer for omitempty semantics.
LastSeenAt *time.Time `json:"last_seen_at,omitempty"`
// Duration is the cumulative time this signal has been active.
// Used by the temporal-ordering gate (cause must precede effect
// by at least Duration).
Duration time.Duration `json:"duration,omitempty"`
// Severity is how serious this single fact is.
Severity FactSeverity `json:"severity"`
// Confidence is the measurement-quality score in [0,1]. See the
// FactConfidence type docstring for the rubric.
Confidence FactConfidence `json:"confidence"`
// BaselineDelta, when set, is how far above baseline this value is.
// Units match Value's Unit. Used by the baseline-deviation gate.
// Zero when no baseline is available — distinguish from "exactly
// at baseline" via Tags["baseline_known"]="true".
BaselineDelta float64 `json:"baseline_delta,omitempty"`
// Tags carries free-form metadata that doesn't fit the schema.
// Examples: device name, cgroup path, weight class, app type.
Tags map[string]string `json:"tags,omitempty"`
}
Fact is one typed observation with provenance. Every field is settable from a composite literal; no required helpers.
Field naming uses snake_case JSON tags for compatibility with the future replay format (line-delimited JSON, one Fact per line).
type FactConfidence ¶
type FactConfidence float64
FactConfidence is how much we trust this fact's measurement, on [0,1]. Distinct from the verifier's downstream "this fact supports cause X" reasoning — FactConfidence is about the MEASUREMENT itself.
Guidance:
1.00 — direct kernel counter (procfs, sysfs, cgroup) 0.90 — derived from kernel counters with simple arithmetic 0.80 — eBPF event stream 0.70 — secondary metrics needing interpretation 0.50 — single-sample heuristics (snapshot-only) 0.30 — log-line parsing / unstructured input 0.10 — wild guess, present mostly for completeness
type FactKind ¶
type FactKind string
FactKind tags a Fact by its evidence class. Per NEXTGEN §2 "Evidence classes", facts fall into six buckets. The kind determines which verifier gates apply and what counter-evidence to look for.
const ( // FactKindSymptom is a user-visible impact signal (latency, error rate, // queue depth from the requesting process's POV). FactKindSymptom FactKind = "symptom" // FactKindSaturation is a resource-pressure signal (CPU%, mem%, PSI, // disk %util, conntrack fill). FactKindSaturation FactKind = "saturation" // FactKindOwnership is an attribution signal (this PID is the top CPU // consumer; this cgroup owns the disk IO). FactKindOwnership FactKind = "ownership" // FactKindDependency is a relational signal (service X talks to Y; // process P holds lock L). FactKindDependency FactKind = "dependency" // FactKindChange is a "something changed" signal (config drift, deploy, // package upgrade, service restart). FactKindChange FactKind = "change" // FactKindProbe is the output of an active confirmation probe — never // passive observation. Triggered only by the verifier when needed. FactKindProbe FactKind = "probe" // FactKindLogEvidence is a structured finding extracted from journald / // syslog entries — e.g. crash-restart loops, OOM kills, segfaults, or // dependency-connection failures. Produced by InjectJournalEvidence. FactKindLogEvidence FactKind = "log_evidence" // FactKindConfigChange is a "something changed" signal specifically for // OS/kernel runtime parameter drift — sysctl values, hugepage settings, // CPU governor, cgroup limits, etc. Produced by InjectConfigDriftEvidence. // Confidence is moderate (~0.6) because config drift is derived/interpreted; // onset-correlation (Phase 4.4) is what gives it RCA weight. FactKindConfigChange FactKind = "config_change" )
type FactSeverity ¶
type FactSeverity string
FactSeverity describes how serious THIS fact is — independent of how it combines with others. The verifier rolls severities up.
Note: distinct from the pre-existing Severity type (which uses integer levels for HealthLevel transitions). FactSeverity is a string for the line-delimited replay format.
const ( FactSeverityInfo FactSeverity = "info" FactSeverityWarn FactSeverity = "warn" FactSeverityCrit FactSeverity = "crit" )
type FailedAuthSource ¶
FailedAuthSource holds a source IP and its failed authentication count.
type FilelessProcess ¶
type FilelessProcess struct {
PID int
Comm string
ExePath string // readlink result, e.g. "/memfd:payload (deleted)"
IsMemFD bool // /memfd: prefix
IsDeleted bool // (deleted) suffix, not memfd
NetConns int // ESTABLISHED + SYN_SENT outbound connections
RemoteIPs []string // up to 5 unique remote IPs
RSS uint64 // resident memory, for context
}
FilelessProcess represents a process running from memory with no on-disk binary.
type FleetAgentConfig ¶
type FleetAgentConfig struct {
HubURL string `json:"hub_url"` // e.g. "https://hub.example:9200"
Token string `json:"token"` // auth token
Tags []string `json:"tags,omitempty"`
QueuePath string `json:"queue_path,omitempty"` // offline queue, default ~/.xtop/fleet-queue.jsonl
MaxQueueSize int `json:"max_queue_size,omitempty"` // default 10_000
// InsecureSkipVerify allows self-signed certs for the hub. Defaults to
// false (secure); only set via --insecure / --fleet-insecure flags.
InsecureSkipVerify bool `json:"insecure_skip_verify,omitempty"`
}
FleetAgentConfig holds the agent-side fleet config (loaded from ~/.xtop/fleet.json or via --fleet-hub / --fleet-token flags).
type FleetHeartbeat ¶
type FleetHeartbeat struct {
// Identity
Hostname string `json:"hostname"`
AgentID string `json:"agent_id"` // stable UUID persisted in ~/.xtop/agent-id
Tags []string `json:"tags,omitempty"`
// Versions
AgentVersion string `json:"agent_version"`
Kernel string `json:"kernel,omitempty"`
OS string `json:"os,omitempty"`
// Timestamp
Timestamp time.Time `json:"timestamp"`
// Health summary
Health HealthLevel `json:"health"`
PrimaryBottleneck string `json:"primary_bottleneck,omitempty"`
PrimaryScore int `json:"primary_score"`
Confidence int `json:"confidence"`
// Top culprit (if any)
CulpritProcess string `json:"culprit_process,omitempty"`
CulpritPID int `json:"culprit_pid,omitempty"`
CulpritApp string `json:"culprit_app,omitempty"`
// Compact metric summary (for fleet-wide sparklines on hub)
CPUBusyPct float64 `json:"cpu_busy_pct"`
MemUsedPct float64 `json:"mem_used_pct"`
IOWorstUtil float64 `json:"io_worst_util"`
LoadAvg1 float64 `json:"load_avg_1"`
NumCPUs int `json:"num_cpus"`
MemTotalBytes uint64 `json:"mem_total_bytes"`
// Incident cross-ref (set when this tick is part of an ongoing incident)
ActiveIncidentID string `json:"active_incident_id,omitempty"`
// Self-resource reporting — xtop's own footprint on the agent host.
// The web dashboard renders these inline on the host card so operators
// can verify at a glance that the observability tool isn't competing
// with the workload it's observing. Zero-valued when the guardian is
// disabled (XTOP_GUARD off) — UIs should hide the row in that case.
XtopOwnCPUPct float64 `json:"xtop_cpu_pct,omitempty"`
XtopOwnRSSMB float64 `json:"xtop_rss_mb,omitempty"`
XtopGuardLevel int `json:"xtop_guard_level,omitempty"`
XtopMode string `json:"xtop_mode,omitempty"` // "lean" or "rich"
}
FleetHeartbeat is the small per-tick payload from agent → hub. Sent every collection tick (default 3s). ~500 bytes per host.
type FleetHost ¶
type FleetHost struct {
Hostname string `json:"hostname"`
AgentID string `json:"agent_id"`
Tags []string `json:"tags,omitempty"`
AgentVersion string `json:"agent_version"`
Kernel string `json:"kernel,omitempty"`
OS string `json:"os,omitempty"`
// Connection liveness
FirstSeen time.Time `json:"first_seen"`
LastSeen time.Time `json:"last_seen"`
Status HostStatus `json:"status"` // live / stale / expired
// Latest health snapshot (from last heartbeat)
Health HealthLevel `json:"health"`
PrimaryBottleneck string `json:"primary_bottleneck,omitempty"`
PrimaryScore int `json:"primary_score"`
Confidence int `json:"confidence"`
CulpritProcess string `json:"culprit_process,omitempty"`
CulpritApp string `json:"culprit_app,omitempty"`
// Latest metrics
CPUBusyPct float64 `json:"cpu_busy_pct"`
MemUsedPct float64 `json:"mem_used_pct"`
IOWorstUtil float64 `json:"io_worst_util"`
LoadAvg1 float64 `json:"load_avg_1"`
NumCPUs int `json:"num_cpus"`
// Active incident, if any
ActiveIncidentID string `json:"active_incident_id,omitempty"`
// Mirrored self-resource fields (latest known) so UIs can show xtop's
// own footprint per host without joining against heartbeats.
XtopOwnCPUPct float64 `json:"xtop_cpu_pct,omitempty"`
XtopOwnRSSMB float64 `json:"xtop_rss_mb,omitempty"`
XtopGuardLevel int `json:"xtop_guard_level,omitempty"`
XtopMode string `json:"xtop_mode,omitempty"`
}
FleetHost is the hub-side state for a single agent. This is what the TUI / web UI reads to render the fleet table.
type FleetHubConfig ¶
type FleetHubConfig struct {
ListenAddr string `json:"listen_addr"` // default ":9200"
TLSCert string `json:"tls_cert,omitempty"`
TLSKey string `json:"tls_key,omitempty"`
AuthToken string `json:"auth_token"` // shared secret for all agents
AllowNoAuth bool `json:"allow_no_auth,omitempty"` // skip empty-token guard (NOT for production)
// Postgres connection (e.g. "postgres://xtop:pw@localhost/xtopfleet")
PostgresDSN string `json:"postgres_dsn"`
// SQLite hot cache path (default ~/.xtop/hub-cache.sqlite)
SQLiteCachePath string `json:"sqlite_cache_path,omitempty"`
// Retention (default 30d)
IncidentRetentionDays int `json:"incident_retention_days,omitempty"`
HeartbeatRetentionHours int `json:"heartbeat_retention_hours,omitempty"`
}
FleetHubConfig holds the hub-side configuration (loaded from ~/.xtop/hub.json).
type FleetIncident ¶
type FleetIncident struct {
// Identity (same as heartbeat)
Hostname string `json:"hostname"`
AgentID string `json:"agent_id"`
// Incident ID — stable across status updates of the same incident.
// Format: "{hostname}-{unix_sec_start}-{short_signature_hash}"
IncidentID string `json:"incident_id"`
// Timestamps
StartedAt time.Time `json:"started_at"`
ResolvedAt *time.Time `json:"resolved_at,omitempty"` // nil while active
Timestamp time.Time `json:"timestamp"` // when this payload was generated
// Core diagnosis
Bottleneck string `json:"bottleneck"`
PeakScore int `json:"peak_score"`
Confidence int `json:"confidence"`
Health HealthLevel `json:"health"`
Pattern string `json:"pattern,omitempty"`
// Culprit
Culprit string `json:"culprit,omitempty"`
CulpritPID int `json:"culprit_pid,omitempty"`
CulpritApp string `json:"culprit_app,omitempty"`
// Narrative
RootCause string `json:"root_cause,omitempty"`
Evidence []string `json:"evidence,omitempty"`
Impact string `json:"impact,omitempty"`
// Signature — stable hash of (bottleneck + top-3 evidence IDs) for
// cross-incident similarity matching.
Signature string `json:"signature"`
// Context at incident peak: top 10 CPU/memory/IO processes.
// Smaller than full snapshot but enough for drill-down.
TopProcesses []FleetProcess `json:"top_processes,omitempty"`
// Raw evidence list for hub-side re-analysis (optional — only sent if
// FleetClient.IncludeRawEvidence is set).
RawEvidence []Evidence `json:"raw_evidence,omitempty"`
// Structured diff vs prior similar incidents on this host. Nil when this
// signature hasn't been seen before. Lets hub UIs show "this is worse/milder
// than usual" and "new signals firing" without recomputing.
Diff *IncidentDiff `json:"diff,omitempty"`
// Update type — lets hub distinguish new incidents from updates
UpdateType IncidentUpdateType `json:"update_type"`
// Lifecycle (Phase 1 of RCA overhaul; Phase 5 of TODOs adds hub schema).
// State: "suspected" | "confirmed" | "resolved" | "" (legacy).
// Hubs that don't know these fields ignore them; they live in JSONB too.
State string `json:"state,omitempty"`
ConfirmedAt time.Time `json:"confirmed_at,omitempty"`
ChangesAtConfirm []SystemChange `json:"changes_at_confirm,omitempty"`
FleetPeersAtConfirm string `json:"fleet_peers_at_confirm,omitempty"`
}
FleetIncident is the richer payload sent once when an incident starts (and again when it changes — e.g., bottleneck domain flips, or confidence jumps). Typical size: 5-30 KB depending on evidence + process list.
type FleetProcess ¶
type FleetProcess struct {
PID int `json:"pid"`
Comm string `json:"comm"`
CPUPct float64 `json:"cpu_pct"`
RSS uint64 `json:"rss"`
State string `json:"state,omitempty"`
}
FleetProcess is a lightweight process record for fleet-wide incident context.
type FlowRateEntry ¶
type FlowRateEntry struct {
PID int `json:"pid"`
Comm string `json:"comm"`
DstIP string `json:"dst_ip"`
ConnectCount uint64 `json:"connect_count"`
CloseCount uint64 `json:"close_count"`
UniqueDestCount int `json:"unique_dest_count"`
Rate float64 `json:"rate"`
}
FlowRateEntry holds BPF-detected connection flow rate data per process/destination.
type GPUDevice ¶
type GPUDevice struct {
Index int
Name string // e.g., "NVIDIA GeForce RTX 3090"
Driver string // driver version
UtilGPU float64 // GPU utilization %
UtilMem float64 // memory controller utilization %
MemUsed uint64 // bytes
MemTotal uint64 // bytes
Temperature int // celsius
PowerDraw float64 // watts
PowerLimit float64 // watts
FanSpeed int // percent (-1 if N/A)
Processes []GPUProcess
}
GPUDevice represents one GPU's metrics.
type GPUProcess ¶
GPUProcess represents a process using the GPU.
type GPUSnapshot ¶
GPUSnapshot holds all GPU data for one collection cycle.
type GateResult ¶
type GateResult struct {
// GateID identifies the gate ("signal_quality", "baseline_deviation",
// "temporal_ordering", "ownership_consistency", "blast_radius",
// "counter_evidence", "deep_probe"). Stable across versions.
GateID string `json:"gate_id"`
// Passed is true if the candidate cleared this gate.
Passed bool `json:"passed"`
// Reason is a single-sentence operator-facing explanation. When
// Passed=false, this is the abstention reason.
Reason string `json:"reason,omitempty"`
// FactsUsed lists the Fact.ID values consulted. Lets the operator
// re-run the gate against the same evidence in replay.
FactsUsed []string `json:"facts_used,omitempty"`
}
GateResult is the outcome of one verifier gate evaluating one candidate. Carries the verdict + the evidence the verifier used so the operator can audit WHY a gate passed or failed.
type GlobalMetrics ¶
type GlobalMetrics struct {
PSI PSIMetrics
CPU CPUMetrics
Memory MemoryMetrics
VMStat VMStatMetrics
Disks []DiskStats
Network []NetworkStats
TCP TCPMetrics
UDP UDPMetrics
Sockets SocketStats
TCPStates TCPConnState
SoftIRQ SoftIRQStats
Conntrack ConntrackStats
ConntrackDissect ConntrackDissection
ConntrackTimeouts ConntrackTimeouts
FD FDStats
EphemeralPorts EphemeralPorts
TopRemoteIPs []RemoteIPStats
CloseWaitLeakers []CloseWaitLeaker
CloseWaitTrend CloseWaitTrend
Mounts []MountStats
DeletedOpen []DeletedOpenFile
BigFiles []BigFile
BigDirs []BigDir
BigDirsPartial bool // scan stat-budget exhausted: BigDirs sizes are lower bounds
FilelessProcs []FilelessProcess
Security SecurityMetrics
Logs LogMetrics
HealthChecks HealthCheckMetrics
Sessions []ActiveSession
Diagnostics DiagMetrics
Sentinel SentinelData
DotNet []DotNetProcessMetrics
Runtimes RuntimeMetrics
Apps AppMetrics
AppIdentities map[int]AppIdentity // PID → resolved identity
Proxmox *ProxmoxMetrics // non-nil only on Proxmox hosts
Profile *ServerProfile // system profiler (optimization audit)
GPU GPUSnapshot // NVIDIA GPU metrics (empty if no GPU)
PHPFPM PHPFPMMetrics // per-pool/per-worker PHP-FPM view
}
GlobalMetrics is the full system-wide metric snapshot.
type GoldenSignalSummary ¶
type GoldenSignalSummary struct {
// Latency proxies
DiskLatencyMs float64 // worst disk await
TCPRTTMs float64 // smoothed TCP RTT (if BPF available)
PSIStallPct float64 // max PSI stall across domains
// Traffic proxies
TCPSegmentsPerSec float64 // in + out segments
NetBytesPerSec float64 // total interface throughput
ConnAcceptRate float64 // passive opens / sec
// Error proxies
ErrorRate float64 // drops + retrans + resets + OOM combined rate
// Saturation proxies
SaturationPct float64 // max of: conntrack%, ephemeral%, runqueue ratio, PSI
SaturationBreakdown SaturationDetail // per-component saturation detail
}
GoldenSignalSummary approximates Google SRE Golden Signals from /proc data.
type GuardStatus ¶
type GuardStatus struct {
Level int `json:"level"` // 0 none, 1 caution, 2 degraded, 3 minimal
Reason string `json:"reason,omitempty"`
IntervalSec int `json:"interval_sec"`
OwnCPUPct float64 `json:"own_cpu_pct"`
HostLoadRatio float64 `json:"host_load_ratio"`
Skipped []string `json:"skipped,omitempty"` // human-readable list of what was skipped
}
GuardStatus is the per-tick report from the engine's ResourceGuard. The UI reads this to show a compact status strip when xtop is self-throttling: "[GUARD L2: host load 4.1x] skipping log-tailer, traces, watchdog".
type HAProxyInfo ¶
type HAProxyInfo struct {
Running bool `json:"running"`
Version string `json:"version,omitempty"`
ConfigFile string `json:"config_file"`
Mode string `json:"mode"` // "reverse_proxy", "forward_proxy", "both", "tcp_lb"
Frontends []string `json:"frontends,omitempty"`
Backends []string `json:"backends,omitempty"`
BindPorts []int `json:"bind_ports,omitempty"`
Evidence []string `json:"evidence,omitempty"` // reasons for classification
}
HAProxyInfo holds HAProxy analysis results.
type HealthCheckMetrics ¶
type HealthCheckMetrics struct {
Probes []HealthProbeResult
}
HealthCheckMetrics holds active health probe results.
type HealthLevel ¶
type HealthLevel int
HealthLevel represents overall system health.
const ( HealthOK HealthLevel = 0 HealthInconclusive HealthLevel = 1 HealthDegraded HealthLevel = 2 HealthCritical HealthLevel = 3 )
func (HealthLevel) String ¶
func (h HealthLevel) String() string
type HealthProbeResult ¶
type HealthProbeResult struct {
Name string
ProbeType string // "http", "tcp", "dns", "cert"
Target string // URL, host:port, or domain
Status string // "OK", "WARN", "CRIT", "UNKNOWN"
LatencyMs float64
StatusCode int // HTTP only
Detail string
LastCheck time.Time
CertDaysLeft int // -1 if N/A
}
HealthProbeResult holds the result of one health probe.
type HostIncident ¶
type HostIncident struct {
HostID string
Health HealthLevel
PrimaryBottleneck string
PrimaryScore int
Timestamp time.Time
EvidenceIDs []string
}
HostIncident holds cross-host correlation data for a single peer host.
type HostStatus ¶
type HostStatus string
HostStatus reflects the liveness of an agent relative to its expected interval.
const ( HostStatusLive HostStatus = "live" // seen within 3× interval HostStatusStale HostStatus = "stale" // seen >3× interval ago, <10min HostStatusExpired HostStatus = "expired" // >10min since last seen )
type ImpactScore ¶
type ImpactScore struct {
PID int
Rank int
Comm string
Service string // resolved from cgroup: k8s pod, systemd unit, or docker container
Cgroup string
// Actual metrics
CPUPct float64 // actual CPU% from process rates
// Component scores (0-1 normalized)
CPUSaturation float64
PSIContrib float64
IOWait float64
MemGrowth float64
NetRetrans float64
// Penalties
NewnessPenalty float64 // +0.15 for processes started <60s ago
ChangePenalty float64 // reserved for future use
// Final score
Composite float64 // 0-100 weighted sum
// Context
Threads int
RSS uint64
WriteMBs float64
}
ImpactScore represents the composite impact a process has on system health.
type IncidentDiff ¶
type IncidentDiff struct {
MatchCount int `json:"match_count"`
FirstSeen time.Time `json:"first_seen,omitempty"`
LastSeen time.Time `json:"last_seen,omitempty"`
MedianPeakScore int `json:"median_peak_score,omitempty"`
MaxPeakScore int `json:"max_peak_score,omitempty"`
CurrentPeakScore int `json:"current_peak_score,omitempty"`
ScoreDeltaFromMedian int `json:"score_delta_from_median,omitempty"`
MedianDurationSec int `json:"median_duration_sec,omitempty"`
CulpritFrequency map[string]int `json:"culprit_frequency,omitempty"`
TopCulprit string `json:"top_culprit,omitempty"`
TopCulpritCount int `json:"top_culprit_count,omitempty"`
CurrentCulprit string `json:"current_culprit,omitempty"`
CulpritIsRepeat bool `json:"culprit_is_repeat,omitempty"`
NewEvidence []string `json:"new_evidence,omitempty"`
MissingEvidence []string `json:"missing_evidence,omitempty"`
SameHourOfDay int `json:"same_hour_of_day,omitempty"`
DriftHint string `json:"drift_hint,omitempty"`
}
IncidentDiff is a structured comparison of the current incident against recent similar ones. Populated by the engine's IncidentRecorder when at least one past incident shares the same signature.
What the fields mean:
- MatchCount / FirstSeen / LastSeen: how often and how long we've seen this.
- MedianPeakScore / MaxPeakScore: baseline severity; compare with current.
- CurrentPeakScore / ScoreDeltaFromMedian: is THIS incident worse than usual?
- MedianDurationSec: how long these typically last — sets expectations.
- CulpritFrequency: "mysqld was the culprit 4/5 times" signals a repeat offender.
- NewEvidence / MissingEvidence: evidence IDs firing now that weren't in prior incidents (or vice versa) — the most actionable signal: "this time it's different because X is also firing."
- SameHourOfDay: count of past occurrences in the same hour-of-day — hints at scheduled causes (backups, cron jobs).
- DriftHint: plain-English one-liner the UI can show inline.
type IncidentFrame ¶
type IncidentFrame struct {
// SchemaVersion is the on-disk format version. v1 = the initial
// Phase 5 ship. Bumped on breaking changes.
SchemaVersion int `json:"schema_version"`
// HostID is the host this incident occurred on (snap.HostID).
HostID string `json:"host_id,omitempty"`
// EngineVersion is the xtop version that produced this frame.
// Replays produced by a different version are flagged in the
// harness output.
EngineVersion string `json:"engine_version,omitempty"`
// CapturedAt is wall-clock time when this frame was written. Used
// for chronological ordering of the corpus, NOT as part of the
// determinism contract (replay doesn't depend on this).
CapturedAt time.Time `json:"captured_at"`
// AnalysisTime is the snap.Timestamp the verifier saw. Replay
// MUST use this — most facts are wall-clock relative.
AnalysisTime time.Time `json:"analysis_time"`
// Facts is the exact Fact slice the verifier received. Lossless
// JSON roundtrip is enforced by TestFactJSONRoundtrip.
Facts []Fact `json:"facts"`
// Entities is the exact entity graph snapshot. The byID index is
// rebuilt via Reindex() after deserialization.
Entities *EntityGraph `json:"entities"`
// VerifiedCauses is what the verifier emitted at capture time.
// Replay compares fresh output against this for determinism.
VerifiedCauses []VerifiedCause `json:"verified_causes"`
// HealthAtCapture is the headline AnalysisResult.Health value at
// capture time. Used to filter the corpus ("show me all the
// frames where xtop said Critical").
HealthAtCapture HealthLevel `json:"health_at_capture"`
// PrimaryBottleneck and PrimaryScore preserve the legacy verdict
// for cross-referencing with the audit trail. Phase 4 verdicts
// (VerifiedCauses) supersede these, but the legacy field is what
// goes into incident notifications + the fleet hub, so it must
// be in the corpus.
PrimaryBottleneck string `json:"primary_bottleneck,omitempty"`
PrimaryScore int `json:"primary_score,omitempty"`
// Label is operator-provided ground truth ("was this a real
// incident, and did the engine call it correctly?"). Empty when
// unlabeled. Used by the replay harness to compute precision per
// mechanism. See LabelKind for the enum.
Label LabelKind `json:"label,omitempty"`
// LabelReason is the operator's free-text note explaining the
// label. Optional but recommended for FN cases ("real incident
// the engine missed because X").
LabelReason string `json:"label_reason,omitempty"`
}
IncidentFrame is the persisted, replayable record of one analysis tick that produced a non-OK verdict. Stored as one JSON object per file in ~/.xtop/incidents/<unix_nano>.json.
Schema version is embedded so future format changes are detectable; older frames can be rejected by an upgraded engine cleanly.
type IncidentUpdateType ¶
type IncidentUpdateType string
IncidentUpdateType signals what kind of update the hub is receiving.
const ( IncidentStarted IncidentUpdateType = "started" IncidentUpdated IncidentUpdateType = "updated" // confidence/score changed, same bottleneck IncidentEscalated IncidentUpdateType = "escalated" // bottleneck changed while incident still active IncidentResolved IncidentUpdateType = "resolved" // health returned to OK )
type JA3Entry ¶
type JA3Entry struct {
Hash string `json:"hash"`
Count uint64 `json:"count"`
SampleSrc string `json:"sample_src"`
SampleDst string `json:"sample_dst"`
Known string `json:"known"`
}
JA3Entry holds a TLS JA3 fingerprint hash and its occurrence data.
type JournalFinding ¶
type JournalFinding struct {
Signature string
Severity DiagSeverity
Count int
Sample string // representative message; truncation to ~120 chars is applied by journal.Classify (the producer), not enforced by this field
PID int
FirstSeen time.Time
LastSeen time.Time
}
JournalFinding is a structured finding produced by Tier-2 journal RCA. It mirrors collector/journal.JournalFinding but lives in model to avoid an import cycle (model must not import collector).
type K8sNodeInfo ¶
type K8sNodeInfo struct {
NodeRole string `json:"node_role"`
PodCount int `json:"pod_count"`
Namespaces []string `json:"namespaces,omitempty"`
}
K8sNodeInfo holds Kubernetes node information.
type KeepalivedInfo ¶
type KeepalivedInfo struct {
Running bool `json:"running"`
VIPs []string `json:"vips,omitempty"`
State string `json:"state,omitempty"` // "MASTER", "BACKUP"
Interface string `json:"interface,omitempty"`
Priority int `json:"priority,omitempty"`
}
KeepalivedInfo holds keepalived/VRRP analysis results.
type LabelKind ¶
type LabelKind string
LabelKind is the operator's ground-truth verdict on an IncidentFrame. Used to compute precision (TP / (TP+FP)) and recall (TP / (TP+FN)) per mechanism over the corpus.
const ( // LabelUnlabeled is the default — operator hasn't reviewed yet. LabelUnlabeled LabelKind = "" // LabelTruePositive: engine called X, X really was happening. LabelTruePositive LabelKind = "TP" // LabelFalsePositive: engine called X, X was NOT happening. // These are what the 0.1% target measures. LabelFalsePositive LabelKind = "FP" // LabelTrueNegative: engine abstained (Tier D or OK), nothing // was wrong. Rarely interesting per-incident — the corpus only // captures non-OK ticks — but included for completeness. LabelTrueNegative LabelKind = "TN" // LabelFalseNegative: engine missed a real incident. Operators // add these manually when xtop didn't catch something but should // have. Recall metric depends on FN counts. LabelFalseNegative LabelKind = "FN" )
type LogExcerpt ¶
type LogExcerpt struct {
App string `json:"app"` // e.g. "mysql", "nginx"
Path string `json:"path"` // source file
Line string `json:"line"` // the matched line (trimmed)
Severity string `json:"severity,omitempty"` // "ERROR", "WARN", "SLOW", "FATAL", "OOM"
Timestamp time.Time `json:"timestamp,omitempty"` // best-effort parse; zero if unparseable
}
LogExcerpt is one notable line from an application log file, correlated with a live incident. The engine fills these in when an incident is active and the culprit matches a known app — so operators see the database's own "ERROR: slow query" line beside xtop's "mysqld is the culprit" verdict.
type LogMetrics ¶
type LogMetrics struct {
Services []ServiceLogStats
}
LogMetrics holds log analysis data for tracked services.
type MemoryMetrics ¶
type MemoryMetrics struct {
Total uint64
Free uint64
Available uint64
Buffers uint64
Cached uint64
SwapTotal uint64
SwapFree uint64
SwapUsed uint64
SwapCached uint64
Dirty uint64
Writeback uint64
Slab uint64
SReclaimable uint64
SUnreclaim uint64
AnonPages uint64
Mapped uint64
Shmem uint64
KernelStack uint64
PageTables uint64
Bounce uint64
HugePages_Total uint64
HugePages_Free uint64
HugepageSize uint64
DirectMap4k uint64
DirectMap2M uint64
DirectMap1G uint64
Mlocked uint64
Active uint64
Inactive uint64
ActiveAnon uint64
InactiveAnon uint64
ActiveFile uint64
InactiveFile uint64
Unevictable uint64
VmallocTotal uint64
VmallocUsed uint64
}
MemoryMetrics holds /proc/meminfo data.
type MetricChange ¶
type MetricChange struct {
Name string // e.g. "mysql IO"
Delta float64 // absolute change value
DeltaPct float64 // percentage change
Current string // current value string
Unit string // e.g. "%", "MB/s", "/s"
Rising bool // true if increasing, false if decreasing
ZScore float64 // statistical significance (0 = not computed)
}
MetricChange represents a notable metric delta for the "what changed?" engine.
type MetricCorrelation ¶
type MetricCorrelation struct {
MetricA string // evidence ID A
MetricB string // evidence ID B
Coefficient float64 // Pearson R (-1 to +1)
Samples int64 // number of samples
Strength string // "strong"/"moderate"/"weak"
}
MetricCorrelation represents a discovered Pearson correlation between two metrics.
type ModLoadEntry ¶
ModLoadEntry holds a BPF-traced kernel module load event.
type MountRate ¶
type MountRate struct {
MountPoint string
Device string
FSType string
TotalBytes uint64
UsedPct float64
FreePct float64
FreeBytes uint64
InodeUsedPct float64
GrowthBytesPerSec float64 // EWMA-smoothed
PrevGrowthBPS float64 // previous tick's smoothed rate (for trend detection)
ETASeconds float64 // seconds until full (-1 = not growing)
GrowthStarted time.Time // when sustained growth first detected
State string // "OK", "WARN", "CRIT"
}
MountRate holds computed per-filesystem rates.
type MountStats ¶
type MountStats struct {
MountPoint string
Device string
FSType string
TotalBytes uint64
FreeBytes uint64
AvailBytes uint64 // available to non-root (statvfs f_bavail)
UsedBytes uint64
TotalInodes uint64
FreeInodes uint64
UsedInodes uint64
}
MountStats holds per-filesystem stats from statfs(2).
type Narrative ¶
type Narrative struct {
RootCause string // e.g. "CPU throttle cascade — cgroup limits saturating run queue"
Evidence []string // top 3-4 evidence lines with values
Impact string // e.g. "CPU stall 42%; disk latency +120ms"
Confidence int
Pattern string // matched pattern name (empty if none)
Temporal string // temporal chain summary
}
Narrative is the human-readable root cause explanation produced by the narrative engine.
type NetRate ¶
type NetRate struct {
Name string
RxMBs float64
TxMBs float64
RxPPS float64
TxPPS float64
RxDropsPS float64
TxDropsPS float64
RxErrorsPS float64
TxErrorsPS float64
// Metadata (passed through from NetworkStats)
OperState string // "up", "down", "unknown"
SpeedMbps int // -1 if unknown
Master string // bridge/bond master name
IfType string // "physical", "bridge", "bond", "veth", etc.
// Computed
UtilPct float64 // link utilization % ((RxMBs+TxMBs)*8*1024/SpeedMbps*100), -1 if unknown
}
NetRate holds computed per-interface rates.
type NetworkStats ¶
type NetworkStats struct {
Name string
RxBytes uint64
RxPackets uint64
RxErrors uint64
RxDrops uint64
RxFifo uint64
RxFrame uint64
TxBytes uint64
TxPackets uint64
TxErrors uint64
TxDrops uint64
TxFifo uint64
TxColls uint64
TxCarrier uint64
// Metadata from /sys/class/net/
OperState string // "up", "down", "unknown"
SpeedMbps int // link speed in Mbps (-1 if unknown)
Master string // bridge/bond master interface name (empty if none)
IfType string // "physical", "bridge", "bond", "veth", "vlan", "tunnel", "virtual"
}
NetworkStats holds per-interface counters from /proc/net/dev plus metadata from /sys/class/net/.
type NewListeningPort ¶
NewListeningPort holds a newly detected listening port.
type OOMKillEntry ¶
type OOMKillEntry struct {
VictimPID uint32
VictimComm string
TotalVM uint64
AnonRSS uint64
Timestamp int64
}
OOMKillEntry holds a BPF-traced OOM kill event.
type OptDomain ¶
type OptDomain string
OptDomain groups related optimization audit rules. Distinct from Domain (RCA domain in snapshot.go).
type OutboundEntry ¶
type OutboundEntry struct {
PID int `json:"pid"`
Comm string `json:"comm"`
DstIP string `json:"dst_ip"`
TotalBytes uint64 `json:"total_bytes"`
PacketCount uint64 `json:"packet_count"`
BytesPerSec float64 `json:"bytes_per_sec"`
}
OutboundEntry holds BPF-detected top outbound data transfer per process/destination.
type OwnerAttribution ¶
type OwnerAttribution struct {
Kind string // "cgroup", "service", "pid"
ID string // cgroup path, service name, or "pid:1234"
Confidence float64 // 0..1
}
OwnerAttribution identifies a resource consumer associated with evidence.
type PHPFPMApp ¶
type PHPFPMApp struct {
App string
PHPVersion string
DocRoot string
DocRootMissing bool // configured DocRoot does not exist on disk (stale vhost)
AccessLog string
WorkerCount int
RunningCount int
IdleCount int
CPUPct float64 // sum of LiveCPUPct (% of one core)
RSSKB int64
DiskReadBps float64
DiskWriteBps float64
RequestsTotal int64
AvgDurationMs float64
TopURI string
// From access log
AccessReqs int
AccessBytes int64
Status2xx int
Status3xx int
Status4xx int
Status5xx int
TopIPs []PHPFPMIPHit // top IPs hitting this site
TopAccessURIs []PHPFPMURIHit // top URIs in access log
TopIPURIs []PHPFPMIPURIHit
// From slow log
SlowBlocksTotal int
TopSlowScripts []PHPFPMScriptHit
TopSlowFns []PHPFPMFunctionHit
WebShellHits []PHPFPMWebShellSuspect
// Filesystem scan of the docroot
FSWebShells []PHPFPMFSFinding
FSBinaries []PHPFPMFSFinding
// From live FPM status (which scripts have most workers right now)
TopRunningScripts []PHPFPMScriptHit
// Issues fired by RCA rules
Issues []PHPFPMIssue
}
PHPFPMApp aggregates workers by app (derived docroot dirname).
type PHPFPMFSFinding ¶
type PHPFPMFunctionHit ¶
type PHPFPMIPHit ¶
type PHPFPMIPURIHit ¶
type PHPFPMIssue ¶
type PHPFPMMaster ¶
type PHPFPMMaster struct {
PID int
PHPVersion string // "8.3", "8.2", "8.1", "7.4" — derived from ConfigPath
ConfigPath string // e.g. /www/server/php/83/etc/php-fpm.conf
ListenAddr string // unix:/tmp/php-cgi-83.sock or 127.0.0.1:9000
StatusPath string // /phpfpm_83_status (if pm.status_path set)
PoolName string
WorkerCount int
StatusOK bool // true if last status fetch succeeded
StatusError string // non-empty on failure
// State is a high-level classification: "ok" (status fetch worked),
// "no-status" (pool exists, pm.status_path not configured — fine
// but uninspectable), "socket-missing", "connect-failed".
State string
}
PHPFPMMaster is one running php-fpm master process — identified by its config path (which encodes the PHP version on most distros).
type PHPFPMMetrics ¶
type PHPFPMMetrics struct {
Masters []PHPFPMMaster
Workers []PHPFPMWorker
Apps []PHPFPMApp
}
PHPFPMMetrics carries the per-worker breakdown that PHP-FPM's `pm.status_path?full` exposes, joined with /proc for live cost.
type PHPFPMScriptHit ¶
type PHPFPMURIHit ¶
type PHPFPMWebShellSuspect ¶
type PHPFPMWorker ¶
type PHPFPMWorker struct {
PID int
MasterPID int
PHPVersion string
PoolName string
State string // "Running" | "Idle"
Script string // /www/wwwroot/<app>/index.php
App string // derived from Script — first dir under /www/wwwroot or docroot
RequestURI string
RequestMethod string
DurationUs int64 // microseconds — current request when Running, last when Idle
LastReqCPUPct float64 // %
LastReqMemKB int64
RequestsTotal int64
// Live values from /proc
LiveCPUPct float64
LiveRSSKB int64
DiskReadBps float64 // bytes/sec via /proc/<pid>/io rchar delta
DiskWriteBps float64 // bytes/sec via /proc/<pid>/io wchar delta
}
PHPFPMWorker is one row from the FPM `?full` status block, augmented with live /proc data.
type PSILine ¶
type PSILine struct {
Avg10 float64
Avg60 float64
Avg300 float64
Total uint64 // cumulative microseconds
}
PSILine holds one line of PSI data (some or full).
type PSIMetrics ¶
type PSIMetrics struct {
CPU PSIResource
Memory PSIResource
IO PSIResource
}
PSIMetrics holds all PSI data.
type PSIResource ¶
PSIResource holds PSI data for one resource (cpu, memory, or io).
type PktDropEntry ¶
type PktDropEntry struct {
Reason uint32
ReasonStr string
Count uint64
Rate float64
Benign bool // true if this is a normal TCP lifecycle drop, not a real problem
}
PktDropEntry holds a BPF-traced packet drop reason and count.
type PktDropLocation ¶
type PktDropLocation struct {
Function string // resolved kernel function name
Rate float64 // drops/s at this location
Count uint64
}
PktDropLocation holds where in the kernel packets are being dropped.
type PktDropProto ¶
type PktDropProto struct {
Proto string // "IPv4", "IPv6", "ARP"
Rate float64 // drops/s for this protocol
Count uint64
}
PktDropProto holds which protocol's packets are being dropped.
type PortScanEntry ¶
type PortScanEntry struct {
SrcIP string `json:"src_ip"`
RSTCount uint64 `json:"rst_count"`
UniquePortBuckets int `json:"unique_port_buckets"`
DurationSec float64 `json:"duration_sec"`
Rate float64 `json:"rate"`
}
PortScanEntry holds BPF-detected port scan indicators per source IP.
type PortUser ¶
type PortUser struct {
PID int
Comm string
Ports int // total ephemeral ports held
Established int
TimeWait int // note: TIME_WAIT usually has inode 0, so this tracks indirect attribution
CloseWait int
}
PortUser holds per-process ephemeral port consumption.
type ProbeResult ¶
type ProbeResult struct {
Name string `json:"name"` // probe class, e.g. "top_cpu_processes"
EvidenceID string `json:"evidence_id"` // evidence ID that triggered this probe
StartedAt time.Time `json:"started_at"`
DurationMs int `json:"duration_ms"`
ExitCode int `json:"exit_code"`
Output string `json:"output"` // stdout (truncated to 64 KB)
Stderr string `json:"stderr,omitempty"`
Truncated bool `json:"truncated,omitempty"`
Error string `json:"error,omitempty"`
}
ProbeResult is the captured output of one Phase 6 active investigation. Probes are short, read-only shell or eBPF commands run on demand to disambiguate Suspected→Confirmed transitions; results are attached to the next trace dump for forensic inspection.
type ProcessAnomaly ¶
type ProcessAnomaly struct {
PID int
Comm string
Metric string // "cpu_pct", "rss_mb", "io_mbs"
Current float64
Baseline float64
StdDev float64
Sigma float64
}
ProcessAnomaly represents a process whose resource usage deviates from its learned profile.
type ProcessMetrics ¶
type ProcessMetrics struct {
PID int
Comm string
State string
PPID int
CgroupPath string
// CPU (in ticks)
UTime uint64
STime uint64
NumThreads int
Processor int
// Memory (in bytes)
RSS uint64
VmSize uint64
VmSwap uint64
MinFault uint64
MajFault uint64
// IO (in bytes)
ReadBytes uint64
WriteBytes uint64
SyscR uint64
SyscW uint64
// Context switches
VoluntaryCtxSwitches uint64
NonVoluntaryCtxSwitches uint64
// File descriptors
FDCount int // count of open FDs from /proc/PID/fd
FDSoftLimit uint64 // soft limit from /proc/PID/limits
// Start time (clock ticks since boot, from /proc/PID/stat field 22)
StartTimeTicks uint64
}
ProcessMetrics holds metrics for a single process.
type ProcessRate ¶
type ProcessRate struct {
PID int
Comm string
State string
CgroupPath string
ServiceName string // resolved from cgroup: k8s pod, systemd unit, or docker container
CPUPct float64
MemPct float64
ReadMBs float64
WriteMBs float64
FaultRate float64
MajFaultRate float64
CtxSwitchRate float64
RSS uint64
VmSwap uint64
NumThreads int
FDCount int
FDSoftLimit uint64
FDPct float64 // FDCount / FDSoftLimit * 100
WritePath string // primary file being written to (resolved from /proc/PID/fd)
}
ProcessRate holds computed per-process rates.
type ProxmoxDiskConf ¶
type ProxmoxDiskConf struct {
Bus string // "scsi0", "ide2", "virtio0"
Path string // "local-lvm:vm-100-disk-0", "/var/lib/vz/..."
SizeGB int
Cache string // "none", "writeback", etc.
}
ProxmoxDiskConf holds VM disk configuration.
type ProxmoxFirewall ¶
type ProxmoxFirewall struct {
ClusterEnabled bool // whether cluster-level firewall is active
VMFirewalls map[int]bool // VMID → firewall enabled
}
ProxmoxFirewall holds cluster and per-VM firewall state.
type ProxmoxHAResource ¶
type ProxmoxHAResource struct {
VMID int
State string // "started", "stopped", "error", "fence"
Group string // HA group name
MaxRestart int // max restart attempts before giving up
MaxRelocate int // max relocate attempts before giving up
}
ProxmoxHAResource represents an HA-managed VM or container.
type ProxmoxMetrics ¶
type ProxmoxMetrics struct {
IsProxmoxHost bool
NodeName string
PVEVersion string
VMs []ProxmoxVM
Storage []ProxmoxStorage
// HA and cluster-level features
HAEnabled bool
HAResources []ProxmoxHAResource
Replication []ProxmoxReplication
Firewall ProxmoxFirewall
}
ProxmoxMetrics holds Proxmox VE host-level data.
type ProxmoxNetConf ¶
type ProxmoxNetConf struct {
ID string // "net0"
Model string // "virtio", "e1000"
MAC string
Bridge string // "vmbr0"
Tag int // VLAN tag (0=none)
}
ProxmoxNetConf holds VM network configuration.
type ProxmoxReplication ¶
type ProxmoxReplication struct {
VMID int
Target string // target node name
Schedule string // cron-style schedule (e.g. "*/15")
LastSync string // timestamp of last successful sync
Status string // "ok", "error", "syncing"
}
ProxmoxReplication holds replication job status for a VM.
type ProxmoxStorage ¶
type ProxmoxStorage struct {
Name string
Type string // "lvmthin", "dir", "nfs", "zfspool"
Path string // mount path or VG/LV
TotalGB float64
UsedGB float64
AvailGB float64
UsedPct float64
}
ProxmoxStorage holds storage pool info.
type ProxmoxVM ¶
type ProxmoxVM struct {
VMID int
Name string
Status string // "running", "stopped", "paused"
PID int // KVM process PID (0 if stopped)
// Config (from .conf) — basic resources
CoresAlloc int
SocketsAlloc int // CPU sockets
MemAllocMB int
BalloonMinMB int // balloon minimum (0 = ballooning disabled)
BalloonOn bool
DiskConfigs []ProxmoxDiskConf
NetConfigs []ProxmoxNetConf
// Config (from .conf) — advanced settings
CPULimit float64 // cpulimit (0=unlimited)
CPUUnits int // cpuunits (1024=default)
NUMA bool // NUMA topology enabled
Machine string // q35, i440fx
BIOS string // seabios, ovmf
Protection bool // prevent accidental removal
StartupOrder string // startup: order=1,up=30
Description string // VM description/notes
SnapCount int // number of snapshots
Features string // features field (e.g. "fuse=1,nesting=1")
HostPCI []string // PCI passthrough devices
// Live metrics (from cgroups + /proc) — CPU
CPUPct float64 // current CPU% of total host
CPUUserPct float64 // user-space CPU%
CPUSysPct float64 // kernel-space CPU%
CPUThrottledPeriods uint64 // number of throttled periods
CPUThrottledPct float64 // percentage of time throttled
// Live metrics — memory
MemUsedMB int // current RSS
MemBalloonMB int // actual memory after ballooning (from cgroup limit or QMP)
MemPeakMB int // peak RSS watermark
MemSwapMB int // swap usage
MemLimitMB int // memory.max in MB (0=unlimited)
MemHighMB int // memory.high in MB (0=unlimited)
MemOOMKills uint64 // total OOM kills
MemOOMEvents uint64 // total OOM events (attempts)
// Live metrics — PSI (per-VM pressure stall info, avg10)
PSICPUSome float64
PSIMemSome float64
PSIIOSome float64
// Live metrics — IO and network
IOReadMBs float64
IOWriteMBs float64
NetRxMBs float64
NetTxMBs float64
// Live metrics — cgroup resource controls
PIDCount int // current number of processes
PIDLimit int // pids.max
IOMaxBps string // io.max limit if set
CPUWeight int // cpu.weight
CPUMaxQuota string // cpu.max (e.g. "200000 100000")
// Live metrics — misc
UptimeSec int64
// Computed health
HealthScore int // 0-100 composite health score
HealthIssues []string // detected issue descriptions
}
ProxmoxVM represents a single QEMU/KVM virtual machine.
type PtraceEventEntry ¶
type PtraceEventEntry struct {
TracerPID uint32
TracerComm string
TargetPID uint32
TargetComm string
Request uint64
RequestStr string
Count uint64
Timestamp int64
}
PtraceEventEntry holds a BPF-traced ptrace syscall event.
type RCAEntry ¶
type RCAEntry struct {
Bottleneck string
Score int
EvidenceGroups int // how many independent evidence groups fired
TopCgroup string
TopProcess string
TopPID int
TopAppName string // resolved app name from identity (empty = use TopProcess)
Evidence []string
Checks []EvidenceCheck // structured evidence with pass/fail
Chain []string
EvidenceV2 []Evidence // v2 evidence objects (parallel to legacy Checks)
DomainConf float64 // v2 domain confidence 0..0.98
// Facts is the NEXTGEN Phase 2 typed-evidence layer for this domain.
// Each entry is one observation. Emitted in parallel to EvidenceV2;
// Phase 4 makes Facts the canonical input to verifier gates.
Facts []Fact `json:"facts,omitempty"`
}
RCAEntry holds one bottleneck analysis result.
type RateSnapshot ¶
type RateSnapshot struct {
DeltaSec float64
// CPU pcts
CPUBusyPct float64
CPUUserPct float64
CPUSystemPct float64
CPUIOWaitPct float64
CPUSoftIRQPct float64
CPUIRQPct float64
CPUStealPct float64
CPUNicePct float64
// Scheduling
CtxSwitchRate float64 // total estimated
// Memory rates (pages/s → MB/s)
SwapInRate float64 // MB/s
SwapOutRate float64
PgFaultRate float64 // pages/s
MajFaultRate float64
DirectReclaimRate float64 // pages/s
KswapdRate float64
OOMKillDelta uint64 // OOM kills since last tick (delta, not cumulative)
AllocStallRate float64 // alloc stalls/s from VMStat.AllocStall delta
SUnreclaimDelta int64 // SUnreclaim change in bytes (slab leak detection)
// Disks
DiskRates []DiskRate
MountRates []MountRate
// Network
NetRates []NetRate
RetransRate float64
InSegRate float64
OutSegRate float64
TCPResetRate float64
TCPAttemptFailRate float64 // TCP connection attempt failures/s from /proc/net/snmp
TCPResetRateAgg float64 // aggregate TCP reset rate from /proc/net/snmp (EstabResets)
// UDP
UDPInRate float64
UDPOutRate float64
UDPErrRate float64
// Conntrack rates
ConntrackInsertRate float64
ConntrackInsertFailRate float64 // insert_failed/s — table full indicator
ConntrackDeleteRate float64
ConntrackDropRate float64
ConntrackEarlyDropRate float64 // forced evictions/s
ConntrackInvalidRate float64
ConntrackSearchRestartRate float64 // hash contention/s
ConntrackGrowthRate float64 // insert - delete (net change)
// SoftIRQ rates
SoftIRQNetRxRate float64
SoftIRQNetTxRate float64
SoftIRQBlockRate float64
// Cgroups
CgroupRates []CgroupRate
// Processes
ProcessRates []ProcessRate
}
RateSnapshot holds all computed rates between two snapshots.
type RemoteIPStats ¶
RemoteIPStats holds aggregated connection counts per remote IP.
type ReverseShellProc ¶
ReverseShellProc holds a candidate reverse shell process.
type RoleScore ¶
type RoleScore struct {
Role ServerRole `json:"role"`
Score int `json:"score"`
MaxScore int `json:"max_score"`
Confidence int `json:"confidence"` // percentage 0-100
Evidence []string `json:"evidence"`
}
RoleScore holds evidence-based confidence for a role classification.
type RuleStatus ¶
type RuleStatus int
RuleStatus indicates whether an audit rule passed.
const ( RulePass RuleStatus = iota RuleWarn RuleFail RuleSkip // not applicable to this role )
func (RuleStatus) String ¶
func (s RuleStatus) String() string
type RunbookMatch ¶
type RunbookMatch struct {
Name string `json:"name"`
Path string `json:"path"`
Score int `json:"score"`
Preview string `json:"preview,omitempty"`
}
RunbookMatch is the lightweight reference to a matched runbook file. The full markdown body stays in the engine's in-memory library and on disk — only the preview + path + score travel with AnalysisResult so the payload stays small and fleet-serializable.
type RuntimeEntry ¶
type RuntimeEntry struct {
Name string `json:"name"` // "jvm", "dotnet", etc.
DisplayName string `json:"display_name"` // "JVM", ".NET", etc.
Active bool `json:"active"`
Processes []RuntimeProcessMetrics `json:"processes"`
}
RuntimeEntry represents one detected language runtime and its processes.
type RuntimeMetrics ¶
type RuntimeMetrics struct {
Entries []RuntimeEntry `json:"entries,omitempty"`
}
RuntimeMetrics holds all detected language runtime data.
type RuntimeProcessMetrics ¶
type RuntimeProcessMetrics struct {
PID int `json:"pid"`
Comm string `json:"comm"`
Runtime string `json:"runtime"` // "jvm", "dotnet", "python", "node", "go"
WorkingSetMB float64 `json:"working_set_mb"`
ThreadCount int `json:"thread_count"`
GCHeapMB float64 `json:"gc_heap_mb,omitempty"`
GCPausePct float64 `json:"gc_pause_pct,omitempty"`
GCCount uint64 `json:"gc_count,omitempty"`
AllocRateMBs float64 `json:"alloc_rate_mbs,omitempty"`
Extra map[string]string `json:"extra,omitempty"`
}
RuntimeProcessMetrics holds metrics for a single process detected by a language runtime module.
type SMARTDisk ¶
type SMARTDisk struct {
Device string // e.g., "/dev/sda", "/dev/nvme0n1"
Name string // short name: "sda", "nvme0n1"
ModelFamily string
ModelNumber string
SerialNum string
DiskType DiskType
// Core health
HealthOK bool // SMART overall health passed
Temperature int // Celsius
PowerOnHours int
// Wear / endurance (NVMe + SSD)
WearLevelPct int // % life remaining (100=new, 0=dead, -1=unknown)
PercentUsed int // % endurance consumed (NVMe: 0=new, 100=EOL, -1=unknown)
// NVMe-specific health indicators
AvailableSpare int // % spare capacity remaining (NVMe only, -1=unknown)
AvailableSpareThreshold int // vendor-set minimum spare % (NVMe only, -1=unknown)
CriticalWarning uint8 // NVMe critical_warning bitmap
MediaErrors uint64 // NVMe media and data integrity errors
UnsafeShutdowns uint64
// Write endurance data
DataUnitsWritten uint64 // NVMe: each unit = 512KB (1000 × 512B sectors)
TotalBytesWritten uint64 // computed: total bytes written (all disk types)
TotalBytesRead uint64
// SATA-specific
ReallocSectors int // reallocated sector count
PendingSectors int // current pending sector count
// Life estimation (computed)
EstLifeDays int // estimated days of remaining life (-1=unknown)
WriteTBW float64 // total terabytes written
WriteRateTBPerYear float64 // write rate in TB/year (from total_written / power_on_hours)
// Source of data
Source string // "nvme_ioctl", "sata_ioctl", "smartctl"
ErrorString string // non-empty if collection failed
}
SMARTDisk holds SMART health data for a single disk.
func (*SMARTDisk) HealthVerdict ¶
HealthVerdict returns a human-readable health status.
type SUIDBinary ¶
SUIDBinary holds a SUID binary detected on the filesystem.
type SaturationDetail ¶
type SaturationDetail struct {
ConntrackPct float64 // conntrack table usage % (0-100)
EphemeralPct float64 // ephemeral port usage % (0-100)
RunqueueRatio float64 // load1 / nCPUs (0-1, clamped)
PSIMax float64 // max PSI stall across domains (0-1, normalized from %)
}
SaturationDetail breaks down saturation into individual components.
type SecurityMetrics ¶
type SecurityMetrics struct {
FailedAuthRate float64
FailedAuthTotal int
FailedAuthIPs []FailedAuthSource
NewPorts []NewListeningPort
SUIDAnomalies []SUIDBinary
ReverseShells []ReverseShellProc
BruteForce bool
Score string // "OK", "WARN", "CRIT"
// Network security watchdog results
TCPFlagAnomalies []TCPFlagAnomaly `json:"tcp_flag_anomalies,omitempty"`
DNSTunnelIndicators []DNSTunnelIndicator `json:"dns_tunnel_indicators,omitempty"`
JA3Fingerprints []JA3Entry `json:"ja3_fingerprints,omitempty"`
BeaconIndicators []BeaconIndicator `json:"beacon_indicators,omitempty"`
ThreatScore string `json:"threat_score"`
ActiveWatchdogs []string `json:"active_watchdogs,omitempty"`
}
SecurityMetrics holds real-time security signal data.
type SentinelConnLatEntry ¶
type SentinelConnLatEntry struct {
PID uint32
Comm string
Count uint32
AvgMs float64
MaxMs float64
DstStr string
}
SentinelConnLatEntry holds always-on BPF TCP connect latency per PID.
type SentinelData ¶
type SentinelData struct {
Active bool
AttachErr string
// Network sentinels
PktDrops []PktDropEntry
PktDropLocs []PktDropLocation // kernel functions where drops happen
PktDropProto []PktDropProto // which protocols are being dropped
TCPResets []TCPResetEntry
StateChanges []SockStateEntry
// Sentinel-promoted existing probes
Retransmits []SentinelRetransEntry
ConnLatency []SentinelConnLatEntry
// Security
ModLoads []ModLoadEntry
ExecEvents []ExecEventEntry
PtraceEvents []PtraceEventEntry
// Memory
OOMKills []OOMKillEntry
DirectReclaim []DirectReclaimEntry
// CPU
CgThrottles []CgThrottleEntry
// Network security sentinels
SynFlood []SynFloodEntry `json:"syn_flood,omitempty"`
PortScans []PortScanEntry `json:"port_scans,omitempty"`
DNSAnomaly []DNSAnomalyEntry `json:"dns_anomaly,omitempty"`
FlowRates []FlowRateEntry `json:"flow_rates,omitempty"`
OutboundTop []OutboundEntry `json:"outbound_top,omitempty"`
// Aggregate rates (computed from deltas)
PktDropRate float64
TCPResetRate float64
RetransRate float64
ReclaimStallMs float64
ThrottleRate float64
}
SentinelData holds always-on eBPF sentinel probe data.
type SentinelRetransEntry ¶
SentinelRetransEntry holds always-on BPF TCP retransmit data per PID.
type ServerIdentity ¶
type ServerIdentity struct {
DiscoveredAt time.Time `json:"discovered_at"`
Roles []ServerRole `json:"roles"`
RoleScores []RoleScore `json:"role_scores,omitempty"`
Services []DetectedService `json:"services"`
Containers []DockerContainer `json:"containers,omitempty"`
K8s *K8sNodeInfo `json:"k8s,omitempty"`
Websites []WebsiteInfo `json:"websites,omitempty"`
Databases []DatabaseInfo `json:"databases,omitempty"`
HAProxy *HAProxyInfo `json:"haproxy,omitempty"`
Keepalived *KeepalivedInfo `json:"keepalived,omitempty"`
VPN *VPNInfo `json:"vpn,omitempty"`
IPForward bool `json:"ip_forward"`
HasNFTables bool `json:"has_nftables"`
HasIPTables bool `json:"has_iptables"`
}
ServerIdentity is the complete result of server identity discovery.
func (*ServerIdentity) HasRole ¶
func (id *ServerIdentity) HasRole(role ServerRole) bool
HasRole returns true if the server has the given role.
func (*ServerIdentity) ServiceByName ¶
func (id *ServerIdentity) ServiceByName(name string) *DetectedService
ServiceByName returns the first service matching the given name, or nil.
type ServerProfile ¶
type ServerProfile struct {
Role ServerRole `json:"role"`
RoleDetail string `json:"role_detail"`
PanelName string `json:"panel_name"`
OverallScore int `json:"overall_score"`
Domains []DomainScore `json:"domains"`
Services []ServiceCensus `json:"services"`
}
ServerProfile is the complete system profiler output.
type ServerRole ¶
type ServerRole string
ServerRole identifies what purpose a server serves.
const ( RoleNATGateway ServerRole = "nat_gateway" RoleRouter ServerRole = "router" RoleFirewall ServerRole = "firewall" RoleWebServer ServerRole = "web_server" RoleDatabaseServer ServerRole = "database_server" RoleDockerHost ServerRole = "docker_host" RoleK8sNode ServerRole = "k8s_node" RoleMailServer ServerRole = "mail_server" RoleDNSServer ServerRole = "dns_server" RoleLoadBalancer ServerRole = "load_balancer" RoleCICDRunner ServerRole = "cicd_runner" RoleMonitoringServer ServerRole = "monitoring_server" RoleAppServer ServerRole = "app_server" RoleVPNServer ServerRole = "vpn_server" )
const ( RoleWebHosting ServerRole = "web_hosting" RoleHypervisor ServerRole = "hypervisor" RoleContainer ServerRole = "container_platform" RoleDatabase ServerRole = RoleDatabaseServer // alias for consistency RoleMixed ServerRole = "mixed_workload" RoleUnknown ServerRole = "unknown" )
Profiler-specific roles (extend ServerRole from identity.go)
type ServiceCensus ¶
type ServiceCensus struct {
Name string `json:"name"`
DisplayName string `json:"display_name"`
CPUPct float64 `json:"cpu_pct"`
RSSMB float64 `json:"rss_mb"`
IOPSRead float64 `json:"iops_read"`
IOPSWrite float64 `json:"iops_write"`
Connections int `json:"connections"`
Processes int `json:"processes"`
}
ServiceCensus describes a detected service and its resource usage.
type ServiceDiag ¶
type ServiceDiag struct {
Name string
Available bool
Findings []DiagFinding
WorstSev DiagSeverity
LastCheck time.Time
Metrics map[string]string // key metrics for TUI display
}
ServiceDiag holds diagnostic results for one service.
type ServiceLogStats ¶
type ServiceLogStats struct {
Name string
Unit string
ErrorRate float64
WarnRate float64
TotalErrors int
TotalWarns int
LastError string
RateHistory []float64 // ring buffer, 60 entries for sparkline
Findings []JournalFinding // Tier-2 structured findings; nil when quiet
}
ServiceLogStats holds per-service log error/warning stats.
type Snapshot ¶
type Snapshot struct {
HostID string // unique identifier for this host (hostname or user-configured)
Timestamp time.Time
Global GlobalMetrics
Cgroups []CgroupMetrics
Processes []ProcessMetrics
SysInfo *SysInfo
Errors []string
CollectionHealth *CollectionHealth
}
Snapshot holds a point-in-time system state.
type SockStateEntry ¶
type SockStateEntry struct {
OldState uint16
NewState uint16
OldStr string
NewStr string
Count uint64
Rate float64
}
SockStateEntry holds a BPF-traced TCP state transition count.
type SocketStats ¶
type SocketStats struct {
SocketsUsed int
TCPInUse int
TCPOrphan int
TCPTimeWait int
TCPAlloc int
TCPMem int // pages
UDPInUse int
UDPMem int
RawInUse int
FragInUse int
FragMem int
}
SocketStats holds socket counts from /proc/net/sockstat.
type SoftIRQStats ¶
type SoftIRQStats struct {
HI uint64
TIMER uint64
NET_TX uint64
NET_RX uint64
BLOCK uint64
IRQ_POLL uint64
TASKLET uint64
SCHED uint64
HRTIMER uint64
RCU uint64
}
SoftIRQStats holds per-type softirq counts from /proc/softirqs.
type SynFloodEntry ¶
type SynFloodEntry struct {
SrcIP string `json:"src_ip"`
SynCount uint64 `json:"syn_count"`
SynAckRetrans uint64 `json:"synack_retrans"`
HalfOpenRatio float64 `json:"half_open_ratio"`
Rate float64 `json:"rate"`
}
SynFloodEntry holds BPF-detected SYN flood indicators per source IP.
type SysInfo ¶
type SysInfo struct {
Hostname string
IPs []string
Virtualization string // "Bare Metal", "VM (KVM)", "VM (VMware)", "Container (Docker)", etc.
CloudProvider string // "AWS", "Hetzner", "DigitalOcean", "GCP", "Azure", etc.
Kernel string // kernel version
OS string // OS name from /etc/os-release
Arch string // architecture
CPUModel string // CPU model name
}
SysInfo holds host identity information (collected once).
type SystemChange ¶
type SystemChange struct {
Type string `json:"type"`
Detail string `json:"detail"`
When time.Time `json:"when"`
// Domain is the RCA domain for config_drift_* types (memory/cpu/network/io/limits/unknown).
// Empty for non-drift change types. Omitted from JSON when empty to keep
// the wire format additive and backward-compatible.
Domain string `json:"domain,omitempty"`
}
SystemChange represents a detected change on the system between ticks. Type vocabulary:
- "new_process" / "stopped_process" — process lifecycle (ChangeDetector)
- "package_install" / "package_upgrade" — dpkg/rpm events (ChangeDetector)
- "config_added" / "config_modified" / "config_removed" — file-level config drift (ConfigDriftDetector)
- "config_drift_memory" / "config_drift_cpu" / "config_drift_network" / "config_drift_io" / "config_drift_limits" / "config_drift_unknown" — kernel-parameter value drift vs persisted baseline (ParamDriftDetector, P4.3)
type TCPConnState ¶
type TCPConnState struct {
Established int
SynSent int
SynRecv int
FinWait1 int
FinWait2 int
TimeWait int
Close int
CloseWait int
LastAck int
Listen int
Closing int
}
TCPConnState holds counts per TCP state from /proc/net/tcp.
type TCPFlagAnomaly ¶
type TCPFlagAnomaly struct {
SrcIP string `json:"src_ip"`
FlagCombo string `json:"flag_combo"`
Count uint64 `json:"count"`
}
TCPFlagAnomaly holds BPF-detected unusual TCP flag combinations.
type TCPMetrics ¶
type TCPMetrics struct {
RetransSegs uint64
InSegs uint64
OutSegs uint64
ActiveOpens uint64
PassiveOpens uint64
CurrEstab uint64
AttemptFails uint64
EstabResets uint64
InErrs uint64
OutRsts uint64
}
TCPMetrics holds TCP-level counters from /proc/net/snmp.
type TCPResetEntry ¶
TCPResetEntry holds a BPF-traced TCP RST event per PID.
type TemporalChain ¶
type TemporalChain struct {
Events []TemporalEvent
Summary string // e.g. "retransmits (T+0s) → drops (T+3s) → threads blocked (T+12s)"
FirstMover string // evidence ID that fired first
}
TemporalChain tracks the order in which signals fired to establish causality.
type TemporalEvent ¶
TemporalEvent is a single signal onset in the temporal chain.
type TimelineEntry ¶
TimelineEntry is a timestamped milestone within an incident.
type TraceSample ¶
type TraceSample struct {
TraceID string `json:"trace_id"`
SpanID string `json:"span_id,omitempty"`
Service string `json:"service,omitempty"`
Operation string `json:"operation,omitempty"`
DurationMs float64 `json:"duration_ms"`
StatusCode string `json:"status_code,omitempty"` // "OK", "ERROR", "UNSET"
StatusError string `json:"status_error,omitempty"`
StartTime time.Time `json:"start_time"`
URL string `json:"url,omitempty"` // optional deep-link into Jaeger/Tempo
}
TraceSample is one OpenTelemetry trace summary correlated with an incident. The engine reads a simple JSONL feed at ~/.xtop/otel-samples.jsonl — an operator can produce it from their existing OTel collector via a processor that emits just these fields. Fields are intentionally minimal: everything needed to link to the full trace in whatever UI the operator already uses (Jaeger/Tempo/etc) plus enough context to display inline.
type UDPMetrics ¶
type UDPMetrics struct {
InDatagrams uint64
OutDatagrams uint64
InErrors uint64
NoPorts uint64
RcvbufErrors uint64
SndbufErrors uint64
}
UDPMetrics holds UDP counters from /proc/net/snmp.
type USECheck ¶
type USECheck struct {
Resource string // "CPU", "Memory", "Disk sda", "Network"
Utilization float64 // percentage (0-100)
Saturation float64 // queue length or pressure
Errors float64 // error count/rate
UtilStatus string // "ok", "warn", "crit"
SatStatus string
ErrStatus string
UtilDetail string // "25.3% busy"
SatDetail string // "runqueue 2/6 (33%)"
ErrDetail string // "0 errors"
}
USECheck represents one USE method check for a resource (Utilization, Saturation, Errors).
type VMStatMetrics ¶
type VMStatMetrics struct {
PgFault uint64
PgMajFault uint64
PgPgIn uint64
PgPgOut uint64
PswpIn uint64
PswpOut uint64
PgStealDirect uint64
PgStealKswapd uint64
PgScanDirect uint64
PgScanKswapd uint64
AllocStall uint64
CompactStall uint64
OOMKill uint64
NrDirtied uint64
NrWritten uint64
ThpFaultAlloc uint64
ThpCollapseAlloc uint64
}
VMStatMetrics holds selected /proc/vmstat counters.
type VPNInfo ¶
type VPNInfo struct {
Type string `json:"type"` // "wireguard", "openvpn", "ipsec"
Interface string `json:"interface,omitempty"` // "wg0", "tun0", etc.
Port int `json:"port,omitempty"`
Peers int `json:"peers,omitempty"`
Container string `json:"container,omitempty"` // container name if containerized
Evidence []string `json:"evidence,omitempty"`
}
VPNInfo holds VPN detection results.
type VerificationTier ¶
type VerificationTier string
VerificationTier classifies how strongly we believe a candidate cause. Strict ordering: A > B > C > D. Only Tier A counts toward the 0.1% precision goal — everything else MUST be treated as "the engine abstained from a strong call."
const ( // TierAConfirmed requires: // - 3 or more independent evidence families // - correct temporal sequence (cause before effect, sustained) // - owner match (claimed root entity owns the stressed resource) // - no counter-evidence // - deep-probe confirmation if the gate demanded one TierAConfirmed VerificationTier = "A_confirmed" // TierBVerified requires: // - 2 strong evidence families // - no major contradictions // - ownership-consistency passes TierBVerified VerificationTier = "B_verified" // TierCProbable means the mechanism is plausible but at least one // proof gate did not pass strongly. Callers should treat as a // "best guess" — do not act on it without operator review. TierCProbable VerificationTier = "C_probable" // TierDInconclusive is the abstain output. Impact exists, but the // engine refuses to commit to a root cause. This is the HEALTHY // default for weak evidence. Per NEXTGEN: "If proof is weak, // return INCONCLUSIVE." TierDInconclusive VerificationTier = "D_inconclusive" )
type VerifiedCause ¶
type VerifiedCause struct {
// Mechanism is the proposed causal mechanism. Human-readable, e.g.
// "CPU contention in service mongod" or "swap-induced disk stall
// impacting nginx". The verifier doesn't generate this — the
// hypothesis engine (Phase 4 candidate generator) does. The
// verifier inherits it.
Mechanism string `json:"mechanism"`
// Tier is the headline output. Tier A = trust this. Tier D = the
// engine refused to commit.
Tier VerificationTier `json:"tier"`
// RootEntityID is the claimed root cause's entity (from EntityGraph).
// Empty when no root entity could be identified (Tier D outputs
// often have an empty root).
RootEntityID string `json:"root_entity_id,omitempty"`
// BlastRadius lists the affected entities. Empty for host-scope
// causes.
BlastRadius []string `json:"blast_radius,omitempty"`
// Confidence is a derived 0-100 score combining the gates that
// passed. NOT a probability — a calibrated heuristic. Operators
// should use Tier as the trust signal, not this number.
Confidence int `json:"confidence"`
// Gates is the full record of every gate that ran against this
// candidate, in evaluation order. Tier-degradation can be traced
// back to the first failing gate.
Gates []GateResult `json:"gates"`
// EvaluatedAt is when the verifier produced this output.
EvaluatedAt time.Time `json:"evaluated_at"`
}
VerifiedCause is one verifier output — a candidate that's been put through every applicable gate, with the resulting tier.
Phase 4 emits one of these per RCA candidate; AnalyzeRCA aggregates them into AnalysisResult.VerifiedCauses. Phase 5 (replay) serializes the full set so an offline harness can re-derive the same tier from the same facts + graph.
type Warning ¶
type Warning struct {
Severity string // "info", "warn", "crit"
Signal string // short label
Detail string // explanation
Value string // current value string
}
Warning represents an early-warning signal.
type WatchdogState ¶
WatchdogState holds auto-trigger state from the watchdog.
type WebsiteInfo ¶
type WebsiteInfo struct {
Domain string `json:"domain"`
Port int `json:"port"`
ConfigFile string `json:"config_file"`
SSLExpiry string `json:"ssl_expiry,omitempty"`
}
WebsiteInfo holds discovered website/vhost information.
type WebsiteMetrics ¶
type WebsiteMetrics struct {
Domain string `json:"domain"`
Active bool `json:"active"`
CPUPct float64 `json:"cpu_pct"`
RSSMB float64 `json:"rss_mb"`
Workers int `json:"workers"`
MaxWorkers int `json:"max_workers"`
HitsPerMin int `json:"hits_per_min"`
DBSizeMB float64 `json:"db_size_mb"`
DiskMB float64 `json:"disk_mb"`
PHPVersion string `json:"php_version,omitempty"`
}
WebsiteMetrics holds per-website resource usage.
type ZScoreAnomaly ¶
type ZScoreAnomaly struct {
EvidenceID string // e.g. "cpu.busy"
Value float64 // current value
WindowMean float64 // mean over sliding window
WindowStd float64 // stddev over sliding window
ZScore float64 // (value - mean) / std
}
ZScoreAnomaly represents a value that is statistically unusual vs recent history.