inspector

package
v0.102.1-nightly Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Feb 11, 2026 License: AGPL-3.0 Imports: 16 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source
var SubsystemCheckers = map[string]CheckFunc{}

SubsystemCheckers maps subsystem names to their check functions. Populated by checks/ package init or by explicit registration.

Functions

func PrintAnalysis

func PrintAnalysis(result *AnalysisResult, w io.Writer)

PrintAnalysis writes the AI analysis to the output, one section per subsystem.

func PrintJSON

func PrintJSON(results *Results, w io.Writer)

PrintJSON writes check results as JSON.

func PrintTable

func PrintTable(results *Results, w io.Writer)

PrintTable writes a human-readable table of check results.

func RegisterChecker

func RegisterChecker(subsystem string, fn CheckFunc)

RegisterChecker registers a check function for a subsystem.

func SummaryLine

func SummaryLine(results *Results) string

SummaryLine returns a one-line summary string.

func WriteResults

func WriteResults(baseDir, env string, results *Results, data *ClusterData, analysis *AnalysisResult) (string, error)

WriteResults saves inspection results as markdown files to a timestamped directory. Returns the output directory path.

Types

type AnalysisResult

type AnalysisResult struct {
	Model    string
	Analyses []SubsystemAnalysis
	Duration time.Duration
}

AnalysisResult holds the AI's analysis of check failures.

func Analyze

func Analyze(results *Results, data *ClusterData, model, apiKey string) (*AnalysisResult, error)

Analyze sends failures and cluster context to OpenRouter for AI analysis. Each subsystem with issues gets its own API call, run in parallel.

func AnalyzeGroups

func AnalyzeGroups(groups []FailureGroup, results *Results, data *ClusterData, model, apiKey string) (*AnalysisResult, error)

AnalyzeGroups sends each failure group to OpenRouter for focused AI analysis. Unlike Analyze which sends one call per subsystem, this sends one call per unique failure pattern, producing more focused and actionable results.

type AnyoneData

type AnyoneData struct {
	RelayActive      bool            // debros-anyone-relay systemd service active
	ClientActive     bool            // debros-anyone-client systemd service active
	ORPortListening  bool            // port 9001 bound locally
	SocksListening   bool            // port 9050 bound locally (client SOCKS5)
	ControlListening bool            // port 9051 bound locally (control port)
	Bootstrapped     bool            // relay has bootstrapped to 100%
	BootstrapPct     int             // bootstrap percentage (0-100)
	Fingerprint      string          // relay fingerprint
	Nickname         string          // relay nickname
	UptimeStr        string          // uptime from control port
	ORPortReachable  map[string]bool // host IP → whether we can TCP connect to their 9001 from this node
}

AnyoneData holds parsed Anyone relay/client status from a node.

type CheckFunc

type CheckFunc func(data *ClusterData) []CheckResult

CheckFunc is the signature for a subsystem check function.

type CheckResult

type CheckResult struct {
	ID        string   `json:"id"`        // e.g. "rqlite.leader_exists"
	Name      string   `json:"name"`      // "Cluster has exactly one leader"
	Subsystem string   `json:"subsystem"` // "rqlite"
	Severity  Severity `json:"severity"`
	Status    Status   `json:"status"`
	Message   string   `json:"message"`        // human-readable detail
	Node      string   `json:"node,omitempty"` // which node (empty for cluster-wide)
}

CheckResult holds the outcome of a single health check.

func Fail

func Fail(id, name, subsystem, node, msg string, sev Severity) CheckResult

Fail creates a failing check result.

func Pass

func Pass(id, name, subsystem, node, msg string, sev Severity) CheckResult

Pass creates a passing check result.

func Skip

func Skip(id, name, subsystem, node, msg string, sev Severity) CheckResult

Skip creates a skipped check result.

func Warn

func Warn(id, name, subsystem, node, msg string, sev Severity) CheckResult

Warn creates a warning check result.

type ClusterData

type ClusterData struct {
	Nodes    map[string]*NodeData // keyed by host IP
	Duration time.Duration
}

ClusterData holds all collected data from the cluster.

func Collect

func Collect(ctx context.Context, nodes []Node, subsystems []string, verbose bool) *ClusterData

Collect gathers data from all nodes in parallel.

type DNSData

type DNSData struct {
	CoreDNSActive   bool
	CaddyActive     bool
	Port53Bound     bool
	Port80Bound     bool
	Port443Bound    bool
	CoreDNSMemMB    int
	CoreDNSRestarts int
	LogErrors       int // error count in recent CoreDNS logs
	// Resolution tests (dig results)
	SOAResolves      bool
	NSResolves       bool
	NSRecordCount    int
	WildcardResolves bool
	BaseAResolves    bool
	// TLS
	BaseTLSDaysLeft int // -1 = failed to check
	WildTLSDaysLeft int // -1 = failed to check
	// Corefile
	CorefileExists bool
}

DNSData holds parsed DNS/CoreDNS status from a nameserver node.

type FailureGroup

type FailureGroup struct {
	ID        string
	Name      string // from first check in group
	Status    Status
	Severity  Severity
	Subsystem string
	Nodes     []string // affected node names (deduplicated)
	Messages  []string // unique messages (capped at 5)
	Count     int      // total raw occurrence count (before dedup)
}

FailureGroup groups identical check failures/warnings across nodes.

func GroupFailures

func GroupFailures(results *Results) []FailureGroup

GroupFailures collapses CheckResults into unique failure groups keyed by (ID, Status). Only failures and warnings are grouped; passes and skips are ignored.

type IPFSData

type IPFSData struct {
	DaemonActive     bool
	ClusterActive    bool
	SwarmPeerCount   int
	ClusterPeerCount int
	RepoSizeBytes    int64
	RepoMaxBytes     int64
	KuboVersion      string
	ClusterVersion   string
	ClusterErrors    int // peers reporting errors
	HasSwarmKey      bool
	BootstrapEmpty   bool // true if bootstrap list is empty (private swarm)
}

IPFSData holds parsed IPFS status from a single node.

type NamespaceData

type NamespaceData struct {
	Name          string // namespace name (from systemd unit)
	PortBase      int    // starting port of the 5-port block
	RQLiteUp      bool   // RQLite HTTP port responding
	RQLiteState   string // Raft state (Leader/Follower)
	RQLiteReady   bool   // /readyz
	OlricUp       bool   // Olric memberlist port listening
	GatewayUp     bool   // Gateway HTTP port responding
	GatewayStatus int    // HTTP status code from gateway health
}

NamespaceData holds data for a single namespace on a node.

type NetworkData

type NetworkData struct {
	InternetReachable bool
	TCPEstablished    int
	TCPTimeWait       int
	TCPRetransRate    float64 // retransmission % from /proc/net/snmp
	DefaultRoute      bool
	WGRouteExists     bool
	PingResults       map[string]bool // WG peer IP → ping success
}

NetworkData holds parsed network-level data from a node.

type Node

type Node struct {
	Environment string // devnet, testnet
	User        string // SSH user
	Host        string // IP or hostname
	Password    string // SSH password
	Role        string // node, nameserver-ns1, nameserver-ns2, nameserver-ns3
	SSHKey      string // optional path to SSH key
}

Node represents a remote node parsed from remote-nodes.conf.

func FilterByEnv

func FilterByEnv(nodes []Node, env string) []Node

FilterByEnv returns only nodes matching the given environment.

func FilterByRole

func FilterByRole(nodes []Node, rolePrefix string) []Node

FilterByRole returns only nodes matching the given role prefix.

func LoadNodes

func LoadNodes(path string) ([]Node, error)

LoadNodes parses a remote-nodes.conf file into a slice of Nodes. Format: environment|user@host|password|role|ssh_key (ssh_key optional)

func RegularNodes

func RegularNodes(nodes []Node) []Node

RegularNodes returns non-nameserver nodes.

func (Node) IsNameserver

func (n Node) IsNameserver() bool

IsNameserver returns true if the node has a nameserver role.

func (Node) Name

func (n Node) Name() string

Name returns a short display name for the node (user@host).

type NodeData

type NodeData struct {
	Node       Node
	RQLite     *RQLiteData
	Olric      *OlricData
	IPFS       *IPFSData
	DNS        *DNSData
	WireGuard  *WireGuardData
	System     *SystemData
	Network    *NetworkData
	Anyone     *AnyoneData
	Namespaces []NamespaceData // namespace instances on this node
	Errors     []string        // collection errors for this node
}

NodeData holds collected data for a single node.

type OlricData

type OlricData struct {
	ServiceActive bool
	MemberlistUp  bool
	MemberCount   int
	Members       []string // memberlist member addresses
	Coordinator   string   // current coordinator address
	LogErrors     int      // error count in recent logs
	LogSuspects   int      // "suspect" or "Marking as failed" count
	LogFlapping   int      // rapid join/leave count
	ProcessMemMB  int      // RSS memory in MB
	RestartCount  int      // NRestarts from systemd
}

OlricData holds parsed Olric status from a single node.

type RQLiteData

type RQLiteData struct {
	Responsive bool
	StatusRaw  string                 // raw JSON from /status
	NodesRaw   string                 // raw JSON from /nodes?nonvoters
	ReadyzRaw  string                 // raw response from /readyz
	DebugRaw   string                 // raw JSON from /debug/vars
	Status     *RQLiteStatus          // parsed /status
	Nodes      map[string]*RQLiteNode // parsed /nodes
	Readyz     *RQLiteReadyz          // parsed /readyz
	DebugVars  *RQLiteDebugVars       // parsed /debug/vars
	StrongRead bool                   // SELECT 1 with level=strong succeeded
}

RQLiteData holds parsed RQLite status from a single node.

type RQLiteDebugVars

type RQLiteDebugVars struct {
	QueryErrors      uint64
	ExecuteErrors    uint64
	RemoteExecErrors uint64
	LeaderNotFound   uint64
	SnapshotErrors   uint64
	ClientRetries    uint64
	ClientTimeouts   uint64
}

RQLiteDebugVars holds metrics from /debug/vars.

type RQLiteNode

type RQLiteNode struct {
	Addr      string
	Reachable bool
	Leader    bool
	Voter     bool
	Time      float64 // response time
	Error     string
}

RQLiteNode holds parsed fields from /nodes response per node.

type RQLiteReadyz

type RQLiteReadyz struct {
	Ready   bool
	Store   string // "ready" or error
	Leader  string // "ready" or error
	Node    string // "ready" or error
	RawBody string
}

RQLiteReadyz holds parsed readiness state.

type RQLiteStatus

type RQLiteStatus struct {
	RaftState      string // Leader, Follower, Candidate, Shutdown
	LeaderNodeID   string // store.leader.node_id
	LeaderAddr     string // store.leader.addr
	NodeID         string // store.node_id
	Term           uint64 // store.raft.term (current_term)
	AppliedIndex   uint64 // store.raft.applied_index
	CommitIndex    uint64 // store.raft.commit_index
	FsmPending     uint64 // store.raft.fsm_pending
	LastContact    string // store.raft.last_contact (followers only)
	LastLogIndex   uint64 // store.raft.last_log_index
	LastLogTerm    uint64 // store.raft.last_log_term
	NumPeers       int    // store.raft.num_peers (string in JSON)
	LastSnapshot   uint64 // store.raft.last_snapshot_index
	Voter          bool   // store.raft.voter
	DBSize         int64  // store.sqlite3.db_size
	DBSizeFriendly string // store.sqlite3.db_size_friendly
	DBAppliedIndex uint64 // store.db_applied_index
	FsmIndex       uint64 // store.fsm_index
	Uptime         string // http.uptime
	Version        string // build.version
	GoVersion      string // runtime.GOARCH + runtime.version
	Goroutines     int    // runtime.num_goroutine
	HeapAlloc      uint64 // runtime.memory.heap_alloc (bytes)
}

RQLiteStatus holds parsed fields from /status.

type Results

type Results struct {
	Checks   []CheckResult `json:"checks"`
	Duration time.Duration `json:"duration"`
}

Results holds all check outcomes.

func RunChecks

func RunChecks(data *ClusterData, subsystems []string) *Results

RunChecks executes checks for the requested subsystems against collected data.

func (*Results) Failures

func (r *Results) Failures() []CheckResult

Failures returns only failed checks.

func (*Results) FailuresAndWarnings

func (r *Results) FailuresAndWarnings() []CheckResult

FailuresAndWarnings returns failed and warning checks.

func (*Results) Summary

func (r *Results) Summary() (passed, failed, warned, skipped int)

Summary returns counts by status.

type SSHResult

type SSHResult struct {
	Stdout   string
	Stderr   string
	ExitCode int
	Duration time.Duration
	Err      error
	Retries  int // how many retries were needed
}

SSHResult holds the output of an SSH command execution.

func RunSSH

func RunSSH(ctx context.Context, node Node, command string) SSHResult

RunSSH executes a command on a remote node via SSH with retry on connection failure. Uses sshpass for password auth, falls back to -i for key-based auth. The -n flag is used to prevent SSH from reading stdin.

func RunSSHMulti

func RunSSHMulti(ctx context.Context, node Node, commands []string) SSHResult

RunSSHMulti executes a multi-command string on a remote node. Commands are joined with " && " so failure stops execution.

func (SSHResult) OK

func (r SSHResult) OK() bool

OK returns true if the command succeeded (exit code 0, no error).

type Severity

type Severity int

Severity levels for check results.

const (
	Low Severity = iota
	Medium
	High
	Critical
)

func (Severity) String

func (s Severity) String() string

type Status

type Status string

Status represents the outcome of a check.

const (
	StatusPass Status = "pass"
	StatusFail Status = "fail"
	StatusWarn Status = "warn"
	StatusSkip Status = "skip"
)

type SubsystemAnalysis

type SubsystemAnalysis struct {
	Subsystem string
	GroupID   string // e.g. "anyone.bootstrapped" — empty when analyzing whole subsystem
	Analysis  string
	Duration  time.Duration
	Error     error
}

SubsystemAnalysis holds the AI analysis for a single subsystem or failure group.

type SystemData

type SystemData struct {
	Services       map[string]string // service name → status
	FailedUnits    []string          // systemd units in failed state
	MemTotalMB     int
	MemUsedMB      int
	MemFreeMB      int
	DiskTotalGB    string
	DiskUsedGB     string
	DiskAvailGB    string
	DiskUsePct     int
	UptimeRaw      string
	LoadAvg        string
	CPUCount       int
	OOMKills       int
	SwapUsedMB     int
	SwapTotalMB    int
	InodePct       int   // inode usage percentage
	ListeningPorts []int // ports from ss -tlnp
	UFWActive      bool
	ProcessUser    string // user running debros-node (e.g. "debros")
	PanicCount     int    // panic/fatal in recent logs
}

SystemData holds parsed system-level data from a node.

type WGPeer

type WGPeer struct {
	PublicKey       string
	Endpoint        string
	AllowedIPs      string
	LatestHandshake int64 // seconds since epoch, 0 = never
	TransferRx      int64
	TransferTx      int64
	Keepalive       int
}

WGPeer holds parsed data for a single WireGuard peer.

type WireGuardData

type WireGuardData struct {
	InterfaceUp   bool
	ServiceActive bool
	WgIP          string
	PeerCount     int
	Peers         []WGPeer
	MTU           int
	ListenPort    int
	ConfigExists  bool
	ConfigPerms   string // e.g. "600"
}

WireGuardData holds parsed WireGuard status from a node.

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL