Documentation
¶
Overview ¶
Package collect is github-scout's scan orchestrator. One Collector instance lives for the process lifetime; each Scan gathers the four actionable GitHub signals across all of an owner's repos and emits them as structured log lines for Alloy to ship to Loki.
Two emission models (see internal/model):
- Event-once (Actions runs): each completed run ID is emitted a single time as msg="workflow run" with its conclusion, so a plain LogQL count equals the number of distinct runs and the dashboard filters that stream for failures and computes the failure rate. The dedup set is in-memory, pruned to the lookback window.
- Snapshot (open PRs, open issues, code-scanning alerts): the full current set is emitted every scan. A closed/merged/fixed item simply stops appearing in later snapshots, so the dashboard reads the most recent scan as "what is open right now" — no dedup state needed.
The only cross-scan state is the run dedup set. Production (main.go) sets Deps.StatePath wherever a scan runs -- the scheduled daemon and every `trigger` exec (in resident-idle mode the resident daemon never scans, so only the trigger execs persist) -- so the set is persisted to a small JSON file (e.g. /tmp/seen-runs.json, shared across `docker exec` triggers of the same running container) at the end of each scan and reloaded at the next process start; a plain restart or a fresh `trigger` then re-emits nothing. Persistence is a flock'd single-slot read-modify-write transaction (scheduler.SlotFile) whose save merges the on-disk set with the in-memory one, so concurrent writers (the scheduled daemon racing an exec'd trigger) never lose each other's entries to a last-writer-wins overwrite. Leaving Deps.StatePath empty keeps the set in memory only (used in tests). Either way a cold start — the first run, or a container recreate that clears /tmp — at worst re-emits runs still inside the lookback window (the dashboard dedups run counts by run_id), so persistence is a best-effort optimization, never a correctness dependency.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Collector ¶
type Collector struct {
// contains filtered or unexported fields
}
Collector holds the cross-scan state: the GitHub client, scan parameters, and the in-memory failed-run dedup set. Construct via New.
func New ¶
New constructs a Collector. When Deps.StatePath is set, the persisted run dedup set is loaded from it (a missing or corrupt file simply starts the set empty); otherwise the set starts empty. Takes *Deps to avoid copying the (large) struct.
func (*Collector) Scan ¶
Scan runs one full cycle and returns whether it was healthy. Health is a LIVENESS verdict: true when repo discovery succeeded, false only when it failed (bad token, rate limit) — the one condition a restart might clear. Per-signal collection failures deliberately do NOT flip health: a restart cannot fix a missing token scope or a sustained rate limit, so flapping the container would be noise. Instead the scan reports DATA INTEGRITY on its own channel via scanIntegrity. A signal that could not be read makes its reported "0" mean "could not check", not "nothing there" — and for code scanning that is a security false-negative — so "scan complete" carries `errors`, `degraded`, and `failed_signals`, and a SYSTEMIC failure (a rejected token or rate limit, or a signal blind across every repo) is escalated to a distinct ERROR-level "scan degraded" line that the shared error panel and the Loki ruler alert key on.
type Deps ¶
type Deps struct {
Client apiClient
Logger *slog.Logger
Now func() time.Time
Exclude map[string]bool
CodeScanningExclude map[string]bool
Owner string
PRExclude string
IssueExclude string
// StatePath, when non-empty, is where the run dedup set is persisted
// (JSON in a flock'd scheduler.SlotFile, merged with any concurrent
// writer's entries) at the end of each scan and reloaded by New. Set it for
// trigger-mode deployments (each trigger is a fresh process) to a path
// shared across execs of the same container, e.g. /tmp/seen-runs.json.
// Production (main.go) sets it wherever a scan runs -- the scheduled
// daemon and every `trigger` exec (in resident-idle mode the resident
// daemon never scans, so only the trigger execs persist). The dedup set
// therefore survives a plain restart and each completed run is emitted
// once regardless of process lifetime.
// Leave empty only for in-memory-only dedup (used in tests).
StatePath string
Lookback time.Duration
}
Deps are the constructor arguments for New. A nil Logger falls back to slog.Default; a nil Now falls back to time.Now.
Source Files