Documentation
¶
Overview ¶
Package incident turns production error reports into journaled facts, folds those facts into incidents, and reacts by minting agent fix tasks (ADR-0021) — the command/event spine of error-driven auto-fix:
- Command: Report (from a web-client beacon or server panic/5xx middleware) arrives via the API and is recorded ONCE as an error.observed fact on the synthetic incident:<fingerprint> stream.
- Projection: the fold derives incident state (occurrences, status, minted tasks, regressions) purely from journal facts — never a mirrored column.
- Policy: the watermark-cursor sweep (the sweeper seam shared with review/dlqfix/status, ADR-0009 exact-consumer semantics) reacts to each new fact: first occurrence mints a hot-band agent fix task, occurrences on an open incident only count, an error observed after the fix task completed is a regression and mints at machine band.
Storm safety: 1,000 identical reports in one day are 1,000 facts but at most ONE task per incident lifecycle stage; replayed pages (crash between consumption and checkpoint) converge on task dedup keys instead of minting duplicates — the same idempotency contract as every sweeper.
Index ¶
- Constants
- func Fingerprint(r Report) string
- func FingerprintOfID(id string) string
- func IncidentID(fingerprint string) string
- func TaskDedupKey(fingerprint string, seq int64) string
- func TopFrame(stack string) string
- type FactSink
- type Incident
- type Kind
- type Mint
- type Policy
- type PolicyConfig
- type PolicyStats
- type RecordResult
- type Recorder
- type Report
- type State
- type Status
Constants ¶
const ( MaxMessage = 4096 MaxStack = 8192 MaxBreadcrumbs = 20 MaxCrumb = 256 MaxShort = 256 )
Field caps enforced by Clip before anything is journaled — the journal is append-only and the dashboard renders payloads, so reports are bounded at the boundary, not trusted from the wire.
const ( // TaskDedupPrefix namespaces minted fix-task dedup keys. TaskDedupPrefix = "err:" // DefaultFirstPriority mints a fresh incident's fix task in the ADR-0015 // hot band (100-149): production errors jump the backlog. DefaultFirstPriority = 120 // DefaultRegressionPriority mints a regression (the previous fix did // not hold) in the machine band (150+): operational work. DefaultRegressionPriority = 150 // DefaultMaxAttempts bounds burned failures on a minted fix task — a // broken repo must not drain the attempt budget. DefaultMaxAttempts = 2 )
const ConsumerKey = "incident-policy"
ConsumerKey is the policy's identity in the watermarks table: the persisted cursor shared by every policy run over the same database, so error facts observed while no policy was running still mint their fix tasks on the next pool start.
const IDPrefix = "incident:"
IDPrefix marks synthetic incident-stream fact TaskIDs (the session:* precedent: an identity no enqueue/claim machinery can ever pick up).
Variables ¶
This section is empty.
Functions ¶
func Fingerprint ¶
Fingerprint groups reports that share one bug: project + kind + the normalized message + the top stack frame (numbers, quoted strings, and whitespace collapsed). The release is deliberately EXCLUDED so the same bug groups across releases — a recurrence after a fix is the regression signal, not a new release's fresh incident. Returns 16 hex chars.
func FingerprintOfID ¶
FingerprintOfID extracts the fingerprint from an incident stream ID ("" when the ID does not carry the prefix).
func IncidentID ¶
IncidentID is the synthetic journal-stream identity for one fingerprint: the fact TaskID every error.observed and incident.task-minted fact of this incident carries.
func TaskDedupKey ¶
TaskDedupKey identifies one mint: fingerprint + the observed fact's Seq. Replayed deliveries of that fact converge on the stored task; a NEW observation (regression) carries a new Seq and mints afresh.
Types ¶
type FactSink ¶
FactSink appends NON-task journal facts (Store.AppendFact satisfies it; the session-bridge seam precedent — facts ride the same append-only log as task facts, in their own transaction).
type Incident ¶
type Incident struct {
Fingerprint string
Project string
Kind Kind
Message string
FirstSeen time.Time
LastSeen time.Time
Occurrences int
Regressions int
Status Status
Mints []Mint
}
Incident is one fingerprint's folded state — the read model of the error.observed / incident.task-minted fact family.
type Mint ¶
type Mint struct {
TaskID string `json:"taskID"`
SourceSeq int64 `json:"sourceSeq"`
Regression bool `json:"regression"`
Priority int `json:"priority"`
// Outcome is the minted task's terminal state folded from its own
// lifecycle facts: "" while running, "completed", or "dead".
Outcome string `json:"outcome"`
// DoneSeq is the Seq of the terminal fact (ordering evidence for the
// regression rule).
DoneSeq int64 `json:"doneSeq,omitempty"`
}
Mint records one fix task minted for an incident, as journaled by the incident.task-minted fact.
type Policy ¶
type Policy struct {
// contains filtered or unexported fields
}
Policy is the reaction half of the pipeline: a watermark-cursor sweeper (the seam shared with review/dlqfix/status; ADR-0009 exact-consumer semantics) that folds every incident-family fact into the State and mints agent fix tasks for incidents that need one. Safe for concurrent use (the cursor serializes sweeps).
art-dupl:accept mirrored sweeper shell: config fields and fact validation differ per sweeper; the shared pump already lives in watermark.Cursor.
func NewPolicy ¶
func NewPolicy(ctx context.Context, store queue.Store, sink FactSink, cfg PolicyConfig) (*Policy, error)
NewPolicy returns a policy over store (facts, enqueue, watermarks) and sink (non-task fact appends; the same object satisfies both in production). The cursor resumes from the persisted checkpoint — error facts observed while no policy was running mint on the next start. A FIRST run deliberately replays the whole journal (see the checkpoint note in the constructor): the family is new, so nothing predates it.
The full journal is folded into the state so the projection is current; only facts after the persisted cursor are REACTED to — the seq guard makes the overlap idempotent.
func (*Policy) Sweep ¶
func (p *Policy) Sweep(ctx context.Context) (PolicyStats, error)
Sweep consumes new facts since the last pass and mints fix tasks for incidents that need one. Idempotent: a replayed page converges on task dedup keys and the fold's seq guard instead of duplicating work.
A mint refusal — a store failure while enqueueing the fix task or appending its mint fact — aborts the sweep BEFORE the checkpoint: the observation redelivers on the next pass instead of silently dropping the mint forever (dlqfix 13-15 §e pattern). Redelivery is dedup-safe: the enqueued task converges on its stored record, and the pending guard is re-established. A completed fix-task dedup key (ErrTaskDone) counts known instead of refusing.
type PolicyConfig ¶
type PolicyConfig struct {
// Log receives mint failures and skips. nil logs nothing.
Log *slog.Logger
// PageSize bounds one fact-stream page; 0 selects the default.
PageSize int
// FirstPriority mints first-occurrence fix tasks; 0 selects the default.
FirstPriority int
// RegressionPriority mints regression fix tasks; 0 selects the default.
RegressionPriority int
// MaxAttempts bounds minted fix tasks; 0 selects the default.
MaxAttempts int
}
PolicyConfig controls one Policy.
type PolicyStats ¶
type PolicyStats struct {
// Facts is the number of facts consumed.
Facts int
// TasksMinted counts fix tasks enqueued (dedup converges replays).
TasksMinted int
// Occurrences counts error.observed facts delivered (storms included).
Occurrences int
// Regressions counts observations that reopened a terminal incident.
Regressions int
// Known counts deliveries whose fix task was already minted (replay or
// in-flight mint).
Known int
// Skipped counts facts that could not yield a mint (bad detail, foreign
// shape) — the incident stays open and the next occurrence re-arms. A
// refused MINT is not counted here: it aborts the sweep before the
// checkpoint so the observation redelivers.
Skipped int
}
PolicyStats summarizes one sweep pass.
type RecordResult ¶
type RecordResult struct {
// Incident is the incident stream identity ("incident:<fingerprint>").
Incident string `json:"incident"`
}
RecordResult reports where the report landed.
type Recorder ¶
type Recorder struct {
// contains filtered or unexported fields
}
Recorder is the ReportError command handler: validate, cap, fingerprint, and append ONE error.observed fact to the journal. It reacts to nothing — the policy sweeper owns the reactions, so recording stays cheap enough for the error path of a production service.
func NewRecorder ¶
NewRecorder returns a Recorder appending through sink.
type Report ¶
type Report struct {
// Project is the repo (or logical owner) the error belongs to and the
// unit the minted fix task's Project/Repo derive from. Required.
Project string `json:"project"`
// Kind selects client or server. Empty defaults to server.
Kind Kind `json:"kind"`
// Message is the error message (exception text or panic value).
Message string `json:"message"`
// Stack is the raw stack (minified frames are fine; the agent resolves
// source maps from the pinned Release in-repo).
Stack string `json:"stack,omitempty"`
// Release is the commit SHA (or version) the error was observed on.
Release string `json:"release,omitempty"`
// Route is the route pattern (server) or path (client), not the raw
// URL — query strings never enter the journal.
Route string `json:"route,omitempty"`
// Method is the HTTP method, server errors only.
Method string `json:"method,omitempty"`
// TraceURL is a deep link to the full trace in the observability stack
// (SigNoz) — the query surface; this pipeline is the action surface.
TraceURL string `json:"traceUrl,omitempty"`
// SessionID correlates reports from one browser session.
SessionID string `json:"sessionId,omitempty"`
// Breadcrumbs are the last user actions/requests before the error.
Breadcrumbs []string `json:"breadcrumbs,omitempty"`
// OccurredAt is the observation time (zero = recording time).
OccurredAt time.Time `json:"occurredAt"`
}
Report is the ReportError command payload. Secrets are unrepresentable by construction: the type has no place to carry headers, cookies, or request bodies — only the fields a fix needs. Free-text fields are length-capped by Clip.
type State ¶
type State struct {
// contains filtered or unexported fields
}
State is the fold over the journal: incidents keyed by fingerprint, first-appearance order preserved, plus the task→incident mint index. A replayed fact (Seq at or below the folded high-water mark) is a no-op, so at-least-once delivery never double-counts.
func FoldAll ¶
FoldAll builds the incident read model from the whole journal without touching watermarks — the read-only fold behind `tq incidents`. Unlike NewPolicy it reacts to nothing and persists nothing, so CLI reads never move a consumer cursor.
func (*State) Apply ¶
Apply folds one fact. Returns false when the fact was already folded (Seq guard) or does not belong to a known family — both are no-ops.
func (*State) FingerprintOfTask ¶
FingerprintOfTask resolves which incident a task was minted for ("" when the task is not a minted fix task).
type Status ¶
type Status string
Status is an incident's derived lifecycle state. It is NEVER stored — the fold recomputes it from facts, so "resolved" means "the latest minted fix task completed", by construction.
const ( // StatusOpen: observed, no fix task minted yet (or regressed after a // terminal fix task — the regression path reopens the incident). StatusOpen Status = "open" // StatusFixDispatched: a fix task is minted and not terminal. StatusFixDispatched Status = "fix-dispatched" // StatusResolved: the latest minted fix task completed. StatusResolved Status = "resolved" // StatusFixFailed: the latest minted fix task dead-lettered. StatusFixFailed Status = "fix-failed" )