Documentation
¶
Overview ¶
Package panicdiag provides panic recovery helpers and crash artifact writing.
Index ¶
- Constants
- func ApplyGoMemLimit(pct int) (int64, error)
- func BreadcrumbStreamInterceptor() grpclib.StreamServerInterceptor
- func BreadcrumbUnaryInterceptor() grpclib.UnaryServerInterceptor
- func DefaultArtifactRoot() string
- func DefaultMaxArtifacts() int
- func ForkMutableBreadcrumbs(ctx context.Context) context.Context
- func GRPCRecoveryHandler(log *logger.Logger, component string) func(context.Context, any) error
- func GoWithRecovery(ctx context.Context, opts RecoveryOptions, reporter Reporter, ...)
- func PruneArtifacts(rootDir string, maxArtifacts int) error
- func SetDefaultAbortFunc(f AbortFunc)
- func SetDefaultArtifactRoot(root string)
- func SetDefaultArtifactSink(s *ArtifactSink)
- func SetDefaultMaxArtifacts(n int)
- func SetDefaultPanicCounter(counter meter.Counter)
- func SetDefaultReporter(r Reporter)
- func WithBreadcrumb(ctx context.Context, stage string, component string, fields map[string]string) context.Context
- func WithMutableBreadcrumbs(ctx context.Context) context.Context
- type AbortFunc
- type ArtifactJob
- type ArtifactSink
- type ArtifactWriter
- func (aw *ArtifactWriter) ArtifactDirPath(record *PanicRecord) string
- func (aw *ArtifactWriter) MkdirArtifact(record *PanicRecord) (string, error)
- func (aw *ArtifactWriter) Write(record *PanicRecord) (string, error)
- func (aw *ArtifactWriter) WriteRecord(artifactDir string, record *PanicRecord) error
- func (aw *ArtifactWriter) WriteStateDump(artifactDir string, value any, limitBytes int64) (bool, string, error)
- type BoundedStateWriter
- type Breadcrumb
- type Collection
- type CrashOutputConfig
- type PanicRecord
- type RecoveryOptions
- type RecoveryOutcome
- type RecoveryResult
- type Reporter
- type StateDumpStatus
- type StateDumper
- type StateDumperFunc
Constants ¶
const DefaultSinkQueueDepth = 64
DefaultSinkQueueDepth is the default ArtifactSink queue capacity.
Variables ¶
This section is empty.
Functions ¶
func ApplyGoMemLimit ¶
ApplyGoMemLimit sets GOMEMLIMIT to pct percent of the cgroup memory limit, reserving the remaining capacity as headroom for diagnostic work after a panic. pct is clamped to [1, 99]. Returns the value applied, or 0 when the cgroup memory limit is unavailable (non-Linux or no cgroup configured).
func BreadcrumbStreamInterceptor ¶
func BreadcrumbStreamInterceptor() grpclib.StreamServerInterceptor
BreadcrumbStreamInterceptor returns a gRPC stream server interceptor that pre-installs a mutable breadcrumb store. It wraps the ServerStream so that stream.Context() returns the enriched context — this is required because the recovery interceptor calls stream.Context() in its defer rather than capturing a ctx variable.
func BreadcrumbUnaryInterceptor ¶
func BreadcrumbUnaryInterceptor() grpclib.UnaryServerInterceptor
BreadcrumbUnaryInterceptor returns a gRPC unary server interceptor that pre-installs a mutable breadcrumb store into the request context. Place this BEFORE the recovery interceptor so that breadcrumbs appended inside the handler are visible to the recovery handler's defer, which captures the context at interceptor entry.
func DefaultArtifactRoot ¶
func DefaultArtifactRoot() string
DefaultArtifactRoot returns the process-wide default artifact root.
func DefaultMaxArtifacts ¶
func DefaultMaxArtifacts() int
DefaultMaxArtifacts returns the process-wide default max artifact count.
func ForkMutableBreadcrumbs ¶
ForkMutableBreadcrumbs returns ctx with a fresh breadcrumb store seeded from ctx. Use it at goroutine boundaries to prevent sibling breadcrumb leakage.
func GRPCRecoveryHandler ¶
GRPCRecoveryHandler returns a handler for the gRPC recovery interceptors that reports the panic through panicdiag before converting it to a codes.Internal error. Servers previously hand-rolled a handler that only logged, which left every panic recovered on a request path out of banyandb_panic_total, out of the crash reporters and without an artifact -- the panics most likely to matter were the ones nothing counted.
The returned function is deliberately untyped so this package does not depend on the gRPC middleware; it satisfies recovery.RecoveryHandlerFuncContext directly:
recovery.WithRecoveryHandlerContext(panicdiag.GRPCRecoveryHandler(log, "grpc.liaison"))
func GoWithRecovery ¶
func GoWithRecovery(ctx context.Context, opts RecoveryOptions, reporter Reporter, fn func(*context.Context))
GoWithRecovery starts fn in a goroutine protected by WithRecovery. Use pkg/run.Go when the launcher needs a Task outcome.
func PruneArtifacts ¶
PruneArtifacts removes the oldest crash artifact directories under rootDir until at most maxArtifacts remain. Only directories containing a panic.json are considered; other directories are left untouched. When maxArtifacts is zero the call is a no-op.
func SetDefaultAbortFunc ¶
func SetDefaultAbortFunc(f AbortFunc)
SetDefaultAbortFunc registers the process-wide AbortFunc that WithRecovery invokes once per recovered panic. Pass nil to clear a previously registered default. Call once during process initialization.
func SetDefaultArtifactRoot ¶
func SetDefaultArtifactRoot(root string)
SetDefaultArtifactRoot stores the process-wide default artifact root.
func SetDefaultArtifactSink ¶
func SetDefaultArtifactSink(s *ArtifactSink)
SetDefaultArtifactSink registers a process-wide ArtifactSink that WithRecovery uses to write panic artifacts asynchronously. When set, the recovery defer fires signals (counter, log, reporter, default abort) before submitting the artifact job to the sink, so a stuck disk no longer pins the recovering goroutine while observers wait for the panic signal. Pass nil to clear a previously registered sink (recovery falls back to synchronous writes, the pre-sink behavior). Call once during process initialization. The caller is responsible for invoking Sink.Start() before registering and Sink.Close() during shutdown.
func SetDefaultMaxArtifacts ¶
func SetDefaultMaxArtifacts(n int)
SetDefaultMaxArtifacts sets the process-wide default maximum number of crash artifact directories to retain. When a new artifact is written, directories beyond this count are removed oldest-first. Zero disables pruning.
func SetDefaultPanicCounter ¶
SetDefaultPanicCounter registers a process-wide panic counter used by WithRecovery when RecoveryOptions.Counter is nil. Call once during process initialization.
func SetDefaultReporter ¶
func SetDefaultReporter(r Reporter)
SetDefaultReporter registers a process-wide Reporter that WithRecovery invokes for every recovered panic, in addition to any per-call reporter supplied by the caller. Pass nil to clear a previously registered default. Call once during process initialization.
Types ¶
type AbortFunc ¶
type AbortFunc func(context.Context, RecoveryResult)
AbortFunc is the process-wide control-flow hook fired once per recovered panic. It is registered through SetDefaultAbortFunc and intended for process-level concerns: canceling a supervising context, signaling lifecycle groups to stop, closing a notify channel that gates shutdown. It runs after every Reporter and before any Repanic, so the registered hook may both fail the supervisor and re-raise the panic. Per-call AbortFunc is not supported; use the *RecoveryOutcome returned by WithRecovery, or pkg/run.Task, to react locally. Implementations must not block.
type ArtifactJob ¶
type ArtifactJob struct {
// Record is the panic record to serialize.
Record *PanicRecord
// StateDump is the value returned by StateDumper.
StateDump any
// StateErr is recorded instead of writing StateDump when non-nil.
StateErr error
// Logger receives worker errors.
Logger *logger.Logger
// Done is closed after the job finishes.
Done chan<- struct{}
// RootDir is the artifact root. Empty means no-op.
RootDir string
// StateLimit caps the size of the deep state dump. Zero means no cap.
StateLimit int64
// HasStateDump is true when recovery attempted a state dump.
HasStateDump bool
}
ArtifactJob carries the inputs for one panic artifact write.
type ArtifactSink ¶
type ArtifactSink struct {
// contains filtered or unexported fields
}
ArtifactSink writes panic artifacts through a bounded async queue. Submit is non-blocking, and Submit/Close are synchronized to avoid sending on a closed queue.
func DefaultArtifactSink ¶
func DefaultArtifactSink() *ArtifactSink
DefaultArtifactSink returns the process-wide sink registered via SetDefaultArtifactSink, or nil when none is set.
func NewArtifactSink ¶
func NewArtifactSink(queueDepth int) *ArtifactSink
NewArtifactSink creates a sink. Non-positive depth uses the default.
func (*ArtifactSink) Close ¶
func (s *ArtifactSink) Close(ctx context.Context) error
Close stops submissions and waits for queued jobs to drain.
func (*ArtifactSink) QueueDepth ¶
func (s *ArtifactSink) QueueDepth() int
QueueDepth returns the current queue length.
func (*ArtifactSink) Start ¶
func (s *ArtifactSink) Start()
Start launches the worker goroutine. It is idempotent.
func (*ArtifactSink) Submit ¶
func (s *ArtifactSink) Submit(job ArtifactJob) bool
Submit enqueues a job without blocking. It returns false when full or closed.
type ArtifactWriter ¶
type ArtifactWriter struct {
// contains filtered or unexported fields
}
ArtifactWriter writes panic artifacts to disk.
func NewArtifactWriter ¶
func NewArtifactWriter(rootDir string) *ArtifactWriter
NewArtifactWriter returns a new ArtifactWriter.
func (*ArtifactWriter) ArtifactDirPath ¶
func (aw *ArtifactWriter) ArtifactDirPath(record *PanicRecord) string
ArtifactDirPath returns the deterministic artifact directory path for the given record without performing any I/O. The recovery defer uses this to populate RecoveryResult.ArtifactDir before the synchronous mkdir runs (or before the sink worker creates the directory asynchronously), so observers see a path even when the actual files have not been written yet.
func (*ArtifactWriter) MkdirArtifact ¶
func (aw *ArtifactWriter) MkdirArtifact(record *PanicRecord) (string, error)
MkdirArtifact creates the artifact directory for the given record and returns its path. OccurredAt is back-filled when zero. Call WriteRecord once all fields (including StateDump) have been populated.
func (*ArtifactWriter) Write ¶
func (aw *ArtifactWriter) Write(record *PanicRecord) (string, error)
Write persists the given panic record and returns the artifact directory. Use MkdirArtifact + WriteRecord directly when additional fields (e.g. StateDump) must be populated before the file is written.
func (*ArtifactWriter) WriteRecord ¶
func (aw *ArtifactWriter) WriteRecord(artifactDir string, record *PanicRecord) error
WriteRecord serializes record as panic.json inside artifactDir.
func (*ArtifactWriter) WriteStateDump ¶
func (aw *ArtifactWriter) WriteStateDump(artifactDir string, value any, limitBytes int64) (bool, string, error)
WriteStateDump persists a deep state dump into an existing artifact directory.
type BoundedStateWriter ¶
type BoundedStateWriter interface {
WriteJSON(path string, value any, limitBytes int64) (truncated bool, err error)
}
BoundedStateWriter writes JSON snapshots under a fixed size limit.
func NewBoundedStateWriter ¶
func NewBoundedStateWriter() BoundedStateWriter
NewBoundedStateWriter returns the default bounded JSON state writer.
type Breadcrumb ¶
type Breadcrumb struct {
Fields map[string]string `json:"fields,omitempty"`
Time time.Time `json:"time"`
Stage string `json:"stage"`
Component string `json:"component,omitempty"`
}
Breadcrumb stores a semantic execution marker attached to a context.
func BreadcrumbsFromContext ¶
func BreadcrumbsFromContext(ctx context.Context) []Breadcrumb
BreadcrumbsFromContext returns breadcrumbs ordered oldest to newest.
type Collection ¶
type Collection struct {
ArtifactDir string `json:"artifactDir"`
Record *PanicRecord `json:"record,omitempty"`
Files []string `json:"files"`
}
Collection contains the persisted diagnosis data for a single panic artifact.
func ListCollections ¶
func ListCollections(root string) ([]Collection, error)
ListCollections returns persisted diagnosis collections from the given artifact root.
type CrashOutputConfig ¶
CrashOutputConfig controls structured panic diagnostics.
func NewCrashOutputConfig ¶
func NewCrashOutputConfig() CrashOutputConfig
NewCrashOutputConfig returns the default global crash-output configuration. Structured panic diagnostics are enabled by default.
func (CrashOutputConfig) InstallGlobalCrashOutput ¶
func (c CrashOutputConfig) InstallGlobalCrashOutput() error
InstallGlobalCrashOutput configures structured panic diagnostics when enabled.
func (*CrashOutputConfig) RegisterFlags ¶
func (c *CrashOutputConfig) RegisterFlags(flags *pflag.FlagSet)
RegisterFlags registers the crash-output flags on the provided flag set.
type PanicRecord ¶
type PanicRecord struct {
ProcessMetadata map[string]string `json:"processMetadata,omitempty"`
StateDump *StateDumpStatus `json:"stateDump,omitempty"`
Component string `json:"component"`
PanicValue string `json:"panicValue"`
GoroutineStack string `json:"goroutineStack"`
OccurredAt time.Time `json:"occurredAt"`
Breadcrumbs []Breadcrumb `json:"breadcrumbs,omitempty"`
Recovered bool `json:"recovered"`
}
PanicRecord stores the structured panic information captured by recovery helpers.
type RecoveryOptions ¶
type RecoveryOptions struct {
Counter meter.Counter
Logger *logger.Logger
StateDumper StateDumper
ProcessMetadata map[string]string
Component string
ArtifactRoot string
StateLimitBytes int64
}
RecoveryOptions configures how panic recovery writes diagnostics.
Neither AbortFunc nor a Repanic toggle is configurable per-call: callers that need to react to a recovered panic should consume the *RecoveryOutcome returned by WithRecovery, or watch a Task launched by pkg/run.Go. Process-wide abort behavior remains available via SetDefaultAbortFunc; callers that need to re-raise the panic with full fidelity can do so themselves with the typed RecoveryOutcome.PanicValue after WithRecovery returns.
type RecoveryOutcome ¶
type RecoveryOutcome struct {
PanicValue any
ArtifactDone <-chan struct{}
Result RecoveryResult
Panicked bool
}
RecoveryOutcome describes whether a recovered panic occurred during a WithRecovery call. The pointer return makes the result composable: callers that want to inspect outcomes locally, fail a parent lifecycle, signal a notify channel, or set an error variable can read fields directly instead of smuggling state through an OnAbort closure. A non-nil outcome is always returned; Panicked is false when fn ran to completion without recovery.
PanicValue holds the original (typed) panic argument when Panicked is true. It is intended for callers that need to re-raise with full fidelity, e.g. `panic(outcome.PanicValue)` after WithRecovery returns. The string-form is available via Result.Record.PanicValue for logging and reporting.
ArtifactDone is closed when the artifact write completes or is skipped. It is nil when no panic was recovered.
func WithRecovery ¶
func WithRecovery(ctx context.Context, opts RecoveryOptions, reporter Reporter, fn func(*context.Context)) (outcome *RecoveryOutcome)
WithRecovery executes fn and recovers panics with diagnostics. fn may update the context pointer before recovery reads it. The outcome is always non-nil.
type RecoveryResult ¶
type RecoveryResult struct {
Record *PanicRecord
ArtifactDir string
}
RecoveryResult contains the outcome of a recovered panic.
func RecoverExternal ¶
func RecoverExternal(ctx context.Context, opts RecoveryOptions, reporter Reporter, panicValue any, stack []byte) (result RecoveryResult)
RecoverExternal fires panicdiag's standard signals for a panic that the caller has already recovered itself, so recovery paths outside WithRecovery -- notably the gRPC servers' recovery interceptors -- are not invisible to the panic counter, the crash reporters and the artifact writer. Call it from inside the recovering defer (or a handler it invokes) so stack still describes the panic site. It never panics: every hook it runs is isolated, and a failure inside panicdiag itself falls back to stderr.
type Reporter ¶
type Reporter func(context.Context, RecoveryResult)
Reporter receives the result of a recovered panic. Reporters are intended for observability: recording, logging, or shipping the panic record, and must not block. Use AbortFunc when control flow needs to react to a panic.
type StateDumpStatus ¶
type StateDumpStatus struct {
Path string `json:"path,omitempty"`
Error string `json:"error,omitempty"`
Truncated bool `json:"truncated,omitempty"`
}
StateDumpStatus describes the result of deep state serialization.
type StateDumper ¶
StateDumper returns a bounded diagnostic snapshot after a recovered panic.
type StateDumperFunc ¶
StateDumperFunc is a function adapter for StateDumper. Closures that capture named-return variables can be used directly as a StateDumper, preserving function-local state across a panic boundary.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package lintrawgo provides a go/analysis Analyzer that fails the build when production code launches a goroutine via a raw `go` statement instead of run.Go, run.GoOrDie, or run.GoWithSignal.
|
Package lintrawgo provides a go/analysis Analyzer that fails the build when production code launches a goroutine via a raw `go` statement instead of run.Go, run.GoOrDie, or run.GoWithSignal. |