Documentation
¶
Overview ¶
Package ops is a self-hosted operations dashboard: errors, slow requests, slow queries, the job queue, cron and health, on a page you run yourself.
OpenTelemetry has won as the wire format and this framework emits it. The unsolved part is that OTel is miserable for a small team: a collector to run, a backend to pay for, and a bill that is famously not small. This is the other end of that -- the twenty percent of observability that answers "is it broken and why", with no ingestion, no retention and nothing to pay for.
It is not a competitor to anything. It is the thing that means a solo developer does not need a $200/month dependency to find out that the cron entry stopped firing three weeks ago.
Mounting it ¶
recorder := ops.NewRecorder(0)
app.Logging.OTel.TracerProvider().RegisterSpanProcessor(recorder)
panel.AddPage(ops.Page(ops.Config{
Recorder: recorder,
Queues: []*jobs.SQLQueue{queue},
Health: database.NewHealthChecker(db, 2*time.Second),
}))
It is a page in the admin panel, which means it inherits the panel's authorizer and the panel's rule that an unauthenticated visitor gets 404. That is deliberate and it is the one hard requirement in the issue this implements: a dashboard that leaks slow queries and error messages to the internet is not a dashboard, it is a reconnaissance endpoint.
What it does not do ¶
No ingestion, no alerting, no long-term storage, nothing with a bill attached. The retained window is however many spans the Recorder holds, and the page says so rather than implying it has seen everything.
Index ¶
- Constants
- func Page(cfg Config) admin.Page
- type Config
- type CronFunc
- type CronReporter
- type CronRun
- type ErrorGroup
- type Recorder
- func (r *Recorder) Count() (held, capacity int)
- func (r *Recorder) Errors(limit int) []ErrorGroup
- func (r *Recorder) ForceFlush(context.Context) error
- func (r *Recorder) HasSpansOfKind(kind trace.SpanKind) bool
- func (r *Recorder) OnEnd(s sdktrace.ReadOnlySpan)
- func (r *Recorder) OnStart(context.Context, sdktrace.ReadWriteSpan)
- func (r *Recorder) Shutdown(context.Context) error
- func (r *Recorder) SlowQueries(limit int) []Timing
- func (r *Recorder) SlowRequests(limit int) []Timing
- type Timing
Constants ¶
const DefaultCapacity = 1000
DefaultCapacity is how many finished spans are kept. At roughly 200 bytes of retained summary each, a thousand is a fifth of a megabyte.
const DefaultRows = 10
DefaultRows is how many entries each panel shows.
Variables ¶
This section is empty.
Functions ¶
Types ¶
type Config ¶
type Config struct {
// Recorder supplies errors, slow requests and slow queries. Register it on
// the tracer provider; see NewRecorder.
Recorder *Recorder
// Queues are the job queues to report on.
Queues []*jobs.SQLQueue
// Workflows shows which step a durable workflow is stuck on.
Workflows *jobs.Workflows
// Health is the database health check.
Health *database.HealthChecker
// Cron reports the last run of each scheduled job. See CronReporter.
Cron CronReporter
// Title overrides the page heading.
Title string
// Rows caps each table. Zero means DefaultRows.
Rows int
}
Config is what the dashboard has been given to look at.
Every field is optional. A panel with nothing configured says so, per panel, rather than showing an empty table that looks like good news.
type CronFunc ¶
type CronFunc func() []CronRun
CronFunc adapts a function.
func (CronFunc) CronStatus ¶
type CronReporter ¶
type CronReporter interface {
CronStatus() []CronRun
}
CronReporter reports scheduled-job runs.
This package does not import the framework root, so it cannot name *tjo.BackgroundService. CronRun has the same fields as tjo.CronRun in the same order, which makes a direct struct conversion legal and the adapter one line:
Cron: ops.CronFunc(func() []ops.CronRun {
runs := app.Background.CronStatus()
out := make([]ops.CronRun, 0, len(runs))
for _, r := range runs {
out = append(out, ops.CronRun(r))
}
return out
}),
Six lines in the application beats this package depending on the whole framework, and it keeps the dashboard usable from a program that is not a Tjo application at all.
type CronRun ¶
type CronRun struct {
Name string
LastRun time.Time
Duration time.Duration
Runs int
Failures int
LastError string
}
CronRun is the last run of a scheduled job. It mirrors tjo.CronRun field for field, deliberately.
type ErrorGroup ¶
ErrorGroup is one kind of error and how often it happened.
type Recorder ¶
type Recorder struct {
// contains filtered or unexported fields
}
Recorder keeps the most recent finished spans.
It implements sdktrace.SpanProcessor, so it is registered on the tracer provider the otel module already builds:
recorder := ops.NewRecorder(0) provider.TracerProvider().RegisterSpanProcessor(recorder)
func NewRecorder ¶
NewRecorder returns a recorder holding capacity spans. Zero means DefaultCapacity.
func (*Recorder) Count ¶
Count reports how many spans are held, and the capacity.
Shown on the dashboard, because "no errors in the last 1000 spans" and "no errors ever" are different claims and only one of them is true.
func (*Recorder) Errors ¶
func (r *Recorder) Errors(limit int) []ErrorGroup
Errors returns the failures, grouped rather than listed.
Grouped because a list of five hundred identical timeouts is not information. The count and the last occurrence are.
func (*Recorder) ForceFlush ¶
ForceFlush implements sdktrace.SpanProcessor. There is nothing to flush: the buffer is the destination.
func (*Recorder) HasSpansOfKind ¶
HasSpansOfKind reports whether anything of a kind has been seen.
The difference between "no slow queries" and "the database is not instrumented" is the difference between a green panel and a broken one, and the dashboard says which it is.
func (*Recorder) OnEnd ¶
func (r *Recorder) OnEnd(s sdktrace.ReadOnlySpan)
OnEnd records a finished span.
func (*Recorder) OnStart ¶
func (r *Recorder) OnStart(context.Context, sdktrace.ReadWriteSpan)
OnStart implements sdktrace.SpanProcessor and does nothing: a span is only interesting once it has a duration and an outcome.
func (*Recorder) SlowQueries ¶
SlowQueries returns the slowest database spans held.
func (*Recorder) SlowRequests ¶
SlowRequests returns the slowest HTTP spans held.
Source Files