Documentation
¶
Overview ¶
Package metrics defines fabriq's Prometheus instruments. They live on a PRIVATE registry that fabriq's forge extension serves at GET /metrics, so every series keeps its fabriq_ name with no forge_ prefix. forge's own series (HTTP, and grove's db_client_operation_duration_seconds for the databases fabriq opens) are on forge's /_/metrics instead; scrape both. Library code receives the instruments it needs as plain callbacks or observer interfaces so core stays dependency-free.
Operational meaning (see docs/OPERATIONS.md for runbooks):
- fabriq_outbox_backlog: unpublished outbox rows. Sustained growth means the relay is down or Redis is unreachable. Worker only.
- fabriq_tenant_hook_trips_total: the tenant backstop fired in this process, read at scrape time on every node. Any non-zero value in production is a fabriq bug. Page.
- fabriq_conflation_depth: deltas buffered in this node's subscription hub, read at scrape time. Sustained growth means subscribers cannot keep up.
- fabriq_projection_lag_events{projection}: entries the graph or search consumer group has not yet acknowledged (lag plus pending). Worker only.
- fabriq_relay_published_total: relay publish throughput.
- fabriq_command_duration_seconds{op,result}, fabriq_relay_batch_duration_seconds{result}, fabriq_projection_apply_duration_seconds{projection,result} and fabriq_model_call_duration_seconds{kind,result}: latency of the work the matching spans cover.
- fabriq_worker_restarts_total{runner}: a background runner exited and was restarted. Steady growth means it keeps crashing.
Index ¶
- type Metrics
- func (m *Metrics) ObserveApply(projection, result string, elapsed time.Duration)
- func (m *Metrics) ObserveCommand(op, result string, elapsed time.Duration)
- func (m *Metrics) ObserveModelCall(kind, result string, elapsed time.Duration)
- func (m *Metrics) ObserveRelayBatch(result string, elapsed time.Duration)
- type Option
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Metrics ¶
type Metrics struct {
OutboxBacklog prometheus.Gauge
ProjectionLag *prometheus.GaugeVec
RelayPublished prometheus.Counter
// Durations of the work fabriq traces, and runner restarts.
CommandDuration *prometheus.HistogramVec
RelayBatchDuration *prometheus.HistogramVec
ProjectionApplyDuration *prometheus.HistogramVec
ModelCallDuration *prometheus.HistogramVec
WorkerRestarts *prometheus.CounterVec
// Blob CAS garbage-collection instruments.
BlobGCBytesFreed prometheus.Counter
BlobGCCollected prometheus.Counter
BlobGCRefDriftCorrected prometheus.Counter
BlobGCBroken prometheus.Gauge
BlobGCOrphans prometheus.Counter
// Embedding worker (proj:embed) instruments.
EmbedEventsTotal prometheus.Counter
EmbedFailuresTotal prometheus.Counter
// Analytics consumer (proj:analytics) instruments.
AnalyticsAppliedTotal prometheus.Counter
AnalyticsFailuresTotal prometheus.Counter
AnalyticsLagSeconds prometheus.Gauge
AnalyticsTenantsBehind prometheus.Gauge
AnalyticsEventsPrunedTotal prometheus.Counter
// Catalog-mode sweeper instruments.
SweepPassDuration prometheus.Histogram
SweepTenantsTracked prometheus.Gauge
SweepEligible prometheus.Gauge
SweepSweptTotal prometheus.Counter
SweepBusyTotal prometheus.Counter
SweepErrorsTotal prometheus.Counter
PoolShardsOpen prometheus.Gauge
PoolShardsHeld prometheus.Gauge
PoolCap prometheus.Gauge
PoolScaleEvents *prometheus.CounterVec
// Distillation worker (proj:distill) instruments.
DistillNodesTotal prometheus.Counter
DistillSummariesTotal prometheus.Counter
DistillShortCircuitTotal prometheus.Counter
DistillGuardBlockedTotal prometheus.Counter
DistillFailuresTotal prometheus.Counter
DistillSplitsTotal prometheus.Counter
DistillDedupTotal prometheus.Counter
DistillIntermediateGCTotal prometheus.Counter
}
Metrics bundles fabriq's instruments.
func New ¶
func New(reg prometheus.Registerer, opts ...Option) (*Metrics, error)
New creates and registers the instruments on reg.
func (*Metrics) ObserveApply ¶ added in v1.8.1
ObserveApply implements telemetry.ApplyObserver.
func (*Metrics) ObserveCommand ¶ added in v1.8.1
ObserveCommand implements telemetry.CommandObserver.
func (*Metrics) ObserveModelCall ¶ added in v1.8.1
ObserveModelCall records one Embedder (kind "embed") or Summarizer (kind "summarize") call.
type Option ¶ added in v1.8.1
type Option func(*sources)
Option supplies the live readings behind fabriq's scrape-time series.
func WithBackstopTrips ¶ added in v1.8.1
WithBackstopTrips supplies this process's cumulative tenant-guard backstop trips, read at scrape time as fabriq_tenant_hook_trips_total. The trips happen wherever a query runs, so every node reports its own.
func WithCatalogReads ¶ added in v1.8.1
WithCatalogReads supplies the routing catalog's cumulative read counters (catalog.Failover.ReadStats). They are read at scrape time as the fabriq_catalog_read_*_total counters; ok=false reads as zero.
func WithConflationDepth ¶ added in v1.8.1
WithConflationDepth supplies the subscription hub's buffered delta count, read at scrape time as fabriq_conflation_depth.
Source Files