odbsafetyprocessor

odbsafetyprocessor

odbsafetyprocessor is an OpenTelemetry Collector logs processor that applies oteldb log safety limiting to pdata logs. It operates after logs have been batched into pdata, in contrast to the stanza odbsafety operator which protects at the receiver level.

Registration

Use odbsafetyprocessor.NewFactory() in a custom collector factory set. The oteldb collector factory map already registers it for oteldb builds.

Configuration

processors:
  odbsafety:
    # Labels for rate limiting and metrics (optional).
    workload: my-service
    namespace: my-team

    # Soft throttle — 0 means no limit. Above this, `on_excess` engages.
    soft_max_rate_per_second: 5000
    # Hard throttle — 0 means no limit. Above this, `hard_on_excess` engages,
    # escalating past the soft limit.
    hard_max_rate_per_second: 10000
    # What to do when the soft rate is exceeded.
    on_excess: compact
    # What to do when the hard rate is exceeded.
    hard_on_excess: drop
    # First N entries logged unconditionally for 'sample' mode, then
    # 1-in-sample_thereafter after that.
    sample_first: 100
    sample_thereafter: 100
    # Compaction window and thresholds.
    compact_window: 30s
    compact_threshold: 100
    compact_max_buckets: 10000
    compact_key_fields: [body]
    # Min suppressed entries in a window before emitting a synthetic
    # truncation record for 'truncate' mode (0 = always emit).
    truncate_threshold: 1000
    # Fields whose values are redacted.
    redact_fields: [password, token]

Config fields

Records with the oteldb.passthrough attribute set to true bypass excess handling entirely.

Single-tier rate limiting and probabilistic sampling

The old config had a single max_rate_per_second threshold and a probabilistic sample_rate. To get the same behaviour with the current fields, leave the hard tier disabled and use sample_thereafter alone — sampling 1-in-N approximates a 1/N probabilistic rate without needing randomness:

    # Single threshold, same as the old max_rate_per_second.
    soft_max_rate_per_second: 10000
    hard_max_rate_per_second: 0 # disabled — no escalation tier
    on_excess: sample
    # ~1% sampling, like the old sample_rate: 0.01.
    sample_first: 0
    sample_thereafter: 100

On-excess modes

Pipeline example

receivers:
  filelog:
    include: [/var/log/app/*.log]

processors:
  odbsafety:
    workload: my-service
    namespace: my-team
    soft_max_rate_per_second: 10000
    on_excess: compact

exporters:
  otlp:
    endpoint: oteldb:4317

service:
  pipelines:
    logs:
      receivers: [filelog]
      processors: [odbsafety]
      exporters: [otlp]