exportercreator

Exporter Creator

Status
Stability	development: logs, metrics, traces
Distributions	[]
Issues
Code coverage
Code Owners	@stu

The exporter_creator exporter dynamically creates other exporters at runtime when observer extensions report endpoints (for example Kubernetes pods, CRDs, or JSON-defined targets). It evaluates rules against each endpoint, starts matching exporter templates with configuration expanded from endpoint data, and routes incoming logs, metrics, and traces to the right sub-exporter using resource attributes. Use it when destinations are not fixed at config time—similar to dynamic receiver wiring, but on the export side of the pipeline.

Relationship to OpenTelemetry Collector Contrib

In opentelemetry-collector-contrib, receiver creator (receiver_creator) watches observers and spawns receivers from templates. Exporter creator is the same pattern for exporters: watch observers, match rules, instantiate exporters. This repository tracks the implementation from contrib’s exporter/exportercreator, published as a standalone module (github.com/stuart23/exportercreator) for custom collector builds.

Building a collector with this exporter

exporter_creator is not included in the default core or contrib collector distributions. Build a custom binary with the OpenTelemetry Collector Builder (ocb). See the OCB documentation.

Add the module under exporters in builder-config.yaml (use a real version tag once you publish; v0.0.0 is common with a local replace during development):

exporters:
  - gomod: go.opentelemetry.io/collector/exporter/debugexporter v0.151.0
  - gomod: go.opentelemetry.io/collector/exporter/otlpexporter v0.151.0
  - gomod: github.com/stuart23/exportercreator v0.1.0

You also need at least one observer extension in the same manifest (for example k8s_observer, jsonfile_observer) so watch_observers has something to attach to. This module depends on extension/observer from contrib; your go.mod or builder manifest often needs replace directives for contrib paths (including observer submodules such as jsonfileobserver). Copy and adapt the replaces block from builder-config.example.yaml—paths are relative to the builder output_path directory.

Go module and dependencies

• Module path: github.com/stuart23/exportercreator
• Shared state: includes a copy of contrib’s internal/sharedcomponent, which is not importable from outside the contrib module tree as a normal dependency.
• Observer API: depends on a tagged extension/observer release (see go.mod). Rules accept k8s.crd and jsonfile endpoint kinds via ObserverEndpointTypeK8sCRD / ObserverEndpointTypeJSONFile so behavior stays aligned with real observers even when those kinds are not yet named constants in the observer module.

How it works

1. Observer Extensions: The exporter watches observer extensions (like k8s_observer) for endpoint discovery events.

2. Rule Matching: When endpoints are discovered, rules are evaluated to determine which exporter templates should be instantiated.

3. Dynamic Exporter Creation: Matching exporter templates are used to create sub-exporters with endpoint-specific configuration.

4. Telemetry Routing: Incoming telemetry is routed to the appropriate sub-exporter based on resource attribute matching against endpoint properties.

Configuration

Every deployment needs three things: an observer to discover endpoints, at least one exporter template with a rule saying which endpoints it applies to, and at least one routing rule saying which telemetry belongs to which endpoint.

extensions:
  k8s_observer:
    observe_pods: true

exporters:
  exporter_creator:
    # Observer extensions to watch for endpoint discovery.
    watch_observers: [k8s_observer]

    # Which telemetry belongs to which endpoint. The resource attribute is read from the
    # telemetry, the endpoint property from the discovered endpoint, and they must be equal.
    routing:
      rules:
        - resource_attribute: k8s.pod.labels.app
          endpoint_property: labels.app

    # Exporter templates. One exporter is created per endpoint the rule matches.
    exporters:
      otlp/per-app:
        rule: type == "pod" && labels["otel-export"] == "true"
        config:
          endpoint: '`labels["collector-endpoint"]`'

service:
  extensions: [k8s_observer]
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [exporter_creator]

See Examples for this worked through end to end, and for a larger configuration.

Configuration Options

default_exporters is validated and stored, but the lookup that would resolve those IDs to running exporters is not implemented, so the set is always empty at runtime. Telemetry that matches no endpoint is counted by the non-routable metrics and dropped, whether or not this is configured. The examples below do not use it.

Routing Rules

Each routing rule maps a resource attribute to an endpoint property:

Exporter Template

Signals

Omit signals and the exporter handles metrics, logs and traces. Provide it and it becomes an allowlist: only the signals set to true are enabled, and any signal you leave out is disabled.

exporters:
  otlp/metrics-only:
    rule: type == "pod" && labels["otel-export"] == "true"
    signals:
      metrics: true
    config:
      endpoint: '`labels["collector-endpoint"]`'

Because unlisted signals are disabled rather than left alone, signals cannot be used to turn a single signal off: signals: {logs: false} disables all three, not just logs. To handle everything except logs, name what you want:

    signals:
      metrics: true
      traces: true

A block that enables nothing is rejected at startup, as is an unrecognised signal name or a non-boolean value.

Signals an exporter cannot handle

Routing matches on resource attributes alone, so it can select an endpoint whose exporter does not handle that signal - a prometheusremotewrite template matched by a logs pipeline, for example. That exporter cannot consume the telemetry, and because the resource already matched, it does not fall through to default_exporters. The telemetry is dropped only if no other matched exporter handles the signal.

Dropped telemetry is reported by otelcol_exporter_creator_nonroutable_log_records_total, otelcol_exporter_creator_nonroutable_metric_points_total or otelcol_exporter_creator_nonroutable_spans_total. Each incompatible matched exporter also logs a warning once per signal.

To avoid the drop, route only the signals your templates handle through exporter_creator, or give the endpoint an exporter template that covers them.

Endpoint Value Expansion

Configuration values can reference endpoint properties using backtick expressions:

config:
  endpoint: '`labels["collector-endpoint"]`'
  topic: '`annotations["kafka.topic"]`'

Examples

Simple: one exporter per application

A collector receiving OTLP from many applications, forwarding each application's telemetry to the collector running beside it.

k8s_observer reports a pod endpoint for every pod. The template creates an otlp exporter for each pod carrying the otel-export: "true" label, pointed at the address in that pod's collector-endpoint label. The routing rule then sends a resource to that exporter when the telemetry's k8s.pod.labels.app equals the pod's app label.

extensions:
  k8s_observer:
    observe_pods: true

exporters:
  exporter_creator:
    watch_observers: [k8s_observer]
    routing:
      rules:
        - resource_attribute: k8s.pod.labels.app
          endpoint_property: labels.app
    exporters:
      otlp/per-app:
        rule: type == "pod" && labels["otel-export"] == "true"
        config:
          endpoint: '`labels["collector-endpoint"]`'
          tls:
            insecure: true

service:
  extensions: [k8s_observer]
  pipelines:
    metrics:
      receivers: [otlp]
      exporters: [exporter_creator]
    traces:
      receivers: [otlp]
      exporters: [exporter_creator]

A pod labelled app: checkout, otel-export: "true", collector-endpoint: checkout-collector:4317 gets an exporter sending to checkout-collector:4317, and every resource whose k8s.pod.labels.app is checkout is routed to it.

Complex: several templates, observers and signals

extensions:
  k8s_observer:
    observe_pods: true
    observe_services: true
  host_observer:

exporters:
  exporter_creator:
    # Endpoints from either observer are matched against every template below.
    watch_observers: [k8s_observer, host_observer]

    # Every rule must match for a resource to reach an exporter, so this pair means
    # "same application, same region".
    routing:
      rules:
        - resource_attribute: k8s.pod.labels.app
          endpoint_property: pod.labels.app
        - resource_attribute: deployment.region
          endpoint_property: region

    exporters:
      # A per-pod OTLP collector, discovered by the port it listens on. Port endpoints nest
      # the pod, so its labels are reached through `pod.` rather than at the top level.
      otlp/sidecar:
        rule: type == "port" && port == 4317 && pod.labels["otel-export"] == "true"
        config:
          endpoint: '`endpoint`'
          tls:
            insecure: true
        # Merged into the endpoint's properties, so routing rules can match on them as well.
        # `region` is the second routing rule's endpoint_property.
        resource_attributes:
          region: '`pod.labels["topology.kubernetes.io/region"]`'

      # Metrics-only backend. Naming a signal disables the ones left out, so logs and traces
      # matching this endpoint are not sent here - and are dropped unless another matched
      # exporter takes them.
      prometheusremotewrite/metrics:
        rule: type == "k8s.service" && annotations["prometheus.io/remote-write"] == "true"
        signals:
          metrics: true
        config:
          endpoint: 'http://`endpoint`:`"prometheus.io/port" in annotations ? annotations["prometheus.io/port"] : 9090`/api/v1/write'
        resource_attributes:
          region: '`labels["topology.kubernetes.io/region"]`'

      # An agent found on the host rather than in Kubernetes, taking logs and traces only.
      otlp/host-agent:
        rule: type == "hostport" && port == 4318 && string(transport) == "TCP"
        signals:
          logs: true
          traces: true
        config:
          endpoint: '`endpoint`'
        # A host endpoint carries nothing to derive a region from, so it is stated. Without it
        # the second routing rule could never match and this exporter would receive nothing.
        resource_attributes:
          region: us-east-1

service:
  extensions: [k8s_observer, host_observer]
  pipelines:
    metrics:
      receivers: [otlp]
      exporters: [exporter_creator]
    logs:
      receivers: [otlp]
      exporters: [exporter_creator]
    traces:
      receivers: [otlp]
      exporters: [exporter_creator]

Three things in that configuration are worth calling out.

One endpoint can produce several exporters. Every template whose rule matches an endpoint gets its own exporter for it, and telemetry matching that endpoint is delivered to all of them.

Routing rules are ANDed. A resource reaches an exporter only when every rule matches, so adding a rule narrows what is routed rather than widening it. A resource missing one of the attributes named on the left matches nothing at all.

resource_attributes are routable. They are expanded per endpoint and merged into that endpoint's properties, so a rule's endpoint_property can name one of them. That is how region above is matched against telemetry even though no observer reports a region property: each template derives it from whatever that endpoint type does provide.

Endpoint properties available to rules

rule expressions and endpoint_property paths both read the endpoint's properties, which differ by endpoint type. The two most common:

The full set for every type is documented by the observer extensions. Note the difference above: a pod endpoint carries labels at the top level, while a port endpoint nests them under pod.

transport is not a plain string but a named type, so transport == "TCP" is false rather than an error. Convert it first:

rule: type == "hostport" && string(transport) == "TCP"

Keys that contain dots

An endpoint_property names keys separated by dots, so a key that itself contains dots - which every namespaced Kubernetes label does - goes in brackets, double quoted:

routing:
  rules:
    - resource_attribute: k8s.pod.labels["app.kubernetes.io/name"]
      endpoint_property: pod.labels["app.kubernetes.io/name"]

On the endpoint_property side the brackets are load bearing. Without them the path reads as pod → labels → app → kubernetes → io/name and matches nothing. A path that cannot be parsed is rejected at startup rather than silently matching nothing for the life of the collector.

On the resource_attribute side they are only punctuation. A resource attribute is a flat name, not a path: k8sattributes emits one attribute called k8s.pod.labels.app.kubernetes.io/name, and nothing is nested. Written out, it is a run of dots with nothing to show where the prefix ends and the label key begins, so the brackets are accepted there too and mean exactly the same name:

resource_attribute: k8s.pod.labels["app.kubernetes.io/name"]   # same attribute
resource_attribute: k8s.pod.labels.app.kubernetes.io/name      # as this one

Both spellings name the same attribute, and the bracketed one is what the examples here use.

A [ anywhere in the value makes it a bracketed path, so the brackets are not merely cosmetic:

A value with no [ in it is the attribute name verbatim, which is every attribute name that follows OpenTelemetry's conventions - they are dotted, and brackets do not appear in them. But an attribute whose name genuinely contains a bracket can no longer be written as itself: the last two rows above changed meaning, one silently. If you have such an attribute, this syntax cannot address it.

Where several endpoint types have to be routed by one rule, deriving a resource_attributes entry per template is still the way to normalise them - a port endpoint nests labels under pod., a k8s.service carries them at the top level, and a hostport has none. That is what the complex example above does with region.

Development

See DEVELOPMENT.md for running the tests and regenerating the files built from metadata.yaml.