parser

Parser Processor Plugin

This plugin parses defined fields or tags containing the specified data format and creates new metrics based on the resulting fields and tags.

⭐ Telegraf v1.8.0 🏷️ transformation 💻 all

Global configuration options

Plugins support additional global and plugin configuration settings for tasks such as modifying metrics, tags, and fields, creating aliases, and configuring plugin ordering. See CONFIGURATION.md for more details.

Configuration

# Parse a value in a specified field(s)/tag(s) and add the result in a new metric
[[processors.parser]]
  ## The name of the fields whose value will be parsed.
  parse_fields = ["message"]

  ## Fields to base64 decode.
  ## These fields do not need to be specified in parse_fields.
  ## Fields specified here will have base64 decode applied to them.
  # parse_fields_base64 = []

  ## The name of the tags whose value will be parsed.
  # parse_tags = []

  ## If true, incoming metrics are not emitted.
  # drop_original = false

  ## Merge Behavior
  ## Possible options are:
  ##  - override: emit a single metric with all tags and fields of newly parsed
  ##    merged but retaining the first timestamp. If drop_original is
  ##    false, all metrics are merged into the original metric.
  ##    NOTE: Existing field or tag values will be overridden.
  ##  - override-with-timestamp: same as "override", but the timestamp is set
  ##    based on the new metrics if present.
  # merge = ""

  ## The dataformat to be read from files
  ## Each data format has its own unique set of configuration options, read
  ## more about them here:
  ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
  data_format = "influx"

Merge strategies

When parsing multiple metrics from a field or tag you can use the merge strategy to combine the newly parsed metrics.

override

This strategy will merge all parsed metrics, i.e. the plugin will emit only one metric containing the superset of all fields and tags of the parsed metric. If drop_original is false, the parent metric is also merged in.

[!IMPORTANT] In case identical field or tag names exist among the set of metrics those fields or tags will override each other and only the latest value will be emitted.

For example the parent metric

test,source=foo message="...",additional=true 1773258782000000000

and parsed metrics

metric,status=ok value1=1i 1773239679000000000
metric,status=warn value2=23i 1773239679100000000
metric,status=ok value3=19i 1773239679200000000
metric,status=fault value4=42i 1773239679300000000

will result in

metric,status=fault value1=1i,value2=23i,value3=19i,value4=42i 1773258782000000000

with drop_original = true

and

metric,source=foo,status=fault value1=1i,value2=23i,value3=19i,value4=42i,additional=true 1773258782000000000

with drop_original = false

override-with-timestamp

This strategy will behave the same way as override but will also override the timestamp with the one of the latest parsed metric.

Example

[[processors.parser]]
  parse_fields = ["message"]
  merge = "override"
  data_format = "logfmt"

Input

syslog,appname=influxd,facility=daemon,hostname=http://influxdb.example.org\ (influxdb.example.org),severity=info facility_code=3i,message=" ts=2018-08-09T21:01:48.137963Z lvl=info msg=\"Executing query\" log_id=09p7QbOG000 service=query query=\"SHOW DATABASES\"",procid="6629",severity_code=6i,timestamp=1533848508138040000i,version=1i

Output

syslog,appname=influxd,facility=daemon,hostname=http://influxdb.example.org\ (influxdb.example.org),severity=info facility_code=3i,log_id="09p7QbOG000",lvl="info",message=" ts=2018-08-09T21:01:48.137963Z lvl=info msg=\"Executing query\" log_id=09p7QbOG000 service=query query=\"SHOW DATABASES\"",msg="Executing query",procid="6629",query="SHOW DATABASES",service="query",severity_code=6i,timestamp=1533848508138040000i,ts="2018-08-09T21:01:48.137963Z",version=1i