piper

piper

piper contains three commands for piping json to Azure EventGrid, AWS EventBridge and Splunk

eventbridge-pipe

Install

go get github.com/willabides/piper/cmd/eventbridge-pipe

Usage

Usage: eventbridge-pipe --type=STRING --source=STRING

eventbridge-pipe posts events to AWS EventBridge.

example:

    $ AWS_ACCESS_KEY='AKIA****************'
    $ AWS_SECRET_KEY='shhh_this_is_a_secret'
    $ data="$(cat <<"EOF"
        {"action": "obj.add", "@timestamp": 1604953432032, "el_name": "foo", "doc_id": "asdf"}
        {"action": "obj.rem", "@timestamp": 1604953732032, "el_name": "bar", "doc_id": "fdsa"}
      EOF
      )"
    $ echo "$data" | \
      eventbridge-pipe -s 'test-source' -t 'jp:action' -b 'my-bus' -T 'jp:"@timestamp"' \
      -r 'jp:"el_name"'

Learn about JMESPath syntax at https://jmespath.org

Flags:
  -h, --help                     Show context-sensitive help.
      --region="us-east-1"       The aws region to publish events to.
  -t, --type=STRING              Value for the DetailType field. JMESPath
                                 expressions allowed with "jp:" prefix.
  -b, --event-bus=STRING         Value for the "EventBusName" field.
  -r, --resource=RESOURCE,...    An element for the list in the "Resources"
                                 array. JMESPath expressions allowed with "jp:"
                                 prefix.
  -s, --source=STRING            Value for the "Source" field. JMESPath
                                 expressions allowed with "jp:" prefix.
  -T, --timestamp=STRING         Value for the "Time" field converted from epoch
                                 milliseconds. JMESPath expressions allowed with
                                 "jp:" prefix.
      --batch-size=10            Number of events to send in a batch.
      --flush-interval=2000      Time in milliseconds to wait before sending a
                                 partial batch. Set to 0 to never send a partial
                                 batch.

eventgrid-pipe

Install

go get github.com/willabides/piper/cmd/eventgrid-pipe

Usage

Usage: eventgrid-pipe --subject=STRING --type=STRING <topic-endpoint>

eventgrid-pipe posts events to Azure Event Grid.

example:

    $ topic_endpoint='https://mytopicendpoint.westus2-1.eventgrid.azure.net'
    $ topic_key='shhh_secret_topic_key'
    $ data="$(cat <<"EOF"
        {"action": "obj.add", "@timestamp": 1604953432032, "el_name": "foo", "doc_id": "asdf"}
        {"action": "obj.rem", "@timestamp": 1604953732032, "el_name": "bar", "doc_id": "fdsa"}
      EOF
      )"
    $ echo "$data" | \
      eventgrid-pipe "$topic_endpoint" \
      -H "aeg-sas-key: $topic_key" \
      -T 'jp:"@timestamp"' \
      -t 'audit-log' \
      -s 'jp:action' \
      -i 'jp:doc_id'

Learn about JMESPath syntax at https://jmespath.org

Arguments:
  <topic-endpoint>    Endpoint for posting events

Flags:
  -h, --help                   Show context-sensitive help.
  -i, --id=STRING              Value for the "id" field. If unset, a uuid will
                               be generated for each event. JMESPath expressions
                               allowed with "jp:" prefix.
  -s, --subject=STRING         Value for the "subject" field. JMESPath
                               expressions allowed with "jp:" prefix.
  -t, --type=STRING            Value for the "eventType" field. JMESPath
                               expressions allowed with "jp:" prefix.
  -T, --timestamp="now"        Value for the "eventTime" field converted from
                               epoch milliseconds. If unset, the current system
                               time will be used.JMESPath expressions allowed
                               with "jp:" prefix.
  -H, --header=HEADER,...      Header to sent with the request in the same
                               format as curl. e.g. '-H "aeg-sas-key: $EVKEY"'
      --data-version="1.0"     Value for the "dataVersion" field. JMESPath
                               expressions allowed with "jp:" prefix.
      --batch-size=10          Number of events to send in a batch.
      --flush-interval=2000    Time in milliseconds to wait before sending a
                               partial batch. Set to 0 to never send a partial
                               batch.

splunk-pipe

Install

go get github.com/willabides/piper/cmd/splunk-pipe

Usage

Usage: splunk-pipe <endpoint>

splunk-pipe posts events to splunk.

example:

    $ splunk_endpoint="http://localhost:8080"
    $ splunk_hec_token="shhh_secret_token"
    $ data="$(cat <<"EOF"
        {"action": "obj.add", "@timestamp": 1604953432032, "el_name": "foo", "doc_id": "asdf"}
        {"action": "obj.rem", "@timestamp": 1604953732032, "el_name": "bar", "doc_id": "fdsa"}
      EOF
      )"
    $ echo "$data" | \
      splunk-pipe "$splunk_endpoint" \
      -H "Authorization: Splunk $splunk_hec_token" \
      -T 'jp:"@timestamp"'

Learn about JMESPath syntax at https://jmespath.org

Arguments:
  <endpoint>    Endpoint for posting events

Flags:
  -h, --help                   Show context-sensitive help.
  -t, --sourcetype=STRING      Value for the "sourcetype" field. JMESPath
                               expressions allowed with "jp:" prefix.
  -s, --source=STRING          Value for the "source" field. JMESPath
                               expressions allowed with "jp:" prefix.
  -T, --timestamp=STRING       Value for the "eventTime" field converted from
                               epoch milliseconds. JMESPath expressions allowed
                               with "jp:" prefix.
  -H, --header=HEADER,...      Header to sent with the request in the same
                               format as curl. e.g. '-H "Authorization: Splunk
                               $HEC_KEY"'
  -h, --host=STRING            Value for the "host" field. JMESPath expressions
                               allowed with "jp:" prefix.
      --index=STRING           Value for the "index" field. JMESPath expressions
                               allowed with "jp:" prefix.
      --batch-size=10          Number of events to send in a batch.
      --flush-interval=2000    Time in milliseconds to wait before sending a
                               partial batch. Set to 0 to never send a partial
                               batch.