kinesumer

Kinesumer

Kinesumer is a simple Go client library for Amazon AWS Kinesis. It aims to be a native Go alternative to Amazon's KCL. Kinesumer includes a tool (called kinesumer) that lets you tail Kinesis streams and check the status of Kinesumer workers.

Features

• Automatically manages one consumer goroutine per shard.
• Handles shard splitting and merging properly.
• Provides a simple channel interface for incoming Kinesis records.
• Provides a tool for managing Kinesis streams:
  • Tailing a stream

Using the package

Install

go get github.com/remind101/kinesumer

Example Program

package main

import (
	"fmt"
	"os"

	"github.com/remind101/kinesumer"
)

func main() {
	k, err := kinesumer.NewDefaultKinesumer(
		"Stream",
	)
	if err != nil {
		panic(err)
	}
	k.Begin()
	defer k.End()
	for i := 0; i < 100; i++ {
		rec := <-k.Records()
		fmt.Println(string(rec.Data()))
	}
}

Using the tool

Install

go get -u github.com/remind101/kinesumer/cmd/kinesumer

To tail a stream make sure you have AWS credentials ready (either in ~/.aws or in env vars) and run:

kinesumer tail -s STREAM_NAME