kafka

package module
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 7, 2026 License: MIT Imports: 13 Imported by: 2

Documentation

Overview

Package kafka lazily deserializes AWS Lambda Kafka event records.

New creates a Consumer with Config for key, value and header behavior. WrapHandler adapts the consumer to a JSON Lambda handler. ConsumerRecords retains event metadata and records expose decoded fields on demand using context.

Decoder composition

Primitive and JSON modes are available in this module. Independent kafka/avro and kafka/protobuf modules supply optional binary field decoders; importing the core consumer does not load them. Custom decoders own their concurrency, result ownership and input validation.

This package parses events delivered by Lambda; it does not poll brokers or manage consumer offsets. Lazy decoding failures retain typed consumer errors. The guide distinguishes supported raw and registry modes and compatibility limits.

For usage and compatibility details, see the user guide.

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func WrapHandler

func WrapHandler[R any](consumer *Consumer, handler func(context.Context, *ConsumerRecords) (R, error)) func(context.Context, jsonv1.RawMessage) (R, error)

WrapHandler preserves context, returned values, errors and panics. It reads no record fields automatically; handlers decide whether to inspect a failing field.

Example
package main

import (
	"context"
	"encoding/json"
	"fmt"

	"github.com/rambow-cloud/powertools-lambda-go/kafka"
)

func main() {
	consumer := kafka.New(kafka.Config{})
	handler := kafka.WrapHandler(consumer, func(ctx context.Context, event *kafka.ConsumerRecords) (any, error) {
		return event.Records[0].Value(ctx)
	})
	value, err := handler(context.Background(), json.RawMessage(
		`{"records":{"orders-0":[{"value":"aGVsbG8=","headers":[]}]}}`))
	fmt.Println(value, err)
}
Output:
hello <nil>

Types

type Config

type Config struct {
	Key, Value *FieldConfig
	// Diagnostic reports JSON parse failures before returning the decoded text.
	// The default writes to stderr. Callbacks must be concurrency-safe.
	Diagnostic func(context.Context, string, error)
}

type Consumer

type Consumer struct {
	// contains filtered or unexported fields
}

Consumer has immutable configuration and no per-invocation mutable state.

func New

func New(config Config) *Consumer

func (*Consumer) Deserialize

func (c *Consumer) Deserialize(ctx context.Context, input any) (*ConsumerRecords, error)

Deserialize validates the event and prepares records without reading any payload. RawMessage preserves topic insertion order; Go maps have deterministic sorted keys. Like the reference guard, only the records object and its array values are required.

type ConsumerError

type ConsumerError struct {
	Message string
	Cause   error
}

ConsumerError reports an invalid event or consumer configuration.

func (*ConsumerError) As

func (e *ConsumerError) As(target any) bool

As lets errors.As match the common consumer base for embedded error variants.

func (*ConsumerError) Error

func (e *ConsumerError) Error() string

func (*ConsumerError) ErrorName

func (e *ConsumerError) ErrorName() string

func (*ConsumerError) Unwrap

func (e *ConsumerError) Unwrap() error

type ConsumerRecords

type ConsumerRecords struct {
	Fields  map[string]any
	Records []*Record
}

ConsumerRecords keeps event metadata separate from the flattened records. Fields is a shallow copy; nested input values remain application-owned.

type Decoder

type Decoder func(context.Context, string, any, any) (any, error)

Decoder is the optional binary codec boundary. Metadata is passed unchanged. Implementations must be safe for concurrent invocations and return fresh values.

type DeserializationError

type DeserializationError struct{ ConsumerError }

DeserializationError reports an unsupported format or binary codec failure.

func (*DeserializationError) ErrorName

func (e *DeserializationError) ErrorName() string

type FieldConfig

type FieldConfig struct {
	Type    SchemaType
	Schema  any
	Decoder Decoder
	Parser  Parser
}

type MissingSchemaError

type MissingSchemaError struct{ ConsumerError }

MissingSchemaError is raised on field access, unless that field is absent or null.

func (*MissingSchemaError) ErrorName

func (e *MissingSchemaError) ErrorName() string

type ParseResult

type ParseResult struct {
	Value  any
	Issues []any
}

ParseResult follows Standard Schema: a non-nil Issues value is a validation failure, including an empty slice. Errors returned by Parser propagate unchanged.

type Parser

type Parser func(context.Context, any) (ParseResult, error)

Parser synchronously validates or transforms a decoded field.

type ParserError

type ParserError struct {
	ConsumerError
	Issues []any
}

ParserError retains Standard Schema issues, including an empty issue array.

func (*ParserError) ErrorName

func (e *ParserError) ErrorName() string

type Record

type Record struct {
	Fields                                      map[string]any
	OriginalKey, OriginalValue, OriginalHeaders any
	KeySchemaMetadata, ValueSchemaMetadata      any
	// contains filtered or unexported fields
}

Record exposes original fields and lazy reads. Reads are deliberately not cached: repeated access decodes and validates again, matching the reference getters.

func (*Record) Headers

func (r *Record) Headers(ctx context.Context) ([]map[string]string, error)

Headers decodes byte arrays using Buffer-style byte coercion and UTF-8 replacement. Null is retained. Missing headers fail only when this method is called.

func (*Record) Key

func (r *Record) Key(ctx context.Context) (any, error)

func (*Record) Value

func (r *Record) Value(ctx context.Context) (any, error)

type SchemaType

type SchemaType string
const (
	JSON     SchemaType = "json"
	Avro     SchemaType = "avro"
	Protobuf SchemaType = "protobuf"
)

type TypeError

type TypeError struct{ Message string }

TypeError represents a reference input error outside the consumer error hierarchy.

func (*TypeError) Error

func (e *TypeError) Error() string

func (*TypeError) ErrorName

func (e *TypeError) ErrorName() string

type Undefined

type Undefined struct{}

Undefined distinguishes a missing field from a null value or tombstone.

Directories

Path Synopsis
avro module
protobuf module

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL