soe

package
v2.31.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Dec 30, 2025 License: MIT Imports: 5 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source
var ErrUnknownSchema = errors.New("unknown schema")

ErrUnknownSchema is an API error that must be returned if the requested schema is not known to a resolver.

View Source
var Magic = []byte{0xc3, 0x01}

Magic is the two-byte magic marker described in: https://avro.apache.org/docs/1.10.2/spec.html#single_object_encoding

Functions

func BuildHeader

func BuildHeader(schema avro.Schema) ([]byte, error)

BuildHeader builds an SOE header from a schema's fingerprint.

func BuildHeaderForFingerprint

func BuildHeaderForFingerprint(fingerprint []byte) ([]byte, error)

BuildHeaderForFingerprint builds an SOE header from a fingerprint.

func ComputeFingerprint

func ComputeFingerprint(schema avro.Schema) ([]byte, error)

ComputeFingerprint returns an SOE-compatible (CRC64, little-endian) schema fingerprint.

func GetSchema

func GetSchema[T AvroGenerated]() avro.Schema

GetSchema returns the avro.Schema associated with type T.

func ParseHeader

func ParseHeader(data []byte) ([]byte, []byte, error)

ParseHeader validates SOE magic and splits data into fingerprint,rest.

Types

type AvroGenerated

type AvroGenerated interface {
	Unmarshal(b []byte) error
	Marshal() ([]byte, error)
	Schema() avro.Schema
}

AvroGenerated is implemented by all avrogen-generated Avro types.

type Codec

type Codec struct {
	// contains filtered or unexported fields
}

Codec marshals values to/from bytes, with the Avro binary payload wrapped in an SOE frame containing the writer schema fingerprint.

func NewCodec

func NewCodec(schema avro.Schema) (*Codec, error)

NewCodec creates a new Codec for a Schema and the default config.

func NewCodecWithAPI

func NewCodecWithAPI(schema avro.Schema, api avro.API) (*Codec, error)

NewCodecWithAPI creates a new Codec for a Schema and an API.

func (*Codec) Decode

func (c *Codec) Decode(data []byte, v any) error

Decode unmarshals a value from SOE-encoded Avro binary, and fails if the schema fingerprint doesn't match the held schema.

func (*Codec) DecodeUnverified

func (c *Codec) DecodeUnverified(data []byte, v any) error

DecodeUnverified unmarshals a value from SOE-encoded Avro binary without validating the schema fingerprint.

func (*Codec) Encode

func (c *Codec) Encode(v any) ([]byte, error)

Encode marshals a value to SOE-encoded Avro binary.

type DynamicDecoder

type DynamicDecoder struct {
	// contains filtered or unexported fields
}

DynamicDecoder unmarshals SOE-framed records. It will use a schema resolver to lookup schemas for every record, and deserialize using exactly the writer schema. This makes it possible to decode records from entirely disparate schemas into the same Go type.

func NewDynamicDecoder

func NewDynamicDecoder(resolver SchemaResolver) *DynamicDecoder

NewDynamicDecoder returns a new DynamicDecoder for a resolver the default config.

func NewDynamicDecoderWithAPI

func NewDynamicDecoderWithAPI(resolver SchemaResolver, api avro.API) *DynamicDecoder

NewDynamicDecoderWithAPI returns a new DynamicDecoder for the given resolver and API.

func (*DynamicDecoder) Decode

func (d *DynamicDecoder) Decode(ctx context.Context, data []byte, v any) error

Decode unmarshals a value from SOE-encoded Avro binary using the schema specified in the SOE header. Fails if schema is not known to resolver.

type SchemaResolver

type SchemaResolver interface {
	// GetSchema must return ErrUnknownSchema if no schema is found for
	// fingerprint. All other errors are unexpected.
	GetSchema(ctx context.Context, fingerprint []byte) (avro.Schema, error)
}

SchemaResolver implementations are expected to be callable for every record, so should cache expensive schema lookups.

type TypedCodec

type TypedCodec[T any] struct {
	// contains filtered or unexported fields
}

TypedCodec marshals/unmarshals AvroGenerated values to/from SOE-framed Avro binary payloads. Must be instantiated with a pointer type, e.g.

c, _ := NewTypedCodec[*MyType]()
var val MyType
c.Encode(&val)
c.Decode(&val)

var badVal any
c.Encode(badVal) // cannot use badVal (variable of type any)...

It is a strongly typed version of soe.Codec.

func NewTypedCodec

func NewTypedCodec[T AvroGenerated]() (*TypedCodec[T], error)

NewTypedCodec creates a new TypedCodec for type T and the default config.

func NewTypedCodecWithAPI

func NewTypedCodecWithAPI[T AvroGenerated](api avro.API) (*TypedCodec[T], error)

NewTypedCodecWithAPI creates a new TypedCodec for type T and an API.

func (*TypedCodec[T]) Decode

func (c *TypedCodec[T]) Decode(data []byte, v T) error

Decode unmarshals a typed value from SOE-encoded Avro binary, and fails if the schema fingerprint doesn't match the held schema.

func (*TypedCodec[T]) DecodeUnverified

func (c *TypedCodec[T]) DecodeUnverified(data []byte, v T) error

DecodeUnverified unmarshals a typed value from SOE-encoded Avro binary without validating the schema fingerprint.

func (*TypedCodec[T]) Encode

func (c *TypedCodec[T]) Encode(v T) ([]byte, error)

Encode marshals a typed value to SOE-encoded Avro binary.

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL