jpegxe

package
v1.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 30, 2026 License: MIT Imports: 8 Imported by: 0

Documentation

Overview

Package jpegxe provides JPEG XE (ISO/IEC 21122-5) decoding support for event camera data.

JPEG XE is a codec designed for event camera (neuromorphic sensor) data streams. Unlike traditional frame-based imaging, event cameras output asynchronous events that represent brightness changes at specific pixel locations with microsecond-level temporal resolution.

Event Camera Data Model

Event cameras detect logarithmic brightness changes and generate events when the change exceeds a threshold. Each event contains:

  • X, Y coordinates: The pixel location where the change occurred
  • Polarity: Whether brightness increased (ON) or decreased (OFF)
  • Timestamp: Microsecond-precision time since stream start

This asynchronous data model enables very high temporal resolution (>10,000 events/second per pixel is possible) while maintaining low power consumption and high dynamic range.

MIPI ESP Compatibility

JPEG XE provides compatibility with the MIPI Event Sensing Pixel (ESP) interface, which defines the hardware interface between event sensors and image processing pipelines. The ESPPacket type represents packets from this interface.

Coding Architecture

JPEG XE uses lossless coding techniques optimized for event data:

  • Differential timestamp coding: Encodes time differences between consecutive events
  • Spatial prediction: Predicts event locations based on temporal locality
  • Context-based arithmetic coding: Entropy coding adapted to event statistics

Security Considerations

This package enforces security limits defined in the security package:

  • Maximum events per frame (MaxEventsPerFrame)
  • Maximum event stream size (MaxEventStreamSize)
  • Maximum timestamp delta (MaxTimestampDelta)

All integer conversions use the safeconv package to prevent overflow.

Standards Reference

This implementation follows ISO/IEC 21122-5 (JPEG XE) for event-based image coding.

Usage

To decode a JPEG XE event stream:

decoder, err := jpegxe.NewDecoder(data)
if err != nil {
    // Handle error
}

// Decode event stream
events, err := decoder.Decode()
if err != nil {
    // Handle error
}

// Process events
for _, event := range events.Events {
    fmt.Printf("Event at (%d, %d) polarity=%v time=%d\n",
        event.X, event.Y, event.Polarity, event.Timestamp)
}

Index

Constants

View Source
const (
	// SignatureJPEGXE is the JPEG XE file signature "JPXE".
	SignatureJPEGXE = "JPXE"

	// Version1 is JPEG XE version 1.0.
	Version1 uint8 = 1

	// MaxCoordinate is the maximum pixel coordinate value (16-bit).
	MaxCoordinate = 65535
)

JPEG XE marker and signature constants.

View Source
const HeaderSize = 32

HeaderSize is the size of the StreamHeader in bytes.

Variables

View Source
var (
	// ErrInvalidEventStream indicates the event stream data is malformed or invalid.
	ErrInvalidEventStream = errors.New("invalid event stream")

	// ErrTimestampError indicates an error in event timestamp decoding or validation.
	ErrTimestampError = errors.New("event timestamp error")

	// ErrPolarityError indicates an error decoding event polarity values.
	ErrPolarityError = errors.New("event polarity decoding error")

	// ErrCoordinateError indicates invalid or out-of-range event coordinates.
	ErrCoordinateError = errors.New("event coordinate error")

	// ErrStreamLimitExceeded indicates the event stream exceeds maximum allowed size.
	ErrStreamLimitExceeded = errors.New("event stream size limit exceeded")

	// ErrTimestampDeltaExceeded indicates a timestamp delta between events exceeds the limit.
	ErrTimestampDeltaExceeded = errors.New("timestamp delta limit exceeded")

	// ErrInvalidHeader indicates the JPEG XE stream header is malformed.
	ErrInvalidHeader = errors.New("invalid JPEG XE header")

	// ErrTruncatedStream indicates the event stream data is incomplete.
	ErrTruncatedStream = errors.New("truncated event stream")

	// ErrTooManyEvents indicates the event count exceeds the maximum allowed per frame.
	ErrTooManyEvents = errors.New("too many events in frame")

	// ErrInvalidESPPacket indicates a malformed MIPI ESP packet.
	ErrInvalidESPPacket = errors.New("invalid ESP packet")

	// ErrUnsupportedVersion indicates an unsupported JPEG XE version.
	ErrUnsupportedVersion = errors.New("unsupported JPEG XE version")

	// ErrEntropyDecodingError indicates an error during entropy decoding.
	ErrEntropyDecodingError = errors.New("entropy decoding error")

	// ErrInvalidContext indicates an invalid arithmetic coding context.
	ErrInvalidContext = errors.New("invalid coding context")

	// ErrDimensionError indicates invalid sensor dimensions.
	ErrDimensionError = errors.New("invalid sensor dimensions")
)

JPEG XE specific errors. These errors correspond to translation keys in locales/en-US.json under jpeg.jpegxe.error.*

Functions

func ValidateEventStream

func ValidateEventStream(data []byte) error

ValidateEventStream validates an event stream without fully decoding it. Returns nil if the stream is valid.

Types

type CodingMode

type CodingMode uint8

CodingMode represents the event coding mode.

const (
	// CodingModeRaw indicates raw event encoding (no compression).
	CodingModeRaw CodingMode = 0

	// CodingModeDifferential indicates differential timestamp encoding.
	CodingModeDifferential CodingMode = 1

	// CodingModeSpatialPrediction indicates spatial prediction mode.
	CodingModeSpatialPrediction CodingMode = 2

	// CodingModeEntropy indicates entropy-coded mode (arithmetic coding).
	CodingModeEntropy CodingMode = 3
)

func (CodingMode) String

func (m CodingMode) String() string

String returns a human-readable name for the coding mode.

type Decoder

type Decoder struct {
	// contains filtered or unexported fields
}

Decoder decodes JPEG XE event streams.

func NewDecoder

func NewDecoder(data []byte) (*Decoder, error)

NewDecoder creates a new JPEG XE decoder for the given data.

func NewDecoderWithOptions

func NewDecoderWithOptions(data []byte, options *DecoderOptions) (*Decoder, error)

NewDecoderWithOptions creates a new JPEG XE decoder with custom options.

func (*Decoder) Decode

func (d *Decoder) Decode() (*EventStream, error)

Decode decodes the event stream and returns an EventStream.

func (*Decoder) DecodeToFrames

func (d *Decoder) DecodeToFrames(frameDuration int64) ([]*EventFrame, error)

DecodeToFrames decodes the event stream and groups events into frames. The frameDuration parameter specifies the duration of each frame in microseconds.

func (*Decoder) GetHeader

func (d *Decoder) GetHeader() *StreamHeader

GetHeader returns the parsed stream header.

type DecoderOptions

type DecoderOptions struct {
	// MaxEvents limits the maximum number of events to decode.
	// 0 means use the default limit from security package.
	MaxEvents int

	// ValidateTimestamps enables strict timestamp ordering validation.
	ValidateTimestamps bool

	// FrameDuration is the duration for frame-based event grouping (microseconds).
	// 0 means no frame grouping.
	FrameDuration int64
}

DecoderOptions contains options for the JPEG XE decoder.

func DefaultDecoderOptions

func DefaultDecoderOptions() *DecoderOptions

DefaultDecoderOptions returns the default decoder options.

type DifferentialDecoder

type DifferentialDecoder struct {
	// contains filtered or unexported fields
}

DifferentialDecoder decodes differentially encoded timestamps.

func NewDifferentialDecoder

func NewDifferentialDecoder() *DifferentialDecoder

NewDifferentialDecoder creates a new differential decoder.

func (*DifferentialDecoder) DecodeTimestamp

func (d *DifferentialDecoder) DecodeTimestamp(encoded int64) int64

DecodeTimestamp decodes a single timestamp incrementally.

func (*DifferentialDecoder) DecodeTimestamps

func (d *DifferentialDecoder) DecodeTimestamps(encoded []int64) []int64

DecodeTimestamps decodes a slice of differentially encoded timestamps.

func (*DifferentialDecoder) Reset

func (d *DifferentialDecoder) Reset()

Reset resets the decoder state.

type DifferentialEncoder

type DifferentialEncoder struct {
	// contains filtered or unexported fields
}

DifferentialEncoder encodes timestamps using differential coding. Instead of storing absolute timestamps, it stores the delta between consecutive events.

func NewDifferentialEncoder

func NewDifferentialEncoder() *DifferentialEncoder

NewDifferentialEncoder creates a new differential encoder.

func (*DifferentialEncoder) EncodeTimestamp

func (e *DifferentialEncoder) EncodeTimestamp(timestamp int64) int64

EncodeTimestamp encodes a single timestamp incrementally.

func (*DifferentialEncoder) EncodeTimestamps

func (e *DifferentialEncoder) EncodeTimestamps(timestamps []int64) []int64

EncodeTimestamps encodes a slice of timestamps differentially. The first timestamp is stored as-is, subsequent values are deltas.

func (*DifferentialEncoder) Reset

func (e *DifferentialEncoder) Reset()

Reset resets the encoder state.

type ESPPacket

type ESPPacket struct {
	// PacketType indicates the type of ESP packet.
	PacketType ESPPacketType

	// SequenceNumber is the packet sequence number for ordering.
	SequenceNumber uint16

	// Timestamp is the packet timestamp in microseconds.
	Timestamp int64

	// Events contains the events in this packet.
	Events []Event

	// Flags contains packet-level flags.
	Flags uint8

	// Reserved bytes for future use.
	Reserved [2]byte
}

ESPPacket represents a MIPI Event Sensing Pixel (ESP) packet. ESP defines the hardware interface between event sensors and image processing.

func (*ESPPacket) Validate

func (p *ESPPacket) Validate() error

Validate checks that the ESP packet is valid.

type ESPPacketType

type ESPPacketType uint8

ESPPacketType indicates the type of ESP packet.

const (
	// ESPPacketTypeEvent indicates an event data packet.
	ESPPacketTypeEvent ESPPacketType = 0

	// ESPPacketTypeSync indicates a synchronization packet.
	ESPPacketTypeSync ESPPacketType = 1

	// ESPPacketTypeConfig indicates a configuration packet.
	ESPPacketTypeConfig ESPPacketType = 2

	// ESPPacketTypeStatus indicates a sensor status packet.
	ESPPacketTypeStatus ESPPacketType = 3
)

func (ESPPacketType) String

func (t ESPPacketType) String() string

String returns a human-readable name for the packet type.

type EntropyDecoder

type EntropyDecoder struct {
	// contains filtered or unexported fields
}

EntropyDecoder decodes entropy-coded event residuals.

func NewEntropyDecoder

func NewEntropyDecoder(data []byte) *EntropyDecoder

NewEntropyDecoder creates a new entropy decoder for the given data.

func (*EntropyDecoder) DecodeResidual

func (d *EntropyDecoder) DecodeResidual() (EventResidual, error)

DecodeResidual decodes a single event residual.

func (*EntropyDecoder) HasMore

func (d *EntropyDecoder) HasMore() bool

HasMore returns true if there is more data to decode.

type EntropyEncoder

type EntropyEncoder struct {
	// contains filtered or unexported fields
}

EntropyEncoder performs entropy coding on event residuals. Uses a simple variable-length encoding scheme.

func NewEntropyEncoder

func NewEntropyEncoder() *EntropyEncoder

NewEntropyEncoder creates a new entropy encoder.

func (*EntropyEncoder) EncodeResidual

func (e *EntropyEncoder) EncodeResidual(r EventResidual)

EncodeResidual encodes a single event residual.

func (*EntropyEncoder) Finalize

func (e *EntropyEncoder) Finalize() []byte

Finalize returns the encoded data.

type Event

type Event struct {
	// X is the horizontal pixel coordinate (0-indexed from left).
	X uint16

	// Y is the vertical pixel coordinate (0-indexed from top).
	Y uint16

	// Polarity indicates whether brightness increased (ON) or decreased (OFF).
	Polarity EventPolarity

	// Timestamp is the event time in microseconds since stream start.
	// Event cameras typically have microsecond-level temporal resolution.
	Timestamp int64
}

Event represents a single event from an event camera. Events are asynchronous notifications of brightness changes at specific pixels.

func (*Event) Validate

func (e *Event) Validate(width, height int) error

Validate checks that the event values are within acceptable ranges.

type EventFrame

type EventFrame struct {
	// FrameIndex is the sequential index of this frame.
	FrameIndex int

	// StartTimestamp is the start time of this frame in microseconds.
	StartTimestamp int64

	// EndTimestamp is the end time of this frame in microseconds.
	EndTimestamp int64

	// Events contains the events within this frame's time window.
	Events []Event

	// Width is the sensor width in pixels.
	Width int

	// Height is the sensor height in pixels.
	Height int
}

EventFrame represents a group of events within a time window. Event frames are useful for integrating events into frame-like representations.

func NewEventFrame

func NewEventFrame(index int, start, end int64, width, height int) (*EventFrame, error)

NewEventFrame creates a new EventFrame with the given parameters.

func (*EventFrame) AddEvent

func (f *EventFrame) AddEvent(e Event) error

AddEvent adds an event to this frame.

func (*EventFrame) GetDuration

func (f *EventFrame) GetDuration() int64

GetDuration returns the frame duration in microseconds.

type EventPolarity

type EventPolarity uint8

EventPolarity represents the polarity of a brightness change event. Event cameras detect both positive (ON) and negative (OFF) brightness changes.

const (
	// PolarityOFF indicates a brightness decrease (negative change).
	PolarityOFF EventPolarity = 0

	// PolarityON indicates a brightness increase (positive change).
	PolarityON EventPolarity = 1
)

func (EventPolarity) IsValid

func (p EventPolarity) IsValid() bool

IsValid returns true if the polarity is a valid value.

func (EventPolarity) String

func (p EventPolarity) String() string

String returns a human-readable representation of the polarity.

type EventResidual

type EventResidual struct {
	// X is the residual for X coordinate.
	X uint16

	// Y is the residual for Y coordinate.
	Y uint16

	// Polarity is the event polarity (not predicted, stored directly).
	Polarity EventPolarity

	// Timestamp is the timestamp or timestamp delta.
	Timestamp int64
}

EventResidual represents the residual (prediction error) for an event. Used in spatial prediction encoding.

type EventStream

type EventStream struct {
	// Width is the sensor width in pixels.
	Width int

	// Height is the sensor height in pixels.
	Height int

	// Events contains the decoded events in timestamp order.
	Events []Event

	// StartTimestamp is the timestamp of the first event in microseconds.
	StartTimestamp int64

	// EndTimestamp is the timestamp of the last event in microseconds.
	EndTimestamp int64

	// TotalDuration is the duration of the stream in microseconds.
	TotalDuration int64
}

EventStream represents a stream of events from an event camera. Event streams are ordered by timestamp and may contain millions of events.

func NewEventStream

func NewEventStream(width, height int) (*EventStream, error)

NewEventStream creates a new EventStream with the given sensor dimensions.

func (*EventStream) AddEvent

func (s *EventStream) AddEvent(e Event) error

AddEvent adds a new event to the stream with validation.

func (*EventStream) GetEventCount

func (s *EventStream) GetEventCount() int

GetEventCount returns the number of events in the stream.

func (*EventStream) GetEventRate

func (s *EventStream) GetEventRate() float64

GetEventRate returns the average event rate in events per second.

func (*EventStream) Validate

func (s *EventStream) Validate() error

Validate checks that the event stream is valid.

type LosslessCodec

type LosslessCodec struct {
	// contains filtered or unexported fields
}

LosslessCodec combines differential timestamp coding, spatial prediction, and entropy coding into a complete lossless codec for event data.

func NewLosslessCodec

func NewLosslessCodec(width, height int) *LosslessCodec

NewLosslessCodec creates a new lossless codec for the given sensor dimensions.

func (*LosslessCodec) Decode

func (c *LosslessCodec) Decode(data []byte, count int) ([]Event, error)

Decode decodes losslessly compressed events.

func (*LosslessCodec) Encode

func (c *LosslessCodec) Encode(events []Event) ([]byte, error)

Encode encodes a slice of events using lossless compression.

type SpatialPredictor

type SpatialPredictor struct {
	// contains filtered or unexported fields
}

SpatialPredictor predicts event coordinates from previous events. This exploits temporal locality - consecutive events often occur near each other.

func NewSpatialPredictor

func NewSpatialPredictor(width, height int) *SpatialPredictor

NewSpatialPredictor creates a new spatial predictor for the given sensor dimensions.

func (*SpatialPredictor) Decode

func (p *SpatialPredictor) Decode(residual EventResidual) Event

Decode reconstructs coordinates from a residual.

func (*SpatialPredictor) Encode

func (p *SpatialPredictor) Encode(event Event) EventResidual

Encode predicts coordinates and returns the residual.

func (*SpatialPredictor) Reset

func (p *SpatialPredictor) Reset()

Reset resets the predictor state.

type StreamHeader

type StreamHeader struct {
	// Signature is the file signature (should be "JPXE").
	Signature [4]byte

	// Version is the JPEG XE version number.
	Version uint8

	// Flags contains stream-level flags.
	Flags uint8

	// SensorWidth is the event sensor width in pixels.
	SensorWidth uint16

	// SensorHeight is the event sensor height in pixels.
	SensorHeight uint16

	// TimestampResolution is the timestamp resolution in nanoseconds.
	// Typical values: 1000 (microsecond), 1 (nanosecond).
	TimestampResolution uint32

	// EventCount is the total number of events in the stream.
	EventCount uint32

	// DurationMicros is the total stream duration in microseconds.
	DurationMicros int64

	// Reserved bytes for future use.
	Reserved [8]byte
}

StreamHeader represents the JPEG XE stream header.

func ParseEventStreamHeader

func ParseEventStreamHeader(data []byte) (*StreamHeader, error)

ParseEventStreamHeader parses just the header without decoding events. Useful for quickly reading stream metadata.

func (*StreamHeader) GetSensorDimensions

func (h *StreamHeader) GetSensorDimensions() (int, int, error)

GetSensorDimensions returns the sensor width and height as ints.

func (*StreamHeader) IsValid

func (h *StreamHeader) IsValid() bool

IsValid checks if the header signature matches JPEG XE.

func (*StreamHeader) Validate

func (h *StreamHeader) Validate() error

Validate checks that the header values are within acceptable ranges.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL