Documentation
¶
Overview ¶
Package jpegxe provides JPEG XE (ISO/IEC 21122-5) decoding support for event camera data.
JPEG XE is a codec designed for event camera (neuromorphic sensor) data streams. Unlike traditional frame-based imaging, event cameras output asynchronous events that represent brightness changes at specific pixel locations with microsecond-level temporal resolution.
Event Camera Data Model ¶
Event cameras detect logarithmic brightness changes and generate events when the change exceeds a threshold. Each event contains:
- X, Y coordinates: The pixel location where the change occurred
- Polarity: Whether brightness increased (ON) or decreased (OFF)
- Timestamp: Microsecond-precision time since stream start
This asynchronous data model enables very high temporal resolution (>10,000 events/second per pixel is possible) while maintaining low power consumption and high dynamic range.
MIPI ESP Compatibility ¶
JPEG XE provides compatibility with the MIPI Event Sensing Pixel (ESP) interface, which defines the hardware interface between event sensors and image processing pipelines. The ESPPacket type represents packets from this interface.
Coding Architecture ¶
JPEG XE uses lossless coding techniques optimized for event data:
- Differential timestamp coding: Encodes time differences between consecutive events
- Spatial prediction: Predicts event locations based on temporal locality
- Context-based arithmetic coding: Entropy coding adapted to event statistics
Security Considerations ¶
This package enforces security limits defined in the security package:
- Maximum events per frame (MaxEventsPerFrame)
- Maximum event stream size (MaxEventStreamSize)
- Maximum timestamp delta (MaxTimestampDelta)
All integer conversions use the safeconv package to prevent overflow.
Standards Reference ¶
This implementation follows ISO/IEC 21122-5 (JPEG XE) for event-based image coding.
Usage ¶
To decode a JPEG XE event stream:
decoder, err := jpegxe.NewDecoder(data)
if err != nil {
// Handle error
}
// Decode event stream
events, err := decoder.Decode()
if err != nil {
// Handle error
}
// Process events
for _, event := range events.Events {
fmt.Printf("Event at (%d, %d) polarity=%v time=%d\n",
event.X, event.Y, event.Polarity, event.Timestamp)
}
Index ¶
- Constants
- Variables
- func ValidateEventStream(data []byte) error
- type CodingMode
- type Decoder
- type DecoderOptions
- type DifferentialDecoder
- type DifferentialEncoder
- type ESPPacket
- type ESPPacketType
- type EntropyDecoder
- type EntropyEncoder
- type Event
- type EventFrame
- type EventPolarity
- type EventResidual
- type EventStream
- type LosslessCodec
- type SpatialPredictor
- type StreamHeader
Constants ¶
const ( // SignatureJPEGXE is the JPEG XE file signature "JPXE". SignatureJPEGXE = "JPXE" // Version1 is JPEG XE version 1.0. Version1 uint8 = 1 // MaxCoordinate is the maximum pixel coordinate value (16-bit). MaxCoordinate = 65535 )
JPEG XE marker and signature constants.
const HeaderSize = 32
HeaderSize is the size of the StreamHeader in bytes.
Variables ¶
var ( // ErrInvalidEventStream indicates the event stream data is malformed or invalid. ErrInvalidEventStream = errors.New("invalid event stream") // ErrTimestampError indicates an error in event timestamp decoding or validation. ErrTimestampError = errors.New("event timestamp error") // ErrPolarityError indicates an error decoding event polarity values. ErrPolarityError = errors.New("event polarity decoding error") // ErrCoordinateError indicates invalid or out-of-range event coordinates. ErrCoordinateError = errors.New("event coordinate error") // ErrStreamLimitExceeded indicates the event stream exceeds maximum allowed size. ErrStreamLimitExceeded = errors.New("event stream size limit exceeded") // ErrTimestampDeltaExceeded indicates a timestamp delta between events exceeds the limit. ErrTimestampDeltaExceeded = errors.New("timestamp delta limit exceeded") // ErrInvalidHeader indicates the JPEG XE stream header is malformed. ErrInvalidHeader = errors.New("invalid JPEG XE header") // ErrTruncatedStream indicates the event stream data is incomplete. ErrTruncatedStream = errors.New("truncated event stream") // ErrTooManyEvents indicates the event count exceeds the maximum allowed per frame. ErrTooManyEvents = errors.New("too many events in frame") // ErrInvalidESPPacket indicates a malformed MIPI ESP packet. ErrInvalidESPPacket = errors.New("invalid ESP packet") // ErrUnsupportedVersion indicates an unsupported JPEG XE version. ErrUnsupportedVersion = errors.New("unsupported JPEG XE version") // ErrEntropyDecodingError indicates an error during entropy decoding. ErrEntropyDecodingError = errors.New("entropy decoding error") // ErrInvalidContext indicates an invalid arithmetic coding context. ErrInvalidContext = errors.New("invalid coding context") // ErrDimensionError indicates invalid sensor dimensions. ErrDimensionError = errors.New("invalid sensor dimensions") )
JPEG XE specific errors. These errors correspond to translation keys in locales/en-US.json under jpeg.jpegxe.error.*
Functions ¶
func ValidateEventStream ¶
ValidateEventStream validates an event stream without fully decoding it. Returns nil if the stream is valid.
Types ¶
type CodingMode ¶
type CodingMode uint8
CodingMode represents the event coding mode.
const ( // CodingModeRaw indicates raw event encoding (no compression). CodingModeRaw CodingMode = 0 // CodingModeDifferential indicates differential timestamp encoding. CodingModeDifferential CodingMode = 1 // CodingModeSpatialPrediction indicates spatial prediction mode. CodingModeSpatialPrediction CodingMode = 2 // CodingModeEntropy indicates entropy-coded mode (arithmetic coding). CodingModeEntropy CodingMode = 3 )
func (CodingMode) String ¶
func (m CodingMode) String() string
String returns a human-readable name for the coding mode.
type Decoder ¶
type Decoder struct {
// contains filtered or unexported fields
}
Decoder decodes JPEG XE event streams.
func NewDecoder ¶
NewDecoder creates a new JPEG XE decoder for the given data.
func NewDecoderWithOptions ¶
func NewDecoderWithOptions(data []byte, options *DecoderOptions) (*Decoder, error)
NewDecoderWithOptions creates a new JPEG XE decoder with custom options.
func (*Decoder) Decode ¶
func (d *Decoder) Decode() (*EventStream, error)
Decode decodes the event stream and returns an EventStream.
func (*Decoder) DecodeToFrames ¶
func (d *Decoder) DecodeToFrames(frameDuration int64) ([]*EventFrame, error)
DecodeToFrames decodes the event stream and groups events into frames. The frameDuration parameter specifies the duration of each frame in microseconds.
func (*Decoder) GetHeader ¶
func (d *Decoder) GetHeader() *StreamHeader
GetHeader returns the parsed stream header.
type DecoderOptions ¶
type DecoderOptions struct {
// MaxEvents limits the maximum number of events to decode.
// 0 means use the default limit from security package.
MaxEvents int
// ValidateTimestamps enables strict timestamp ordering validation.
ValidateTimestamps bool
// FrameDuration is the duration for frame-based event grouping (microseconds).
// 0 means no frame grouping.
FrameDuration int64
}
DecoderOptions contains options for the JPEG XE decoder.
func DefaultDecoderOptions ¶
func DefaultDecoderOptions() *DecoderOptions
DefaultDecoderOptions returns the default decoder options.
type DifferentialDecoder ¶
type DifferentialDecoder struct {
// contains filtered or unexported fields
}
DifferentialDecoder decodes differentially encoded timestamps.
func NewDifferentialDecoder ¶
func NewDifferentialDecoder() *DifferentialDecoder
NewDifferentialDecoder creates a new differential decoder.
func (*DifferentialDecoder) DecodeTimestamp ¶
func (d *DifferentialDecoder) DecodeTimestamp(encoded int64) int64
DecodeTimestamp decodes a single timestamp incrementally.
func (*DifferentialDecoder) DecodeTimestamps ¶
func (d *DifferentialDecoder) DecodeTimestamps(encoded []int64) []int64
DecodeTimestamps decodes a slice of differentially encoded timestamps.
func (*DifferentialDecoder) Reset ¶
func (d *DifferentialDecoder) Reset()
Reset resets the decoder state.
type DifferentialEncoder ¶
type DifferentialEncoder struct {
// contains filtered or unexported fields
}
DifferentialEncoder encodes timestamps using differential coding. Instead of storing absolute timestamps, it stores the delta between consecutive events.
func NewDifferentialEncoder ¶
func NewDifferentialEncoder() *DifferentialEncoder
NewDifferentialEncoder creates a new differential encoder.
func (*DifferentialEncoder) EncodeTimestamp ¶
func (e *DifferentialEncoder) EncodeTimestamp(timestamp int64) int64
EncodeTimestamp encodes a single timestamp incrementally.
func (*DifferentialEncoder) EncodeTimestamps ¶
func (e *DifferentialEncoder) EncodeTimestamps(timestamps []int64) []int64
EncodeTimestamps encodes a slice of timestamps differentially. The first timestamp is stored as-is, subsequent values are deltas.
func (*DifferentialEncoder) Reset ¶
func (e *DifferentialEncoder) Reset()
Reset resets the encoder state.
type ESPPacket ¶
type ESPPacket struct {
// PacketType indicates the type of ESP packet.
PacketType ESPPacketType
// SequenceNumber is the packet sequence number for ordering.
SequenceNumber uint16
// Timestamp is the packet timestamp in microseconds.
Timestamp int64
// Events contains the events in this packet.
Events []Event
// Flags contains packet-level flags.
Flags uint8
// Reserved bytes for future use.
Reserved [2]byte
}
ESPPacket represents a MIPI Event Sensing Pixel (ESP) packet. ESP defines the hardware interface between event sensors and image processing.
type ESPPacketType ¶
type ESPPacketType uint8
ESPPacketType indicates the type of ESP packet.
const ( // ESPPacketTypeEvent indicates an event data packet. ESPPacketTypeEvent ESPPacketType = 0 // ESPPacketTypeSync indicates a synchronization packet. ESPPacketTypeSync ESPPacketType = 1 // ESPPacketTypeConfig indicates a configuration packet. ESPPacketTypeConfig ESPPacketType = 2 // ESPPacketTypeStatus indicates a sensor status packet. ESPPacketTypeStatus ESPPacketType = 3 )
func (ESPPacketType) String ¶
func (t ESPPacketType) String() string
String returns a human-readable name for the packet type.
type EntropyDecoder ¶
type EntropyDecoder struct {
// contains filtered or unexported fields
}
EntropyDecoder decodes entropy-coded event residuals.
func NewEntropyDecoder ¶
func NewEntropyDecoder(data []byte) *EntropyDecoder
NewEntropyDecoder creates a new entropy decoder for the given data.
func (*EntropyDecoder) DecodeResidual ¶
func (d *EntropyDecoder) DecodeResidual() (EventResidual, error)
DecodeResidual decodes a single event residual.
func (*EntropyDecoder) HasMore ¶
func (d *EntropyDecoder) HasMore() bool
HasMore returns true if there is more data to decode.
type EntropyEncoder ¶
type EntropyEncoder struct {
// contains filtered or unexported fields
}
EntropyEncoder performs entropy coding on event residuals. Uses a simple variable-length encoding scheme.
func NewEntropyEncoder ¶
func NewEntropyEncoder() *EntropyEncoder
NewEntropyEncoder creates a new entropy encoder.
func (*EntropyEncoder) EncodeResidual ¶
func (e *EntropyEncoder) EncodeResidual(r EventResidual)
EncodeResidual encodes a single event residual.
func (*EntropyEncoder) Finalize ¶
func (e *EntropyEncoder) Finalize() []byte
Finalize returns the encoded data.
type Event ¶
type Event struct {
// X is the horizontal pixel coordinate (0-indexed from left).
X uint16
// Y is the vertical pixel coordinate (0-indexed from top).
Y uint16
// Polarity indicates whether brightness increased (ON) or decreased (OFF).
Polarity EventPolarity
// Timestamp is the event time in microseconds since stream start.
// Event cameras typically have microsecond-level temporal resolution.
Timestamp int64
}
Event represents a single event from an event camera. Events are asynchronous notifications of brightness changes at specific pixels.
type EventFrame ¶
type EventFrame struct {
// FrameIndex is the sequential index of this frame.
FrameIndex int
// StartTimestamp is the start time of this frame in microseconds.
StartTimestamp int64
// EndTimestamp is the end time of this frame in microseconds.
EndTimestamp int64
// Events contains the events within this frame's time window.
Events []Event
// Width is the sensor width in pixels.
Width int
// Height is the sensor height in pixels.
Height int
}
EventFrame represents a group of events within a time window. Event frames are useful for integrating events into frame-like representations.
func NewEventFrame ¶
func NewEventFrame(index int, start, end int64, width, height int) (*EventFrame, error)
NewEventFrame creates a new EventFrame with the given parameters.
func (*EventFrame) AddEvent ¶
func (f *EventFrame) AddEvent(e Event) error
AddEvent adds an event to this frame.
func (*EventFrame) GetDuration ¶
func (f *EventFrame) GetDuration() int64
GetDuration returns the frame duration in microseconds.
type EventPolarity ¶
type EventPolarity uint8
EventPolarity represents the polarity of a brightness change event. Event cameras detect both positive (ON) and negative (OFF) brightness changes.
const ( // PolarityOFF indicates a brightness decrease (negative change). PolarityOFF EventPolarity = 0 // PolarityON indicates a brightness increase (positive change). PolarityON EventPolarity = 1 )
func (EventPolarity) IsValid ¶
func (p EventPolarity) IsValid() bool
IsValid returns true if the polarity is a valid value.
func (EventPolarity) String ¶
func (p EventPolarity) String() string
String returns a human-readable representation of the polarity.
type EventResidual ¶
type EventResidual struct {
// X is the residual for X coordinate.
X uint16
// Y is the residual for Y coordinate.
Y uint16
// Polarity is the event polarity (not predicted, stored directly).
Polarity EventPolarity
// Timestamp is the timestamp or timestamp delta.
Timestamp int64
}
EventResidual represents the residual (prediction error) for an event. Used in spatial prediction encoding.
type EventStream ¶
type EventStream struct {
// Width is the sensor width in pixels.
Width int
// Height is the sensor height in pixels.
Height int
// Events contains the decoded events in timestamp order.
Events []Event
// StartTimestamp is the timestamp of the first event in microseconds.
StartTimestamp int64
// EndTimestamp is the timestamp of the last event in microseconds.
EndTimestamp int64
// TotalDuration is the duration of the stream in microseconds.
TotalDuration int64
}
EventStream represents a stream of events from an event camera. Event streams are ordered by timestamp and may contain millions of events.
func NewEventStream ¶
func NewEventStream(width, height int) (*EventStream, error)
NewEventStream creates a new EventStream with the given sensor dimensions.
func (*EventStream) AddEvent ¶
func (s *EventStream) AddEvent(e Event) error
AddEvent adds a new event to the stream with validation.
func (*EventStream) GetEventCount ¶
func (s *EventStream) GetEventCount() int
GetEventCount returns the number of events in the stream.
func (*EventStream) GetEventRate ¶
func (s *EventStream) GetEventRate() float64
GetEventRate returns the average event rate in events per second.
func (*EventStream) Validate ¶
func (s *EventStream) Validate() error
Validate checks that the event stream is valid.
type LosslessCodec ¶
type LosslessCodec struct {
// contains filtered or unexported fields
}
LosslessCodec combines differential timestamp coding, spatial prediction, and entropy coding into a complete lossless codec for event data.
func NewLosslessCodec ¶
func NewLosslessCodec(width, height int) *LosslessCodec
NewLosslessCodec creates a new lossless codec for the given sensor dimensions.
type SpatialPredictor ¶
type SpatialPredictor struct {
// contains filtered or unexported fields
}
SpatialPredictor predicts event coordinates from previous events. This exploits temporal locality - consecutive events often occur near each other.
func NewSpatialPredictor ¶
func NewSpatialPredictor(width, height int) *SpatialPredictor
NewSpatialPredictor creates a new spatial predictor for the given sensor dimensions.
func (*SpatialPredictor) Decode ¶
func (p *SpatialPredictor) Decode(residual EventResidual) Event
Decode reconstructs coordinates from a residual.
func (*SpatialPredictor) Encode ¶
func (p *SpatialPredictor) Encode(event Event) EventResidual
Encode predicts coordinates and returns the residual.
func (*SpatialPredictor) Reset ¶
func (p *SpatialPredictor) Reset()
Reset resets the predictor state.
type StreamHeader ¶
type StreamHeader struct {
// Signature is the file signature (should be "JPXE").
Signature [4]byte
// Version is the JPEG XE version number.
Version uint8
// Flags contains stream-level flags.
Flags uint8
// SensorWidth is the event sensor width in pixels.
SensorWidth uint16
// SensorHeight is the event sensor height in pixels.
SensorHeight uint16
// TimestampResolution is the timestamp resolution in nanoseconds.
// Typical values: 1000 (microsecond), 1 (nanosecond).
TimestampResolution uint32
// EventCount is the total number of events in the stream.
EventCount uint32
// DurationMicros is the total stream duration in microseconds.
DurationMicros int64
// Reserved bytes for future use.
Reserved [8]byte
}
StreamHeader represents the JPEG XE stream header.
func ParseEventStreamHeader ¶
func ParseEventStreamHeader(data []byte) (*StreamHeader, error)
ParseEventStreamHeader parses just the header without decoding events. Useful for quickly reading stream metadata.
func (*StreamHeader) GetSensorDimensions ¶
func (h *StreamHeader) GetSensorDimensions() (int, int, error)
GetSensorDimensions returns the sensor width and height as ints.
func (*StreamHeader) IsValid ¶
func (h *StreamHeader) IsValid() bool
IsValid checks if the header signature matches JPEG XE.
func (*StreamHeader) Validate ¶
func (h *StreamHeader) Validate() error
Validate checks that the header values are within acceptable ranges.