jpegai

package
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 30, 2026 License: MIT Imports: 6 Imported by: 0

Documentation

Overview

Package jpegai provides parsing and decoding support for JPEG AI (ISO/IEC 6046) neural network-based image compression.

JPEG AI is a learning-based image compression standard that uses neural networks for encoding and decoding. Unlike traditional transform-based codecs (like DCT in JPEG), JPEG AI employs:

  • Learned transforms via neural networks (encoder/decoder networks)
  • Latent representations in a learned feature space
  • Entropy coding with hyperprior models
  • End-to-end optimized rate-distortion performance

Architecture Overview

The JPEG AI codec architecture consists of:

  1. Encoder Network: Transforms input image to latent representation
  2. Hyperprior Network: Models latent statistics for entropy coding
  3. Entropy Coder: Compresses latents using learned probability models
  4. Decoder Network: Reconstructs image from decoded latents

Important Note on Decoding

This package provides the bitstream parsing infrastructure for JPEG AI. Actual neural network inference requires an external backend (such as ONNX Runtime, TensorFlow, or PyTorch) which must be configured separately.

Without a configured inference backend, this package can:

  • Parse JPEG AI bitstream headers
  • Extract model metadata
  • Decode latent representations
  • Validate bitstream structure

Full image decoding requires calling the inference backend with the extracted latent data.

Pre-Normative Signature Notice

The 4-byte 'JAI\x00' magic used by ParseHeader is a pre-normative placeholder. ISO/IEC 6046 targets a JUMBF-contained 'jaic' box signature, not a raw 4-byte magic. To avoid silently accepting non-conformant bitstreams, ParseHeader rejects the placeholder signature by default. Call-sites that need to exercise the legacy layout must opt in by setting AllowPlaceholderSignature to true. Once JUMBF 'jaic' routing is wired in via internal/jpegsystems/jumbf, the placeholder path will be removed.

Security Considerations

All parsing operations enforce security limits defined in the security package to prevent denial-of-service attacks from malformed bitstreams. Model loading validates model size against MaxModelSize limits.

Usage

Basic parsing without inference:

data, err := os.ReadFile("image.jai")
if err != nil {
    log.Fatal(err)
}

header, err := jpegai.ParseHeader(data)
if err != nil {
    log.Fatal(err)
}
fmt.Printf("Image: %dx%d, Model: %s\n", header.Width, header.Height, header.ModelID)

With external inference backend:

decoder := jpegai.NewDecoder(data)
decoder.SetInferenceEngine(myONNXBackend)
pixels, err := decoder.Decode()

Reference

ISO/IEC 6046: Information technology - Learned Image Coding (JPEG AI, SC29/WG1). The working designation during CD/FDIS is in the ISO/IEC 6046 family; the earlier "ISO/IEC 23090-4" citation referred to the MPEG Immersive Media series (SC29/WG11) and did not apply to JPEG AI.

Index

Constants

View Source
const (
	// MarkerLatentCodestream marks the start of latent codestream data.
	MarkerLatentCodestream = 0xAA

	// MarkerHyperprior marks the start of hyperprior data.
	MarkerHyperprior = 0xBB

	// MarkerSideInfo marks the start of side information.
	MarkerSideInfo = 0xCC

	// MarkerEndOfBitstream marks the end of the bitstream.
	MarkerEndOfBitstream = 0xFF
)

Marker bytes used in JPEG AI bitstream.

View Source
const (
	// SignatureSize is the byte length of the pre-normative 'JAI\x00' magic.
	// ISO/IEC 6046 targets a JUMBF-contained 'jaic' box signature, not a raw
	// 4-byte magic; SignatureSize is retained only for the placeholder path.
	SignatureSize = 4

	// MinHeaderSize is the minimum valid header size in bytes.
	MinHeaderSize = 32

	// MaxLatentDimension is the maximum allowed latent tensor dimension.
	MaxLatentDimension = 4096

	// MaxChannels is the maximum number of latent channels.
	MaxChannels = 1024

	// CurrentVersion is the currently supported bitstream version.
	CurrentVersion = 1
)

JPEG AI Constants and Magic Numbers

Variables

View Source
var (
	// ErrInvalidSignature indicates the data does not have a valid JPEG AI signature.
	ErrInvalidSignature = errors.New("jpegai: invalid JPEG AI signature")

	// ErrTruncatedBitstream indicates the bitstream is truncated.
	ErrTruncatedBitstream = errors.New("jpegai: truncated bitstream")

	// ErrInvalidHeader indicates the header structure is malformed.
	ErrInvalidHeader = errors.New("jpegai: invalid header structure")

	// ErrUnsupportedVersion indicates the bitstream version is not supported.
	ErrUnsupportedVersion = errors.New("jpegai: unsupported bitstream version")

	// ErrUnsupportedProfile indicates the coding profile is not supported.
	ErrUnsupportedProfile = errors.New("jpegai: unsupported coding profile")

	// ErrInvalidModelID indicates the model identifier is invalid or unknown.
	ErrInvalidModelID = errors.New("jpegai: invalid model identifier")

	// ErrInvalidDimensions indicates the image dimensions are invalid.
	ErrInvalidDimensions = errors.New("jpegai: invalid image dimensions")

	// ErrInvalidQuality indicates the quality parameter is invalid.
	ErrInvalidQuality = errors.New("jpegai: invalid quality parameter")

	// ErrInvalidLatentShape indicates the latent tensor shape is invalid.
	ErrInvalidLatentShape = errors.New("jpegai: invalid latent tensor shape")

	// ErrLatentDecodeFailed indicates entropy decoding of latents failed.
	ErrLatentDecodeFailed = errors.New("jpegai: latent decoding failed")

	// ErrHyperpriorDecodeFailed indicates hyperprior decoding failed.
	ErrHyperpriorDecodeFailed = errors.New("jpegai: hyperprior decoding failed")

	// ErrNoInferenceEngine indicates no inference backend is configured.
	ErrNoInferenceEngine = errors.New("jpegai: no inference engine configured")

	// ErrModelLoadFailed indicates the neural network model could not be loaded.
	ErrModelLoadFailed = errors.New("jpegai: model loading failed")

	// ErrModelNotLoaded indicates an operation was attempted without loading a model.
	ErrModelNotLoaded = errors.New("jpegai: model not loaded")

	// ErrInferenceFailed indicates neural network inference failed.
	ErrInferenceFailed = errors.New("jpegai: inference failed")

	// ErrTensorDimensionMismatch indicates tensor dimensions don't match expected shape.
	ErrTensorDimensionMismatch = errors.New("jpegai: tensor dimension mismatch")

	// ErrOutputSizeTooLarge indicates the decoded output would exceed limits.
	ErrOutputSizeTooLarge = errors.New("jpegai: output size exceeds maximum limit")

	// ErrModelTooLarge indicates the model size exceeds security limits.
	ErrModelTooLarge = errors.New("jpegai: model size exceeds maximum limit")

	// ErrIterationLimitExceeded indicates too many iterations in decoding loop.
	ErrIterationLimitExceeded = errors.New("jpegai: iteration limit exceeded")

	// ErrInvalidEntropyData indicates the entropy coded data is corrupted.
	ErrInvalidEntropyData = errors.New("jpegai: invalid entropy coded data")

	// ErrPlaceholderSignatureDisabled indicates the pre-normative 'JAI\x00'
	// magic was rejected because AllowPlaceholderSignature is false. ISO/IEC
	// 6046 targets a JUMBF-contained 'jaic' box signature; set
	// AllowPlaceholderSignature to true only in transitional call-sites.
	ErrPlaceholderSignatureDisabled = errors.New("jpegai: placeholder JAI signature disabled; set AllowPlaceholderSignature")

	// ErrChecksumMismatch indicates a checksum verification failed.
	ErrChecksumMismatch = errors.New("jpegai: checksum mismatch")
)

JPEG AI specific errors.

View Source
var AllowPlaceholderSignature = false

AllowPlaceholderSignature opts in to parsing the pre-normative 4-byte 'JAI\x00' magic. It is false by default so that production call-sites do not silently accept non-conformant bitstreams. Test harnesses and transitional tooling may set it to true; ISO/IEC 6046-conformant parsing should consume the JUMBF 'jaic' box via internal/jpegsystems/jumbf once that routing lands.

View Source
var Signature = [4]byte{'J', 'A', 'I', 0x00}

Signature holds the 4-byte 'JAI\x00' magic used by the pre-normative placeholder header path. ISO/IEC 6046 targets a JUMBF-contained 'jaic' box signature; this raw magic is not normative and is only accepted when AllowPlaceholderSignature is set.

Functions

This section is empty.

Types

type ColorSpace

type ColorSpace uint8

ColorSpace represents the color space of the image.

const (
	// ColorSpaceRGB is the RGB color space.
	ColorSpaceRGB ColorSpace = 0

	// ColorSpaceYCbCr is the YCbCr color space.
	ColorSpaceYCbCr ColorSpace = 1

	// ColorSpaceGrayscale is single-channel grayscale.
	ColorSpaceGrayscale ColorSpace = 2

	// ColorSpaceRGBA is RGB with alpha channel.
	ColorSpaceRGBA ColorSpace = 3
)

func (ColorSpace) String

func (c ColorSpace) String() string

String returns a human-readable color space name.

type DecodeInfo

type DecodeInfo struct {
	Width         int
	Height        int
	ColorSpace    ColorSpace
	BitDepth      int
	ModelID       string
	Quality       int
	LatentShape   *LatentShape
	HasHyperprior bool
}

DecodeInfo represents information about a decoded JPEG AI image.

type Decoder

type Decoder struct {
	// contains filtered or unexported fields
}

Decoder handles JPEG AI decoding with optional neural network backend.

func NewDecoder

func NewDecoder(data []byte) *Decoder

NewDecoder creates a new JPEG AI decoder.

func (*Decoder) Decode

func (d *Decoder) Decode() ([]float32, error)

Decode performs full JPEG AI decoding. This requires both a valid bitstream and a configured inference backend.

func (*Decoder) GetDecodeInfo

func (d *Decoder) GetDecodeInfo() (*DecodeInfo, error)

GetDecodeInfo returns information about the image without full decoding.

func (*Decoder) GetHeader

func (d *Decoder) GetHeader() *JPEGAIHeader

GetHeader returns the parsed header, or nil if not yet parsed.

func (*Decoder) Infer

func (d *Decoder) Infer(latentData []float32) ([]float32, error)

Infer runs neural network inference on latent data. Requires an InferenceEngine to be set via SetInferenceEngine.

func (*Decoder) IsModelLoaded

func (d *Decoder) IsModelLoaded() bool

IsModelLoaded returns true if a neural network model is loaded.

func (*Decoder) LoadModel

func (d *Decoder) LoadModel(path string) (*ModelMetadata, error)

LoadModel loads a neural network model for decoding. Requires a ModelLoader to be set via SetModelLoader.

func (*Decoder) ParseHeader

func (d *Decoder) ParseHeader() (*JPEGAIHeader, error)

ParseHeader parses the JPEG AI header from the data.

func (*Decoder) SetInferenceEngine

func (d *Decoder) SetInferenceEngine(engine InferenceEngine)

SetInferenceEngine sets the inference engine implementation.

func (*Decoder) SetModelLoader

func (d *Decoder) SetModelLoader(loader ModelLoader)

SetModelLoader sets the model loader implementation.

func (*Decoder) UnloadModel

func (d *Decoder) UnloadModel() error

UnloadModel releases the currently loaded model.

type DecoderNetwork

type DecoderNetwork interface {
	// Decode transforms latent representation back to image pixels.
	// Input shape: [batch, latent_channels, latent_height, latent_width]
	// Output shape: [batch, channels, height, width]
	Decode(latents []float32, shape *LatentShape) ([]float32, error)

	// GetModelInfo returns metadata about the decoder model.
	GetModelInfo() (*ModelMetadata, error)
}

DecoderNetwork defines the interface for the decoder neural network. This interface abstracts the decoder for external implementation.

type DecodingContext

type DecodingContext struct {
	// Header is the parsed header.
	Header *JPEGAIHeader

	// Latents is the decoded latent representation.
	Latents *LatentRepresentation

	// Hyperprior is the decoded hyperprior data (if present).
	Hyperprior *HyperpriorData

	// Position tracks the current read position.
	Position int

	// Iterations tracks decoding iterations for security.
	Iterations int
}

DecodingContext holds state during the decoding process.

type EncoderNetwork

type EncoderNetwork interface {
	// Encode transforms input image data to latent representation.
	// Input shape: [batch, channels, height, width]
	// Output shape: [batch, latent_channels, latent_height, latent_width]
	Encode(input []float32, width, height, channels int) ([]float32, *LatentShape, error)

	// GetModelInfo returns metadata about the encoder model.
	GetModelInfo() (*ModelMetadata, error)
}

EncoderNetwork defines the interface for the encoder neural network. This interface abstracts the encoder for external implementation.

type EntropyMode

type EntropyMode uint8

EntropyMode represents the entropy coding method.

const (
	// EntropyModeRange uses range coding (ANS variant).
	EntropyModeRange EntropyMode = 0

	// EntropyModeArithmetic uses arithmetic coding.
	EntropyModeArithmetic EntropyMode = 1

	// EntropyModeHybrid uses a hybrid coding scheme.
	EntropyModeHybrid EntropyMode = 2
)

func (EntropyMode) String

func (m EntropyMode) String() string

String returns a human-readable entropy mode name.

type HyperpriorData

type HyperpriorData struct {
	// Shape describes the hyperprior latent shape.
	Shape *LatentShape

	// Data is the encoded hyperprior data.
	Data []byte

	// ScaleParams contains scale parameters for latent distributions.
	ScaleParams []float32

	// MeanParams contains mean parameters for latent distributions.
	MeanParams []float32
}

HyperpriorData contains the hyperprior side information.

type InferenceEngine

type InferenceEngine interface {
	// Infer runs the decoder neural network on latent data.
	// Input is the quantized latent representation.
	// Output is the reconstructed pixel data.
	Infer(latentData []float32) ([]float32, error)

	// GetModelInfo returns metadata about the loaded model.
	GetModelInfo() (*ModelMetadata, error)
}

InferenceEngine defines the interface for neural network inference. External backends (ONNX, TensorFlow, etc.) must implement this interface.

type JPEGAIHeader

type JPEGAIHeader struct {
	// Version is the bitstream version number.
	Version uint16

	// Profile is the coding profile.
	Profile Profile

	// Width is the image width in pixels.
	Width int

	// Height is the image height in pixels.
	Height int

	// ColorSpace indicates the color space.
	ColorSpace ColorSpace

	// BitDepth is the bit depth per sample.
	BitDepth int

	// ModelID identifies the neural network model.
	ModelID string

	// Quality is the quality parameter (0-100).
	Quality int

	// LatentShape describes the latent tensor dimensions.
	LatentShape *LatentShape

	// HasHyperprior indicates if hyperprior data is present.
	HasHyperprior bool

	// HasSideInfo indicates if additional side information is present.
	HasSideInfo bool

	// Checksum is the header checksum for validation.
	Checksum uint32
}

JPEGAIHeader contains the parsed header information.

func ParseHeader

func ParseHeader(data []byte) (*JPEGAIHeader, error)

ParseHeader parses the JPEG AI header from data.

type LatentCodestream

type LatentCodestream struct {
	// Data is the raw entropy-coded bitstream.
	Data []byte

	// Shape describes the expected decoded shape.
	Shape *LatentShape

	// EntropyMode indicates the entropy coding method.
	EntropyMode EntropyMode

	// TableOffset is the offset to entropy coding tables.
	TableOffset int

	// DataOffset is the offset to actual coded data.
	DataOffset int
}

LatentCodestream holds the entropy-coded latent data.

type LatentRepresentation

type LatentRepresentation struct {
	// Shape describes the tensor dimensions.
	Shape *LatentShape

	// Data contains the quantized latent values.
	// Stored as int16 after quantization.
	Data []int16

	// Scale is the quantization scale factor for dequantization.
	Scale float32

	// Offset is the quantization offset for dequantization.
	Offset float32
}

LatentRepresentation holds the quantized latent data.

func (*LatentRepresentation) Dequantize

func (l *LatentRepresentation) Dequantize() []float32

Dequantize converts quantized latents back to float32.

type LatentShape

type LatentShape struct {
	// Batch is the batch dimension (typically 1).
	Batch int

	// Channels is the number of latent channels.
	Channels int

	// Height is the spatial height of the latent tensor.
	Height int

	// Width is the spatial width of the latent tensor.
	Width int
}

LatentShape describes the shape of latent tensor data.

func (*LatentShape) Size

func (s *LatentShape) Size() int

Size returns the total number of elements in the latent tensor.

func (*LatentShape) Validate

func (s *LatentShape) Validate() error

Validate checks if the latent shape is valid.

type ModelLoader

type ModelLoader interface {
	// LoadModel loads a neural network model from the specified path.
	// Returns metadata about the loaded model.
	LoadModel(path string) (*ModelMetadata, error)

	// UnloadModel releases the currently loaded model.
	UnloadModel() error

	// IsModelLoaded returns true if a model is currently loaded.
	IsModelLoaded() bool
}

ModelLoader defines the interface for loading neural network models. External backends must implement this interface to enable JPEG AI decoding.

type ModelMetadata

type ModelMetadata struct {
	// ModelID is the unique identifier for the model architecture.
	ModelID string

	// ModelVersion is the version of the model.
	ModelVersion uint16

	// Architecture describes the network architecture type.
	Architecture string

	// InputChannels is the number of input channels expected.
	InputChannels int

	// LatentChannels is the number of channels in latent space.
	LatentChannels int

	// DownsampleFactor is the spatial downsampling factor.
	DownsampleFactor int

	// HyperLatentChannels is the number of hyperprior latent channels.
	HyperLatentChannels int

	// Precision indicates the numerical precision (e.g., "float32", "float16").
	Precision string

	// ModelSize is the size of the model in bytes (for validation).
	ModelSize int64
}

ModelMetadata contains information about the neural network model.

type ParseError

type ParseError struct {
	Offset  int    // Byte offset where the error occurred
	Message string // Human-readable error description
	Cause   error  // Underlying error if any
}

ParseError represents an error during JPEG AI parsing with context.

func NewParseError

func NewParseError(offset int, message string, cause error) *ParseError

NewParseError creates a new parse error with context.

func (*ParseError) Error

func (e *ParseError) Error() string

Error implements the error interface.

func (*ParseError) Unwrap

func (e *ParseError) Unwrap() error

Unwrap returns the underlying error.

type Parser

type Parser struct {
	// contains filtered or unexported fields
}

Parser handles parsing of JPEG AI bitstreams.

func NewParser

func NewParser(data []byte) *Parser

NewParser creates a new JPEG AI parser.

func (*Parser) DecodeLatents

func (p *Parser) DecodeLatents(codestream *LatentCodestream) (*LatentRepresentation, error)

DecodeLatents performs entropy decoding on the latent codestream. This is a stub that returns an error if the latent data is empty.

func (*Parser) ParseHyperprior

func (p *Parser) ParseHyperprior() (*HyperpriorData, error)

ParseHyperprior parses hyperprior data from the bitstream.

func (*Parser) ParseLatentCodestream

func (p *Parser) ParseLatentCodestream(expectedShape *LatentShape) (*LatentCodestream, error)

ParseLatentCodestream parses the latent codestream data.

func (*Parser) Position

func (p *Parser) Position() int

Position returns the current read position.

func (*Parser) Remaining

func (p *Parser) Remaining() int

Remaining returns the number of bytes remaining.

func (*Parser) SetPosition

func (p *Parser) SetPosition(pos int)

SetPosition sets the current read position.

type Profile

type Profile uint8

Profile represents the JPEG AI coding profile.

const (
	// ProfileBase is the baseline profile with standard neural codec.
	ProfileBase Profile = 0

	// ProfileHigh is the high-quality profile with enhanced network.
	ProfileHigh Profile = 1

	// ProfileLowLatency is optimized for low-latency streaming.
	ProfileLowLatency Profile = 2

	// ProfileScalable supports progressive/scalable decoding.
	ProfileScalable Profile = 3
)

func (Profile) String

func (p Profile) String() string

String returns a human-readable profile name.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL