Documentation
¶
Overview ¶
Package jpegai provides parsing and decoding support for JPEG AI (ISO/IEC 6046) neural network-based image compression.
JPEG AI is a learning-based image compression standard that uses neural networks for encoding and decoding. Unlike traditional transform-based codecs (like DCT in JPEG), JPEG AI employs:
- Learned transforms via neural networks (encoder/decoder networks)
- Latent representations in a learned feature space
- Entropy coding with hyperprior models
- End-to-end optimized rate-distortion performance
Architecture Overview ¶
The JPEG AI codec architecture consists of:
- Encoder Network: Transforms input image to latent representation
- Hyperprior Network: Models latent statistics for entropy coding
- Entropy Coder: Compresses latents using learned probability models
- Decoder Network: Reconstructs image from decoded latents
Important Note on Decoding ¶
This package provides the bitstream parsing infrastructure for JPEG AI. Actual neural network inference requires an external backend (such as ONNX Runtime, TensorFlow, or PyTorch) which must be configured separately.
Without a configured inference backend, this package can:
- Parse JPEG AI bitstream headers
- Extract model metadata
- Decode latent representations
- Validate bitstream structure
Full image decoding requires calling the inference backend with the extracted latent data.
Pre-Normative Signature Notice ¶
The 4-byte 'JAI\x00' magic used by ParseHeader is a pre-normative placeholder. ISO/IEC 6046 targets a JUMBF-contained 'jaic' box signature, not a raw 4-byte magic. To avoid silently accepting non-conformant bitstreams, ParseHeader rejects the placeholder signature by default. Call-sites that need to exercise the legacy layout must opt in by setting AllowPlaceholderSignature to true. Once JUMBF 'jaic' routing is wired in via internal/jpegsystems/jumbf, the placeholder path will be removed.
Security Considerations ¶
All parsing operations enforce security limits defined in the security package to prevent denial-of-service attacks from malformed bitstreams. Model loading validates model size against MaxModelSize limits.
Usage ¶
Basic parsing without inference:
data, err := os.ReadFile("image.jai")
if err != nil {
log.Fatal(err)
}
header, err := jpegai.ParseHeader(data)
if err != nil {
log.Fatal(err)
}
fmt.Printf("Image: %dx%d, Model: %s\n", header.Width, header.Height, header.ModelID)
With external inference backend:
decoder := jpegai.NewDecoder(data) decoder.SetInferenceEngine(myONNXBackend) pixels, err := decoder.Decode()
Reference ¶
ISO/IEC 6046: Information technology - Learned Image Coding (JPEG AI, SC29/WG1). The working designation during CD/FDIS is in the ISO/IEC 6046 family; the earlier "ISO/IEC 23090-4" citation referred to the MPEG Immersive Media series (SC29/WG11) and did not apply to JPEG AI.
Index ¶
- Constants
- Variables
- type ColorSpace
- type DecodeInfo
- type Decoder
- func (d *Decoder) Decode() ([]float32, error)
- func (d *Decoder) GetDecodeInfo() (*DecodeInfo, error)
- func (d *Decoder) GetHeader() *JPEGAIHeader
- func (d *Decoder) Infer(latentData []float32) ([]float32, error)
- func (d *Decoder) IsModelLoaded() bool
- func (d *Decoder) LoadModel(path string) (*ModelMetadata, error)
- func (d *Decoder) ParseHeader() (*JPEGAIHeader, error)
- func (d *Decoder) SetInferenceEngine(engine InferenceEngine)
- func (d *Decoder) SetModelLoader(loader ModelLoader)
- func (d *Decoder) UnloadModel() error
- type DecoderNetwork
- type DecodingContext
- type EncoderNetwork
- type EntropyMode
- type HyperpriorData
- type InferenceEngine
- type JPEGAIHeader
- type LatentCodestream
- type LatentRepresentation
- type LatentShape
- type ModelLoader
- type ModelMetadata
- type ParseError
- type Parser
- func (p *Parser) DecodeLatents(codestream *LatentCodestream) (*LatentRepresentation, error)
- func (p *Parser) ParseHyperprior() (*HyperpriorData, error)
- func (p *Parser) ParseLatentCodestream(expectedShape *LatentShape) (*LatentCodestream, error)
- func (p *Parser) Position() int
- func (p *Parser) Remaining() int
- func (p *Parser) SetPosition(pos int)
- type Profile
Constants ¶
const ( // MarkerLatentCodestream marks the start of latent codestream data. MarkerLatentCodestream = 0xAA // MarkerHyperprior marks the start of hyperprior data. MarkerHyperprior = 0xBB // MarkerSideInfo marks the start of side information. MarkerSideInfo = 0xCC // MarkerEndOfBitstream marks the end of the bitstream. MarkerEndOfBitstream = 0xFF )
Marker bytes used in JPEG AI bitstream.
const ( // SignatureSize is the byte length of the pre-normative 'JAI\x00' magic. // ISO/IEC 6046 targets a JUMBF-contained 'jaic' box signature, not a raw // 4-byte magic; SignatureSize is retained only for the placeholder path. SignatureSize = 4 // MinHeaderSize is the minimum valid header size in bytes. MinHeaderSize = 32 // MaxLatentDimension is the maximum allowed latent tensor dimension. MaxLatentDimension = 4096 // MaxChannels is the maximum number of latent channels. MaxChannels = 1024 // CurrentVersion is the currently supported bitstream version. CurrentVersion = 1 )
JPEG AI Constants and Magic Numbers
Variables ¶
var ( // ErrInvalidSignature indicates the data does not have a valid JPEG AI signature. ErrInvalidSignature = errors.New("jpegai: invalid JPEG AI signature") // ErrTruncatedBitstream indicates the bitstream is truncated. ErrTruncatedBitstream = errors.New("jpegai: truncated bitstream") // ErrInvalidHeader indicates the header structure is malformed. ErrInvalidHeader = errors.New("jpegai: invalid header structure") // ErrUnsupportedVersion indicates the bitstream version is not supported. ErrUnsupportedVersion = errors.New("jpegai: unsupported bitstream version") // ErrUnsupportedProfile indicates the coding profile is not supported. ErrUnsupportedProfile = errors.New("jpegai: unsupported coding profile") // ErrInvalidModelID indicates the model identifier is invalid or unknown. ErrInvalidModelID = errors.New("jpegai: invalid model identifier") // ErrInvalidDimensions indicates the image dimensions are invalid. ErrInvalidDimensions = errors.New("jpegai: invalid image dimensions") // ErrInvalidQuality indicates the quality parameter is invalid. ErrInvalidQuality = errors.New("jpegai: invalid quality parameter") // ErrInvalidLatentShape indicates the latent tensor shape is invalid. ErrInvalidLatentShape = errors.New("jpegai: invalid latent tensor shape") // ErrLatentDecodeFailed indicates entropy decoding of latents failed. ErrLatentDecodeFailed = errors.New("jpegai: latent decoding failed") // ErrHyperpriorDecodeFailed indicates hyperprior decoding failed. ErrHyperpriorDecodeFailed = errors.New("jpegai: hyperprior decoding failed") // ErrNoInferenceEngine indicates no inference backend is configured. ErrNoInferenceEngine = errors.New("jpegai: no inference engine configured") // ErrModelLoadFailed indicates the neural network model could not be loaded. ErrModelLoadFailed = errors.New("jpegai: model loading failed") // ErrModelNotLoaded indicates an operation was attempted without loading a model. ErrModelNotLoaded = errors.New("jpegai: model not loaded") // ErrInferenceFailed indicates neural network inference failed. ErrInferenceFailed = errors.New("jpegai: inference failed") // ErrTensorDimensionMismatch indicates tensor dimensions don't match expected shape. ErrTensorDimensionMismatch = errors.New("jpegai: tensor dimension mismatch") // ErrOutputSizeTooLarge indicates the decoded output would exceed limits. ErrOutputSizeTooLarge = errors.New("jpegai: output size exceeds maximum limit") // ErrModelTooLarge indicates the model size exceeds security limits. ErrModelTooLarge = errors.New("jpegai: model size exceeds maximum limit") // ErrIterationLimitExceeded indicates too many iterations in decoding loop. ErrIterationLimitExceeded = errors.New("jpegai: iteration limit exceeded") // ErrInvalidEntropyData indicates the entropy coded data is corrupted. ErrInvalidEntropyData = errors.New("jpegai: invalid entropy coded data") // ErrPlaceholderSignatureDisabled indicates the pre-normative 'JAI\x00' // magic was rejected because AllowPlaceholderSignature is false. ISO/IEC // 6046 targets a JUMBF-contained 'jaic' box signature; set // AllowPlaceholderSignature to true only in transitional call-sites. ErrPlaceholderSignatureDisabled = errors.New("jpegai: placeholder JAI signature disabled; set AllowPlaceholderSignature") // ErrChecksumMismatch indicates a checksum verification failed. ErrChecksumMismatch = errors.New("jpegai: checksum mismatch") )
JPEG AI specific errors.
var AllowPlaceholderSignature = false
AllowPlaceholderSignature opts in to parsing the pre-normative 4-byte 'JAI\x00' magic. It is false by default so that production call-sites do not silently accept non-conformant bitstreams. Test harnesses and transitional tooling may set it to true; ISO/IEC 6046-conformant parsing should consume the JUMBF 'jaic' box via internal/jpegsystems/jumbf once that routing lands.
var Signature = [4]byte{'J', 'A', 'I', 0x00}
Signature holds the 4-byte 'JAI\x00' magic used by the pre-normative placeholder header path. ISO/IEC 6046 targets a JUMBF-contained 'jaic' box signature; this raw magic is not normative and is only accepted when AllowPlaceholderSignature is set.
Functions ¶
This section is empty.
Types ¶
type ColorSpace ¶
type ColorSpace uint8
ColorSpace represents the color space of the image.
const ( // ColorSpaceRGB is the RGB color space. ColorSpaceRGB ColorSpace = 0 // ColorSpaceYCbCr is the YCbCr color space. ColorSpaceYCbCr ColorSpace = 1 // ColorSpaceGrayscale is single-channel grayscale. ColorSpaceGrayscale ColorSpace = 2 // ColorSpaceRGBA is RGB with alpha channel. ColorSpaceRGBA ColorSpace = 3 )
func (ColorSpace) String ¶
func (c ColorSpace) String() string
String returns a human-readable color space name.
type DecodeInfo ¶
type DecodeInfo struct {
Width int
Height int
ColorSpace ColorSpace
BitDepth int
ModelID string
Quality int
LatentShape *LatentShape
HasHyperprior bool
}
DecodeInfo represents information about a decoded JPEG AI image.
type Decoder ¶
type Decoder struct {
// contains filtered or unexported fields
}
Decoder handles JPEG AI decoding with optional neural network backend.
func (*Decoder) Decode ¶
Decode performs full JPEG AI decoding. This requires both a valid bitstream and a configured inference backend.
func (*Decoder) GetDecodeInfo ¶
func (d *Decoder) GetDecodeInfo() (*DecodeInfo, error)
GetDecodeInfo returns information about the image without full decoding.
func (*Decoder) GetHeader ¶
func (d *Decoder) GetHeader() *JPEGAIHeader
GetHeader returns the parsed header, or nil if not yet parsed.
func (*Decoder) Infer ¶
Infer runs neural network inference on latent data. Requires an InferenceEngine to be set via SetInferenceEngine.
func (*Decoder) IsModelLoaded ¶
IsModelLoaded returns true if a neural network model is loaded.
func (*Decoder) LoadModel ¶
func (d *Decoder) LoadModel(path string) (*ModelMetadata, error)
LoadModel loads a neural network model for decoding. Requires a ModelLoader to be set via SetModelLoader.
func (*Decoder) ParseHeader ¶
func (d *Decoder) ParseHeader() (*JPEGAIHeader, error)
ParseHeader parses the JPEG AI header from the data.
func (*Decoder) SetInferenceEngine ¶
func (d *Decoder) SetInferenceEngine(engine InferenceEngine)
SetInferenceEngine sets the inference engine implementation.
func (*Decoder) SetModelLoader ¶
func (d *Decoder) SetModelLoader(loader ModelLoader)
SetModelLoader sets the model loader implementation.
func (*Decoder) UnloadModel ¶
UnloadModel releases the currently loaded model.
type DecoderNetwork ¶
type DecoderNetwork interface {
// Decode transforms latent representation back to image pixels.
// Input shape: [batch, latent_channels, latent_height, latent_width]
// Output shape: [batch, channels, height, width]
Decode(latents []float32, shape *LatentShape) ([]float32, error)
// GetModelInfo returns metadata about the decoder model.
GetModelInfo() (*ModelMetadata, error)
}
DecoderNetwork defines the interface for the decoder neural network. This interface abstracts the decoder for external implementation.
type DecodingContext ¶
type DecodingContext struct {
// Header is the parsed header.
Header *JPEGAIHeader
// Latents is the decoded latent representation.
Latents *LatentRepresentation
// Hyperprior is the decoded hyperprior data (if present).
Hyperprior *HyperpriorData
// Position tracks the current read position.
Position int
// Iterations tracks decoding iterations for security.
Iterations int
}
DecodingContext holds state during the decoding process.
type EncoderNetwork ¶
type EncoderNetwork interface {
// Encode transforms input image data to latent representation.
// Input shape: [batch, channels, height, width]
// Output shape: [batch, latent_channels, latent_height, latent_width]
Encode(input []float32, width, height, channels int) ([]float32, *LatentShape, error)
// GetModelInfo returns metadata about the encoder model.
GetModelInfo() (*ModelMetadata, error)
}
EncoderNetwork defines the interface for the encoder neural network. This interface abstracts the encoder for external implementation.
type EntropyMode ¶
type EntropyMode uint8
EntropyMode represents the entropy coding method.
const ( // EntropyModeRange uses range coding (ANS variant). EntropyModeRange EntropyMode = 0 // EntropyModeArithmetic uses arithmetic coding. EntropyModeArithmetic EntropyMode = 1 // EntropyModeHybrid uses a hybrid coding scheme. EntropyModeHybrid EntropyMode = 2 )
func (EntropyMode) String ¶
func (m EntropyMode) String() string
String returns a human-readable entropy mode name.
type HyperpriorData ¶
type HyperpriorData struct {
// Shape describes the hyperprior latent shape.
Shape *LatentShape
// Data is the encoded hyperprior data.
Data []byte
// ScaleParams contains scale parameters for latent distributions.
ScaleParams []float32
// MeanParams contains mean parameters for latent distributions.
MeanParams []float32
}
HyperpriorData contains the hyperprior side information.
type InferenceEngine ¶
type InferenceEngine interface {
// Infer runs the decoder neural network on latent data.
// Input is the quantized latent representation.
// Output is the reconstructed pixel data.
Infer(latentData []float32) ([]float32, error)
// GetModelInfo returns metadata about the loaded model.
GetModelInfo() (*ModelMetadata, error)
}
InferenceEngine defines the interface for neural network inference. External backends (ONNX, TensorFlow, etc.) must implement this interface.
type JPEGAIHeader ¶
type JPEGAIHeader struct {
// Version is the bitstream version number.
Version uint16
// Profile is the coding profile.
Profile Profile
// Width is the image width in pixels.
Width int
// Height is the image height in pixels.
Height int
// ColorSpace indicates the color space.
ColorSpace ColorSpace
// BitDepth is the bit depth per sample.
BitDepth int
// ModelID identifies the neural network model.
ModelID string
// Quality is the quality parameter (0-100).
Quality int
// LatentShape describes the latent tensor dimensions.
LatentShape *LatentShape
// HasHyperprior indicates if hyperprior data is present.
HasHyperprior bool
// HasSideInfo indicates if additional side information is present.
HasSideInfo bool
// Checksum is the header checksum for validation.
Checksum uint32
}
JPEGAIHeader contains the parsed header information.
func ParseHeader ¶
func ParseHeader(data []byte) (*JPEGAIHeader, error)
ParseHeader parses the JPEG AI header from data.
type LatentCodestream ¶
type LatentCodestream struct {
// Data is the raw entropy-coded bitstream.
Data []byte
// Shape describes the expected decoded shape.
Shape *LatentShape
// EntropyMode indicates the entropy coding method.
EntropyMode EntropyMode
// TableOffset is the offset to entropy coding tables.
TableOffset int
// DataOffset is the offset to actual coded data.
DataOffset int
}
LatentCodestream holds the entropy-coded latent data.
type LatentRepresentation ¶
type LatentRepresentation struct {
// Shape describes the tensor dimensions.
Shape *LatentShape
// Data contains the quantized latent values.
// Stored as int16 after quantization.
Data []int16
// Scale is the quantization scale factor for dequantization.
Scale float32
// Offset is the quantization offset for dequantization.
Offset float32
}
LatentRepresentation holds the quantized latent data.
func (*LatentRepresentation) Dequantize ¶
func (l *LatentRepresentation) Dequantize() []float32
Dequantize converts quantized latents back to float32.
type LatentShape ¶
type LatentShape struct {
// Batch is the batch dimension (typically 1).
Batch int
// Channels is the number of latent channels.
Channels int
// Height is the spatial height of the latent tensor.
Height int
// Width is the spatial width of the latent tensor.
Width int
}
LatentShape describes the shape of latent tensor data.
func (*LatentShape) Size ¶
func (s *LatentShape) Size() int
Size returns the total number of elements in the latent tensor.
func (*LatentShape) Validate ¶
func (s *LatentShape) Validate() error
Validate checks if the latent shape is valid.
type ModelLoader ¶
type ModelLoader interface {
// LoadModel loads a neural network model from the specified path.
// Returns metadata about the loaded model.
LoadModel(path string) (*ModelMetadata, error)
// UnloadModel releases the currently loaded model.
UnloadModel() error
// IsModelLoaded returns true if a model is currently loaded.
IsModelLoaded() bool
}
ModelLoader defines the interface for loading neural network models. External backends must implement this interface to enable JPEG AI decoding.
type ModelMetadata ¶
type ModelMetadata struct {
// ModelID is the unique identifier for the model architecture.
ModelID string
// ModelVersion is the version of the model.
ModelVersion uint16
// Architecture describes the network architecture type.
Architecture string
// InputChannels is the number of input channels expected.
InputChannels int
// LatentChannels is the number of channels in latent space.
LatentChannels int
// DownsampleFactor is the spatial downsampling factor.
DownsampleFactor int
// HyperLatentChannels is the number of hyperprior latent channels.
HyperLatentChannels int
// Precision indicates the numerical precision (e.g., "float32", "float16").
Precision string
// ModelSize is the size of the model in bytes (for validation).
ModelSize int64
}
ModelMetadata contains information about the neural network model.
type ParseError ¶
type ParseError struct {
Offset int // Byte offset where the error occurred
Message string // Human-readable error description
Cause error // Underlying error if any
}
ParseError represents an error during JPEG AI parsing with context.
func NewParseError ¶
func NewParseError(offset int, message string, cause error) *ParseError
NewParseError creates a new parse error with context.
func (*ParseError) Error ¶
func (e *ParseError) Error() string
Error implements the error interface.
func (*ParseError) Unwrap ¶
func (e *ParseError) Unwrap() error
Unwrap returns the underlying error.
type Parser ¶
type Parser struct {
// contains filtered or unexported fields
}
Parser handles parsing of JPEG AI bitstreams.
func (*Parser) DecodeLatents ¶
func (p *Parser) DecodeLatents(codestream *LatentCodestream) (*LatentRepresentation, error)
DecodeLatents performs entropy decoding on the latent codestream. This is a stub that returns an error if the latent data is empty.
func (*Parser) ParseHyperprior ¶
func (p *Parser) ParseHyperprior() (*HyperpriorData, error)
ParseHyperprior parses hyperprior data from the bitstream.
func (*Parser) ParseLatentCodestream ¶
func (p *Parser) ParseLatentCodestream(expectedShape *LatentShape) (*LatentCodestream, error)
ParseLatentCodestream parses the latent codestream data.
func (*Parser) SetPosition ¶
SetPosition sets the current read position.
type Profile ¶
type Profile uint8
Profile represents the JPEG AI coding profile.
const ( // ProfileBase is the baseline profile with standard neural codec. ProfileBase Profile = 0 // ProfileHigh is the high-quality profile with enhanced network. ProfileHigh Profile = 1 // ProfileLowLatency is optimized for low-latency streaming. ProfileLowLatency Profile = 2 // ProfileScalable supports progressive/scalable decoding. ProfileScalable Profile = 3 )