Documentation
¶
Overview ¶
Package extract implements the F5 steganographic algorithm for extracting hidden messages from JPEG DCT coefficients.
The F5 algorithm was developed by Andreas Westfeld (2001) and is designed to resist both visual and statistical detection attacks while maintaining high embedding capacity. This implementation produces byte-identical output to the Java reference implementation for interoperability.
Algorithm Overview ¶
F5 extraction reverses the embedding process:
- Jsteg: LSB replacement (detectable via chi-square attack)
- F3: Decrements instead of overwrites (detectable via even coefficient surplus)
- F4: Sign-based encoding (resistant to statistical attacks)
- F5: F4 + matrix encoding + permutative straddling (optimal efficiency)
The extraction process follows these steps:
- Seed PRNG with password bytes
- Generate Fisher-Yates permutation of coefficient indices
- Extract 32-bit header (k parameter and file size) from permuted coefficients
- XOR header with PRNG bytes to decode
- Extract message bits using matrix decoding with parameter k
- XOR message bytes with PRNG stream to recover original data
Key Features ¶
- Matrix Decoding: Uses (1, n, k) codes where n = 2^k - 1. Extracts k message bits from n coefficients by computing XOR hash of coefficient positions.
- Permutative Straddling: Uses same Fisher-Yates permutation as embedding to access coefficients in the correct order for extraction.
- Header Parsing: Extracts 32-bit header containing k parameter (8 bits) and file size (23 bits) for proper message recovery.
- De-Zigzag Transformation: Applies JPEG de-zigzag table to access coefficients in their natural 8x8 block positions.
Security Warning ¶
This package uses SHA1PRNG for pseudorandom number generation to maintain compatibility with the Java reference implementation. SHA1PRNG is NOT cryptographically secure by modern standards. The algorithm relies on SHA-1 which has known collision vulnerabilities.
DO NOT use this package for security-critical applications requiring cryptographic randomness. The PRNG is deterministic and predictable given the password/seed.
This package is intended for:
- Digital forensics and steganography analysis
- Compatibility with existing F5-encoded images (e.g., PixelKnot)
- Research and educational purposes
- Applications where statistical undetectability is prioritized over cryptographic security
Basic Usage ¶
Extracting a message from JPEG DCT coefficients:
import (
"github.com/0verkilll/f5/extract"
"github.com/0verkilll/fisheryates"
"github.com/0verkilll/sha1"
)
// Create dependencies
hasher := sha1.NewSHA1(sha1.NewBigEndian())
permutator := fisheryates.NewFisherYates()
// Create extractor
extractor := extract.NewExtractor(hasher, permutator)
// Extract hidden message from JPEG DCT coefficients
result, err := extractor.Extract(coefficients, "password")
if err != nil {
log.Fatal(err)
}
fmt.Printf("Extracted %d bytes with k=%d\n", result.FileSize, result.K)
fmt.Println(string(result.Data))
Advanced Options ¶
Using options for custom file size limits:
extractor := extract.NewExtractor(
hasher,
permutator,
extract.WithMaxFileSize(50 * 1024 * 1024), // 50MB limit
)
Dependencies ¶
This package depends on the following sibling packages:
- securerandom: SHA1PRNG implementation for deterministic randomness
- fisheryates: Fisher-Yates shuffle for coefficient permutation
- sha1: SHA-1 hasher for PRNG seeding
- f5core: Shared constants and de-zigzag table
- f5coefficient: Coefficient bit extraction functions (via internal/coefficient)
- logger: Optional structured logging support (via internal/logx)
- i18n: Internationalized error messages
Reference ¶
Westfeld, A. (2001). F5 - A Steganographic Algorithm: High Capacity Despite Better Steganalysis. Lecture Notes in Computer Science, 2137, 289-302.
Index ¶
- Constants
- Variables
- func GetLogger() logger.Logger
- func SetLogger(l logger.Logger)
- func SetTranslator(translator TranslatorProvider)
- func Translate(key string) string
- func ValidateCoefficients(coefficients []int16) error
- func ValidatePassword(password string) error
- type ExtractionResult
- type Extractor
- type F5Extractor
- type Hasher
- type Option
- type PRNGFactory
- type Permutator
- type RandomSource
- type TranslatorProvider
- type ValidationError
Constants ¶
const ( // MaxFileSize is the maximum allowed size for extracted files. // This prevents memory exhaustion attacks from malformed headers. // See f5core.DefaultMaxFileSize for documentation. MaxFileSize = f5core.DefaultMaxFileSize // MaxCoefficientArraySize is the maximum allowed size for coefficient arrays. // This prevents memory exhaustion when processing untrusted input. MaxCoefficientArraySize = 100_000_000 // 100 million coefficients // MinKParameter is the minimum valid k value. // k=1 uses simple (1,1,1) codes: 1 coefficient per bit (no matrix encoding). // See f5core.MinKParameter for documentation. MinKParameter = f5core.MinKParameter // MaxKParameter is the maximum valid k value. // k=8 uses (1,255,8) codes: 255 coefficients to extract 8 bits. // See f5core.MaxKParameter for documentation. MaxKParameter = f5core.MaxKParameter // HeaderSize is the number of bits in the F5 message header. // The header contains the k parameter (8 bits) and file size (23 bits). // See f5core.HeaderSize for documentation. HeaderSize = f5core.HeaderSize // MaxMessageSize is the maximum message size in bytes that can be embedded/extracted. // This is limited by the 23-bit file size field in the F5 header (2^23 - 1 = 8,388,607 bytes). // See f5core.MaxMessageSize for documentation. MaxMessageSize = f5core.MaxMessageSize )
Re-export constants from f5core for backward compatibility and convenience. These constants define the limits and parameters for the F5 extraction algorithm.
const ( ErrKeyEmptyCoefficients = "f5messageextract.error.empty_coefficients" //nolint:gosec // G101: translation key, not credential ErrKeyEmptyPassword = "f5messageextract.error.empty_password" //nolint:gosec // G101: translation key, not credential ErrKeyInvalidCoefficientVal = "f5messageextract.error.invalid_coefficient_range" ErrKeyInvalidKParameter = "f5messageextract.error.invalid_k_parameter" ErrKeyInvalidFileSize = "f5messageextract.error.invalid_file_size" ErrKeyFileSizeExceedsMax = "f5messageextract.error.file_size_exceeds_max" ErrKeyInsufficientCoeffs = "f5messageextract.error.insufficient_coefficients" ErrKeyIncompleteExtraction = "f5messageextract.error.incomplete_extraction" ErrKeyPermutationFailed = "f5messageextract.error.permutation_failed" ErrKeySeedingFailed = "f5messageextract.error.seeding_failed" //nolint:gosec // G101: translation key, not credential )
Error keys for i18n translation. These keys correspond to entries in locales/*.json. #nosec G101 -- These are translation keys, not hardcoded credentials
const ( // CoefficientMin is the minimum valid JPEG quantized DCT coefficient value. // See f5core.CoefficientMin for documentation. CoefficientMin = valid.CoefficientMin // CoefficientMax is the maximum valid JPEG quantized DCT coefficient value. // See f5core.CoefficientMax for documentation. CoefficientMax = valid.CoefficientMax )
Coefficient range constants re-exported from the shared valid package (which in turn re-exports f5core) for validation.
Variables ¶
var ( // ErrNoCoefficients is returned when the coefficient array is empty. // F5 extraction requires at least enough coefficients to extract the header. ErrNoCoefficients = errors.New("f5messageextract: coefficient array is empty") // ErrEmptyPassword is returned when the password string is empty. // The password is required for PRNG seeding to generate the coefficient permutation. ErrEmptyPassword = errors.New("f5messageextract: password cannot be empty") // ErrInsufficientCoefficients is returned when there are not enough // usable coefficients (non-zero, non-DC) to extract the header or message. // This typically indicates the image is too small or has too few non-zero coefficients. ErrInsufficientCoefficients = errors.New("f5messageextract: insufficient usable coefficients for extraction") // ErrInvalidKParameter is returned when the k parameter from the header // is outside the valid range [1, 8]. // k=0 is invalid as it would mean no bits per group. // k>8 is impractical as it would require 255+ coefficients per group. ErrInvalidKParameter = errors.New("f5messageextract: k parameter outside valid range [1, 8]") // ErrInvalidFileSize is returned when the file size from the header is invalid. // This includes: // - Zero file size (nothing to extract) // - Negative file size (corrupted header) // - File size exceeding MaxFileSize limit (potential attack or corruption) ErrInvalidFileSize = errors.New("f5messageextract: invalid file size in header") // ErrIncompleteExtraction is returned when extraction terminates before // all expected bytes have been extracted. // This typically indicates coefficient exhaustion or data corruption. ErrIncompleteExtraction = errors.New("f5messageextract: extraction incomplete, not all bytes extracted") // ErrSeedingFailed is returned when the PRNG cannot be initialized with // the derived key (password-hash combination). This is rare in practice — // SHA-1 seeding cannot fail for any input the Extract surface accepts — // but consumers should still propagate it rather than silently swallowing. ErrSeedingFailed = errors.New("f5messageextract: PRNG seeding failed") )
Sentinel errors for F5 extraction failures. These errors can be checked using errors.Is() and wrapped with additional context using wrapError() or fmt.Errorf() with %w verb.
Example:
result, err := extractor.Extract(coeffs, password)
if errors.Is(err, extract.ErrNoCoefficients) {
// Handle empty coefficient array
}
Functions ¶
func GetLogger ¶
GetLogger returns the package logger, or NopLogger if not set. This function is thread-safe and can be called from any goroutine.
func SetLogger ¶
SetLogger sets the logger for the extract package. Pass nil to disable logging and reset to the default NopLogger. The logger is shared across all goroutines and is thread-safe.
By default, the package uses NopLogger which discards all log messages. Set a logger to enable debug output for extraction operations.
Example:
// Enable logging with a custom logger extract.SetLogger(myLogger) // Disable logging extract.SetLogger(nil)
func SetTranslator ¶
func SetTranslator(translator TranslatorProvider)
SetTranslator sets the global translator for this package. This allows the application to provide translations for error messages and other user-facing strings.
Pass nil to disable translations and use English defaults.
This function is thread-safe and can be called from multiple goroutines.
Example:
translator, _ := i18n.New(
i18n.WithFileSystemLoader("locales"),
i18n.WithDefaultLocale("en-US"),
)
extract.SetTranslator(translator)
func Translate ¶
Translate returns a translated message for the given key. If no translator is set or the key is not found, returns the default English message. This is the public translation helper function for i18n support.
func ValidateCoefficients ¶
ValidateCoefficients validates that the coefficient slice is valid for extraction. It checks that:
- The slice is not empty
- All coefficient values are within the valid JPEG DCT range [-2048, 2047]
The validation rule itself is delegated to the shared valid package; this function maps the shared valid.Reason back onto this package's i18n keys and ValidationError model.
Returns nil if validation passes, or a ValidationError if it fails.
func ValidatePassword ¶
ValidatePassword validates that the password is valid for extraction. It checks that the password is not an empty string.
The rule is delegated to the shared valid package; presentation as a ValidationError stays in this package.
Returns nil if validation passes, or a ValidationError if it fails.
Types ¶
type ExtractionResult ¶
type ExtractionResult struct {
// Data contains the extracted message bytes after XOR decoding
// with the PRNG stream.
Data []byte
// K is the k parameter from the F5 header, indicating the matrix
// encoding scheme used. Valid range is [1, 8].
// With k bits per n=2^k-1 coefficients, higher k values provide
// better embedding efficiency but require more coefficients.
K int
// FileSize is the original file size from the F5 header.
// This indicates how many bytes were embedded (before any padding).
FileSize int
}
ExtractionResult contains the result of an F5 extraction operation. This structured result type provides access to both the extracted data and the extraction parameters used.
type Extractor ¶
type Extractor interface {
// Extract extracts a hidden message from JPEG DCT coefficients.
// The extraction uses the provided password to seed the PRNG for
// generating the coefficient permutation.
//
// The method follows the F5 algorithm:
// 1. Seed PRNG with password bytes
// 2. Generate Fisher-Yates permutation of coefficient indices
// 3. Extract 32 header bits from permuted coefficients
// 4. Parse k parameter and file size from header
// 5. Extract message using matrix encoding
//
// Parameters:
// coefficients - JPEG DCT coefficients as int16 array
// password - Password used to seed the PRNG
//
// Returns:
// *ExtractionResult containing the extracted data, k parameter, and file size
// error if extraction fails (empty coefficients, invalid header, etc.)
//
// Errors:
// - ErrNoCoefficients: coefficients slice is empty
// - ErrEmptyPassword: password string is empty
// - ErrInsufficientCoefficients: not enough coefficients for extraction
// - ErrInvalidKParameter: k parameter outside valid range (1-8)
// - ErrInvalidFileSize: file size is invalid (zero, negative, or too large)
// - ErrIncompleteExtraction: extraction did not complete successfully
Extract(coefficients []int16, password string) (*ExtractionResult, error)
}
Extractor defines the interface for F5 steganographic extraction. This exportable interface enables mock injection in user tests and follows the Dependency Inversion Principle by providing an abstraction for extraction.
The interface is designed for use in forensic analysis and security research applications where hidden messages need to be extracted from JPEG images.
type F5Extractor ¶
type F5Extractor struct {
// contains filtered or unexported fields
}
F5Extractor implements the F5 steganographic extraction algorithm.
This implementation follows the SOLID Adapter pattern, using dependency injection for all cryptographic and algorithmic components. The extractor coordinates the F5 algorithm's steps:
- Convert password to seed and initialize PRNG
- Generate Fisher-Yates permutation of coefficients
- Extract 32-bit header (k parameter and file size)
- XOR header with PRNG bytes
- Extract message using matrix encoding
- XOR message bytes with PRNG stream
The algorithm matches the Java F5 implementation exactly, including:
- Signed arithmetic for PRNG and XOR operations
- DC coefficient skipping (multiples of 64)
- De-zigzag transformation for JPEG blocks
- Matrix encoding with Hamming codes
Dependencies are injected to enable testing and portability to other languages (TypeScript, Rust, JavaScript, WebAssembly).
Thread Safety: Each extraction creates a fresh PRNG instance, eliminating shared mutable state and making the extractor fully thread-safe. When WithPermutationPool is enabled, the shared sync.Pool is the only cross-call mutable state and it is safe for concurrent use by design.
func NewExtractor ¶
func NewExtractor(hasher Hasher, permutator Permutator, opts ...Option) *F5Extractor
NewExtractor creates a new F5 message extractor with injected dependencies.
Parameters:
- hasher: SHA1 hasher for PRNG seeding (used when no PRNGFactory is provided)
- permutator: Fisher-Yates permutation generator
- opts: Optional configuration options (e.g., WithMaxFileSize, WithPRNGFactory)
Returns an *F5Extractor that implements the Extractor interface.
Note: This extractor is fully thread-safe. Each password test creates a fresh PRNG instance, eliminating shared mutable state.
Example:
extractor := extract.NewExtractor(sha1Hasher, fisherYates)
result, err := extractor.Extract(coefficients, "password")
// With PRNGFactory for dependency injection (recommended):
factory := f5prng.NewDefaultFactory()
extractor := extract.NewExtractor(sha1Hasher, fisherYates,
extract.WithPRNGFactory(factory))
func (*F5Extractor) Extract ¶
func (f *F5Extractor) Extract(coefficients []int16, password string) (*ExtractionResult, error)
Extract extracts a hidden message from JPEG DCT coefficients using F5 steganography.
The method performs the complete F5 extraction algorithm:
- Seeds PRNG with password
- Generates coefficient permutation
- Extracts and decodes 32-bit header
- Extracts message using matrix encoding
- XORs message with PRNG stream
PRNG Source: If a PRNGFactory was provided via WithPRNGFactory, the factory-created PRNG is used. Otherwise, an internal securerandom.SecureRandom is created using the hasher for backward compatibility.
Parameters:
- coefficients: Quantized DCT coefficients from JPEG
- password: Password used for extraction (matches embedding password)
Returns:
- *ExtractionResult: Contains extracted data, k parameter, and file size
- error: Any extraction errors (invalid header, insufficient coefficients, etc.)
Errors:
- ErrNoCoefficients: coefficients slice is empty
- ErrEmptyPassword: password string is empty
- ErrInsufficientCoefficients: not enough coefficients for extraction
- ErrInvalidKParameter: k parameter outside valid range (1-8)
- ErrInvalidFileSize: file size is invalid (zero, negative, or too large)
- ErrIncompleteExtraction: extraction did not complete successfully
func (*F5Extractor) ExtractBytes ¶
func (f *F5Extractor) ExtractBytes(coefficients []int16, password string) ([]byte, error)
ExtractBytes is a convenience method that extracts only the message bytes. This is useful for simple use cases where only the extracted data is needed.
Parameters:
- coefficients: Quantized DCT coefficients from JPEG
- password: Password used for extraction
Returns:
- []byte: The extracted message bytes
- error: Any extraction errors
Example:
data, err := extractor.ExtractBytes(coefficients, "password")
if err != nil {
log.Fatal(err)
}
fmt.Println(string(data))
type Hasher ¶
type Hasher interface {
// Sum computes the cryptographic hash of the input data.
// For F5 extraction, this should be SHA-1 (returning 20 bytes).
//
// Parameters:
// data - The input bytes to hash
//
// Returns:
// The hash digest as a byte slice
Sum(data []byte) []byte
}
Hasher defines the interface for cryptographic hash computation. This abstraction enables different hash implementations while maintaining the Interface Segregation Principle by providing only the essential method.
The interface is designed to support SHA-1 hashing for PRNG seeding in the F5 steganographic algorithm.
type Option ¶
type Option func(*F5Extractor)
Option is a functional option for configuring F5Extractor.
func WithMaxFileSize ¶
WithMaxFileSize sets the maximum allowed file size for extraction. If the extracted file size exceeds this limit, Extract returns ErrInvalidFileSize.
Default: MaxFileSize (10MB)
Example:
extractor := extract.NewExtractor(hasher, permutator,
extract.WithMaxFileSize(5 * 1024 * 1024)) // 5MB limit
func WithPRNGFactory ¶
func WithPRNGFactory(factory PRNGFactory) Option
WithPRNGFactory sets a factory for creating PRNG instances during extraction. When a PRNGFactory is provided, the Extract method will use factory-created PRNGs instead of the internal securerandom implementation.
The factory parameter accepts f5prng.PRNGFactory (or the type alias PRNGFactory) to enable seamless integration with the unified f5prng package.
This enables:
- Custom PRNG implementations for testing with mock PRNGs
- Consistent PRNG behavior across embed/extract operations
- Dependency injection from higher-level packages (e.g., f5stegokit)
If no PRNGFactory is provided (or nil is passed), the extractor falls back to creating securerandom.SecureRandom instances internally for backward compatibility.
The factory-created PRNG will be seeded with the password bytes during extraction. The factory should return fresh, uninitialized PRNG instances.
Example with f5prng:
factory := f5prng.NewDefaultFactory()
extractor := extract.NewExtractor(hasher, permutator,
extract.WithPRNGFactory(factory))
result, err := extractor.Extract(coefficients, password)
func WithPermutationPool ¶
func WithPermutationPool() Option
WithPermutationPool enables a sync.Pool of permutation buffers shared by this extractor. This eliminates the per-call []int allocation for the Fisher-Yates permutation, which for a 256K-coefficient image is a 2 MB allocation PER Extract call.
Recommended for:
- Bruteforce / dictionary-attack workloads that call Extract in a tight loop against the same image (same coefficient count).
- Any batch extraction scenario where the same *F5Extractor processes many images of similar size.
Not recommended for one-shot extractions: the pool's bookkeeping overhead does not pay off if only a handful of calls are made.
The pool is safe for concurrent use. Each Extract call Gets a buffer, passes it to the permutator (which may reuse or grow it), and Puts the resulting buffer back on return. Buffers of different sizes may end up in the pool; GenerateInto handles growing as needed.
Example:
extractor := extract.NewExtractor(hasher, permutator,
extract.WithPermutationPool())
for _, candidate := range candidates {
result, err := extractor.Extract(coefficients, candidate)
// ...
}
type PRNGFactory ¶
type PRNGFactory = f5prng.PRNGFactory
PRNGFactory is a clean alias to f5prng.PRNGFactory. New code may use f5prng.PRNGFactory directly. When supplied to NewExtractor via WithPRNGFactory, the factory creates the PRNG instances used during extraction.
type Permutator ¶
type Permutator interface {
// Generate creates and returns a permutation of integers from 0 to size-1.
// The permutation is determined by the provided RandomSource, which must
// be properly seeded before calling this method.
//
// The returned slice contains each integer from 0 to size-1 exactly once,
// arranged in a pseudo-random order determined by the RandomSource.
// The same RandomSource state must always produce the same permutation.
//
// Parameters:
// size - The number of elements in the permutation (must be >= 0)
// random - The RandomSource used to generate the permutation order
//
// Returns:
// A slice of size integers containing a permutation of [0, size-1]
// An error if size is negative or exceeds maximum limits
//
// Example:
// Generate(5, seededRandom) might return [2, 4, 0, 3, 1], nil
Generate(size int, random f5prng.RandomSource) ([]int, error)
// GenerateInto generates a permutation into the provided buffer.
// This is a zero-allocation variant that reuses an existing buffer.
// The buffer will be resized if needed to fit 'size' elements.
//
// Parameters:
// buf - Reusable buffer (will be resized if needed)
// size - The number of elements in the permutation (must be >= 0)
// random - The RandomSource used to generate the permutation order
//
// Returns:
// The buffer containing the permutation (may be reallocated if too small)
// An error if size is negative or exceeds maximum limits
GenerateInto(buf []int, size int, random f5prng.RandomSource) ([]int, error)
}
Permutator defines the interface for generating permutations of integers. This abstraction follows the Single Responsibility Principle by focusing solely on permutation generation, and the Dependency Inversion Principle by depending on the f5prng.RandomSource abstraction rather than concrete implementations.
The interface supports both allocating and zero-allocation variants to enable performance optimization in hot paths.
type RandomSource ¶
type RandomSource = f5prng.RandomSource
RandomSource is a clean alias to f5prng.RandomSource. New code may use f5prng.RandomSource directly; the alias is retained because the Permutator and PRNG abstractions in this package are expressed in terms of it.
type TranslatorProvider ¶
type TranslatorProvider interface {
// Translate looks up a translation key in the current locale.
// If the key is not found, it tries the fallback chain.
// Returns the key itself if not found in any locale.
Translate(key string) string
// TranslateWithArgs looks up a translation key and formats it with arguments.
// Uses fmt.Sprintf formatting. If the key is not found, returns the key itself.
TranslateWithArgs(key string, args ...interface{}) string
// HasKey checks if a translation key exists in the current locale or fallback chain.
HasKey(key string) bool
// SetLocale changes the current locale for translation lookups.
SetLocale(locale string)
// GetLocale returns the current locale being used for translations.
GetLocale() string
}
TranslatorProvider allows optional translation support. This interface matches github.com/0verkilll/i18n.TranslatorProvider but is defined here to avoid a hard dependency on the i18n package.
Packages using this pattern allow application developers to optionally provide translations without forcing the i18n package on all users.
Example usage:
import "github.com/0verkilll/i18n"
translator, _ := i18n.New(
i18n.WithFileSystemLoader("locales"),
i18n.WithDefaultLocale("en-US"),
)
extract.SetTranslator(translator)
Now all extract error messages will be translated according to the current locale setting in the translator.
func GetTranslator ¶
func GetTranslator() TranslatorProvider
GetTranslator returns the currently configured translator, or nil if none is set.
This function is thread-safe and can be called from multiple goroutines.
type ValidationError ¶
type ValidationError struct {
// Wrapped is the underlying sentinel error for errors.Is() support
Wrapped error
// Key is the i18n key for the error message
Key string
// Message is the translated or fallback error message
Message string
}
ValidationError represents an input validation error with i18n support.
func NewValidationErrorWithSentinel ¶
func NewValidationErrorWithSentinel(key string, sentinel error, args ...interface{}) *ValidationError
NewValidationErrorWithSentinel creates a validation error that wraps a sentinel error. This allows both i18n translation and errors.Is() checking.
Example:
err := NewValidationErrorWithSentinel(ErrKeyInvalidKParameter, ErrInvalidKParameter, k) // errors.Is(err, ErrInvalidKParameter) returns true // err.Error() returns translated message
func ValidationErrorf ¶
func ValidationErrorf(key string, args ...interface{}) *ValidationError
ValidationErrorf creates a validation error with a formatted message. This is useful for validation errors that need dynamic content. The key should be an i18n key, and args are passed to fmt.Sprintf after translation if the translator supports TranslateWithArgs.
func (*ValidationError) Error ¶
func (e *ValidationError) Error() string
Error implements the error interface.
func (*ValidationError) Unwrap ¶
func (e *ValidationError) Unwrap() error
Unwrap returns the underlying sentinel error for errors.Is() support.