Documentation
¶
Overview ¶
Package trix implements the TRIX binary container format (RFC-0002).
The .trix format is a generic, protocol-agnostic container for storing arbitrary binary payloads alongside structured JSON metadata. It consists of:
[Magic Number (4)] [Version (1)] [Header Length (4)] [JSON Header] [Payload]
Key features:
- Custom 4-byte magic number for application-specific identification
- Extensible JSON header for metadata (content type, checksums, timestamps)
- Optional integrity verification via configurable checksum algorithms
- Integration with the Sigil transformation framework for encoding/compression
Example usage:
container := &trix.Trix{
Header: map[string]interface{}{"content_type": "text/plain"},
Payload: []byte("Hello, World!"),
}
encoded, _ := trix.Encode(container, "MYAP", nil)
Index ¶
- Constants
- Variables
- func DecodeHeader(r io.Reader, magicNumber string) (map[string]interface{}, io.Reader, error)
- func DecodeStream(r io.Reader, magicNumber string, payload io.Writer) (map[string]interface{}, int64, error)
- func Encode(trix *Trix, magicNumber string, w io.Writer) ([]byte, error)
- func EncodeStream(header map[string]interface{}, magicNumber string, payload io.Reader, ...) (int64, error)
- type CryptoConfig
- type HeaderInfo
- type Trix
Examples ¶
Constants ¶
const ( // HeaderKeyEncrypted indicates whether the payload is encrypted. HeaderKeyEncrypted = "encrypted" // HeaderKeyAlgorithm stores the encryption algorithm used. HeaderKeyAlgorithm = "encryption_algorithm" // HeaderKeyEncryptedAt stores when the payload was encrypted. HeaderKeyEncryptedAt = "encrypted_at" // HeaderKeyObfuscator stores the obfuscator type used. HeaderKeyObfuscator = "obfuscator" // AlgorithmChaCha20Poly1305 is the identifier for ChaCha20-Poly1305. AlgorithmChaCha20Poly1305 = "xchacha20-poly1305" // ObfuscatorXOR identifies the XOR obfuscator. ObfuscatorXOR = "xor" // ObfuscatorShuffleMask identifies the shuffle-mask obfuscator. ObfuscatorShuffleMask = "shuffle-mask" )
const ( // Version is the current version of the .trix file format. // See RFC-0002 for version history and compatibility notes. Version = 2 // MaxHeaderSize is the maximum allowed size for the header (16 MB). // This limit prevents denial-of-service attacks via large header allocations. MaxHeaderSize = 16 * 1024 * 1024 // 16 MB )
const PrefixSize = 4 + 1 + 4
PrefixSize is the size of the fixed framing that precedes the JSON header: magic (4) + version (1) + header length (4). RFC-0002 §3.1.
Exported because it is the constant a caller needs to reason about payload offsets without decoding anything.
Variables ¶
var ( // ErrNoEncryptionKey is returned when encryption is requested without a key. ErrNoEncryptionKey = errors.New("trix: encryption key not configured") // ErrAlreadyEncrypted is returned when trying to encrypt already encrypted data. ErrAlreadyEncrypted = errors.New("trix: payload is already encrypted") // ErrNotEncrypted is returned when trying to decrypt non-encrypted data. ErrNotEncrypted = errors.New("trix: payload is not encrypted") )
var ( // ErrNilWriter is returned when a streaming call is given no destination. ErrNilWriter = fmt.Errorf("trix: writer cannot be nil") // ErrNilReader is returned when a streaming call is given no source. ErrNilReader = fmt.Errorf("trix: reader cannot be nil") )
var ( // ErrInvalidMagicNumber is returned when the magic number is incorrect. ErrInvalidMagicNumber = errors.New("trix: invalid magic number") // ErrInvalidVersion is returned when the version is incorrect. ErrInvalidVersion = errors.New("trix: invalid version") // ErrMagicNumberLength is returned when the magic number is not 4 bytes long. ErrMagicNumberLength = errors.New("trix: magic number must be 4 bytes long") // ErrNilSigil is returned when a sigil is nil. ErrNilSigil = errors.New("trix: sigil cannot be nil") // ErrChecksumMismatch is returned when the checksum does not match. ErrChecksumMismatch = errors.New("trix: checksum mismatch") // ErrHeaderTooLarge is returned when the header size exceeds the maximum allowed. ErrHeaderTooLarge = errors.New("trix: header size exceeds maximum allowed") )
Functions ¶
func DecodeHeader ¶ added in v0.2.0
DecodeHeader reads and validates a container's framing from r and returns the decoded header plus a reader positioned at the first payload byte.
Exactly PrefixSize+len(header) bytes are consumed — nothing is read ahead and nothing is buffered — so the returned reader (which is r itself) can be streamed, spliced or handed to an mmap-backed consumer without the payload ever being materialised.
header, payload, err := trix.DecodeHeader(f, "KVST")
if err != nil { return err }
_, err = io.Copy(dst, payload)
A checksum recorded in the header is NOT verified: verification needs the whole payload, which is precisely what this path refuses to hold. Callers that want it should hash the payload stream as they consume it.
Example ¶
package main
import (
"bytes"
"fmt"
"log"
"github.com/Snider/Enchantrix/pkg/trix"
)
func main() {
var container bytes.Buffer
if _, err := trix.EncodeStream(map[string]interface{}{"kind": "state-kv"}, "KVST", bytes.NewReader([]byte("binary tail")), &container); err != nil {
log.Fatalf("EncodeStream failed: %v", err)
}
header, payload, err := trix.DecodeHeader(bytes.NewReader(container.Bytes()), "KVST")
if err != nil {
log.Fatalf("DecodeHeader failed: %v", err)
}
// payload is positioned at the first payload byte — stream it anywhere.
var out bytes.Buffer
if _, err := out.ReadFrom(payload); err != nil {
log.Fatalf("copy failed: %v", err)
}
fmt.Printf("Header kind: %v\n", header["kind"])
fmt.Printf("Payload: %s\n", out.String())
}
Output: Header kind: state-kv Payload: binary tail
func DecodeStream ¶ added in v0.2.0
func DecodeStream(r io.Reader, magicNumber string, payload io.Writer) (map[string]interface{}, int64, error)
DecodeStream reads a container from r and copies its payload to payload. It returns the decoded header and the number of payload bytes copied.
The payload is written out byte-for-byte as stored; see DecodeHeader for the checksum caveat.
header, n, err := trix.DecodeStream(f, "KVST", out)
func Encode ¶
Encode serializes a Trix struct into the .trix binary format. It returns the encoded data as a byte slice.
Example ¶
package main
import (
"fmt"
"log"
"github.com/Snider/Enchantrix/pkg/trix"
)
func main() {
t := &trix.Trix{
Header: map[string]interface{}{"author": "Jules"},
Payload: []byte("Hello, Trix!"),
}
encoded, err := trix.Encode(t, "TRIX", nil)
if err != nil {
log.Fatalf("Encode failed: %v", err)
}
fmt.Printf("Encoded data is not empty: %v\n", len(encoded) > 0)
}
Output: Encoded data is not empty: true
func EncodeStream ¶ added in v0.2.0
func EncodeStream(header map[string]interface{}, magicNumber string, payload io.Reader, w io.Writer) (int64, error)
EncodeStream writes an RFC-0002 container to w, copying payload straight through rather than buffering it. It returns the total number of bytes written to w — the full container length, which matches len(Encode(...)) for the same header, magic and payload.
The wire format is byte-for-byte identical to Encode: the two differ only in how the payload reaches the writer. Encode needs the whole payload resident as a []byte; EncodeStream needs a copy buffer, so a 128k-token State log costs the same allocation as a one-byte one.
No transformation is applied to the payload — no sigil, no compression, no checksum, no encryption. The binary tail is exactly the bytes read from payload, so it can later become a zero-copy mmap of the source file. Callers wanting any of those apply them to the reader themselves.
f, _ := os.Create("session.kv")
src, _ := os.Open("session.mvlog")
n, err := trix.EncodeStream(map[string]interface{}{"kind": "state-kv"}, "KVST", src, f)
A nil payload is treated as an empty payload, matching Encode's handling of a nil Trix.Payload.
Example ¶
package main
import (
"bytes"
"fmt"
"log"
"github.com/Snider/Enchantrix/pkg/trix"
)
func main() {
// The payload is copied straight from reader to writer, so a container
// holding a multi-gigabyte log costs the same memory as a tiny one.
payload := bytes.NewReader([]byte("a very long state log"))
var out bytes.Buffer
n, err := trix.EncodeStream(map[string]interface{}{"kind": "state-kv"}, "KVST", payload, &out)
if err != nil {
log.Fatalf("EncodeStream failed: %v", err)
}
fmt.Printf("Container length: %d\n", n)
}
Output: Container length: 49
Types ¶
type CryptoConfig ¶ added in v0.0.2
type CryptoConfig struct {
// Key is the 32-byte encryption key.
Key []byte
// Obfuscator type: "xor" (default) or "shuffle-mask"
Obfuscator string
}
CryptoConfig holds encryption configuration for a Trix container.
type HeaderInfo ¶ added in v0.2.0
type HeaderInfo struct {
// Header is the decoded JSON header.
Header map[string]interface{}
// PayloadOffset is the byte offset of the first payload byte — the
// value an mmap/pread of the binary tail starts from.
PayloadOffset int64
// PayloadBytes is the length of the payload in bytes, or 0 when the
// reader cannot report its size.
PayloadBytes int64
}
HeaderInfo describes a container's framing without touching its payload.
PayloadBytes is best-effort: it is only populated when the reader can report its own size cheaply (an *os.File, *bytes.Reader, *strings.Reader or *io.SectionReader). A zero value means "unknown", never "empty".
func ReadHeaderInfo ¶ added in v0.2.0
func ReadHeaderInfo(r io.ReaderAt, magicNumber string) (HeaderInfo, error)
ReadHeaderInfo reads a container's framing via random access and reports where the payload starts, without reading a single payload byte.
This is what makes a zero-copy view of the binary tail possible: given the offset, a consumer can mmap or pread the payload region directly instead of streaming it through the process.
info, err := trix.ReadHeaderInfo(f, "KVST")
if err != nil { return err }
section := io.NewSectionReader(f, info.PayloadOffset, info.PayloadBytes)
Example ¶
package main
import (
"bytes"
"fmt"
"log"
"github.com/Snider/Enchantrix/pkg/trix"
)
func main() {
var container bytes.Buffer
if _, err := trix.EncodeStream(map[string]interface{}{"kind": "state-kv"}, "KVST", bytes.NewReader([]byte("binary tail")), &container); err != nil {
log.Fatalf("EncodeStream failed: %v", err)
}
// No payload byte is read — the offset is what an mmap would start from.
info, err := trix.ReadHeaderInfo(bytes.NewReader(container.Bytes()), "KVST")
if err != nil {
log.Fatalf("ReadHeaderInfo failed: %v", err)
}
fmt.Printf("Payload offset: %d\n", info.PayloadOffset)
fmt.Printf("Payload bytes: %d\n", info.PayloadBytes)
}
Output: Payload offset: 28 Payload bytes: 11
type Trix ¶
type Trix struct {
// Header contains JSON-serializable metadata about the payload.
Header map[string]interface{}
// Payload is the binary data stored in the container.
Payload []byte
// InSigils lists sigil names to apply during Pack (forward transformation).
InSigils []string `json:"-"`
// OutSigils lists sigil names to apply during Unpack (reverse transformation).
// If empty, InSigils is used in reverse order.
OutSigils []string `json:"-"`
// ChecksumAlgo specifies the hash algorithm for integrity verification.
// If set, a checksum is computed and stored in the header during Encode.
ChecksumAlgo crypt.HashType `json:"-"`
}
Trix represents a .trix container with header metadata and binary payload.
The Header field holds arbitrary JSON-serializable metadata. Common fields include:
- content_type: MIME type of the original payload
- created_at: ISO 8601 timestamp
- encryption_algorithm: Algorithm used for encryption (if applicable)
- checksum: Hex-encoded integrity checksum (auto-populated if ChecksumAlgo is set)
The InSigils and OutSigils fields specify transformation pipelines:
- InSigils: Applied during Pack() in order (e.g., ["gzip", "base64"])
- OutSigils: Applied during Unpack() in reverse order (defaults to InSigils)
func Decode ¶
Decode deserializes the .trix binary format into a Trix struct. It returns the decoded Trix struct. Note: Sigils are not stored in the format and must be re-attached by the caller.
Example ¶
package main
import (
"fmt"
"log"
"github.com/Snider/Enchantrix/pkg/trix"
)
func main() {
t := &trix.Trix{
Header: map[string]interface{}{"author": "Jules"},
Payload: []byte("Hello, Trix!"),
}
encoded, err := trix.Encode(t, "TRIX", nil)
if err != nil {
log.Fatalf("Encode failed: %v", err)
}
decoded, err := trix.Decode(encoded, "TRIX", nil)
if err != nil {
log.Fatalf("Decode failed: %v", err)
}
fmt.Printf("Decoded payload: %s\n", decoded.Payload)
fmt.Printf("Decoded header: %v\n", decoded.Header)
}
Output: Decoded payload: Hello, Trix! Decoded header: map[author:Jules]
func NewEncryptedTrix ¶ added in v0.0.2
NewEncryptedTrix creates a new Trix container with an encrypted payload. This is a convenience function for creating encrypted containers in one step.
func (*Trix) DecryptPayload ¶ added in v0.0.2
func (t *Trix) DecryptPayload(config *CryptoConfig) error
DecryptPayload decrypts the Trix payload using the provided key.
The nonce is extracted from the ciphertext itself - no need to read it from the header separately.
func (*Trix) EncryptPayload ¶ added in v0.0.2
func (t *Trix) EncryptPayload(config *CryptoConfig) error
EncryptPayload encrypts the Trix payload using ChaCha20-Poly1305 with pre-obfuscation.
The nonce is embedded in the ciphertext itself and is NOT stored separately in the header. This is the production-ready approach (not demo-style).
Header metadata is updated to indicate encryption status without exposing cryptographic parameters that are already embedded in the ciphertext.
func (*Trix) GetEncryptionAlgorithm ¶ added in v0.0.2
GetEncryptionAlgorithm returns the encryption algorithm used, if any.
func (*Trix) IsEncrypted ¶ added in v0.0.2
IsEncrypted returns true if the payload is currently encrypted.
func (*Trix) Pack ¶
Pack applies the In method of all attached sigils to the payload. It modifies the Trix struct in place.
Example ¶
package main
import (
"fmt"
"log"
"github.com/Snider/Enchantrix/pkg/trix"
)
func main() {
t := &trix.Trix{
Payload: []byte("secret message"),
InSigils: []string{"base64", "reverse"},
}
err := t.Pack()
if err != nil {
log.Fatalf("Pack failed: %v", err)
}
fmt.Printf("Packed payload: %s\n", t.Payload)
}
Output: Packed payload: =U2ZhN3cl1GI0VmcjV2c
Example (Checksum) ¶
package main
import (
"fmt"
"log"
"github.com/Snider/Enchantrix/pkg/crypt"
"github.com/Snider/Enchantrix/pkg/trix"
)
func main() {
t := &trix.Trix{
Header: map[string]interface{}{},
Payload: []byte("secret message"),
InSigils: []string{"base64", "reverse"},
ChecksumAlgo: crypt.SHA256,
}
encoded, err := trix.Encode(t, "TRIX", nil)
if err != nil {
log.Fatalf("Encode failed: %v", err)
}
decoded, err := trix.Decode(encoded, "TRIX", nil)
if err != nil {
log.Fatalf("Decode failed: %v", err)
}
fmt.Printf("Decoded payload: %s\n", decoded.Payload)
fmt.Printf("Checksum verified: %v\n", decoded.Header["checksum"] != nil)
}
Output: Decoded payload: secret message Checksum verified: true
func (*Trix) Unpack ¶
Unpack applies the Out method of all sigils in reverse order. It modifies the Trix struct in place.
Example ¶
package main
import (
"fmt"
"log"
"github.com/Snider/Enchantrix/pkg/trix"
)
func main() {
t := &trix.Trix{
Payload: []byte("=U2ZhN3cl1GI0VmcjV2c"),
OutSigils: []string{"base64", "reverse"},
}
err := t.Unpack()
if err != nil {
log.Fatalf("Unpack failed: %v", err)
}
fmt.Printf("Unpacked payload: %s\n", t.Payload)
}
Output: Unpacked payload: secret message