trix

package
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 1, 2026 License: EUPL-1.2 Imports: 10 Imported by: 5

Documentation

Overview

Package trix implements the TRIX binary container format (RFC-0002).

The .trix format is a generic, protocol-agnostic container for storing arbitrary binary payloads alongside structured JSON metadata. It consists of:

[Magic Number (4)] [Version (1)] [Header Length (4)] [JSON Header] [Payload]

Key features:

  • Custom 4-byte magic number for application-specific identification
  • Extensible JSON header for metadata (content type, checksums, timestamps)
  • Optional integrity verification via configurable checksum algorithms
  • Integration with the Sigil transformation framework for encoding/compression

Example usage:

container := &trix.Trix{
    Header:  map[string]interface{}{"content_type": "text/plain"},
    Payload: []byte("Hello, World!"),
}
encoded, _ := trix.Encode(container, "MYAP", nil)

Index

Examples

Constants

View Source
const (
	// HeaderKeyEncrypted indicates whether the payload is encrypted.
	HeaderKeyEncrypted = "encrypted"
	// HeaderKeyAlgorithm stores the encryption algorithm used.
	HeaderKeyAlgorithm = "encryption_algorithm"
	// HeaderKeyEncryptedAt stores when the payload was encrypted.
	HeaderKeyEncryptedAt = "encrypted_at"
	// HeaderKeyObfuscator stores the obfuscator type used.
	HeaderKeyObfuscator = "obfuscator"

	// AlgorithmChaCha20Poly1305 is the identifier for ChaCha20-Poly1305.
	AlgorithmChaCha20Poly1305 = "xchacha20-poly1305"
	// ObfuscatorXOR identifies the XOR obfuscator.
	ObfuscatorXOR = "xor"
	// ObfuscatorShuffleMask identifies the shuffle-mask obfuscator.
	ObfuscatorShuffleMask = "shuffle-mask"
)
View Source
const (
	// Version is the current version of the .trix file format.
	// See RFC-0002 for version history and compatibility notes.
	Version = 2
	// MaxHeaderSize is the maximum allowed size for the header (16 MB).
	// This limit prevents denial-of-service attacks via large header allocations.
	MaxHeaderSize = 16 * 1024 * 1024 // 16 MB
)
View Source
const PrefixSize = 4 + 1 + 4

PrefixSize is the size of the fixed framing that precedes the JSON header: magic (4) + version (1) + header length (4). RFC-0002 §3.1.

Exported because it is the constant a caller needs to reason about payload offsets without decoding anything.

Variables

View Source
var (
	// ErrNoEncryptionKey is returned when encryption is requested without a key.
	ErrNoEncryptionKey = errors.New("trix: encryption key not configured")
	// ErrAlreadyEncrypted is returned when trying to encrypt already encrypted data.
	ErrAlreadyEncrypted = errors.New("trix: payload is already encrypted")
	// ErrNotEncrypted is returned when trying to decrypt non-encrypted data.
	ErrNotEncrypted = errors.New("trix: payload is not encrypted")
)
View Source
var (
	// ErrNilWriter is returned when a streaming call is given no destination.
	ErrNilWriter = fmt.Errorf("trix: writer cannot be nil")
	// ErrNilReader is returned when a streaming call is given no source.
	ErrNilReader = fmt.Errorf("trix: reader cannot be nil")
)
View Source
var (
	// ErrInvalidMagicNumber is returned when the magic number is incorrect.
	ErrInvalidMagicNumber = errors.New("trix: invalid magic number")
	// ErrInvalidVersion is returned when the version is incorrect.
	ErrInvalidVersion = errors.New("trix: invalid version")
	// ErrMagicNumberLength is returned when the magic number is not 4 bytes long.
	ErrMagicNumberLength = errors.New("trix: magic number must be 4 bytes long")
	// ErrNilSigil is returned when a sigil is nil.
	ErrNilSigil = errors.New("trix: sigil cannot be nil")
	// ErrChecksumMismatch is returned when the checksum does not match.
	ErrChecksumMismatch = errors.New("trix: checksum mismatch")
	// ErrHeaderTooLarge is returned when the header size exceeds the maximum allowed.
	ErrHeaderTooLarge = errors.New("trix: header size exceeds maximum allowed")
)

Functions

func DecodeHeader added in v0.2.0

func DecodeHeader(r io.Reader, magicNumber string) (map[string]interface{}, io.Reader, error)

DecodeHeader reads and validates a container's framing from r and returns the decoded header plus a reader positioned at the first payload byte.

Exactly PrefixSize+len(header) bytes are consumed — nothing is read ahead and nothing is buffered — so the returned reader (which is r itself) can be streamed, spliced or handed to an mmap-backed consumer without the payload ever being materialised.

header, payload, err := trix.DecodeHeader(f, "KVST")
if err != nil { return err }
_, err = io.Copy(dst, payload)

A checksum recorded in the header is NOT verified: verification needs the whole payload, which is precisely what this path refuses to hold. Callers that want it should hash the payload stream as they consume it.

Example
package main

import (
	"bytes"
	"fmt"
	"log"

	"github.com/Snider/Enchantrix/pkg/trix"
)

func main() {
	var container bytes.Buffer
	if _, err := trix.EncodeStream(map[string]interface{}{"kind": "state-kv"}, "KVST", bytes.NewReader([]byte("binary tail")), &container); err != nil {
		log.Fatalf("EncodeStream failed: %v", err)
	}

	header, payload, err := trix.DecodeHeader(bytes.NewReader(container.Bytes()), "KVST")
	if err != nil {
		log.Fatalf("DecodeHeader failed: %v", err)
	}
	// payload is positioned at the first payload byte — stream it anywhere.
	var out bytes.Buffer
	if _, err := out.ReadFrom(payload); err != nil {
		log.Fatalf("copy failed: %v", err)
	}
	fmt.Printf("Header kind: %v\n", header["kind"])
	fmt.Printf("Payload: %s\n", out.String())
}
Output:
Header kind: state-kv
Payload: binary tail

func DecodeStream added in v0.2.0

func DecodeStream(r io.Reader, magicNumber string, payload io.Writer) (map[string]interface{}, int64, error)

DecodeStream reads a container from r and copies its payload to payload. It returns the decoded header and the number of payload bytes copied.

The payload is written out byte-for-byte as stored; see DecodeHeader for the checksum caveat.

header, n, err := trix.DecodeStream(f, "KVST", out)

func Encode

func Encode(trix *Trix, magicNumber string, w io.Writer) ([]byte, error)

Encode serializes a Trix struct into the .trix binary format. It returns the encoded data as a byte slice.

Example
package main

import (
	"fmt"
	"log"

	"github.com/Snider/Enchantrix/pkg/trix"
)

func main() {
	t := &trix.Trix{
		Header:  map[string]interface{}{"author": "Jules"},
		Payload: []byte("Hello, Trix!"),
	}
	encoded, err := trix.Encode(t, "TRIX", nil)
	if err != nil {
		log.Fatalf("Encode failed: %v", err)
	}
	fmt.Printf("Encoded data is not empty: %v\n", len(encoded) > 0)
}
Output:
Encoded data is not empty: true

func EncodeStream added in v0.2.0

func EncodeStream(header map[string]interface{}, magicNumber string, payload io.Reader, w io.Writer) (int64, error)

EncodeStream writes an RFC-0002 container to w, copying payload straight through rather than buffering it. It returns the total number of bytes written to w — the full container length, which matches len(Encode(...)) for the same header, magic and payload.

The wire format is byte-for-byte identical to Encode: the two differ only in how the payload reaches the writer. Encode needs the whole payload resident as a []byte; EncodeStream needs a copy buffer, so a 128k-token State log costs the same allocation as a one-byte one.

No transformation is applied to the payload — no sigil, no compression, no checksum, no encryption. The binary tail is exactly the bytes read from payload, so it can later become a zero-copy mmap of the source file. Callers wanting any of those apply them to the reader themselves.

f, _ := os.Create("session.kv")
src, _ := os.Open("session.mvlog")
n, err := trix.EncodeStream(map[string]interface{}{"kind": "state-kv"}, "KVST", src, f)

A nil payload is treated as an empty payload, matching Encode's handling of a nil Trix.Payload.

Example
package main

import (
	"bytes"
	"fmt"
	"log"

	"github.com/Snider/Enchantrix/pkg/trix"
)

func main() {
	// The payload is copied straight from reader to writer, so a container
	// holding a multi-gigabyte log costs the same memory as a tiny one.
	payload := bytes.NewReader([]byte("a very long state log"))

	var out bytes.Buffer
	n, err := trix.EncodeStream(map[string]interface{}{"kind": "state-kv"}, "KVST", payload, &out)
	if err != nil {
		log.Fatalf("EncodeStream failed: %v", err)
	}
	fmt.Printf("Container length: %d\n", n)
}
Output:
Container length: 49

Types

type CryptoConfig added in v0.0.2

type CryptoConfig struct {
	// Key is the 32-byte encryption key.
	Key []byte
	// Obfuscator type: "xor" (default) or "shuffle-mask"
	Obfuscator string
}

CryptoConfig holds encryption configuration for a Trix container.

type HeaderInfo added in v0.2.0

type HeaderInfo struct {
	// Header is the decoded JSON header.
	Header map[string]interface{}
	// PayloadOffset is the byte offset of the first payload byte — the
	// value an mmap/pread of the binary tail starts from.
	PayloadOffset int64
	// PayloadBytes is the length of the payload in bytes, or 0 when the
	// reader cannot report its size.
	PayloadBytes int64
}

HeaderInfo describes a container's framing without touching its payload.

PayloadBytes is best-effort: it is only populated when the reader can report its own size cheaply (an *os.File, *bytes.Reader, *strings.Reader or *io.SectionReader). A zero value means "unknown", never "empty".

func ReadHeaderInfo added in v0.2.0

func ReadHeaderInfo(r io.ReaderAt, magicNumber string) (HeaderInfo, error)

ReadHeaderInfo reads a container's framing via random access and reports where the payload starts, without reading a single payload byte.

This is what makes a zero-copy view of the binary tail possible: given the offset, a consumer can mmap or pread the payload region directly instead of streaming it through the process.

info, err := trix.ReadHeaderInfo(f, "KVST")
if err != nil { return err }
section := io.NewSectionReader(f, info.PayloadOffset, info.PayloadBytes)
Example
package main

import (
	"bytes"
	"fmt"
	"log"

	"github.com/Snider/Enchantrix/pkg/trix"
)

func main() {
	var container bytes.Buffer
	if _, err := trix.EncodeStream(map[string]interface{}{"kind": "state-kv"}, "KVST", bytes.NewReader([]byte("binary tail")), &container); err != nil {
		log.Fatalf("EncodeStream failed: %v", err)
	}

	// No payload byte is read — the offset is what an mmap would start from.
	info, err := trix.ReadHeaderInfo(bytes.NewReader(container.Bytes()), "KVST")
	if err != nil {
		log.Fatalf("ReadHeaderInfo failed: %v", err)
	}
	fmt.Printf("Payload offset: %d\n", info.PayloadOffset)
	fmt.Printf("Payload bytes: %d\n", info.PayloadBytes)
}
Output:
Payload offset: 28
Payload bytes: 11

type Trix

type Trix struct {
	// Header contains JSON-serializable metadata about the payload.
	Header map[string]interface{}
	// Payload is the binary data stored in the container.
	Payload []byte
	// InSigils lists sigil names to apply during Pack (forward transformation).
	InSigils []string `json:"-"`
	// OutSigils lists sigil names to apply during Unpack (reverse transformation).
	// If empty, InSigils is used in reverse order.
	OutSigils []string `json:"-"`
	// ChecksumAlgo specifies the hash algorithm for integrity verification.
	// If set, a checksum is computed and stored in the header during Encode.
	ChecksumAlgo crypt.HashType `json:"-"`
}

Trix represents a .trix container with header metadata and binary payload.

The Header field holds arbitrary JSON-serializable metadata. Common fields include:

  • content_type: MIME type of the original payload
  • created_at: ISO 8601 timestamp
  • encryption_algorithm: Algorithm used for encryption (if applicable)
  • checksum: Hex-encoded integrity checksum (auto-populated if ChecksumAlgo is set)

The InSigils and OutSigils fields specify transformation pipelines:

  • InSigils: Applied during Pack() in order (e.g., ["gzip", "base64"])
  • OutSigils: Applied during Unpack() in reverse order (defaults to InSigils)

func Decode

func Decode(data []byte, magicNumber string, r io.Reader) (*Trix, error)

Decode deserializes the .trix binary format into a Trix struct. It returns the decoded Trix struct. Note: Sigils are not stored in the format and must be re-attached by the caller.

Example
package main

import (
	"fmt"
	"log"

	"github.com/Snider/Enchantrix/pkg/trix"
)

func main() {
	t := &trix.Trix{
		Header:  map[string]interface{}{"author": "Jules"},
		Payload: []byte("Hello, Trix!"),
	}
	encoded, err := trix.Encode(t, "TRIX", nil)
	if err != nil {
		log.Fatalf("Encode failed: %v", err)
	}
	decoded, err := trix.Decode(encoded, "TRIX", nil)
	if err != nil {
		log.Fatalf("Decode failed: %v", err)
	}
	fmt.Printf("Decoded payload: %s\n", decoded.Payload)
	fmt.Printf("Decoded header: %v\n", decoded.Header)
}
Output:
Decoded payload: Hello, Trix!
Decoded header: map[author:Jules]

func NewEncryptedTrix added in v0.0.2

func NewEncryptedTrix(payload []byte, key []byte, header map[string]interface{}) (*Trix, error)

NewEncryptedTrix creates a new Trix container with an encrypted payload. This is a convenience function for creating encrypted containers in one step.

func (*Trix) DecryptPayload added in v0.0.2

func (t *Trix) DecryptPayload(config *CryptoConfig) error

DecryptPayload decrypts the Trix payload using the provided key.

The nonce is extracted from the ciphertext itself - no need to read it from the header separately.

func (*Trix) EncryptPayload added in v0.0.2

func (t *Trix) EncryptPayload(config *CryptoConfig) error

EncryptPayload encrypts the Trix payload using ChaCha20-Poly1305 with pre-obfuscation.

The nonce is embedded in the ciphertext itself and is NOT stored separately in the header. This is the production-ready approach (not demo-style).

Header metadata is updated to indicate encryption status without exposing cryptographic parameters that are already embedded in the ciphertext.

func (*Trix) GetEncryptionAlgorithm added in v0.0.2

func (t *Trix) GetEncryptionAlgorithm() string

GetEncryptionAlgorithm returns the encryption algorithm used, if any.

func (*Trix) IsEncrypted added in v0.0.2

func (t *Trix) IsEncrypted() bool

IsEncrypted returns true if the payload is currently encrypted.

func (*Trix) Pack

func (t *Trix) Pack() error

Pack applies the In method of all attached sigils to the payload. It modifies the Trix struct in place.

Example
package main

import (
	"fmt"
	"log"

	"github.com/Snider/Enchantrix/pkg/trix"
)

func main() {
	t := &trix.Trix{
		Payload:  []byte("secret message"),
		InSigils: []string{"base64", "reverse"},
	}
	err := t.Pack()
	if err != nil {
		log.Fatalf("Pack failed: %v", err)
	}
	fmt.Printf("Packed payload: %s\n", t.Payload)
}
Output:
Packed payload: =U2ZhN3cl1GI0VmcjV2c
Example (Checksum)
package main

import (
	"fmt"
	"log"

	"github.com/Snider/Enchantrix/pkg/crypt"
	"github.com/Snider/Enchantrix/pkg/trix"
)

func main() {
	t := &trix.Trix{
		Header:       map[string]interface{}{},
		Payload:      []byte("secret message"),
		InSigils:     []string{"base64", "reverse"},
		ChecksumAlgo: crypt.SHA256,
	}
	encoded, err := trix.Encode(t, "TRIX", nil)
	if err != nil {
		log.Fatalf("Encode failed: %v", err)
	}
	decoded, err := trix.Decode(encoded, "TRIX", nil)
	if err != nil {
		log.Fatalf("Decode failed: %v", err)
	}
	fmt.Printf("Decoded payload: %s\n", decoded.Payload)
	fmt.Printf("Checksum verified: %v\n", decoded.Header["checksum"] != nil)
}
Output:
Decoded payload: secret message
Checksum verified: true

func (*Trix) Unpack

func (t *Trix) Unpack() error

Unpack applies the Out method of all sigils in reverse order. It modifies the Trix struct in place.

Example
package main

import (
	"fmt"
	"log"

	"github.com/Snider/Enchantrix/pkg/trix"
)

func main() {
	t := &trix.Trix{
		Payload:   []byte("=U2ZhN3cl1GI0VmcjV2c"),
		OutSigils: []string{"base64", "reverse"},
	}
	err := t.Unpack()
	if err != nil {
		log.Fatalf("Unpack failed: %v", err)
	}
	fmt.Printf("Unpacked payload: %s\n", t.Payload)
}
Output:
Unpacked payload: secret message

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL