format

func AppendFramedBlock ¶

func AppendFramedBlock(dst, plain []byte, c Compression) ([]byte, error)

AppendFramedBlock frames an already-encoded plain payload into a complete on-disk tx block (fixheader + payload) appended to dst, returning the extended buffer. It compresses the payload (into a pooled scratch) when the Compression policy says to, computes the CRC32C over the on-disk bytes, and writes the fixheader. Callers holding their own reusable scratch should prefer CompressTxInto + EncodeFixheader directly.

func AppendTxBlock ¶

func AppendTxBlock(dst []byte, rows []XRow, opts TxOptions) ([]byte, error)

AppendTxBlock appends a complete on-disk tx block (fixheader + payload) for rows to dst and returns the extended buffer. Compression and CRC follow the opts.Compression policy (zstd at its level iff the payload meets the policy's threshold and it is not Disabled). EncodeTxBlock is AppendTxBlock(nil, …); writers reuse a dst across txs to avoid the per-block allocation.

Note: the payload is encoded into a transient buffer internally, so this path still allocates that scratch. The fully reusable hot path lives in the writer package (AppendTxBlockPayload into owned scratch + framing), which is what high-throughput writers should use.

func AppendTxBlockPayload ¶

func AppendTxBlockPayload(dst []byte, rows []XRow) ([]byte, error)

AppendTxBlockPayload appends the concatenated EncodeXRow output of rows — the *uncompressed* tx payload, with no fixheader — to dst and returns the extended buffer. Passing a reused dst[:0] makes the per-tx payload encode allocation-free in steady state (the buffer grows to the largest tx, then reuses its capacity). It is the building block BatchWriter accumulates into, and the first half of AppendTxBlock.

func CRC32C ¶

func CRC32C(data []byte) uint32

CRC32C computes the Tarantool variant CRC-32C over data:

• polynomial: Castagnoli (0x82F63B78 reflected)
• initial value: 0
• final XOR: none

Used for the `crc32c` field of every tx-block fixheader. The CRC is computed over the on-disk payload bytes — post-compression for ZRowMarker tx blocks, plain bytes for RowMarker tx blocks.

Tarantool's variant runs init=0 with NO final XOR, whereas the stdlib's crc32.Checksum / crc32.New apply the standard CRC frame (init=^0, final ^0), incompatible with `crc32_calc` (src/lib/digest/crc32_impl.c:196-205). We recover the unframed variant from Go's *hardware-accelerated* Castagnoli path with a length-independent identity: seeding crc32.Update with ^0 cancels its entry inversion, and the trailing ^0 cancels its exit inversion. This is ~20x faster than a byte-at-a-time loop on block-sized payloads (≈0.5 GB/s → ≈11 GB/s on arm64); crc32c_test.go pins it against a software reference over fixed and random inputs.

package main

import (
	"fmt"

	"github.com/tarantool/go-xlog/format"
)

func main() {
	a := format.CRC32C([]byte("tarantool"))
	b := format.CRC32C([]byte("tarantool"))
	c := format.CRC32C([]byte("Tarantool"))

	fmt.Println(a == b)
	fmt.Println(a == c)

}

Output:
true
false

func CompressTx ¶

func CompressTx(payload []byte) ([]byte, error)

CompressTx zstd-encodes payload at ZstdLevel (3). The output is a self-contained zstd frame; Tarantool's libzstd decoder accepts it. Note that the output bytes are NOT byte-identical to libzstd's output — only the decompressed content is guaranteed to round-trip.

func CompressTxInto ¶

func CompressTxInto(dst, payload []byte, level int) ([]byte, error)

CompressTxInto zstd-encodes payload at the given level, appending the frame to dst[:0] and returning the result (which may alias dst if it had enough capacity, or a fresh allocation otherwise). Pass a reused dst to amortise the output allocation across calls — the writer's hot path does exactly this.

func DecodeDMLBodyInto ¶ added in v0.2.0

func DecodeDMLBodyInto(b []byte, dst *DMLBody, opts *DecodeOptions) error

DecodeDMLBodyInto parses a DML body map at b[0:] into dst. It is the zero-allocation core of DecodeDMLBody: a hot loop can decode every body into one caller-owned DMLBody. It first resets dst to the zero DMLBody, so a reused dst carries no field of a prior row, then sets the fields of the keys b holds. The raw fields of dst (Tuple, Key, Ops, OldTuple, NewTuple, Arrow) alias b; see Aliasing in the package doc. A key with no field goes to opts.OnUnknown, or is skipped when opts or its OnUnknown is nil. On error dst holds what was decoded before it.

package main

import (
	"fmt"

	"github.com/tarantool/go-xlog/format"
)

func main() {
	bodies := [][]byte{
		// {IPROTO_SPACE_ID: 512, IPROTO_INDEX_BASE: 1, IPROTO_KEY: [1]}
		{0x83, 0x10, 0xcd, 0x02, 0x00, 0x15, 0x01, 0x20, 0x91, 0x01},
		// {IPROTO_SPACE_ID: 513, 0x7e: 5}
		{0x82, 0x10, 0xcd, 0x02, 0x01, 0x7e, 0x05},
	}

	var body format.DMLBody

	for _, raw := range bodies {
		var unknown map[uint64][]byte

		if err := format.DecodeDMLBodyInto(raw, &body, format.CollectUnknown(&unknown)); err != nil {
			panic(err)
		}

		fmt.Printf("space=%d index_base=%d key=%x unknown=%x\n", body.SpaceID, body.IndexBase, body.Key, unknown)
	}

}

Output:
space=512 index_base=1 key=9101 unknown=map[]
space=513 index_base=0 key= unknown=map[7e:05]

func DecodeFixheaderInto ¶

func DecodeFixheaderInto(b [FixheaderSize]byte, h *Fixheader) error

DecodeFixheaderInto parses the 19 bytes into h, fully overwriting it. It is the zero-allocation core of DecodeFixheader: a reader driving a tight block loop reuses a single Fixheader instead of allocating one per tx block.

func DecodeRaftBodyInto ¶ added in v0.2.0

func DecodeRaftBodyInto(b []byte, dst *RaftBody, opts *DecodeOptions) error

DecodeRaftBodyInto parses a RAFT body map into dst, the core of DecodeRaftBody. It first resets dst to the zero RaftBody, so a dst reused across rows carries no field of a prior row, then sets the fields of the keys b holds. A key with no field goes to opts.OnUnknown, or is skipped when opts or its OnUnknown is nil. The VClock map of a prior row is never reused: a body without a vclock decodes with no allocation, but a vclock allocates a new map. VClockArray is the allocation-free alternative to VClock; the body decoders do not fill one. On error dst holds what was decoded before it.

func DecodeSynchroBodyInto ¶ added in v0.2.0

func DecodeSynchroBodyInto(b []byte, dst *SynchroBody, opts *DecodeOptions) error

DecodeSynchroBodyInto parses a synchro body map at b[0:] into dst, the core of DecodeSynchroBody. It first resets dst to the zero SynchroBody, so a dst reused across rows carries no field of a prior row, then sets the fields of the keys b holds. A key with no field goes to opts.OnUnknown, or is skipped when opts or its OnUnknown is nil. The ConfirmedVClock map of a prior row is never reused: a CONFIRM or ROLLBACK decodes with no allocation, but a PROMOTE or DEMOTE with a vclock allocates a new map. VClockArray is the allocation-free alternative to VClock; the body decoders do not fill one. On error dst holds what was decoded before it.

func DecodeTxBlock ¶

func DecodeTxBlock(b []byte) ([][]byte, []byte, int, error)

DecodeTxBlock parses a complete tx block at b[0:] and returns:

• rowSlices: byte slices, one per xrow record, into the *plain* payload
• plain: the decompressed (or as-is) payload bytes
• n: total bytes consumed from b (fixheader + on-disk payload)
• err: on CRC mismatch, malformed header, or truncated input

rowSlices aliases plain, and plain is a fresh allocation for a zrow block and aliases b for a row block; see Aliasing in the package doc.

func DecodeVClockArrayInto ¶ added in v0.2.0

func DecodeVClockArrayInto(b []byte, dst *VClockArray) (int, error)

DecodeVClockArrayInto decodes a msgpack map of {uint replica_id: uint lsn} — the encoding of IPROTO_RAFT_VCLOCK and IPROTO_VCLOCK — into dst and returns the number of bytes consumed. It resets dst first, so nothing of a previous value survives, and allocates nothing on success.

A replica id >= VClockMax fails with ErrVClockReplicaID. A repeated id keeps its last LSN, as the VClock decoder does. On error dst is left reset.

func DecodeVyLogBodyInto ¶ added in v0.2.0

func DecodeVyLogBodyInto(b []byte, dst *VyLogBody, opts *DecodeOptions) error

DecodeVyLogBodyInto parses a VYLOG row body into dst, the core of DecodeVyLogBody. It first resets dst to the zero VyLogBody, so a dst reused across rows carries no field of a prior row. The values of Keys alias b; see Aliasing in the package doc. A body key other than IPROTO_TUPLE goes to opts.OnUnknown, or is skipped when opts or its OnUnknown is nil. The Keys map of a prior row is never reused: every record allocates a new one. On error dst holds what was decoded before it.

func DecodeVyPageInfoBodyInto ¶ added in v0.2.0

func DecodeVyPageInfoBodyInto(b []byte, dst *VyPageInfo, opts *DecodeOptions) error

DecodeVyPageInfoBodyInto parses the body of a VY_INDEX_PAGE_INFO row into dst, the zero-allocation core of DecodeVyPageInfoBody. It first resets dst to the zero VyPageInfo, so a reused dst carries no field of a prior row, then sets the fields of the keys b holds. MinKey aliases b; see Aliasing in the package doc. A key with no field goes to opts.OnUnknown, or is skipped when opts or its OnUnknown is nil. On error dst holds what was decoded before it.

func DecodeVyRowIndexBody ¶ added in v0.2.0

func DecodeVyRowIndexBody(b []byte) ([]uint32, error)

DecodeVyRowIndexBody parses the body of a VY_RUN_ROW_INDEX row: the offset of each statement of the page in the rows of the page, uncompressed (vy_row_index_encode, src/box/vy_run.c).

func DecodeVyRunInfoBodyInto ¶ added in v0.2.0

func DecodeVyRunInfoBodyInto(b []byte, dst *VyRunInfo, opts *DecodeOptions) error

DecodeVyRunInfoBodyInto parses the body of a VY_INDEX_RUN_INFO row into dst, the zero-allocation core of DecodeVyRunInfoBody. It first resets dst to the zero VyRunInfo, so a reused dst carries no field of a prior row, then sets the fields of the keys b holds. The raw fields of dst (MinKey, MaxKey, Bloom) alias b; see Aliasing in the package doc. A key with no field goes to opts.OnUnknown, or is skipped when opts or its OnUnknown is nil. On error dst holds what was decoded before it.

func DecodeVyStmtBodyInto ¶ added in v0.2.0

func DecodeVyStmtBodyInto(b []byte, dst *VyStmtBody, opts *DecodeOptions) error

DecodeVyStmtBodyInto parses the body of a statement of a .run file into dst, the zero-allocation core of DecodeVyStmtBody. It first resets dst to the zero VyStmtBody, so a dst reused across statements carries no field of a prior one, then sets the fields of the keys b holds. The raw fields of dst (Tuple, Key, Ops) alias b; see Aliasing in the package doc. A body key with no field goes to opts.OnUnknown, or is skipped when opts or its OnUnknown is nil. On error dst holds what was decoded before it.

func DecodeXRowInto ¶

func DecodeXRowInto(b []byte, dst *XRow) (int, error)

DecodeXRowInto parses one xrow record at b[0:] into dst, fully overwriting every field (including zeroing BodyRaw / StreamID / Sync / SchemaVersion / Flags / Present so a reused dst carries no state from a prior row), and returns the number of bytes consumed. It is the zero-allocation core of DecodeXRow: a hot read loop can reuse a single *XRow — or an arena slot — across rows with no per-row heap allocation. As with DecodeXRow, dst.BodyRaw aliases b; see Aliasing in the package doc.

func DecompressTx ¶

func DecompressTx(payload, scratch []byte) ([]byte, error)

DecompressTx zstd-decodes payload. The caller may pass a non-nil scratch to amortise allocations across calls; the returned slice is the decompressed bytes (may alias scratch if it was large enough).

func EncodeFixheader ¶

func EncodeFixheader(buf *[FixheaderSize]byte, h *Fixheader)

EncodeFixheader writes a complete 19-byte fixheader into buf.

The three uints are msgpack-encoded with the smallest width that fits (matching `mp_encode_uint` in mp.h). The remaining tail bytes are filled as an mp_str header with N-1 zero bytes of content, where N is the remaining byte count — exactly the recipe in src/box/xlog.c:1100-1105.

EncodeFixheader panics if the encoded uints overflow 15 bytes, which cannot happen for uint32 fields (max 5+5+5 = 15 bytes after the 4-byte magic, leaving exactly 0 padding bytes — and `mp_encode_strl(0)` for a zero-length fixstr is one byte 0xa0, but since `padding == 0` the writer skips the str entirely, matching Tarantool's `if (padding > 0)`).

func EncodeMeta ¶

func EncodeMeta(w io.Writer, m *Meta) error

EncodeMeta writes the meta header. The format-version line is m.FormatVer when set (it must be "0.13" or the legacy "0.12", else ErrMetaBadVersion), and defaults to FormatVersion ("0.13") when m.FormatVer is empty — so the writer's freshly-built metas still emit 0.13, while a rewrite can preserve or retarget the source's version. The VClock line is always emitted (even "VClock: {}", required by Tarantool's signature check); an empty PrevVClock line is omitted. The header ends with a blank line.

package main

import (
	"bytes"
	"os"

	"github.com/google/uuid"

	"github.com/tarantool/go-xlog/format"
)

func main() {
	meta := &format.Meta{
		Filetype:     format.FiletypeXLOG,
		Version:      "example",
		InstanceUUID: uuid.MustParse("11111111-2222-3333-4444-555555555555"),
		VClock:       format.VClock{1: 2},
	}

	var buf bytes.Buffer
	if err := format.EncodeMeta(&buf, meta); err != nil {
		panic(err)
	}

	_, _ = os.Stdout.Write(buf.Bytes())

}

Output:
XLOG
0.13
Version: example
Instance: 11111111-2222-3333-4444-555555555555
VClock: {1: 2}

func EncodeTxBlock ¶

func EncodeTxBlock(rows []XRow, opts TxOptions) ([]byte, error)

EncodeTxBlock encodes rows into a complete on-disk tx block: fixheader + payload. The payload is each row's EncodeXRow output concatenated in order. If opts.Compression says to compress (the payload meets its threshold and it is not Disabled), the payload is zstd-compressed and the fixheader magic is ZRowMarker; otherwise RowMarker is used. The CRC32C in the fixheader is computed over the on-disk payload bytes (post-compression for zrow, plain for row) per src/box/xlog.c:1086,1165.

func EncodeXRow ¶

func EncodeXRow(buf []byte, r *XRow) ([]byte, error)

EncodeXRow appends a complete xrow record to buf and returns the new buf. The record consists of:

• a header map (per src/box/xrow.c:331-444)
• the BodyRaw bytes appended verbatim

Encoding rules (src/box/xrow.c:402-410):

• IPROTO_REQUEST_TYPE is always present.
• IPROTO_SYNC / IPROTO_SCHEMA_VERSION / IPROTO_REPLICA_ID / IPROTO_LSN / IPROTO_GROUP_ID / IPROTO_TIMESTAMP / IPROTO_STREAM_ID present iff nonzero.
• Single-statement tx (TSN == LSN && IsCommit && TSN != 0): omit IPROTO_TSN AND clear IPROTO_FLAG_COMMIT in the encoded flags.
• Multi-statement tx (TSN != LSN || !IsCommit): emit IPROTO_TSN as the diff (LSN - TSN). If IsCommit && TSN != LSN (last row of multi-stmt), OR IPROTO_FLAG_COMMIT back into the encoded flags.
• IPROTO_FLAGS is emitted iff the resulting flags value is non-zero.

The caller is responsible for BodyRaw being a valid msgpack map (or nil for NOP rows).

EncodeXRow never reads r.Present: a key is emitted by the rules above, from the value of its field alone. Present is decode-only, and every XRow built by hand has Present == 0; gating a key on it would encode such rows with an empty header (see XRow.Present).

Caveat: this function mirrors Tarantool's `if (header->tsn != 0)` guard (src/box/xrow.c:402). A row with LSN == TSN == 0 will skip the TSN/COMMIT emission block entirely. For a single-statement tx that is harmless — the decoder also infers `is_commit = true` when IPROTO_TSN is absent. For a multi-row tx where some rows legitimately have LSN == 0, the encoded output would be ambiguous (Tarantool would treat each as an independent single-stmt tx). The writer package's assignTxIDs avoids this by setting TSN = rows[0].LSN before encoding; callers using EncodeXRow directly must ensure rows in a multi-row tx have non-zero LSN.

The keys are emitted in the order of xrow_header_encode (src/box/xrow.c:338-452): REQUEST_TYPE, SYNC, REPLICA_ID, GROUP_ID, LSN, TIMESTAMP, TSN, STREAM_ID, FLAGS, so a header Tarantool would write encodes to the same bytes. xrow_header_encode never writes IPROTO_SCHEMA_VERSION; only iproto_header_encode (src/box/xrow.c:497-515) does, in a response header, right after IPROTO_SYNC, so EncodeXRow emits it there too.

package main

import (
	"fmt"

	"github.com/tarantool/go-iproto"

	"github.com/tarantool/go-xlog/format"
)

func main() {
	// A minimal valid msgpack body: the one-element array [1].
	body := []byte{0x91, 0x01}

	row := &format.XRow{Type: iproto.IPROTO_INSERT, LSN: 1, BodyRaw: body}

	encoded, err := format.EncodeXRow(nil, row)
	if err != nil {
		panic(err)
	}

	decoded, n, err := format.DecodeXRow(encoded)
	if err != nil {
		panic(err)
	}

	fmt.Printf("type=%s lsn=%d commit=%t body=%d bytes (consumed %d of %d)\n",
		format.TypeName(decoded.Type), decoded.LSN, decoded.IsCommit(),
		len(decoded.BodyRaw), n, len(encoded))

}

Output:
type=INSERT lsn=1 commit=true body=2 bytes (consumed 7 of 7)

func TypeName ¶

func TypeName(t iproto.Type) string

TypeName returns the short Tarantool name of a durable request type ("INSERT", "REPLACE", "RAFT", …). It is iproto.Type.String() with the "IPROTO_" prefix trimmed; the vinyl row types are named as Tarantool's iproto_type_name names them ("RUNINFO", "PAGEINFO", "ROWINDEX"); unknown types render as iproto's Type(n) form.