Documentation
¶
Overview ¶
Package lib0 implements the binary encoding format used by Yjs.
lib0 is the npm package by Kevin Jahns that defines Yjs's wire encoding. This package provides byte-for-byte compatible encode/decode primitives for varuint, varint (zigzag-encoded signed), variable-length strings, variable-length byte buffers, and the fixed-width number types used throughout the Yjs sync protocol.
Wire format compatibility with the JS implementation is non-negotiable. See testdata/ for fixtures captured from the JS lib0 reference.
Reference: https://github.com/dmonad/lib0
Index ¶
- Variables
- func ReadFloat32(buf []byte) (float32, int, error)
- func ReadFloat64(buf []byte) (float64, int, error)
- func ReadUint8(buf []byte) (uint8, int, error)
- func ReadUint8Func(buf []byte) (uint64, int, error)
- func ReadUint16(buf []byte) (uint16, int, error)
- func ReadUint32(buf []byte) (uint32, int, error)
- func ReadVarInt(buf []byte) (int64, int, error)
- func ReadVarString(buf []byte) (string, int, error)
- func ReadVarUint(buf []byte) (uint64, int, error)
- func ReadVarUint8Array(buf []byte) ([]byte, int, error)
- func WriteFloat32(buf []byte, f float32) []byte
- func WriteFloat64(buf []byte, f float64) []byte
- func WriteUint8(buf []byte, n uint8) []byte
- func WriteUint8Func(buf []byte, v uint64) []byte
- func WriteUint16(buf []byte, n uint16) []byte
- func WriteUint32(buf []byte, n uint32) []byte
- func WriteVarInt(buf []byte, n int64) []byte
- func WriteVarString(buf []byte, s string) []byte
- func WriteVarUint(buf []byte, n uint64) []byte
- func WriteVarUint8Array(buf, b []byte) []byte
- type IntDiffOptRleDecoder
- type IntDiffOptRleEncoder
- type RleDecoder
- type RleEncoder
- type StringDecoder
- type StringEncoder
- type UintOptRleDecoder
- type UintOptRleEncoder
Constants ¶
This section is empty.
Variables ¶
var ErrOverflow = errors.New("lib0: varint overflow")
ErrOverflow is returned when a varuint/varint exceeds 64 bits.
var ErrTruncated = errors.New("lib0: truncated input")
ErrTruncated is returned when a decoder runs past the end of the input.
Functions ¶
func ReadFloat32 ¶
ReadFloat32 reads a big-endian IEEE-754 float32.
func ReadFloat64 ¶
ReadFloat64 reads a big-endian IEEE-754 float64.
func ReadUint8Func ¶
ReadUint8Func wraps ReadUint8 for the RleDecoder reader signature.
func ReadUint16 ¶
ReadUint16 reads a little-endian uint16.
func ReadUint32 ¶
ReadUint32 reads a little-endian uint32.
func ReadVarInt ¶
ReadVarInt reads a lib0-encoded signed varint (zigzag-on-first-byte) and returns the value plus the number of bytes consumed.
func ReadVarString ¶
ReadVarString reads a varuint-prefixed UTF-8 string.
func ReadVarUint ¶
ReadVarUint reads a LEB128-style varuint from buf and returns the value plus the number of bytes consumed.
func ReadVarUint8Array ¶
ReadVarUint8Array reads a varuint-prefixed byte slice. The returned slice references the input buffer; copy if you need to retain it.
func WriteFloat32 ¶
WriteFloat32 appends a big-endian IEEE-754 32-bit float. lib0 uses big-endian for floats to match the DataView default in JS.
func WriteFloat64 ¶
WriteFloat64 appends a big-endian IEEE-754 64-bit float.
func WriteUint8Func ¶
WriteUint8Func wraps WriteUint8 to fit the RleEncoder writer signature. The value MUST fit in a uint8; higher bits are discarded by the underlying WriteUint8 cast.
func WriteUint16 ¶
WriteUint16 appends a little-endian 16-bit unsigned integer.
func WriteUint32 ¶
WriteUint32 appends a little-endian 32-bit unsigned integer.
func WriteVarInt ¶
WriteVarInt appends the zigzag-then-LEB128 encoding of n to buf and returns the extended slice. The sign is folded into the LSB of the first byte: bit 6 holds the sign, bits 0-5 hold the lowest 6 bits of |n|, bit 7 is continuation. This matches lib0's writeVarInt.
func WriteVarString ¶
WriteVarString appends the lib0 varstring encoding of s to buf: a varuint length prefix in bytes (UTF-8) followed by the UTF-8 bytes themselves.
func WriteVarUint ¶
WriteVarUint appends the LEB128-style varuint encoding of n to buf and returns the extended slice. Bytes use 7 bits of value with the MSB as the continuation flag. This matches lib0's writeVarUint.
func WriteVarUint8Array ¶
WriteVarUint8Array appends the lib0 varbuffer encoding of b to buf: a varuint length prefix followed by the raw bytes.
Types ¶
type IntDiffOptRleDecoder ¶
type IntDiffOptRleDecoder struct {
// contains filtered or unexported fields
}
IntDiffOptRleDecoder is the read counterpart.
func NewIntDiffOptRleDecoder ¶
func NewIntDiffOptRleDecoder(buf []byte) *IntDiffOptRleDecoder
NewIntDiffOptRleDecoder returns a decoder over buf.
func NewIntDiffOptRleDecoderFor ¶
func NewIntDiffOptRleDecoderFor(buf []byte) *IntDiffOptRleDecoder
NewIntDiffOptRleDecoderFor returns a decoder over buf — same as NewIntDiffOptRleDecoder but also takes the buffer in the constructor (the original ctor takes no args, matching the JS API; this helper avoids the Reset dance for callers that have the buffer up front).
func (*IntDiffOptRleDecoder) Read ¶
func (d *IntDiffOptRleDecoder) Read() (int64, error)
Read returns the next reconstructed value.
func (*IntDiffOptRleDecoder) Reset ¶
func (d *IntDiffOptRleDecoder) Reset(buf []byte)
Reset re-points the decoder at buf (e.g. after column-buffer slicing in V2's parallel reader). Internal accumulator state (s) is preserved per lib0 convention — IntDiffOpt decoders are expected to start at s=0 and accumulate from there.
type IntDiffOptRleEncoder ¶
type IntDiffOptRleEncoder struct {
// contains filtered or unexported fields
}
IntDiffOptRleEncoder packs `(diff << 1) | hasCount` into the first varint of each run. Mirrors lib0 encoding.js (lines 853-887). 31-bit diff limit due to the shift.
Receiver maintains its own accumulator and adds each diff repeatedly. Resetting state mid-stream requires writing a record that breaks the current run — the encoder/decoder will then naturally re-sync on a fresh diff.
func NewIntDiffOptRleEncoder ¶
func NewIntDiffOptRleEncoder() *IntDiffOptRleEncoder
NewIntDiffOptRleEncoder returns an empty encoder.
func (*IntDiffOptRleEncoder) Bytes ¶
func (e *IntDiffOptRleEncoder) Bytes() []byte
Bytes flushes and returns.
func (*IntDiffOptRleEncoder) Write ¶
func (e *IntDiffOptRleEncoder) Write(v int64)
Write records v into the current run. If the delta from the running accumulator matches the current run's delta the count grows; otherwise the run flushes and a new one starts.
type RleDecoder ¶
type RleDecoder struct {
// contains filtered or unexported fields
}
RleDecoder is the read counterpart to RleEncoder.
func NewRleDecoder ¶
NewRleDecoder returns a decoder over buf using the supplied value-reader (matches the encoder's writer).
func (*RleDecoder) Read ¶
func (d *RleDecoder) Read() (uint64, error)
Read returns the next value in the run. Re-uses the cached run value until the count exhausts, then reads a fresh value + count pair from the underlying buffer.
Returns an error if the underlying buffer is malformed mid-run.
type RleEncoder ¶
type RleEncoder struct {
// contains filtered or unexported fields
}
RleEncoder pairs `(value, count-1)`. On Write(v): if v matches the current-run value, increment count; else flush previous run as a varuint(count-1) followed by the new value (written via the user-supplied writer func — typically a single byte for V2's info / parentInfo columns).
Mirrors testdata/gen/node_modules/lib0/encoding.js RleEncoder (lines 615-650). Tail-flush happens on Bytes().
func NewRleEncoder ¶
func NewRleEncoder(writer func(buf []byte, v uint64) []byte) *RleEncoder
NewRleEncoder returns an RLE encoder that writes each run-value via the supplied writer. For V2's info/parentInfo columns pass WriteUint8Func — a thin wrapper around WriteUint8.
func (*RleEncoder) Bytes ¶
func (e *RleEncoder) Bytes() []byte
Bytes flushes any pending run and returns the accumulated byte buffer. Must be called exactly once at the end of column construction; subsequent Writes after Bytes are undefined.
func (*RleEncoder) Write ¶
func (e *RleEncoder) Write(v uint64)
Write records v into the current run. If v matches the previous value the count grows; otherwise the previous run flushes and a new one starts.
type StringDecoder ¶
type StringDecoder struct {
// contains filtered or unexported fields
}
StringDecoder reads back what StringEncoder wrote: a varstring of concatenated content followed by a UintOptRle length stream.
func NewStringDecoder ¶
func NewStringDecoder(buf []byte) (*StringDecoder, error)
NewStringDecoder constructs a decoder over the StringEncoder column bytes.
func (*StringDecoder) Read ¶
func (d *StringDecoder) Read() (string, error)
Read returns the next string slice.
type StringEncoder ¶
type StringEncoder struct {
// contains filtered or unexported fields
}
StringEncoder concatenates strings into one big buffer and emits them with a parallel UintOptRle length stream. Output layout: `varstring(concatenated) + uintOptRleLengthBytes` — two segments back-to-back, no inner length prefix on the length stream (outer V2 column wrapper bounds the whole thing). Mirrors lib0 encoding.js StringEncoder (lines 900-922).
func NewStringEncoder ¶
func NewStringEncoder() *StringEncoder
NewStringEncoder returns an empty StringEncoder.
func (*StringEncoder) Bytes ¶
func (e *StringEncoder) Bytes() []byte
Bytes flushes the staging buffer and emits the wire bytes: varstring(concatenated) followed by the UintOptRle length stream (no length prefix on the length stream — outer wrapper bounds the whole thing).
func (*StringEncoder) Write ¶
func (e *StringEncoder) Write(s string)
Write appends one string to the column.
type UintOptRleDecoder ¶
type UintOptRleDecoder struct {
// contains filtered or unexported fields
}
UintOptRleDecoder is the read counterpart to UintOptRleEncoder.
func NewUintOptRleDecoder ¶
func NewUintOptRleDecoder(buf []byte) *UintOptRleDecoder
NewUintOptRleDecoder returns a decoder over buf.
func (*UintOptRleDecoder) Read ¶
func (d *UintOptRleDecoder) Read() (uint64, error)
Read returns the next value.
type UintOptRleEncoder ¶
type UintOptRleEncoder struct {
// contains filtered or unexported fields
}
UintOptRleEncoder is the optimized RLE for absolute uints. A single occurrence emits one varint with positive sign; a run of N (N>=2) emits varint(-value) + varuint(count-2). Mirrors lib0 encoding.js UintOptRleEncoder (lines 742-773).
31-bit value limit: the value-sign bit lives in the leading varint; values larger than 2^31-1 would collide with the sign flag.
func NewUintOptRleEncoder ¶
func NewUintOptRleEncoder() *UintOptRleEncoder
NewUintOptRleEncoder returns an empty UintOptRle encoder.
func (*UintOptRleEncoder) Bytes ¶
func (e *UintOptRleEncoder) Bytes() []byte
Bytes flushes the pending run and returns the buffer. Call once.
func (*UintOptRleEncoder) Write ¶
func (e *UintOptRleEncoder) Write(v uint64)
Write records v into the current run.
Source Files