encoding

package
v0.9.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 24, 2025 License: Apache-2.0 Imports: 10 Imported by: 0

Documentation

Overview

Package encoding implements encoding/decoding data points.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func BytesToInt64 added in v0.6.0 

func BytesToInt64(src []byte) int64

BytesToInt64 converts the first eight bytes of the given byte slice to an int64.

func BytesToInt64List added in v0.6.0 

func BytesToInt64List(dst []int64, src []byte, mt EncodeType, firstValue int64, itemsCount int) ([]int64, error)

BytesToInt64List decodes bytes into a list of int64.

func BytesToUint16 added in v0.6.0 

func BytesToUint16(src []byte) uint16

BytesToUint16 converts the first two bytes of the given byte slice to a uint16.

func BytesToUint32 added in v0.6.0 

func BytesToUint32(src []byte) uint32

BytesToUint32 converts the first four bytes of the given byte slice to a uint32.

func BytesToUint64 added in v0.6.0 

func BytesToUint64(src []byte) uint64

BytesToUint64 converts the first eight bytes of the given byte slice to a uint64.

func BytesToVarInt64 added in v0.6.0 

func BytesToVarInt64(src []byte) ([]byte, int64, error)

BytesToVarInt64 converts the first bytes of the given byte slice to an int64. It uses variable-length encoding.

func BytesToVarInt64List added in v0.6.0 

func BytesToVarInt64List(dst []int64, src []byte) ([]byte, error)

BytesToVarInt64List converts the first bytes of the given byte slice to an int64s. It uses variable-length encoding.

func BytesToVarUint64 added in v0.6.0 

func BytesToVarUint64(src []byte) ([]byte, uint64)

BytesToVarUint64 converts the first bytes of the given byte slice to a uint64. It uses variable-length encoding.

func BytesToVarUint64s added in v0.6.0 

func BytesToVarUint64s(dst []uint64, src []byte) ([]byte, error)

BytesToVarUint64s converts the first bytes of the given byte slice to a uint64s. It uses variable-length encoding.

func DecimalIntListToFloat64List added in v0.9.0 

func DecimalIntListToFloat64List(dst []float64, values []int64, exponent int16, itemsCount int) ([]float64, error)

DecimalIntListToFloat64List restores float64 values from scaled int64s using a decimal exponent.

func DecodeBytes added in v0.6.0 

func DecodeBytes(src []byte) ([]byte, []byte, error)

DecodeBytes decodes a string from src.

func DecodeUint64Block added in v0.9.0 

func DecodeUint64Block(dst []uint64, src []byte, itemsCount uint64) ([]uint64, []byte, error)

DecodeUint64Block decodes a block of uint64 values from src.

func EncodeBytes added in v0.6.0 

func EncodeBytes(dst, b []byte) []byte

EncodeBytes encodes a string into dst.

func EncodeBytesBlock added in v0.6.0 

func EncodeBytesBlock(dst []byte, a [][]byte) []byte

EncodeBytesBlock encodes a block of strings into dst.

func EncodeUint64Block added in v0.9.0 

func EncodeUint64Block(dst []byte, a []uint64) []byte

EncodeUint64Block encodes a block of uint64 values into dst.

func ExtendListCapacity added in v0.7.0 

func ExtendListCapacity[T any](dst []T, additionalItems int) []T

ExtendListCapacity extends the capacity of the given list.

func Float64ListToDecimalIntList added in v0.9.0 

func Float64ListToDecimalIntList(dst []int64, nums []float64) ([]int64, int16, error)

Float64ListToDecimalIntList converts float64 values to int64s with a common decimal scale factor.

func Int64ToBytes added in v0.6.0 

func Int64ToBytes(dst []byte, v int64) []byte

Int64ToBytes appends the bytes of the given int64 to the given byte slice.

func ReleaseInt64List added in v0.6.0 

func ReleaseInt64List(is *Int64List)

ReleaseInt64List releases the given list of int64. The list may be put into a pool for reuse.

func ReleaseUint64List added in v0.6.0 

func ReleaseUint64List(is *Uint64List)

ReleaseUint64List releases the given list of uint64. The list may be put into a pool for reuse.

func Uint16ToBytes added in v0.6.0 

func Uint16ToBytes(dst []byte, u uint16) []byte

Uint16ToBytes appends the bytes of the given uint16 to the given byte slice.

func Uint32ToBytes added in v0.6.0 

func Uint32ToBytes(dst []byte, u uint32) []byte

Uint32ToBytes appends the bytes of the given uint32 to the given byte slice.

func Uint64ToBytes added in v0.6.0 

func Uint64ToBytes(dst []byte, u uint64) []byte

Uint64ToBytes appends the bytes of the given uint64 to the given byte slice.

func VarInt64ListToBytes added in v0.6.0 

func VarInt64ListToBytes(dst []byte, vs []int64) []byte

VarInt64ListToBytes appends the bytes of the given int64s to the given byte slice. It uses variable-length encoding.

func VarInt64ToBytes added in v0.6.0 

func VarInt64ToBytes(dst []byte, v int64) []byte

VarInt64ToBytes appends the bytes of the given int64 to the given byte slice. It uses variable-length encoding.

func VarUint64ToBytes added in v0.6.0 

func VarUint64ToBytes(dst []byte, u uint64) []byte

VarUint64ToBytes appends the bytes of the given uint64 to the given byte slice. It uses variable-length encoding.

func VarUint64sToBytes added in v0.6.0 

func VarUint64sToBytes(dst []byte, us []uint64) []byte

VarUint64sToBytes appends the bytes of the given uint64s to the given byte slice. It uses variable-length encoding.

Types

type BufferWriter added in v0.3.0 

type BufferWriter interface {
	Write(data []byte) (n int, err error)
	WriteByte(b byte) error
	Bytes() []byte
}

BufferWriter allows writing a variable-sized buffer of bytes.

type BytesBlockDecoder added in v0.6.0 

type BytesBlockDecoder struct {
	// contains filtered or unexported fields
}

BytesBlockDecoder decodes a block of strings from src. It reuses the underlying buffer for storing the decoded strings.

func (*BytesBlockDecoder) Decode added in v0.6.0 

func (bbd *BytesBlockDecoder) Decode(dst [][]byte, src []byte, itemsCount uint64) ([][]byte, error)

Decode decodes a block of strings from src.

func (*BytesBlockDecoder) DecodeWithTail added in v0.9.0 

func (bbd *BytesBlockDecoder) DecodeWithTail(dst [][]byte, src []byte, itemsCount uint64) ([][]byte, []byte, error)

DecodeWithTail decodes a block of strings from src and returns the remaining tail.

func (*BytesBlockDecoder) Reset added in v0.6.0 

func (bbd *BytesBlockDecoder) Reset()

Reset resets the decoder.

type Dictionary added in v0.9.0 

type Dictionary struct {
	// contains filtered or unexported fields
}

Dictionary is used for dictionary encoding.

func NewDictionary added in v0.9.0 

func NewDictionary() *Dictionary

NewDictionary creates a dictionary.

func (*Dictionary) Add added in v0.9.0 

func (d *Dictionary) Add(value []byte) bool

Add adds a value to the dictionary.

func (*Dictionary) Decode added in v0.9.0 

func (d *Dictionary) Decode(dst [][]byte, src []byte, itemsCount uint64) ([][]byte, error)

Decode decodes the dictionary.

func (*Dictionary) Encode added in v0.9.0 

func (d *Dictionary) Encode(dst []byte) []byte

Encode encodes the dictionary.

func (*Dictionary) Reset added in v0.9.0 

func (d *Dictionary) Reset()

Reset resets the dictionary.

type EncodeType added in v0.6.0 

type EncodeType byte

EncodeType indicates the encoding type of a series. 

const (
	EncodeTypeUnknown EncodeType = iota
	EncodeTypeConst
	EncodeTypeDeltaConst
	EncodeTypeDelta
	EncodeTypeDeltaOfDelta
	EncodeTypeConstWithVersion
	EncodeTypeDeltaConstWithVersion
	EncodeTypeDeltaWithVersion
	EncodeTypeDeltaOfDeltaWithVersion
	EncodeTypePlain
	EncodeTypeDictionary
)

EncodeType constants.

func GetCommonType added in v0.7.0 

func GetCommonType(et EncodeType) EncodeType

GetCommonType returns the common type of the given encoding type.

func GetVersionType added in v0.7.0 

func GetVersionType(et EncodeType) EncodeType

GetVersionType returns the version type of the given encoding type.

func Int64ListToBytes added in v0.6.0 

func Int64ListToBytes(dst []byte, a []int64) (result []byte, mt EncodeType, firstValue int64)

Int64ListToBytes encodes a list of int64 into bytes.

type Int64List added in v0.6.0 

type Int64List struct {
	L []int64
}

Int64List is a list of int64.

func GenerateInt64List added in v0.6.0 

func GenerateInt64List(size int) *Int64List

GenerateInt64List generates a list of int64 with the given size. The returned list may be from a pool and should be released after use.

type Packer added in v0.3.0 

type Packer struct {
	// contains filtered or unexported fields
}

Packer writes data into a buffer.

func NewPacker added in v0.3.0 

func NewPacker(buf BufferWriter) *Packer

NewPacker returns a new Writer.

func (*Packer) Bytes added in v0.3.0 

func (w *Packer) Bytes() []byte

Bytes outputs the data in the buffer.

func (*Packer) PutUint16 added in v0.3.0 

func (w *Packer) PutUint16(v uint16)

PutUint16 puts uint16 data into the buffer.

func (*Packer) PutUint32 added in v0.3.0 

func (w *Packer) PutUint32(v uint32)

PutUint32 puts uint32 data into the buffer.

func (*Packer) PutUint64 added in v0.3.0 

func (w *Packer) PutUint64(v uint64)

PutUint64 puts uint64 data into the buffer.

func (*Packer) Write added in v0.3.0 

func (w *Packer) Write(p []byte)

Write binaries to the buffer.

type Reader added in v0.3.0 

type Reader struct {
	// contains filtered or unexported fields
}

Reader reads bits from buffer.

func NewReader added in v0.3.0 

func NewReader(in io.ByteReader) *Reader

NewReader crate bit reader.

func (*Reader) ReadBits added in v0.3.0 

func (r *Reader) ReadBits(numBits int) (uint64, error)

ReadBits read number of bits.

func (*Reader) ReadBool added in v0.3.0 

func (r *Reader) ReadBool() (bool, error)

ReadBool reads a bit, 1 returns true, 0 returns false.

func (*Reader) ReadByte added in v0.3.0 

func (r *Reader) ReadByte() (byte, error)

ReadByte reads a byte.

func (*Reader) Reset added in v0.3.0 

func (r *Reader) Reset()

Reset resets the reader to read from a new slice.

type SeriesDecoder 

type SeriesDecoder interface {
	// Decode the time series data
	Decode(key, data []byte) error
	// Len denotes the size of iterator
	Len() int
	// IsFull returns whether the encoded data reached its capacity
	IsFull() bool
	// Get the data point by its time
	Get(ts uint64) ([]byte, error)
	// Iterator returns a SeriesIterator
	Iterator() SeriesIterator
	// Range returns the start and end time of this series
	Range() (start, end uint64)
}

SeriesDecoder decodes encoded time series data.

type SeriesDecoderPool 

type SeriesDecoderPool interface {
	Get(metadata []byte) SeriesDecoder
	Put(encoder SeriesDecoder)
}

SeriesDecoderPool allows putting and getting SeriesDecoder.

type SeriesEncoder 

type SeriesEncoder interface {
	// Append a data point
	Append(ts uint64, value []byte)
	// IsFull returns whether the encoded data reached its capacity
	IsFull() bool
	// Reset the underlying buffer
	Reset(key []byte, buffer BufferWriter)
	// Encode the time series data point to a binary
	Encode() error
	// StartTime indicates the first entry's time
	StartTime() uint64
}

SeriesEncoder encodes time series data point.

type SeriesEncoderPool 

type SeriesEncoderPool interface {
	Get(metadata []byte, buffer BufferWriter) SeriesEncoder
	Put(encoder SeriesEncoder)
}

SeriesEncoderPool allows putting and getting SeriesEncoder.

type SeriesIterator 

type SeriesIterator interface {
	// Next scroll the cursor to the next
	Next() bool
	// Val returns the value of the current data point
	Val() []byte
	// Time returns the time of the current data point
	Time() uint64
	// Error might return an error indicates a decode failure
	Error() error
}

SeriesIterator iterates time series data.

type Uint64List added in v0.6.0 

type Uint64List struct {
	L []uint64
}

Uint64List is a list of uint64.

func GenerateUint64List added in v0.6.0 

func GenerateUint64List(size int) *Uint64List

GenerateUint64List generates a list of uint64 with the given size. The returned list may be from a pool and should be released after use.

type Writer added in v0.3.0 

type Writer struct {
	// contains filtered or unexported fields
}

Writer writes bits to an io.BufferWriter.

func NewWriter added in v0.3.0 

func NewWriter() *Writer

NewWriter create bit writer.

func (*Writer) Flush added in v0.3.0 

func (w *Writer) Flush()

Flush flushes the currently in-process byte.

func (*Writer) Reset added in v0.3.0 

func (w *Writer) Reset(buffer BufferWriter)

Reset writes to a new writer.

func (*Writer) WriteBits added in v0.3.0 

func (w *Writer) WriteBits(u uint64, numBits int)

WriteBits writes number of bits.

func (*Writer) WriteBool added in v0.3.0 

func (w *Writer) WriteBool(b bool)

WriteBool writes a boolean value true: 1 false: 0

func (*Writer) WriteByte added in v0.3.0 

func (w *Writer) WriteByte(b byte) error

WriteByte write a byte.

type XORDecoder 

type XORDecoder struct {
	// contains filtered or unexported fields
}

XORDecoder decodes buffer to uint64 values using xor compress.

func NewXORDecoder 

func NewXORDecoder(br *Reader) *XORDecoder

NewXORDecoder create zstdDecoder decompress buffer using xor.

func (*XORDecoder) Err 

func (d *XORDecoder) Err() error

Err returns error raised in Next().

func (*XORDecoder) Next 

func (d *XORDecoder) Next() bool

Next return if zstdDecoder has value in buffer using xor, do uncompress logic in next method, data format reference zstdEncoder format.

func (*XORDecoder) Reset 

func (d *XORDecoder) Reset()

Reset resets the underlying buffer to decode.

func (*XORDecoder) Value 

func (d *XORDecoder) Value() uint64

Value returns uint64 from buffer.

type XOREncoder 

type XOREncoder struct {
	// contains filtered or unexported fields
}

XOREncoder intends to compress uint64 data https://www.vldb.org/pvldb/vol8/p1816-teller.pdf

func NewXOREncoder 

func NewXOREncoder(bw *Writer) *XOREncoder

NewXOREncoder creates xor zstdEncoder for compressing uint64 data.

func (*XOREncoder) Write 

func (e *XOREncoder) Write(val uint64)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.