block

package
v1.1.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 8, 2025 License: MIT Imports: 7 Imported by: 0

Documentation

Overview

Package block provides reader and writer types for processing Argo data blocks. These types handle different encoding strategies such as length-prefixing (with or without deduplication), fixed-size data, and unlabeled varints.

Package block provides BlockWriter and DeduplicatingBlockWriter types for preparing Argo data blocks. These writers manage the conversion of values to their byte representations and the generation of corresponding labels, which are essential for the Argo binary format.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type AnyBlockWriter 

type AnyBlockWriter interface {
	// Write processes a value `v` of any type. It attempts to assert `v` to the
	// concrete type expected by the underlying block writer. If successful, it delegates
	// to the underlying writer's Write method. It returns the generated label and any error.
	Write(v interface{}) (*label.Label, error)
	// AllValuesAsBytes returns all accumulated byte arrays from the underlying writer.
	AllValuesAsBytes() [][]byte
	// WriteLastToBuf writes the byte representation of the most recently processed value
	// to the provided buffer, by delegating to the underlying writer.
	WriteLastToBuf(buf buf.Write) error
}

AnyBlockWriter defines an interface for block writers that accept `interface{}` (any) values. This allows for type-erased handling of block writing operations, useful in contexts where the specific type of values being written is not known at compile time or varies. Implementations typically wrap a generic BlockWriter[T] or DeduplicatingBlockWriter[T] and perform a type assertion before delegating to the underlying writer.

func NewAnyBlockWriter 

func NewAnyBlockWriter[T any](bw *BlockWriter[T]) AnyBlockWriter

NewAnyBlockWriter creates an AnyBlockWriter by wrapping a given generic BlockWriter[T]. This allows the specifically typed BlockWriter to be used through the type-erased AnyBlockWriter interface.

func NewAnyDeduplicatingBlockWriter 

func NewAnyDeduplicatingBlockWriter[T comparable](dbw *DeduplicatingBlockWriter[T]) AnyBlockWriter

NewAnyDeduplicatingBlockWriter creates an AnyBlockWriter by wrapping a given generic DeduplicatingBlockWriter[T]. The type T must be comparable. This allows the specifically typed DeduplicatingBlockWriter to be used through the type-erased AnyBlockWriter interface.

func NewAnyLengthOfBytesBlockWriter 

func NewAnyLengthOfBytesBlockWriter(valueToBytes ValueToBytesFunc[interface{}]) AnyBlockWriter

NewAnyLengthOfBytesBlockWriter creates a BlockWriter specifically for `interface{}` values (T is interface{}) that generates labels based on byte length, and then wraps it as an AnyBlockWriter. This is a convenience constructor for another common use case.

func NewAnyNoLabelBlockWriter 

func NewAnyNoLabelBlockWriter(valueToBytes ValueToBytesFunc[interface{}]) AnyBlockWriter

NewAnyNoLabelBlockWriter creates a BlockWriter specifically for `interface{}` values (T is interface{}) that generates no labels, and then wraps it as an AnyBlockWriter. This is a convenience constructor for a common use case.

type BlockWriter 

type BlockWriter[In any] struct {
	// contains filtered or unexported fields
}

BlockWriter is responsible for converting input values of type `In` into their byte representations and generating corresponding labels using provided functions. It accumulates the byte representations internally. The actual writing of labels and byte data to output streams is typically handled by the caller.

func NewBlockWriter 

func NewBlockWriter[In any](
	makeLabel MakeLabelFunc[In],
	valueToBytes ValueToBytesFunc[In],
) *BlockWriter[In]

NewBlockWriter creates and returns a new BlockWriter configured with the provided makeLabel and valueToBytes functions. If either function is nil, the Write method will subsequently fail.

func NewLengthOfBytesBlockWriter 

func NewLengthOfBytesBlockWriter[In any](valueToBytes ValueToBytesFunc[In]) *BlockWriter[In]

NewLengthOfBytesBlockWriter creates a BlockWriter that generates labels representing the length (in bytes) of each value's binary representation.

func NewNoLabelBlockWriter 

func NewNoLabelBlockWriter[In comparable](valueToBytes ValueToBytesFunc[In]) *BlockWriter[In]

NewNoLabelBlockWriter creates a BlockWriter that does not generate any labels. Its makeLabelFunc will always return (nil, nil).

func (*BlockWriter[In]) AfterNewWrite 

func (bw *BlockWriter[In]) AfterNewWrite()

AfterNewWrite is a hook method called by Write after a new value's byte representation has been successfully generated and stored. Its default implementation is a no-op.

func (*BlockWriter[In]) AllValuesAsBytes 

func (bw *BlockWriter[In]) AllValuesAsBytes() [][]byte

AllValuesAsBytes returns a new slice containing all byte arrays accumulated by the BlockWriter. This is typically used when constructing a final value block from all processed items. The returned slice is a copy of the slice header, but the underlying byte arrays are shared.

func (*BlockWriter[In]) Write 

func (bw *BlockWriter[In]) Write(v In) (*label.Label, error)

Write converts the given value `v` to its byte representation using valueToBytesFunc, stores these bytes internally, and then generates a label using makeLabelFunc. It returns the generated label (or nil if no label is to be generated) and any error encountered during the process. This method calls AfterNewWrite after successfully storing the bytes.

func (*BlockWriter[In]) WriteLastToBuf 

func (bw *BlockWriter[In]) WriteLastToBuf(buf buf.Write) error

WriteLastToBuf writes the byte representation of the most recently processed value to the provided buf.Write buffer. This method is primarily intended for scenarios like "noBlocks" mode where values are written directly. It does not remove the value from its internal store. For block construction, AllValuesAsBytes is typically used.

type CommonState 

type CommonState struct {
	// DataBuf is the buffer from which the actual data for this block is read.
	// This is distinct from any "parent" buffer that might contain labels or references.
	DataBuf buf.Read
}

CommonState holds shared state for block readers, primarily the `DataBuf` from which block data is read. It also provides a default `AfterNewRead` hook. Embedding CommonState allows concrete reader types to share this buffer and default hook.

func (*CommonState) AfterNewRead 

func (cs *CommonState) AfterNewRead()

AfterNewRead is a hook method called by some block readers after successfully reading and processing a new value from the block. The default implementation is a no-op. Concrete block reader types can provide their own `AfterNewRead` implementation if specific post-read actions are needed (e.g., updating internal state for deduplication).

type DeduplicatingBlockWriter 

type DeduplicatingBlockWriter[In comparable] struct {
	// Embeds BlockWriter. The embedded `makeLabelFunc` is not used by DeduplicatingBlockWriter's `Write` method.
	// `valueToBytesFunc`, `valuesAsBytes`, and `AfterNewWrite` are inherited and used.
	BlockWriter[In]
	// contains filtered or unexported fields
}

DeduplicatingBlockWriter embeds BlockWriter and extends its functionality to support value deduplication. When a value is processed, if it has been seen before, a backreference label is returned. For new values, a unique ID is assigned (and used for future backreferences), and then a label is generated using its labelForNew function. The input type `In` must be comparable to be used as a key for tracking seen values.

func NewDeduplicatingBlockWriter 

func NewDeduplicatingBlockWriter[In comparable](
	labelForNew MakeLabelFunc[In],
	valueToBytes ValueToBytesFunc[In],
) *DeduplicatingBlockWriter[In]

NewDeduplicatingBlockWriter creates and returns a new DeduplicatingBlockWriter. It requires a `labelForNew` function (to generate labels for unique items) and a `valueToBytes` function (to convert values to bytes). Backreference IDs are initialized based on label.LowestReservedValue. If either function is nil, the Write method will subsequently fail.

func NewLengthOfBytesDeduplicatingBlockWriter 

func NewLengthOfBytesDeduplicatingBlockWriter[In comparable](
	valueToBytes ValueToBytesFunc[In],
) *DeduplicatingBlockWriter[In]

NewLengthOfBytesDeduplicatingBlockWriter creates a DeduplicatingBlockWriter where labels for new (non-duplicate) values are generated based on the length of their byte representation.

func ToDeduplicatingWriter 

func ToDeduplicatingWriter[In comparable](
	bw *BlockWriter[In],
) *DeduplicatingBlockWriter[In]

ToDeduplicatingWriter is a package-level utility function that converts a given BlockWriter[In] to a DeduplicatingBlockWriter[In]. The type parameter `In` for the input BlockWriter must be `comparable`. It initializes the new DeduplicatingBlockWriter using the `makeLabelFunc` (as `labelForNew`) and `valueToBytesFunc` from the original BlockWriter.

func (*DeduplicatingBlockWriter[In]) ToDeduplicating 

func (dbw *DeduplicatingBlockWriter[In]) ToDeduplicating() *DeduplicatingBlockWriter[In]

ToDeduplicating on a DeduplicatingBlockWriter returns the receiver `dbw` itself, as it's already a deduplicating writer.

func (*DeduplicatingBlockWriter[In]) Write 

func (dbw *DeduplicatingBlockWriter[In]) Write(v In) (*label.Label, error)

Write processes the value `v` for the DeduplicatingBlockWriter. It first calls `labelForValue` to check if `v` is nil or a duplicate.

  • If `labelForValue` returns a non-nil label (NullMarker or backreference), Write returns that label immediately.
  • If `labelForValue` returns `(nil, nil)` (indicating `v` is new, non-nil, and has been assigned a new backreference ID in the `seen` map), Write then converts `v` to bytes, stores the bytes (appending to the embedded BlockWriter's `valuesAsBytes`), calls `AfterNewWrite`, and finally generates a label for this new item using `dbw.labelForNew`.

This method effectively overrides the Write method of the embedded BlockWriter.

type DeduplicatingLabelBlockReader 

type DeduplicatingLabelBlockReader[Out any] struct {
	CommonState // Embeds DataBuf and default AfterNewRead.
	// contains filtered or unexported fields
}

DeduplicatingLabelBlockReader is similar to LabelBlockReader but adds support for deduplication. It reads values from its `DataBuf` that are preceded by a label (read from `parentBuf`). This label can either specify the length of a new value or be a backreference to a previously read value. New values are stored internally to be targets for future backreferences. `Out` is the target type of the values read.

func NewDeduplicatingLabelBlockReader 

func NewDeduplicatingLabelBlockReader[Out any](dataBuf buf.Read, fromBytes FromBytesFunc[Out], readNullTerminator bool) *DeduplicatingLabelBlockReader[Out]

NewDeduplicatingLabelBlockReader creates and returns a new DeduplicatingLabelBlockReader. - dataBuf: The buffer for reading actual value data. - fromBytes: Function to convert raw bytes to the `Out` type. - values: An initial slice of values; typically empty, will be populated as new unique values are read. - readNullTerminator: If true, an extra null byte is read from `dataBuf` after new string data.

func (*DeduplicatingLabelBlockReader[Out]) Read 

func (r *DeduplicatingLabelBlockReader[Out]) Read(parentBuf buf.Read) (Out, error)

Read decodes a single value, handling potential backreferences. It reads a label from `parentBuf`. If the label is a backreference, it returns the previously stored value. If the label indicates a length, it reads the data from `DataBuf`, converts it, stores it for future backreferences, and returns it. Optionally reads a null terminator for new string data. Returns the decoded value and an error if any step fails. This reader cannot handle null, absent, or error labels directly.

type FixedSizeBlockReader 

type FixedSizeBlockReader[Out any] struct {
	CommonState // Embeds DataBuf.

	ByteLength int // The fixed number of bytes for each value.
	// contains filtered or unexported fields
}

FixedSizeBlockReader reads values of a predetermined fixed byte length from its `DataBuf`. It does not use labels from a `parentBuf` to determine length, as the length is constant. `Out` is the target type of the values read.

func NewFixedSizeBlockReader 

func NewFixedSizeBlockReader[Out any](dataBuf buf.Read, fromBytes FromBytesFunc[Out], byteLength int) *FixedSizeBlockReader[Out]

NewFixedSizeBlockReader creates and returns a new FixedSizeBlockReader. - dataBuf: The buffer from which fixed-size data chunks are read. - fromBytes: Function to convert the fixed-size byte chunks to the `Out` type. - byteLength: The exact number of bytes that constitutes one value. Must be non-negative, or it panics.

func (*FixedSizeBlockReader[Out]) Read 

func (r *FixedSizeBlockReader[Out]) Read(parentBuf buf.Read) (Out, error)

Read decodes a fixed-size block from r.DataBuf. parentBuf is ignored by this implementation but included in the signature to potentially match a common interface derived from the abstract BlockReader<Out>.

type FromBytesFunc 

type FromBytesFunc[Out any] func(bytes []byte) Out

FromBytesFunc is a generic function type that defines a conversion from a byte slice to a specific output type `Out`. It is used by various block readers to transform raw bytes read from a block into the desired data type. This corresponds to the `(bytes: Uint8Array) => Out` callback in the TypeScript Argo implementation.

type LabelBlockReader 

type LabelBlockReader[Out any] struct {
	CommonState // Embeds DataBuf and default AfterNewRead.
	// contains filtered or unexported fields
}

LabelBlockReader reads values from its `DataBuf` where each value is preceded by a length-determining label read from a separate `parentBuf`. It is used for types where each instance in the block is individually length-prefixed. `Out` is the target type of the values read.

func NewLabelBlockReader 

func NewLabelBlockReader[Out any](dataBuf buf.Read, fromBytes FromBytesFunc[Out], readNullTerminator bool) *LabelBlockReader[Out]

NewLabelBlockReader creates and returns a new LabelBlockReader. - dataBuf: The buffer from which the actual value data is read (corresponds to `this.buf` in some TS implementations). - fromBytes: A function to convert the raw bytes (read according to the label) into the target type `Out`. - readNullTerminator: If true, an extra null byte is read and discarded from `dataBuf` after reading the value's data (used for null-terminated strings).

func (*LabelBlockReader[Out]) Read 

func (r *LabelBlockReader[Out]) Read(parentBuf buf.Read) (Out, error)

Read decodes a single value. It first reads a label from `parentBuf` to determine the length of the data. Then, it reads that many bytes from its internal `DataBuf`, converts these bytes to the `Out` type using `fromBytes`, and optionally reads a null terminator. It returns the decoded value and an error if any step fails. This reader expects the label to indicate a length; it cannot handle backreferences, null, absent, or error labels directly.

type MakeLabelFunc 

type MakeLabelFunc[In any] func(value In, data []byte) (l *label.Label, err error)

MakeLabelFunc defines a function signature for creating a Label for a given input value and its byte representation. It returns a pointer to a label, or nil if no label should be generated. It can also return an error if label creation fails.

type UnlabeledVarIntBlockReader 

type UnlabeledVarIntBlockReader struct {
	CommonState // Embeds DataBuf.
}

UnlabeledVarIntBlockReader reads varint-encoded integers directly from its `DataBuf`. It does not use labels from a `parentBuf` as varints are self-delimiting. This reader is specifically for `int64` values after ZigZag decoding.

func NewUnlabeledVarIntBlockReader 

func NewUnlabeledVarIntBlockReader(dataBuf buf.Read) *UnlabeledVarIntBlockReader

NewUnlabeledVarIntBlockReader creates and returns a new UnlabeledVarIntBlockReader. - dataBuf: The buffer from which varint-encoded data is read.

func (*UnlabeledVarIntBlockReader) Read 

func (r *UnlabeledVarIntBlockReader) Read(parentBuf buf.Read) (int64, error)

Read decodes a single varint-encoded integer from `DataBuf` (ignoring `parentBuf`). It performs ZigZag decoding to convert the unsigned varint read from the buffer into a signed `int64`. Returns the decoded `int64` and an error if reading or decoding fails.

type ValueToBytesFunc 

type ValueToBytesFunc[In any] func(value In) (data []byte, err error)

ValueToBytesFunc defines a function signature for converting an input value to its byte representation. It returns the byte data or an error if the conversion fails.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.