streamingdata

package
v1.8.8-0...-6a25a8b Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 16, 2026 License: MIT Imports: 6 Imported by: 0

Documentation

Overview

Package streamingdata is part of the internal baseline and is unsupported for public use. It is provided for internal comparison/testing and may change without notice.

Package streamingdata implements chunked, stream-like storage on top of a B-Tree. It splits large values into ordered chunks and provides Encoder/Reader helpers for IO-like access.

Index

Constants

View Source 
const (
	// MinimumStreamingStoreSlotLength is the smallest node slot length allowed for streaming stores.
	MinimumStreamingStoreSlotLength = 50
)

Variables

This section is empty.

Functions

This section is empty.

Types

type Encoder 

type Encoder[TK btree.Ordered] struct {
	// contains filtered or unexported fields
}

Encoder writes JSON values to chunked storage by delegating to json.Encoder over a custom writer.

func (*Encoder[TK]) Close 

func (e *Encoder[TK]) Close() error

Close finalizes Update/UpdateCurrentValue by deleting any remaining old chunks. It is a no-op for Add/AddIfNotExist.

func (*Encoder[TK]) Encode 

func (e *Encoder[TK]) Encode(v any) error

Encode writes v as JSON to the underlying chunked writer followed by a newline.

func (*Encoder[TK]) SetEscapeHTML 

func (e *Encoder[TK]) SetEscapeHTML(on bool)

SetEscapeHTML toggles escaping of HTML characters in JSON strings.

func (*Encoder[TK]) SetIndent 

func (e *Encoder[TK]) SetIndent(prefix, indent string)

SetIndent controls indentation for subsequent encodes. SetIndent("", "") disables pretty printing.

type StreamingDataKey 

type StreamingDataKey[TK btree.Ordered] struct {
	Key        TK
	ChunkIndex int
}

StreamingDataKey is the B-Tree key for streaming chunks: (Key, ChunkIndex).

func (StreamingDataKey[TK]) Compare 

func (x StreamingDataKey[TK]) Compare(other interface{}) int

Compare implements ordering by Key then ChunkIndex to preserve chunk order.

type StreamingDataStore 

type StreamingDataStore[TK btree.Ordered] struct {
	// Inherit or reuse an Object implementing BtreeInterface. Golang's inheritance is actually better,
	// it alows to inherit or reuse any object implementing a given interface. "loosely" & nicely done.
	btree.BtreeInterface[StreamingDataKey[TK], []byte]
}

StreamingDataStore stores entries split into ordered chunks and provides stream-like read/write operations.

func (*StreamingDataStore[TK]) Add 

func (s *StreamingDataStore[TK]) Add(ctx context.Context, key TK) (*Encoder[TK], error)

Add inserts a new streaming item and returns an Encoder for writing chunks.

func (*StreamingDataStore[TK]) AddChunk 

func (s *StreamingDataStore[TK]) AddChunk(ctx context.Context, key TK, chunkIndex int, chunkValue []byte) (bool, error)

AddChunk inserts a new chunk for the specified key and chunk index.

func (*StreamingDataStore[TK]) AddIfNotExist 

func (s *StreamingDataStore[TK]) AddIfNotExist(ctx context.Context, key TK) (*Encoder[TK], error)

AddIfNotExist inserts an item only when it does not exist and returns an Encoder to write the value.

func (*StreamingDataStore[TK]) FindChunk 

func (s *StreamingDataStore[TK]) FindChunk(ctx context.Context, key TK, chunkIndex int) (bool, error)

FindChunk finds a specific chunk for a key and positions the cursor there.

func (*StreamingDataStore[TK]) FindOne 

func (s *StreamingDataStore[TK]) FindOne(ctx context.Context, key TK) (bool, error)

FindOne finds the first chunk for a key and positions the cursor there.

func (*StreamingDataStore[TK]) FindOneWithID 

func (s *StreamingDataStore[TK]) FindOneWithID(ctx context.Context, key TK, chunkIndex int) (bool, error)

FindOneWithID finds a specific chunk index for a key (alias of FindChunk).

func (*StreamingDataStore[TK]) GetCurrentItem 

func (s *StreamingDataStore[TK]) GetCurrentItem(ctx context.Context) (btree.Item[TK, json.Decoder], error)

GetCurrentItem returns the current key and a JSON decoder that streams its value.

func (*StreamingDataStore[TK]) GetCurrentKey 

func (s *StreamingDataStore[TK]) GetCurrentKey(ctx context.Context) TK

GetCurrentKey returns the logical key of the current item cursor.

func (*StreamingDataStore[TK]) GetCurrentValue 

func (s *StreamingDataStore[TK]) GetCurrentValue(ctx context.Context) (*json.Decoder, error)

GetCurrentValue returns a JSON decoder that streams the current item's value.

func (*StreamingDataStore[TK]) Remove 

func (s *StreamingDataStore[TK]) Remove(ctx context.Context, key TK) (bool, error)

Remove deletes all chunks for the item with the given key.

func (*StreamingDataStore[TK]) RemoveChunk 

func (s *StreamingDataStore[TK]) RemoveChunk(ctx context.Context, key TK, chunkIndex int) (bool, error)

RemoveChunk deletes the chunk for the specified key and chunk index.

func (*StreamingDataStore[TK]) RemoveCurrentItem 

func (s *StreamingDataStore[TK]) RemoveCurrentItem(ctx context.Context) (bool, error)

RemoveCurrentItem deletes all chunks for the current item.

func (*StreamingDataStore[TK]) Update 

func (s *StreamingDataStore[TK]) Update(ctx context.Context, key TK) (*Encoder[TK], error)

Update prepares an Encoder for updating chunks of the item with the given key.

func (*StreamingDataStore[TK]) UpdateChunk 

func (s *StreamingDataStore[TK]) UpdateChunk(ctx context.Context, key TK, chunkIndex int, newChunkValue []byte) (bool, error)

UpdateChunk updates an existing chunk for the specified key and chunk index.

func (*StreamingDataStore[TK]) UpdateCurrentValue 

func (s *StreamingDataStore[TK]) UpdateCurrentValue(ctx context.Context) (*Encoder[TK], error)

UpdateCurrentValue returns an Encoder for updating chunks of the current item.

func (*StreamingDataStore[TK]) Upsert 

func (s *StreamingDataStore[TK]) Upsert(ctx context.Context, key TK) (*Encoder[TK], error)

Upsert updates the item if it exists; otherwise it adds a new one and returns an Encoder.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.