batch

package
v0.0.0-debug-20260702 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 2, 2026 License: Apache-2.0 Imports: 10 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var (
	EmptyBatch = &Batch{rowCount: 0}

	EmptyForConstFoldBatch = &Batch{
		Vecs:     make([]*vector.Vector, 0),
		rowCount: 1,
	}

	CteEndBatch = &Batch{
		Recursive: 2,
		rowCount:  1,
	}
)

special batch that will never been free.

Functions

func BatchToProtoBatch added in v0.6.0 

func BatchToProtoBatch(bat *Batch) (*api.Batch, error)

func SetLength 

func SetLength(bat *Batch, n int)

Types

type Batch 

type Batch struct {
	// For recursive CTE, 1 is last batch, 2 is end of batch
	Recursive  int32
	ShuffleIDX int32 //used only in shuffle
	// Attrs column name list, letter case: origin
	Attrs []string
	// Vecs col data
	Vecs []*vector.Vector

	ExtraBuf []byte
	// contains filtered or unexported fields
}

Batch represents a part of a relationship, it's the basic unit for data transmission in the computing layer of MatrixOne.

this includes information such as column names (Attrs), column data (Vecs), the number of rows (rowCount), aggregation intermediate data (Aggs), and some specific flags like Recursive, ShuffleIDX.

offHeap is true means this batch was allocated outside mpool.

func EmptyBatchWithAttrs 

func EmptyBatchWithAttrs(attrs []string) Batch

func New 

func New(attrs []string) *Batch

func NewOffHeap 

func NewOffHeap(attrs []string) *Batch

func NewOffHeapEmpty 

func NewOffHeapEmpty() *Batch

func NewOffHeapWithSize 

func NewOffHeapWithSize(n int) *Batch

func NewWithSchema 

func NewWithSchema(offHeap bool, attrs []string, attTypes []types.Type) *Batch

func NewWithSize added in v0.5.0 

func NewWithSize(n int) *Batch

func ProtoBatchToBatch added in v0.6.0 

func ProtoBatchToBatch(bat *api.Batch) (*Batch, error)

func (*Batch) AddRowCount added in v1.0.0 

func (bat *Batch) AddRowCount(rowCount int)

func (*Batch) Allocated 

func (bat *Batch) Allocated() int

func (*Batch) Append 

func (bat *Batch) Append(ctx context.Context, mp *mpool.MPool, b *Batch) (*Batch, error)

func (*Batch) AppendWithCopy added in v1.2.0 

func (bat *Batch) AppendWithCopy(ctx context.Context, mp *mpool.MPool, b *Batch) (*Batch, error)

AppendWithCopy is used to append data from batch `b` to another batch `bat`. The function ensures that the batch structure is consistent and copies all vector data to the target batch. WARING: this function will cause a memory allocation.

func (*Batch) CheckLength 

func (bat *Batch) CheckLength() error

func (*Batch) Clean added in v0.5.0 

func (bat *Batch) Clean(m *mpool.MPool)

func (*Batch) CleanOnlyData added in v0.8.0 

func (bat *Batch) CleanOnlyData()

func (*Batch) Clone 

func (bat *Batch) Clone(mp *mpool.MPool, offHeap bool) (*Batch, error)

func (*Batch) CloneSelectedColumns 

func (bat *Batch) CloneSelectedColumns(
	selectCols []int,
	selectAttrs []string,
	mp *mpool.MPool,
) (cloned *Batch, err error)

func (*Batch) CloneSelectedColumnsTo 

func (bat *Batch) CloneSelectedColumnsTo(
	selectCols []int,
	toBat *Batch,
	mp *mpool.MPool,
) (err error)

func (*Batch) CloneTo 

func (bat *Batch) CloneTo(toBat *Batch, mp *mpool.MPool) (err error)

func (*Batch) Dup added in v0.8.0 

func (bat *Batch) Dup(mp *mpool.MPool) (*Batch, error)

Dup used to copy a Batch object, this method will create a new batch and copy all vectors (Vecs) of the current batch to the new batch.

func (*Batch) End added in v1.0.0 

func (bat *Batch) End() bool

func (*Batch) FreeColumns 

func (bat *Batch) FreeColumns(m *mpool.MPool)

func (*Batch) GetSchema 

func (bat *Batch) GetSchema() (attrs []string, attrTypes []types.Type)

func (*Batch) GetVector added in v0.5.0 

func (bat *Batch) GetVector(pos int32) *vector.Vector

func (*Batch) InsertVector 

func (bat *Batch) InsertVector(
	pos int32,
	attr string,
	vec *vector.Vector,
)

func (*Batch) IsDone 

func (bat *Batch) IsDone() bool

func (*Batch) IsEmpty added in v1.0.0 

func (bat *Batch) IsEmpty() bool

func (*Batch) Last added in v1.0.0 

func (bat *Batch) Last() bool

func (*Batch) MarshalBinary added in v0.6.0 

func (bat *Batch) MarshalBinary() ([]byte, error)

func (*Batch) MarshalBinaryWithBuffer 

func (bat *Batch) MarshalBinaryWithBuffer(w *bytes.Buffer, reset bool) ([]byte, error)

func (*Batch) PreExtend added in v1.0.0 

func (bat *Batch) PreExtend(mp *mpool.MPool, rows int) error

func (*Batch) ReplaceVector added in v0.8.0 

func (bat *Batch) ReplaceVector(oldVec *vector.Vector, newVec *vector.Vector, startIndex int)

func (*Batch) RowCount added in v1.0.0 

func (bat *Batch) RowCount() int

func (*Batch) SelectColumns 

func (bat *Batch) SelectColumns(cols []int, attrs []string) *Batch

func (*Batch) SetAttributes added in v0.6.0 

func (bat *Batch) SetAttributes(attrs []string)

func (*Batch) SetEnd added in v1.0.0 

func (bat *Batch) SetEnd()

func (*Batch) SetLast added in v1.0.0 

func (bat *Batch) SetLast()

func (*Batch) SetRowCount added in v1.0.0 

func (bat *Batch) SetRowCount(rowCount int)

func (*Batch) SetVector added in v0.6.0 

func (bat *Batch) SetVector(pos int32, vec *vector.Vector)

func (*Batch) Shrink added in v0.5.0 

func (bat *Batch) Shrink(sels []int64, negate bool)

func (*Batch) ShrinkByMask 

func (bat *Batch) ShrinkByMask(sels *bitmap.Bitmap, negate bool, offset uint64)

func (*Batch) Shuffle added in v0.5.0 

func (bat *Batch) Shuffle(sels []int64, m *mpool.MPool) error

func (*Batch) Size added in v0.6.0 

func (bat *Batch) Size() int

func (*Batch) String 

func (bat *Batch) String() string

func (*Batch) Union 

func (bat *Batch) Union(bat2 *Batch, sels []int64, m *mpool.MPool) error

func (*Batch) UnionOne 

func (bat *Batch) UnionOne(bat2 *Batch, pos int64, m *mpool.MPool) error

func (*Batch) UnionWindow 

func (bat *Batch) UnionWindow(bat2 *Batch, offset, cnt int, m *mpool.MPool) error

func (*Batch) UnmarshalBinary added in v0.6.0 

func (bat *Batch) UnmarshalBinary(data []byte) (err error)

func (*Batch) UnmarshalBinaryWithAnyMp 

func (bat *Batch) UnmarshalBinaryWithAnyMp(data []byte, mp *mpool.MPool) (err error)

func (*Batch) UnmarshalFromReader 

func (bat *Batch) UnmarshalFromReader(r io.Reader, mp *mpool.MPool) (err error)

func (*Batch) VectorCount added in v0.6.0 

func (bat *Batch) VectorCount() int

func (*Batch) Window 

func (bat *Batch) Window(start, end int) (*Batch, error)

type BatchSet 

type BatchSet struct {
	// contains filtered or unexported fields
}

simple batch slice if BatchSet.batches[lastIdx].rowCount < batchMaxRow just fill data to BatchSet.batches[lastIdx] until BatchSet.batches[lastIdx].rowCount to batchMaxRow

func NewBatchSet 

func NewBatchSet(batchMaxRow int) *BatchSet

func (*BatchSet) Clean 

func (bs *BatchSet) Clean(mpool *mpool.MPool)

func (*BatchSet) Extend 

func (bs *BatchSet) Extend(mpool *mpool.MPool, inBatch *Batch, reuseBuf *Batch) (bool, error)

Extend extend one batch'data to BatchSet, returns whether reuseBuf was consumed BatchSet donot obtain ownership of inBatch

func (*BatchSet) Get 

func (bs *BatchSet) Get(idx int) *Batch

func (*BatchSet) Length 

func (bs *BatchSet) Length() int

func (*BatchSet) Pop 

func (bs *BatchSet) Pop() *Batch

func (*BatchSet) PopFront 

func (bs *BatchSet) PopFront() *Batch

func (*BatchSet) Push 

func (bs *BatchSet) Push(mpool *mpool.MPool, inBatch *Batch) error

Push append inBatch to BatchSet. BatchSet will obtain ownership of inBatch

func (*BatchSet) RowCount 

func (bs *BatchSet) RowCount() int

func (*BatchSet) TakeBatches 

func (bs *BatchSet) TakeBatches() []*Batch

func (*BatchSet) Union 

func (bs *BatchSet) Union(mpool *mpool.MPool, inBatch *Batch, sels []int32, reuseBuf *Batch) (bool, error)

Union union some data from one batch to BatchSet, returns whether reuseBuf was consumed

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.