hashmap

package
v0.0.0-debug-20260702 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 2, 2026 License: Apache-2.0 Imports: 8 Imported by: 0

Documentation

Index

Constants

View Source
const (
	UnitLimit             = 256
	HashMapSizeThreshHold = UnitLimit * 128
	HashMapSizeEstimate   = UnitLimit * 32
)
View Source
const MaxStrIteratorCapacity = 64 * 1024

MaxStrIteratorCapacity limits how many bytes of backing storage we keep when reusing a string iterator. Avoids retaining oversized buffers after handling very large strings.

Variables

View Source
var (
	OneUInt8s []uint8
	OneInt64s []int64
)

Functions

func IteratorChangeOwner

func IteratorChangeOwner(itr Iterator, m HashMap)

func IteratorClearOwner

func IteratorClearOwner(itr Iterator)

IteratorClearOwner detaches the iterator from its hashmap to allow the old hashmap to be garbage collected when the iterator is cached.

func StrIteratorCapacity

func StrIteratorCapacity(itr Iterator) int

StrIteratorCapacity reports the total capacity of all key buffers maintained by a string iterator. Used to decide if a cached iterator should be kept.

Types

type HashMap added in v0.6.0

type HashMap interface {
	// HasNull returns whether the hash map considers the null values.
	HasNull() bool
	// Free method frees the hash map.
	Free()
	// AddGroup adds 1 to the row count of hash map.
	AddGroup()
	// AddGroups adds N to the row count of hash map.
	AddGroups(uint64)
	// GroupCount returns the hash map's row count.
	GroupCount() uint64
	// Size returns the hash map's size
	Size() int64
	// MarshalBinary serializes the hash map into a byte slice.
	MarshalBinary() ([]byte, error)
	// UnmarshalBinary deserializes a byte slice into the hash map.
	UnmarshalBinary(data []byte, mp *mpool.MPool) error
	// WriteTo serializes the hash map to a writer.
	WriteTo(w io.Writer) (int64, error)
	// UnmarshalFrom deserializes a byte slice from a reader.
	UnmarshalFrom(r io.Reader, mp *mpool.MPool) (int64, error)
	// FillGroupHashes appends all group hash codes into dst and returns the result.
	FillGroupHashes(dst []uint64) []uint64
}

HashMap is the encapsulated hash table interface exposed to the outside

type IntHashMap added in v0.6.0

type IntHashMap struct {
	// contains filtered or unexported fields
}

IntHashMap key is int64, value is an uint64 (start from 1) before you use the IntHashMap, the user should make sure that sum of vectors' length equal to 8

func NewIntHashMap added in v0.6.0

func NewIntHashMap(hasNull bool, memPool *mpool.MPool) (*IntHashMap, error)

func (*IntHashMap) AddGroup added in v0.6.0

func (m *IntHashMap) AddGroup()

func (*IntHashMap) AddGroups added in v0.6.0

func (m *IntHashMap) AddGroups(rows uint64)

func (*IntHashMap) Cardinality added in v0.6.0

func (m *IntHashMap) Cardinality() uint64

func (*IntHashMap) FillGroupHashes

func (m *IntHashMap) FillGroupHashes(dst []uint64) []uint64

func (*IntHashMap) Free added in v0.6.0

func (m *IntHashMap) Free()

func (*IntHashMap) GroupCount added in v0.6.0

func (m *IntHashMap) GroupCount() uint64

func (*IntHashMap) HasNull added in v0.6.0

func (m *IntHashMap) HasNull() bool

func (*IntHashMap) MarshalBinary

func (m *IntHashMap) MarshalBinary() ([]byte, error)

func (*IntHashMap) NewIterator added in v0.6.0

func (m *IntHashMap) NewIterator() *intHashMapIterator

func (*IntHashMap) PreAlloc added in v1.0.0

func (m *IntHashMap) PreAlloc(n uint64) error

func (*IntHashMap) Size added in v0.7.0

func (m *IntHashMap) Size() int64

func (*IntHashMap) UnmarshalBinary

func (m *IntHashMap) UnmarshalBinary(data []byte, mp *mpool.MPool) error

func (*IntHashMap) UnmarshalFrom

func (m *IntHashMap) UnmarshalFrom(r io.Reader, mp *mpool.MPool) (int64, error)

func (*IntHashMap) WriteTo

func (m *IntHashMap) WriteTo(w io.Writer) (int64, error)

type Iterator added in v0.6.0

type Iterator interface {
	// not safe for multi parallel!!!!
	// Insert vecs[start, start+count) into hashmap
	// vs  : the number of rows corresponding to each value in the hash table (start with 1)
	// zvs : if zvs[i] is 0 indicates the presence null, 1 indicates the absence of a null.
	Insert(start, count int, vecs []*vector.Vector) (vs []uint64, zvs []int64, err error)

	// not safe for multi parallel!!!!
	// Insert a row from multiple columns into the hashmap, return true if it is new, otherwise false
	DetectDup(vecs []*vector.Vector, row int) (bool, error)

	//safe for multi parallel
	// Find vecs[start, start+count) in hashmap
	// vs  : the number of rows corresponding to each value in the hash table (start with 1, and 0 means not found.)
	// zvs : if zvs[i] is 0 indicates the presence null, 1 indicates the absence of a null.
	Find(start, count int, vecs []*vector.Vector) (vs []uint64, zvs []int64)
}

Iterator allows users to do insert or find operations on hash tables in bulk.

type StrHashMap

type StrHashMap struct {
	// contains filtered or unexported fields
}

StrHashMap key is []byte, value is an uint64 value (starting from 1)

each time a new key is inserted, the hashtable returns a last-value+1 or, if the old key is inserted, the value corresponding to that key

func NewStrHashMap

func NewStrHashMap(hasNull bool, memPool *mpool.MPool) (*StrHashMap, error)

func (*StrHashMap) AddGroup added in v0.6.0

func (m *StrHashMap) AddGroup()

func (*StrHashMap) AddGroups added in v0.6.0

func (m *StrHashMap) AddGroups(rows uint64)

func (*StrHashMap) FillGroupHashes

func (m *StrHashMap) FillGroupHashes(dst []uint64) []uint64

func (*StrHashMap) Free added in v0.6.0

func (m *StrHashMap) Free()

func (*StrHashMap) GroupCount added in v0.6.0

func (m *StrHashMap) GroupCount() uint64

func (*StrHashMap) HasNull added in v0.6.0

func (m *StrHashMap) HasNull() bool

func (*StrHashMap) MarshalBinary

func (m *StrHashMap) MarshalBinary() ([]byte, error)

func (*StrHashMap) NewIterator added in v0.6.0

func (m *StrHashMap) NewIterator() Iterator

func (*StrHashMap) PreAlloc added in v1.0.0

func (m *StrHashMap) PreAlloc(n uint64) error

func (*StrHashMap) Size added in v0.7.0

func (m *StrHashMap) Size() int64

func (*StrHashMap) UnmarshalBinary

func (m *StrHashMap) UnmarshalBinary(data []byte, mp *mpool.MPool) error

func (*StrHashMap) UnmarshalFrom

func (m *StrHashMap) UnmarshalFrom(r io.Reader, mp *mpool.MPool) (int64, error)

func (*StrHashMap) WriteTo

func (m *StrHashMap) WriteTo(w io.Writer) (int64, error)

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL