typedcolumn

package
v0.6.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 1, 2026 License: MIT Imports: 16 Imported by: 0

Documentation

Overview

Package typedcolumn contains the transplanted experiments/colgranule typed-column data plane for TreeDB.

The package is intentionally non-authoritative in this PR: it can build, encode, decode, and read typed-column part artifacts, but it does not publish production collection assets, register manifest/recovery state, participate in WAL replay, or own logical fields through typed_storage_layout. Later TreeDB integration layers are expected to adapt this package to #1736 mappedresource handles and the production control plane without reshaping the sectioned column-major part-image model into the existing typed-row asset format.

Index

Constants

View Source
const (
	DefaultAdaptiveMarkTargetBytes = 1 << 20
	DefaultAdaptiveMarkMinRows     = 512
)
View Source
const (
	ColumnPruningEnvelopeVersion = uint16(1)
	Int64PruningPayloadVersion   = uint16(1)
)
View Source
const (
	ColumnPruningReasonSupported            = "supported"
	ColumnPruningReasonMissingMetadata      = "pruning_metadata_missing"
	ColumnPruningReasonUnsupportedPayload   = "pruning_payload_unsupported"
	ColumnPruningReasonOperationUnsupported = "pruning_operation_unsupported"
	ColumnPruningReasonChecksumMismatch     = "pruning_checksum_mismatch"
	ColumnPruningReasonIdentityMismatch     = "pruning_identity_mismatch"
	ColumnPruningReasonRowCountMismatch     = "pruning_row_count_mismatch"
	ColumnPruningReasonNullDefaultMismatch  = "pruning_null_default_mismatch"
	ColumnPruningReasonEntryOrderMismatch   = "pruning_entry_order_mismatch"
	ColumnPruningReasonEntryRowMismatch     = "pruning_entry_row_mismatch"
	ColumnPruningReasonMinMaxMismatch       = "pruning_min_max_mismatch"
	ColumnPruningReasonSumOverflow          = "pruning_sum_overflow"
)
View Source
const (
	ColumnStatsEnvelopeVersion = uint16(1)
	Int64StatsPayloadVersion   = uint16(1)
)
View Source
const (
	ColumnStatsReasonSupported            = "supported"
	ColumnStatsReasonUnsupportedPayload   = "stats_payload_unsupported"
	ColumnStatsReasonOperationUnsupported = "operation_unsupported"
	ColumnStatsReasonSelectionUnsupported = "selection_shape_unsupported"
	ColumnStatsReasonChecksumMismatch     = "checksum_mismatch"
	ColumnStatsReasonIdentityMismatch     = "identity_mismatch"
	ColumnStatsReasonRowCountMismatch     = "row_count_mismatch"
	ColumnStatsReasonNullDefaultMismatch  = "null_default_count_mismatch"
	ColumnStatsReasonVisibilityMismatch   = "visibility_count_mismatch"
	ColumnStatsReasonMinMaxMismatch       = "min_max_mismatch"
	ColumnStatsReasonSumOverflow          = "sum_overflow"
)
View Source
const AggregateMetadataDefinitionVersion uint16 = 1
View Source
const ColumnPartImageManifestHeaderBytes = 32
View Source
const ColumnPartLayoutContractVersion uint16 = 2
View Source
const (
	DefaultRowsPerGranule = 8192
)

Variables

This section is empty.

Functions

func ColumnPartImageManifestLength

func ColumnPartImageManifestLength(header []byte) (int, error)

func DecodeInt64

func DecodeInt64(dst []int64, g EncodedGranule) ([]int64, error)

func DecodeRawBytesOffsetsOffsetsFallback

func DecodeRawBytesOffsetsOffsetsFallback(dst []uint64, offsetsRaw []byte, rows int) ([]uint64, error)

DecodeRawBytesOffsetsOffsetsFallback decodes only the little-endian uint64 offsets substream.

func DecodeRawDenseFixedWidthPayload

func DecodeRawDenseFixedWidthPayload(dst []byte, raw []byte, rows int, elementsPerRow int, elementBytes int) ([]byte, error)

func DecodeRawFixedBytesPayload

func DecodeRawFixedBytesPayload(dst []byte, raw []byte, rows int, bytesPerRow int) ([]byte, error)

func DecodeRawFloat32Payload

func DecodeRawFloat32Payload(dst []float32, raw []byte, rows int) ([]float32, error)

func DecodeRawFloat32VectorPayload

func DecodeRawFloat32VectorPayload(dst []float32, raw []byte, rows int, elementsPerRow int) ([]float32, error)

func DecodeRawFloat64Payload

func DecodeRawFloat64Payload(dst []float64, raw []byte, rows int) ([]float64, error)

func DecodeRawInt8Payload

func DecodeRawInt8Payload(dst []int8, raw []byte, rows int) ([]int8, error)

func DecodeRawInt16Payload

func DecodeRawInt16Payload(dst []int16, raw []byte, rows int) ([]int16, error)

func DecodeRawInt32Payload

func DecodeRawInt32Payload(dst []int32, raw []byte, rows int) ([]int32, error)

func DecodeRawPackedUintPayload

func DecodeRawPackedUintPayload(dst []byte, raw []byte, rows int, elementsPerRow int, bitsPerElement int) ([]byte, error)

func DecodeRawUint8Payload

func DecodeRawUint8Payload(dst []uint8, raw []byte, rows int) ([]uint8, error)

func DecodeRawUint16Payload

func DecodeRawUint16Payload(dst []uint16, raw []byte, rows int, name string) ([]uint16, error)

func DecodeRawUint32DensePayload

func DecodeRawUint32DensePayload(dst []uint32, raw []byte, rows int, elementsPerRow int) ([]uint32, error)

func DecodeRawUint32OffsetsListOffsetsFallback

func DecodeRawUint32OffsetsListOffsetsFallback(dst []uint64, offsetsRaw []byte, rows int) ([]uint64, error)

DecodeRawUint32OffsetsListOffsetsFallback decodes only the little-endian uint64 offsets substream. It validates rows+1 shape, offsets[0]==0, monotonicity, and host-int bounds, but intentionally does not verify the final offset against values bytes.

func DecodeRawUint32Payload

func DecodeRawUint32Payload(dst []uint32, raw []byte, rows int, name string) ([]uint32, error)

func DecodeRawUint64Payload

func DecodeRawUint64Payload(dst []uint64, raw []byte, rows int) ([]uint64, error)

func DecodeTCS1ColumnPartImage

func DecodeTCS1ColumnPartImage(data []byte) (ColumnPartImage, TCS1PartRecord, error)

func DenseFixedWidthEncodingElementWidth

func DenseFixedWidthEncodingElementWidth(encoding Encoding) (int, bool)

func DenseFixedWidthVectorElementWidth

func DenseFixedWidthVectorElementWidth(t ColumnType) (int, bool)

func DenseFloat32Dot

func DenseFloat32Dot(left, right []float32) float32

DenseFloat32Dot returns the float32 dot product for dense typed-column vector rows. The hot kernel only accepts already validated []float32 values; callers that start from column-section bytes should first use mappedresource.Float32View (or another checked decode path) to validate length, alignment, and endian preconditions before slicing rows.

func DenseFloat32DotImplementation

func DenseFloat32DotImplementation() string

DenseFloat32DotImplementation identifies the active optimized dot kernel.

func DenseFloat32DotScalar

func DenseFloat32DotScalar(left, right []float32) float32

DenseFloat32DotScalar is the portable scalar baseline for dense typed-column vector rows.

func EncodePackedUintRows

func EncodePackedUintRows(dst []byte, rows int, elementsPerRow int, bitsPerElement int, values []uint8) ([]byte, error)

EncodePackedUintRows packs flattened row-major element values into bytes. Bits per element must be 1, 2, or 4. Padding bits are written as zero.

func EncodeRawBytesOffsetsOffsets

func EncodeRawBytesOffsetsOffsets(dst []byte, offsets []uint64) ([]byte, error)

EncodeRawBytesOffsetsOffsets writes offsets as little-endian uint64 values. The returned slice may alias dst.

func EncodeRawBytesOffsetsPayload

func EncodeRawBytesOffsetsPayload(dst []byte, rows int, offsets []uint64, values []byte) ([]byte, error)

EncodeRawBytesOffsetsPayload writes the canonical fallback block payload: offsets bytes followed by value bytes. Image writers split this combined block payload into independent offsets and values sections.

func EncodeRawBytesOffsetsValues

func EncodeRawBytesOffsetsValues(dst []byte, values []byte) ([]byte, error)

EncodeRawBytesOffsetsValues writes value bytes exactly as supplied. The returned slice may alias dst but never interprets bytes as text.

func EncodeRawUint32OffsetsListOffsets

func EncodeRawUint32OffsetsListOffsets(dst []byte, offsets []uint64) ([]byte, error)

EncodeRawUint32OffsetsListOffsets writes offsets as little-endian uint64 values. The returned slice may alias dst.

func EncodeRawUint32OffsetsListPayload

func EncodeRawUint32OffsetsListPayload(dst []byte, rows int, offsets []uint64, values []uint32) ([]byte, error)

EncodeRawUint32OffsetsListPayload writes the canonical fallback block payload: offsets bytes followed by values bytes. Image writers split this combined block payload into independent offsets and values sections.

func EncodeRawUint32OffsetsListValues

func EncodeRawUint32OffsetsListValues(dst []byte, values []uint32) ([]byte, error)

EncodeRawUint32OffsetsListValues writes values as little-endian uint32 values. The returned slice may alias dst.

func EstimateBatchUncompressedBytes

func EstimateBatchUncompressedBytes(batch Batch, defs []ColumnDefinition) (int, error)

func FixedBytesPayloadBytes

func FixedBytesPayloadBytes(rows int, bytesPerRow int) (int, error)

FixedBytesPayloadBytes returns rows*bytesPerRow after shape/overflow checks.

func IsDenseFixedWidthVectorColumnType

func IsDenseFixedWidthVectorColumnType(t ColumnType) bool

func IsGenericDenseFixedWidthVectorColumnType

func IsGenericDenseFixedWidthVectorColumnType(t ColumnType) bool

func IsPackedUintVectorColumnType

func IsPackedUintVectorColumnType(t ColumnType) bool

func NewRawBytesOffsetsImageSections

func NewRawBytesOffsetsImageSections(column string, rows int, offsetsBytes int, valuesBytes int) (ColumnPartImageSection, ColumnPartImageSection, error)

NewRawBytesOffsetsImageSections returns image-directory metadata for the split offsets and values sections. Offsets length must be (rows+1)*8. Values bytes may be any length, including zero.

func NewRawUint32OffsetsListImageSections

func NewRawUint32OffsetsListImageSections(column string, rows int, offsetsBytes int, valuesBytes int) (ColumnPartImageSection, ColumnPartImageSection, error)

NewRawUint32OffsetsListImageSections returns image-directory metadata for the split offsets and values sections. Offsets and values byte lengths are checked for exact element sizing; offsets length must be (rows+1)*8.

func PackUintRow

func PackUintRow(dst []byte, values []uint8, bitsPerElement int) ([]byte, error)

PackUintRow packs one row into dst. The returned slice may alias dst.

func PackedUintEncodingBits

func PackedUintEncodingBits(encoding Encoding) (int, bool)

func PackedUintPayloadBytes

func PackedUintPayloadBytes(rows int, elementsPerRow int, bitsPerElement int) (int, error)

PackedUintPayloadBytes returns the physical bytes for a packed payload.

func PackedUintRowBytes

func PackedUintRowBytes(elementsPerRow int, bitsPerElement int) (int, error)

PackedUintRowBytes returns ceil(elementsPerRow*bitsPerElement/8).

func PackedUintRowWords

func PackedUintRowWords(elementsPerRow int, bitsPerElement int) (int, error)

PackedUintRowWords returns ceil(row_bytes/8), the scratch-word count for one row.

func PackedUintVectorBits

func PackedUintVectorBits(t ColumnType) (int, bool)

func RangeScanCount

func RangeScanCount(g EncodedGranule, low, high int64, scratch []int64) (int, []int64, error)

func RawBytesOffsetsBlockPayloadBytes

func RawBytesOffsetsBlockPayloadBytes(rows int, rawBytes int) (offsetsBytes int, valuesBytes int, err error)

RawBytesOffsetsBlockPayloadBytes returns the split byte lengths for a combined block payload. rawBytes must be exactly offsets bytes plus arbitrary value bytes.

func RawUint32OffsetsListBlockPayloadBytes

func RawUint32OffsetsListBlockPayloadBytes(rows int, rawBytes int) (offsetsBytes int, valuesBytes int, err error)

RawUint32OffsetsListBlockPayloadBytes returns the split byte lengths for a combined block payload. rawBytes must be exactly offsets bytes plus a uint32 values byte multiple.

func UnpackUintRow

func UnpackUintRow(dst []uint8, row []byte, elementsPerRow int, bitsPerElement int) ([]uint8, error)

UnpackUintRow unpacks one row into dst after validating row shape and padding. The returned slice may alias dst.

func ValidateColumnPartPruning

func ValidateColumnPartPruning(pruning ColumnPartPruning, desc ColumnPartDescriptor, columns map[string]ColumnPartColumn) error

func ValidateColumnPartSortKeyMarks

func ValidateColumnPartSortKeyMarks(desc ColumnPartDescriptor, marks []SortKeyMark) error

func ValidateColumnPartStats

func ValidateColumnPartStats(stats ColumnPartStats, desc ColumnPartDescriptor, columns map[string]ColumnPartColumn) error

func ValidateInt64ColumnStats

func ValidateInt64ColumnStats(stats Int64ColumnStats, desc ColumnPartDescriptor, columnDesc ColumnPartColumnDescriptor, column ColumnPartColumn) error

func ValidateInt64ValueRowIndex

func ValidateInt64ValueRowIndex(index Int64ValueRowIndex, desc ColumnPartDescriptor, columnDesc ColumnPartColumnDescriptor, column ColumnPartColumn) error

func ValidatePackedUintRowPadding

func ValidatePackedUintRowPadding(row []byte, elementsPerRow int, bitsPerElement int) error

ValidatePackedUintRowPadding validates zero unused high bits in the row's final physical byte.

func ValidateRawBytesOffsetsOffsets

func ValidateRawBytesOffsetsOffsets(rows int, offsets []uint64) error

ValidateRawBytesOffsetsOffsets validates the length-only offsets substream shape without requiring the values substream. It is intended for row-length APIs and offset-only integrity checks; full value integrity still requires ValidateRawBytesOffsetsShape or section validation.

func ValidateRawBytesOffsetsOffsetsBytes

func ValidateRawBytesOffsetsOffsetsBytes(offsetsRaw []byte, rows int) (int, error)

ValidateRawBytesOffsetsOffsetsBytes validates only the byte length of the offsets substream and returns its uint64 element count.

func ValidateRawBytesOffsetsSections

func ValidateRawBytesOffsetsSections(offsetsSection ColumnPartImageSection, valuesSection ColumnPartImageSection, offsetsRaw []byte, valuesRaw []byte, rows int) error

ValidateRawBytesOffsetsSections validates split image section metadata and payload shape for raw_bytes_offsets without exposing unsafe direct views. It fails closed on any metadata or byte-shape mismatch.

func ValidateRawBytesOffsetsShape

func ValidateRawBytesOffsetsShape(rows int, offsets []uint64, values uint64) error

ValidateRawBytesOffsetsShape validates owned offset/value counts before encoding or row slicing. values is the number of payload bytes.

func ValidateRawUint32OffsetsListOffsets

func ValidateRawUint32OffsetsListOffsets(rows int, offsets []uint64) error

ValidateRawUint32OffsetsListOffsets validates the length-only offsets substream shape without requiring the values substream. It is intended for row-length APIs and offset-only integrity checks; full value integrity still requires ValidateRawUint32OffsetsListShape or section validation.

func ValidateRawUint32OffsetsListOffsetsBytes

func ValidateRawUint32OffsetsListOffsetsBytes(offsetsRaw []byte, rows int) (int, error)

ValidateRawUint32OffsetsListOffsetsBytes validates only the byte length of the offsets substream and returns its uint64 element count.

func ValidateRawUint32OffsetsListSections

func ValidateRawUint32OffsetsListSections(offsetsSection ColumnPartImageSection, valuesSection ColumnPartImageSection, offsetsRaw []byte, valuesRaw []byte, rows int) error

ValidateRawUint32OffsetsListSections validates split image section metadata and payload shape for raw_uint32_offsets_list without exposing unsafe direct views. It fails closed on any metadata or byte-shape mismatch.

func ValidateRawUint32OffsetsListShape

func ValidateRawUint32OffsetsListShape(rows int, offsets []uint64, values uint64) error

ValidateRawUint32OffsetsListShape validates owned offset/value counts before encoding or row slicing. values is the number of uint32 values, not bytes.

Types

type AggregateArena

type AggregateArena struct {
	// contains filtered or unexported fields
}

func (*AggregateArena) ExactDistinctCodes

func (a *AggregateArena) ExactDistinctCodes(granules []EncodedGranule, cardinality uint32) (int, error)

func (*AggregateArena) ExactDistinctCodesSelected

func (a *AggregateArena) ExactDistinctCodesSelected(granules []EncodedGranule, selections []RowSelection, cardinality uint32) (int, error)

ExactDistinctCodesSelected returns an exact distinct code count over one selection per granule without materializing strings or full code slices.

func (*AggregateArena) ExactDistinctInt64

func (a *AggregateArena) ExactDistinctInt64(granules []EncodedGranule) (int, error)

func (*AggregateArena) FilteredGroupedCountCodes

func (a *AggregateArena) FilteredGroupedCountCodes(codeGranules []EncodedGranule, filterGranules []EncodedGranule, filter Int64RangePredicate, cardinality uint32) ([]uint64, PredicateDiagnostics, error)

FilteredGroupedCountCodes returns arena-owned count storage. The returned slice is valid only until the next AggregateArena operation or Reset.

func (*AggregateArena) GroupedCountCodes

func (a *AggregateArena) GroupedCountCodes(granules []EncodedGranule, cardinality uint32) ([]uint64, error)

GroupedCountCodes returns arena-owned count storage. The returned slice is valid only until the next AggregateArena operation or Reset.

func (*AggregateArena) GroupedCountCodesSelected

func (a *AggregateArena) GroupedCountCodesSelected(granules []EncodedGranule, selections []RowSelection, cardinality uint32) ([]uint64, error)

GroupedCountCodesSelected returns arena-owned count storage for one selection per granule. It is the dictionary-code reducer substrate for callers that already resolved visibility/category predicates into RowSelection values. The returned slice is valid only until the next AggregateArena operation or Reset.

func (*AggregateArena) MinMaxInt64

func (a *AggregateArena) MinMaxInt64(granules []EncodedGranule, filter *Int64RangePredicate) (int64, int64, bool, error)

func (*AggregateArena) Reset

func (a *AggregateArena) Reset()

func (*AggregateArena) TimeBucketedCountCodes

func (a *AggregateArena) TimeBucketedCountCodes(codeGranules []EncodedGranule, timeGranules []EncodedGranule, bucketWidth int64, cardinality uint32) (TimeBucketedCounts, error)

TimeBucketedCountCodes returns a result whose Counts slice is arena-owned. Counts is valid only until the next AggregateArena operation or Reset.

type AggregateMetadata

type AggregateMetadata struct {
	Definition AggregateMetadataDefinition `json:"definition"`
	Granules   []AggregateMetadataGranule  `json:"granules,omitempty"`
	Stats      AggregateMetadataStats      `json:"stats"`
}

type AggregateMetadataDefinition

type AggregateMetadataDefinition struct {
	Name           string                       `json:"name"`
	Version        uint16                       `json:"version"`
	Kind           AggregateMetadataKind        `json:"kind"`
	Scope          AggregateMetadataScope       `json:"scope"`
	GroupKeys      []string                     `json:"group_keys"`
	Measures       []AggregateMetadataMeasure   `json:"measures"`
	Predicates     []AggregateMetadataPredicate `json:"predicates,omitempty"`
	MaxBytesPerRow float64                      `json:"max_bytes_per_row"`
}

type AggregateMetadataEntry

type AggregateMetadataEntry struct {
	Group uint32 `json:"group"`
	Count uint32 `json:"count"`
	Min   int64  `json:"min"`
	Max   int64  `json:"max"`
}

type AggregateMetadataGranule

type AggregateMetadataGranule struct {
	GranuleOrdinal int                      `json:"granule_ordinal"`
	FirstRow       int                      `json:"first_row"`
	RowCount       int                      `json:"row_count"`
	MatchedRows    int                      `json:"matched_rows"`
	Entries        []AggregateMetadataEntry `json:"entries,omitempty"`
}

type AggregateMetadataKind

type AggregateMetadataKind string
const (
	AggregateMetadataGroupMinMax AggregateMetadataKind = "group_min_max"
)

type AggregateMetadataMeasure

type AggregateMetadataMeasure struct {
	Op     AggregateMetadataMeasureOp `json:"op"`
	Column string                     `json:"column,omitempty"`
}

type AggregateMetadataMeasureOp

type AggregateMetadataMeasureOp string
const (
	AggregateMetadataMeasureCount AggregateMetadataMeasureOp = "count"
	AggregateMetadataMeasureMin   AggregateMetadataMeasureOp = "min"
	AggregateMetadataMeasureMax   AggregateMetadataMeasureOp = "max"
)

type AggregateMetadataPredicate

type AggregateMetadataPredicate struct {
	Column string                       `json:"column"`
	Op     AggregateMetadataPredicateOp `json:"op"`
	Value  int64                        `json:"value"`
}

type AggregateMetadataPredicateOp

type AggregateMetadataPredicateOp string
const (
	AggregateMetadataPredicateEq AggregateMetadataPredicateOp = "eq"
)

type AggregateMetadataScope

type AggregateMetadataScope string
const (
	AggregateMetadataScopeGranule AggregateMetadataScope = "granule"
)

type AggregateMetadataStats

type AggregateMetadataStats struct {
	Admitted            bool          `json:"admitted"`
	RejectedReason      string        `json:"rejected_reason,omitempty"`
	BuildDuration       time.Duration `json:"build_duration"`
	Granules            int           `json:"granules"`
	GranulesWithRows    int           `json:"granules_with_rows"`
	RowsMatched         int           `json:"rows_matched"`
	Entries             int           `json:"entries"`
	ValueBytes          int           `json:"value_bytes"`
	DescriptorBytes     int           `json:"estimated_descriptor_bytes"`
	TotalBytes          int           `json:"estimated_total_bytes"`
	BytesPerPartRow     float64       `json:"bytes_per_part_row"`
	BytesPerMatchedRow  float64       `json:"bytes_per_matched_row"`
	Compression         string        `json:"compression"`
	AdmissionMaxBytes   float64       `json:"admission_max_bytes_per_row"`
	AdmissionMeasuredBy string        `json:"admission_measured_by"`
}

type Batch

type Batch struct {
	Rows                   int
	Columns                map[string][]int64
	Nulls                  map[string][]bool
	Defaults               map[string][]bool
	DefaultValues          map[string]int64
	Float32Columns         map[string][]float32
	Float64Columns         map[string][]float64
	Int8Columns            map[string][]int8
	Uint8Columns           map[string][]uint8
	Int16Columns           map[string][]int16
	Uint16Columns          map[string][]uint16
	Int32Columns           map[string][]int32
	Uint32Columns          map[string][]uint32
	Uint64Columns          map[string][]uint64
	Float16Columns         map[string][]uint16
	BFloat16Columns        map[string][]uint16
	Float32Vectors         map[string][]float32
	DenseFixedWidthVectors map[string]RawDenseFixedWidth
	FixedBytesColumns      map[string]FixedBytesRows
	PackedUintColumns      map[string]PackedUintRows
	Uint32Vectors          map[string][]uint32
	Uint32OffsetsLists     map[string]RawUint32OffsetsList
	BytesColumns           map[string]RawBytesOffsets
}

type BoolSelectionScratch

type BoolSelectionScratch struct {
	Rows   []int
	Ranges []RowRange
	Bitmap []uint64
}

BoolSelectionScratch owns caller/session-scoped temporary storage for bool predicate selections. Returned selections from SelectBool may alias this scratch until the next use.

type BytesColumn

type BytesColumn struct {
	Rows    int
	Offsets []uint64
	Values  []byte
}

BytesColumn is the generic owned representation for the v1 non-null opaque bytes primitive. It is consumer-neutral storage machinery: every row owns one exact byte string, and callers decide higher-level meaning above this layer. Offsets has Rows+1 entries and Values stores all row payload bytes concatenated; row i spans Values[Offsets[i]:Offsets[i+1]].

func (BytesColumn) Row

func (b BytesColumn) Row(row int) ([]byte, error)

Row returns the row slice Values[Offsets[row]:Offsets[row+1]] after validating row bounds and host-int conversions. The returned slice aliases Values.

func (BytesColumn) Validate

func (b BytesColumn) Validate() error

Validate checks the complete offsets/value shape for this bytes column.

type BytesColumnView

type BytesColumnView = BytesColumn

BytesColumnView is the same row-slicing contract used by direct views: Offsets and Values may alias a mmap/heap resource whose lifetime is owned elsewhere.

type CodecReport

type CodecReport struct {
	Encoding                  Encoding
	RequestedCompression      Compression
	ActualCompression         Compression
	CompressionAttempted      bool
	CompressionKept           bool
	CompressionFallbackReason string
	RawBytes                  int
	StoredBytes               int
	CompressionNanos          int64
}

type ColumnAdaptiveMarkEstimate

type ColumnAdaptiveMarkEstimate struct {
	Rows           int     `json:"rows"`
	RawBytes       int     `json:"raw_bytes"`
	RawBytesPerRow float64 `json:"raw_bytes_per_row"`
	TargetBytes    int     `json:"target_bytes"`
	MinRows        int     `json:"min_rows"`
	MaxRows        int     `json:"max_rows"`
	RowsPerMark    int     `json:"rows_per_mark"`
	Marks          int     `json:"marks"`
	ClampedByMin   bool    `json:"clamped_by_min,omitempty"`
	ClampedByMax   bool    `json:"clamped_by_max,omitempty"`
}

func EstimateAdaptiveRowsPerMark

func EstimateAdaptiveRowsPerMark(rows int, rawBytes int, cfg ColumnAdaptiveMarkSizing) (ColumnAdaptiveMarkEstimate, error)

type ColumnAdaptiveMarkSizing

type ColumnAdaptiveMarkSizing struct {
	Enabled     bool `json:"enabled,omitempty"`
	TargetBytes int  `json:"target_bytes,omitempty"`
	MinRows     int  `json:"min_rows,omitempty"`
	MaxRows     int  `json:"max_rows,omitempty"`
}

func NormalizeColumnAdaptiveMarkSizing

func NormalizeColumnAdaptiveMarkSizing(cfg ColumnAdaptiveMarkSizing, rowsPerGranule int) (ColumnAdaptiveMarkSizing, error)

type ColumnBlock

type ColumnBlock struct {
	Descriptor ColumnBlockDescriptor
	Granule    EncodedGranule
}

type ColumnBlockDescriptor

type ColumnBlockDescriptor struct {
	FirstRow          int
	RowCount          int
	FirstGranule      int
	LastGranule       int
	Encoding          Encoding
	Compression       Compression
	RawBytes          int
	StoredBytes       int
	CodecBlockOrdinal int
}

type ColumnCompressionPolicy

type ColumnCompressionPolicy struct {
	Default Compression
}

type ColumnDefinition

type ColumnDefinition struct {
	Name               string
	Type               ColumnType
	Encoding           Encoding
	Compression        Compression
	CompressionSet     bool
	Cardinality        uint32
	FixedWidthElements int
	BitsPerElement     int
	CodecBlockRows     int
	StatsDisabled      bool
}

type ColumnExecutionRole

type ColumnExecutionRole uint8

ColumnExecutionRole classifies how a column participates in a future multi-column execution plan. It is descriptive; it is not a planner.

const (
	ColumnRolePredicate  ColumnExecutionRole = sectionDependencyPredicate
	ColumnRoleMeasure    ColumnExecutionRole = sectionDependencyMeasure
	ColumnRoleProjection ColumnExecutionRole = sectionDependencyProjection
	ColumnRoleVisibility ColumnExecutionRole = sectionDependencyVisibility
	ColumnRoleNull       ColumnExecutionRole = sectionDependencyNull
	ColumnRoleDefault    ColumnExecutionRole = sectionDependencyDefault
)

func (ColumnExecutionRole) String

func (r ColumnExecutionRole) String() string

type ColumnPart

type ColumnPart struct {
	Options           Options
	Descriptor        ColumnPartDescriptor
	Columns           map[string]ColumnPartColumn
	Marks             []SortKeyMark
	Locators          map[int64]RowLocator
	AggregateMetadata map[string]AggregateMetadata
	ColumnStats       ColumnPartStats
	PruningMetadata   ColumnPartPruning
}

func BuildColumnPart

func BuildColumnPart(partID uint64, opts Options, batch Batch) (*ColumnPart, error)

func ColumnPartFromImage

func ColumnPartFromImage(image ColumnPartImage) (*ColumnPart, error)

func ColumnPartFromImageWithOptions

func ColumnPartFromImageWithOptions(image ColumnPartImage, opts ColumnPartImageReadOptions) (*ColumnPart, error)

func (*ColumnPart) AggregateMetadataByName

func (p *ColumnPart) AggregateMetadataByName(name string) (AggregateMetadata, bool)

func (*ColumnPart) ByteAccounting

func (p *ColumnPart) ByteAccounting() ColumnPartByteAccounting

func (*ColumnPart) ByteAccountingFromImage

func (p *ColumnPart) ByteAccountingFromImage(image ColumnPartImage) ColumnPartByteAccounting

func (*ColumnPart) BytesColumn

func (p *ColumnPart) BytesColumn(name string, offsetsDst []uint64, valuesDst []byte) (BytesColumn, error)

func (*ColumnPart) DenseFixedWidthColumn

func (p *ColumnPart) DenseFixedWidthColumn(name string, dst []byte) (DenseFixedWidthColumn, error)

func (*ColumnPart) DenseFloat32VectorColumn

func (p *ColumnPart) DenseFloat32VectorColumn(name string, dst []float32) (DenseFloat32Column, error)

func (*ColumnPart) DenseUint32Column

func (p *ColumnPart) DenseUint32Column(name string, dst []uint32) (DenseUint32Column, error)

func (*ColumnPart) EncodedColumnBlocks

func (p *ColumnPart) EncodedColumnBlocks(name string) ([]EncodedGranule, error)

func (*ColumnPart) FixedBytesColumn

func (p *ColumnPart) FixedBytesColumn(name string, dst []byte) (FixedBytesRows, error)

func (*ColumnPart) LocatePrimaryID

func (p *ColumnPart) LocatePrimaryID(primaryID int64) (RowLocator, bool)

func (*ColumnPart) NewScanner

func (p *ColumnPart) NewScanner() *ColumnPartScanner

func (*ColumnPart) PackedUintColumn

func (p *ColumnPart) PackedUintColumn(name string, dst []byte) (PackedUintRows, error)

func (*ColumnPart) Uint32ListColumn

func (p *ColumnPart) Uint32ListColumn(name string, offsetsDst []uint64, valuesDst []uint32) (Uint32List, error)

func (*ColumnPart) Uint32OffsetsListColumn

func (p *ColumnPart) Uint32OffsetsListColumn(name string, offsetsDst []uint64, valuesDst []uint32) (RawUint32OffsetsList, error)

Uint32OffsetsListColumn is the physical-encoding compatibility reader. It accepts the generic uint32_list type and the legacy adjacency_list offsets-list selector while new callers should prefer Uint32ListColumn for generic data.

func (*ColumnPart) WithImagePayloads

func (p *ColumnPart) WithImagePayloads(image ColumnPartImage) (*ColumnPart, error)

type ColumnPartBuilder

type ColumnPartBuilder struct {
	// contains filtered or unexported fields
}

func NewColumnPartBuilder

func NewColumnPartBuilder(opts Options) (*ColumnPartBuilder, error)

func (*ColumnPartBuilder) Build

func (b *ColumnPartBuilder) Build(partID uint64, batch Batch) (*ColumnPart, error)

type ColumnPartByteAccounting

type ColumnPartByteAccounting struct {
	Rows                       int                                    `json:"rows"`
	Columns                    int                                    `json:"columns"`
	Granules                   int                                    `json:"granules"`
	CodecBlocks                int                                    `json:"codec_blocks"`
	PhysicalFiles              int                                    `json:"physical_files"`
	SerializedImageBytes       int                                    `json:"serialized_image_bytes,omitempty"`
	SerializedManifestBytes    int                                    `json:"serialized_manifest_bytes,omitempty"`
	SerializedPaddingBytes     int                                    `json:"serialized_padding_bytes,omitempty"`
	LogicalValueBytes          int                                    `json:"logical_value_bytes"`
	EncodedRawBytes            int                                    `json:"encoded_raw_bytes"`
	DeclaredColumnBytes        int                                    `json:"declared_column_bytes"`
	DeclaredColumnOffsetsBytes int                                    `json:"declared_column_offsets_bytes,omitempty"`
	DeclaredColumnValuesBytes  int                                    `json:"declared_column_values_bytes,omitempty"`
	DictionaryBytes            int                                    `json:"dictionary_bytes"`
	MarkBytes                  int                                    `json:"mark_bytes"`
	SortKeyMetadataBytes       int                                    `json:"sort_key_metadata_bytes"`
	AggregateMetadataBytes     int                                    `json:"aggregate_metadata_bytes"`
	ColumnStatsBytes           int                                    `json:"column_stats_bytes"`
	PruningMetadataBytes       int                                    `json:"pruning_metadata_bytes"`
	DescriptorBytes            int                                    `json:"descriptor_bytes"`
	LayoutContractBytes        int                                    `json:"layout_contract_bytes"`
	LocatorBytes               int                                    `json:"locator_bytes"`
	TotalStoredBytes           int                                    `json:"total_stored_bytes"`
	BytesPerRow                float64                                `json:"bytes_per_row"`
	RetainedJSONPayload        string                                 `json:"retained_json_payload"`
	ColumnsDetail              []ColumnPartColumnByteAccounting       `json:"columns_detail"`
	CompressionDetail          []ColumnPartCompressionByteAccounting  `json:"compression_detail"`
	SerializedSections         []ColumnPartImageSectionByteAccounting `json:"serialized_sections,omitempty"`
}

func (ColumnPartByteAccounting) CategoryBytes

func (a ColumnPartByteAccounting) CategoryBytes() int

func (*ColumnPartByteAccounting) RecomputeTotals

func (a *ColumnPartByteAccounting) RecomputeTotals()

type ColumnPartColumn

type ColumnPartColumn struct {
	Definition ColumnDefinition
	Blocks     []ColumnBlock
}

type ColumnPartColumnByteAccounting

type ColumnPartColumnByteAccounting struct {
	Column               string         `json:"column"`
	Type                 ColumnType     `json:"type"`
	Rows                 int            `json:"rows"`
	Blocks               int            `json:"blocks"`
	LogicalValueBytes    int            `json:"logical_value_bytes"`
	EncodedRawBytes      int            `json:"encoded_raw_bytes"`
	StoredBytes          int            `json:"stored_bytes"`
	Encoding             Encoding       `json:"encoding"`
	RequestedCompression Compression    `json:"requested_compression"`
	ActualCompressionMix map[string]int `json:"actual_compression_mix"`
	CompressionAttempted int            `json:"compression_attempted"`
	CompressionKept      int            `json:"compression_kept"`
	CompressionRejected  int            `json:"compression_rejected"`
	FallbackReasons      map[string]int `json:"fallback_reasons,omitempty"`
	CompressionNanos     int64          `json:"compression_nanos"`
}

type ColumnPartColumnDescriptor

type ColumnPartColumnDescriptor struct {
	Name               string
	Type               ColumnType
	FixedWidthElements int
	BitsPerElement     int
	Blocks             []ColumnBlockDescriptor
}

type ColumnPartCompressionByteAccounting

type ColumnPartCompressionByteAccounting struct {
	Column                 string      `json:"column"`
	Substream              string      `json:"substream"`
	Encoding               Encoding    `json:"encoding"`
	RequestedCompression   Compression `json:"requested_compression"`
	ActualCompression      Compression `json:"actual_compression"`
	Blocks                 int         `json:"blocks"`
	CompressionAttempted   int         `json:"compression_attempted"`
	CompressionKept        int         `json:"compression_kept"`
	CompressionRejected    int         `json:"compression_rejected"`
	FallbackReason         string      `json:"fallback_reason,omitempty"`
	EncodedRawBytes        int         `json:"encoded_raw_bytes"`
	StoredBytes            int         `json:"stored_bytes"`
	CompressionNanos       int64       `json:"compression_nanos"`
	StoredToEncodedRawRate float64     `json:"stored_to_encoded_raw_rate"`
}

type ColumnPartDescriptor

type ColumnPartDescriptor struct {
	Version           uint8
	PartID            uint64
	SchemaVersion     uint32
	RowCount          int
	VisibleRowCount   int
	LogicalPrimaryKey []string
	SortKey           []SortKeyColumn
	Granules          []GranuleDescriptor
	Columns           []ColumnPartColumnDescriptor
}

func DecodeColumnPartDescriptorSection

func DecodeColumnPartDescriptorSection(data []byte) (ColumnPartDescriptor, map[string]ColumnPartColumn, error)

type ColumnPartImage

type ColumnPartImage struct {
	Version       uint16                   `json:"version"`
	PartID        uint64                   `json:"part_id"`
	Rows          int                      `json:"rows"`
	ManifestBytes int                      `json:"manifest_bytes"`
	Sections      []ColumnPartImageSection `json:"sections"`
	Bytes         []byte                   `json:"-"`
}

func BuildColumnPartImage

func BuildColumnPartImage(part *ColumnPart, opts ColumnPartImageOptions) (ColumnPartImage, error)

func ParseColumnPartImage

func ParseColumnPartImage(data []byte) (ColumnPartImage, error)

func ParseColumnPartImageManifest

func ParseColumnPartImageManifest(data []byte, totalBytes int) (ColumnPartImage, error)

func (ColumnPartImage) CategoryBytes

func (i ColumnPartImage) CategoryBytes(category ColumnPartImageSectionCategory) int

func (ColumnPartImage) ColumnOffsetsListSections

func (i ColumnPartImage) ColumnOffsetsListSections(column string) (ColumnPartImageSection, ColumnPartImageSection, bool)

ColumnOffsetsListSections returns the unique offsets and values sections for the named offsets-list column. The boolean is false when either section is missing or duplicated, so callers fail closed instead of selecting a last matching section from a malformed image.

func (ColumnPartImage) ColumnStatsSection

func (i ColumnPartImage) ColumnStatsSection() (ColumnPartImageSection, bool, error)

func (ColumnPartImage) Dictionaries

func (i ColumnPartImage) Dictionaries() (map[string]map[string]int64, error)

func (ColumnPartImage) LayoutContractSection

func (i ColumnPartImage) LayoutContractSection() (ColumnPartImageSection, error)

func (ColumnPartImage) PaddingBytes

func (i ColumnPartImage) PaddingBytes() int

func (ColumnPartImage) PruningMetadataSection

func (i ColumnPartImage) PruningMetadataSection() (ColumnPartImageSection, bool, error)

func (ColumnPartImage) SectionByteAccounting

func (i ColumnPartImage) SectionByteAccounting() []ColumnPartImageSectionByteAccounting

func (ColumnPartImage) SectionBytes

func (i ColumnPartImage) SectionBytes(section ColumnPartImageSection) ([]byte, error)

func (ColumnPartImage) TotalBytes

func (i ColumnPartImage) TotalBytes() int

type ColumnPartImageOptions

type ColumnPartImageOptions struct {
	Dictionaries        map[string]map[string]int64
	LayoutLogicalTypes  map[string]string
	DictionaryOrder     map[string]bool
	DictionaryCollation map[string]string
	// SectionCompression compresses eligible whole-image sections. It is
	// intentionally limited to sections whose raw length can be recovered from
	// existing manifest fields without a TCIM/TCS1 format change.
	SectionCompression                   Compression
	RowLocatorSectionCompression         Compression
	RowLocatorSectionCompressionSet      bool
	DictionarySectionCompression         Compression
	DictionarySectionCompressionSet      bool
	PruningMetadataSectionCompression    Compression
	PruningMetadataSectionCompressionSet bool
}

type ColumnPartImageReadOptions

type ColumnPartImageReadOptions struct {
	IncludeRowLocators       bool
	ValidateRowLocators      bool
	IncludeAggregateMetadata bool
	IncludeColumnStats       bool
	IncludePruningMetadata   bool
}

type ColumnPartImageSection

type ColumnPartImageSection struct {
	Kind        ColumnPartImageSectionKind     `json:"kind"`
	Category    ColumnPartImageSectionCategory `json:"category"`
	Name        string                         `json:"name,omitempty"`
	Column      string                         `json:"column,omitempty"`
	Offset      int                            `json:"offset"`
	Length      int                            `json:"length"`
	Rows        int                            `json:"rows,omitempty"`
	Granules    int                            `json:"granules,omitempty"`
	Blocks      int                            `json:"blocks,omitempty"`
	Encoding    Encoding                       `json:"encoding,omitempty"`
	Compression Compression                    `json:"compression,omitempty"`
	RawBytes    int                            `json:"raw_bytes,omitempty"`
}

type ColumnPartImageSectionByteAccounting

type ColumnPartImageSectionByteAccounting struct {
	Kind        ColumnPartImageSectionKind     `json:"kind"`
	Category    ColumnPartImageSectionCategory `json:"category"`
	Name        string                         `json:"name,omitempty"`
	Column      string                         `json:"column,omitempty"`
	Bytes       int                            `json:"bytes"`
	Compression Compression                    `json:"compression,omitempty"`
	RawBytes    int                            `json:"raw_bytes,omitempty"`
	StoredBytes int                            `json:"stored_bytes,omitempty"`
}

type ColumnPartImageSectionCategory

type ColumnPartImageSectionCategory string
const (
	ColumnPartImageCategoryManifest              ColumnPartImageSectionCategory = "manifest"
	ColumnPartImageCategoryDescriptor            ColumnPartImageSectionCategory = "descriptor"
	ColumnPartImageCategorySortKeyMetadata       ColumnPartImageSectionCategory = "sort_key_metadata"
	ColumnPartImageCategoryMarks                 ColumnPartImageSectionCategory = "marks"
	ColumnPartImageCategoryLocators              ColumnPartImageSectionCategory = "locators"
	ColumnPartImageCategoryAggregateMetadata     ColumnPartImageSectionCategory = "aggregate_metadata"
	ColumnPartImageCategoryColumnStats           ColumnPartImageSectionCategory = "column_stats"
	ColumnPartImageCategoryPruningMetadata       ColumnPartImageSectionCategory = "pruning_metadata"
	ColumnPartImageCategoryDictionaries          ColumnPartImageSectionCategory = "dictionaries"
	ColumnPartImageCategoryLayoutContract        ColumnPartImageSectionCategory = "layout_contract"
	ColumnPartImageCategoryDeclaredColumns       ColumnPartImageSectionCategory = "declared_columns"
	ColumnPartImageCategoryDeclaredColumnOffsets ColumnPartImageSectionCategory = "declared_column_offsets"
	ColumnPartImageCategoryDeclaredColumnValues  ColumnPartImageSectionCategory = "declared_column_values"
	ColumnPartImageCategoryPadding               ColumnPartImageSectionCategory = "padding"
)

type ColumnPartImageSectionKind

type ColumnPartImageSectionKind string
const (
	ColumnPartImageSectionManifest          ColumnPartImageSectionKind = "manifest"
	ColumnPartImageSectionDescriptor        ColumnPartImageSectionKind = "descriptor"
	ColumnPartImageSectionSortKeyMetadata   ColumnPartImageSectionKind = "sort_key_metadata"
	ColumnPartImageSectionSortKeyMarks      ColumnPartImageSectionKind = "sort_key_marks"
	ColumnPartImageSectionRowLocators       ColumnPartImageSectionKind = "row_locators"
	ColumnPartImageSectionAggregateMetadata ColumnPartImageSectionKind = "aggregate_metadata"
	ColumnPartImageSectionColumnStats       ColumnPartImageSectionKind = "column_stats"
	ColumnPartImageSectionPruningMetadata   ColumnPartImageSectionKind = "pruning_metadata"
	ColumnPartImageSectionDictionaries      ColumnPartImageSectionKind = "dictionaries"
	ColumnPartImageSectionLayoutContract    ColumnPartImageSectionKind = "layout_contract"
	ColumnPartImageSectionColumnData        ColumnPartImageSectionKind = "column_data"
	ColumnPartImageSectionColumnOffsets     ColumnPartImageSectionKind = "column_offsets"
	ColumnPartImageSectionColumnValues      ColumnPartImageSectionKind = "column_values"
	ColumnPartImageSectionPadding           ColumnPartImageSectionKind = "padding"
)

type ColumnPartLayoutCertification

type ColumnPartLayoutCertification struct {
	Contract              ColumnPartLayoutContract
	DirectViewCertified   int
	StreamingCertified    int
	StatsCertified        int
	PruningCertified      int
	CertificationFallback []string
}

func CertifyColumnPartLayoutContract

func CertifyColumnPartLayoutContract(image ColumnPartImage, desc ColumnPartDescriptor, columns map[string]ColumnPartColumn, descriptorRaw []byte, contractRaw []byte) (ColumnPartLayoutCertification, error)

func CertifyColumnPartLayoutContractFromImage

func CertifyColumnPartLayoutContractFromImage(image ColumnPartImage) (ColumnPartLayoutCertification, error)

func (ColumnPartLayoutCertification) Column

type ColumnPartLayoutContract

type ColumnPartLayoutContract struct {
	Version       uint16
	PartID        uint64
	Rows          int
	ImageVersion  uint16
	ManifestBytes int
	Descriptor    ColumnPartLayoutContractSection
	Columns       []ColumnPartLayoutContractColumn
}

func DecodeColumnPartLayoutContract

func DecodeColumnPartLayoutContract(data []byte) (ColumnPartLayoutContract, error)

func (ColumnPartLayoutContract) Column

type ColumnPartLayoutContractBlock

type ColumnPartLayoutContractBlock struct {
	FirstRow      int
	RowCount      int
	FirstGranule  int
	LastGranule   int
	Encoding      Encoding
	Compression   Compression
	RawBytes      int
	StoredBytes   int
	PayloadOffset int
	PayloadLength int
	NullCount     int
	DefaultCount  int
}

type ColumnPartLayoutContractColumn

type ColumnPartLayoutContractColumn struct {
	Name                string
	LogicalType         string
	Type                ColumnType
	Encoding            Encoding
	Compression         Compression
	Rows                int
	Section             ColumnPartLayoutContractSection
	OffsetsSection      ColumnPartLayoutContractSection
	ValuesSection       ColumnPartLayoutContractSection
	OffsetsBytes        int
	ValuesBytes         int
	FixedWidthElements  int
	BitsPerElement      int
	BytesPerRow         int
	LogicalBitsPerRow   int
	ElementSize         int
	Alignment           int
	Endian              ColumnPartLayoutEndian
	LengthMultiple      int
	DirectViewCertified bool
	StreamingCertified  bool
	StatsCertified      bool
	PruningCertified    bool
	Dictionary          bool
	DictionaryOrdered   bool
	DictionaryCollation string
	DictionarySection   ColumnPartLayoutContractSection
	NullMaskPresent     bool
	DefaultMaskPresent  bool
	NullCount           int
	DefaultCount        int
	Blocks              []ColumnPartLayoutContractBlock
}

type ColumnPartLayoutContractSection

type ColumnPartLayoutContractSection struct {
	Offset   int
	Length   int
	Checksum uint32
}

type ColumnPartLayoutEndian

type ColumnPartLayoutEndian uint16

ColumnPartLayoutEndian records the byte order guaranteed by the writer for a certified physical section. Codec-defined variable-width encodings do not expose fixed-width endian-sensitive values to hot direct-view paths.

const (
	ColumnPartLayoutEndianNone ColumnPartLayoutEndian = iota
	ColumnPartLayoutEndianLittle
	ColumnPartLayoutEndianCodecDefined
)

func (ColumnPartLayoutEndian) String

func (e ColumnPartLayoutEndian) String() string

type ColumnPartPolicy

type ColumnPartPolicy struct {
	RowsPerGranule        int
	DefaultCodecBlockRows int
	AdaptiveMarkSizing    ColumnAdaptiveMarkSizing
}

type ColumnPartPruning

type ColumnPartPruning struct {
	Version uint16
	PartID  uint64
	Rows    int
	Int64   map[string]Int64ValueRowIndex
}

func DecodeColumnPartPruningImageSection

func DecodeColumnPartPruningImageSection(section ColumnPartImageSection, payload []byte) (ColumnPartPruning, error)

func DecodeColumnPartPruningSection

func DecodeColumnPartPruningSection(data []byte) (ColumnPartPruning, error)

func (ColumnPartPruning) Empty

func (p ColumnPartPruning) Empty() bool

func (ColumnPartPruning) Int64Column

func (p ColumnPartPruning) Int64Column(name string) (Int64ValueRowIndex, bool)

type ColumnPartScanner

type ColumnPartScanner struct {
	// contains filtered or unexported fields
}

func (*ColumnPartScanner) ScanProjected

func (s *ColumnPartScanner) ScanProjected(columns []string) (ProjectedScanResult, error)

func (*ColumnPartScanner) ScanProjectedInto

func (s *ColumnPartScanner) ScanProjectedInto(dst map[string][]int64, columns []string) (ProjectedScanResult, error)

func (*ColumnPartScanner) ScanProjectedRows

func (s *ColumnPartScanner) ScanProjectedRows(columns []string, rows []int) (ProjectedScanResult, error)

func (*ColumnPartScanner) ScanProjectedRowsInto

func (s *ColumnPartScanner) ScanProjectedRowsInto(dst map[string][]int64, columns []string, rows []int) (ProjectedScanResult, error)

func (*ColumnPartScanner) ValueAt

func (s *ColumnPartScanner) ValueAt(locator RowLocator, columnName string) (int64, error)

type ColumnPartStats

type ColumnPartStats struct {
	Version uint16
	PartID  uint64
	Rows    int
	Int64   map[string]Int64ColumnStats
}

func DecodeColumnPartStatsSection

func DecodeColumnPartStatsSection(data []byte) (ColumnPartStats, error)

func (ColumnPartStats) Empty

func (s ColumnPartStats) Empty() bool

func (ColumnPartStats) Int64Column

func (s ColumnPartStats) Int64Column(name string) (Int64ColumnStats, bool)

type ColumnPruningBlockCandidate

type ColumnPruningBlockCandidate struct {
	BlockIndex     int
	FirstRow       int
	RowCount       int
	Selection      RowSelection
	NeedsPredicate bool
	Exact          bool
	ExactCount     int64
	ExactSum       int64
}

type ColumnPruningCandidatePlan

type ColumnPruningCandidatePlan struct {
	Rows              int
	Blocks            []ColumnPruningBlockCandidate
	CandidateRows     int
	PrunedRows        int
	CandidateBlocks   int
	PrunedBlocks      int
	FalsePositiveRows int
	Exact             bool
	ExactCount        int64
	ExactSum          int64
	Reason            string
}

type ColumnPruningEnvelope

type ColumnPruningEnvelope struct {
	Version         uint16
	PartID          uint64
	ColumnName      string
	ColumnType      ColumnType
	Encoding        Encoding
	Compression     Compression
	Rows            int
	Blocks          int
	NullCount       int
	DefaultCount    int
	PayloadKind     ColumnPruningPayloadKind
	Operations      []ColumnPruningOperation
	PayloadLength   int
	PayloadChecksum uint32
}

func (ColumnPruningEnvelope) SupportsOperation

func (e ColumnPruningEnvelope) SupportsOperation(op ColumnPruningOperation) bool

type ColumnPruningOperation

type ColumnPruningOperation string

ColumnPruningOperation is stored as the semantic predicate operation string advertised by a pruning/index envelope. The values intentionally match columnsemantics.Operation without importing that package into typedcolumn.

const (
	ColumnPruningOpEquality     ColumnPruningOperation = "pruning.equality"
	ColumnPruningOpOrderedRange ColumnPruningOperation = "pruning.ordered_range"
)

type ColumnPruningPayloadKind

type ColumnPruningPayloadKind string
const (
	ColumnPruningPayloadInt64ValueRowsV1 ColumnPruningPayloadKind = "int64_value_rows_v1"
	ColumnPruningPayloadUnsupported      ColumnPruningPayloadKind = "unsupported"
)

type ColumnRowDescriptor

type ColumnRowDescriptor struct {
	Column             string
	Type               ColumnType
	Span               RowSpan
	SnapshotGeneration uint64
	AssetGeneration    uint64
	PartID             uint64
	SchemaVersion      uint32
	AlignmentKey       string
}

ColumnRowDescriptor binds one column role to a row span and optional immutable asset identity. Non-zero identity fields must align across descriptors.

type ColumnSchemaMode

type ColumnSchemaMode string
const (
	ColumnSchemaFixed ColumnSchemaMode = "fixed"
)

type ColumnStatsEnvelope

type ColumnStatsEnvelope struct {
	Version         uint16
	PartID          uint64
	ColumnName      string
	ColumnType      ColumnType
	Encoding        Encoding
	Compression     Compression
	Rows            int
	Blocks          int
	NullCount       int
	DefaultCount    int
	VisibleCount    int
	ValueCount      int
	PayloadKind     ColumnStatsPayloadKind
	Operations      []ColumnStatsOperation
	SelectionShapes []ColumnStatsSelectionShape
	PayloadLength   int
	PayloadChecksum uint32
}

func (ColumnStatsEnvelope) SupportsOperation

func (e ColumnStatsEnvelope) SupportsOperation(op ColumnStatsOperation) bool

func (ColumnStatsEnvelope) SupportsSelectionShape

func (e ColumnStatsEnvelope) SupportsSelectionShape(shape ColumnStatsSelectionShape) bool

type ColumnStatsOperation

type ColumnStatsOperation string

ColumnStatsOperation is stored as the semantic operation string advertised by a stats envelope. The values intentionally match columnsemantics.Operation without importing that package into typedcolumn.

const (
	ColumnStatsOpCountRows    ColumnStatsOperation = "aggregate.count_rows"
	ColumnStatsOpCountNonNull ColumnStatsOperation = "aggregate.count_non_null"
	ColumnStatsOpSum          ColumnStatsOperation = "aggregate.sum"
	ColumnStatsOpAvg          ColumnStatsOperation = "aggregate.avg"
	ColumnStatsOpMin          ColumnStatsOperation = "aggregate.min"
	ColumnStatsOpMax          ColumnStatsOperation = "aggregate.max"
	ColumnStatsOpStatsMinMax  ColumnStatsOperation = "stats.min_max"
	ColumnStatsOpStatsSum     ColumnStatsOperation = "stats.sum"
)

type ColumnStatsPayloadKind

type ColumnStatsPayloadKind string
const (
	ColumnStatsPayloadInt64V1      ColumnStatsPayloadKind = "int64_count_sum_min_max_v1"
	ColumnStatsPayloadUnsupported  ColumnStatsPayloadKind = "unsupported"
	ColumnStatsPayloadFuturePrefix ColumnStatsPayloadKind = "future"
)

type ColumnStatsSelectionShape

type ColumnStatsSelectionShape string

ColumnStatsSelectionShape records the row-selection domains an envelope can answer without decoding values. Narrower/random selections must not use a payload unless a future type-specific stats format advertises that shape.

const (
	ColumnStatsSelectionAllRows   ColumnStatsSelectionShape = "all_rows"
	ColumnStatsSelectionFullBlock ColumnStatsSelectionShape = "full_block"
)

type ColumnType

type ColumnType string
const (
	ColumnTypeInt64              ColumnType = "int64"
	ColumnTypeLowCardinalityCode ColumnType = "low_cardinality_code"
	ColumnTypeBool               ColumnType = "bool"
	ColumnTypeFloat32            ColumnType = "float32"
	ColumnTypeFloat64            ColumnType = "float64"
	ColumnTypeInt8               ColumnType = "int8"
	ColumnTypeUint8              ColumnType = "uint8"
	ColumnTypeInt16              ColumnType = "int16"
	ColumnTypeUint16             ColumnType = "uint16"
	ColumnTypeInt32              ColumnType = "int32"
	ColumnTypeUint32             ColumnType = "uint32"
	ColumnTypeUint64             ColumnType = "uint64"
	// Float16 and BFloat16 are storage-only raw 16-bit bit payloads.
	ColumnTypeFloat16           ColumnType = "float16"
	ColumnTypeBFloat16          ColumnType = "bfloat16"
	ColumnTypeUint8Vector       ColumnType = "uint8_vector"
	ColumnTypeInt8Vector        ColumnType = "int8_vector"
	ColumnTypeUint16Vector      ColumnType = "uint16_vector"
	ColumnTypeInt16Vector       ColumnType = "int16_vector"
	ColumnTypeUint32Vector      ColumnType = "uint32_vector"
	ColumnTypeInt32Vector       ColumnType = "int32_vector"
	ColumnTypeUint64Vector      ColumnType = "uint64_vector"
	ColumnTypeInt64Vector       ColumnType = "int64_vector"
	ColumnTypeFloat16Vector     ColumnType = "float16_vector"
	ColumnTypeBFloat16Vector    ColumnType = "bfloat16_vector"
	ColumnTypeFloat32Vector     ColumnType = "float32_vector"
	ColumnTypeFloat64Vector     ColumnType = "float64_vector"
	ColumnTypeFixedBytes        ColumnType = "fixed_bytes"
	ColumnTypePackedBitVector   ColumnType = "packed_bit_vector"
	ColumnTypePackedUint2Vector ColumnType = "packed_uint2_vector"
	ColumnTypePackedUint4Vector ColumnType = "packed_uint4_vector"
	ColumnTypeUint32List        ColumnType = "uint32_list"
	ColumnTypeBytes             ColumnType = "bytes"
	ColumnTypeAdjacencyList     ColumnType = "adjacency_list"
)

func PackedUintVectorTypeForBits

func PackedUintVectorTypeForBits(bitsPerElement int) (ColumnType, bool)

type Compression

type Compression uint8
const (
	CompressionNone Compression = iota
	CompressionSnappy
	CompressionLZ4
	CompressionZSTD
	CompressionZSTDDict
)

func (Compression) String

func (c Compression) String() string

type CompressionSelection

type CompressionSelection struct {
	Payload []byte
	Actual  Compression
	Scratch []byte
	Report  CodecReport
}

type Config

type Config struct {
	Encoding    Encoding
	Compression Compression
}

type DenseFixedWidthColumn

type DenseFixedWidthColumn struct {
	Rows              int
	ElementsPerRow    int
	ElementWidthBytes int
	Values            []byte
	Direct            bool
}

DenseFixedWidthColumn is a row-major fixed-width byte view. Values may alias a mapped/session-owned section when Direct is true; callers must copy before retaining row slices beyond that handle/session lifetime.

func DenseFixedWidthViewFromBytes

func DenseFixedWidthViewFromBytes(raw []byte, rows int, elementsPerRow int, elementBytes int) (DenseFixedWidthColumn, error)

func (DenseFixedWidthColumn) RowBytes

func (c DenseFixedWidthColumn) RowBytes(row int) ([]byte, error)

func (DenseFixedWidthColumn) RowRangeBytes

func (c DenseFixedWidthColumn) RowRangeBytes(startRow, endRow int) ([]byte, error)

type DenseFloat32Column

type DenseFloat32Column struct {
	Rows           int
	ElementsPerRow int
	Values         []float32
	Direct         bool
}

DenseFloat32Column is a row-major fixed-width float32 matrix section.

type DenseUint32Column

type DenseUint32Column struct {
	Rows           int
	ElementsPerRow int
	Values         []uint32
	Direct         bool
}

DenseUint32Column is a row-major fixed-width uint32 matrix section.

type EncodedGranule

type EncodedGranule struct {
	Rows         int
	NullCount    int
	DefaultCount int
	HasMinMax    bool
	Min          int64
	Max          int64
	Encoding     Encoding
	Compression  Compression
	RawBytes     int
	StoredBytes  int
	PayloadRef   PayloadRef
	CodecReport  CodecReport
	Payload      []byte
}

func EncodeInt64

func EncodeInt64(dst []byte, values []int64, cfg Config) (EncodedGranule, error)

type Encoding

type Encoding uint8
const (
	EncodingRawInt64 Encoding = iota + 1
	EncodingDeltaVarint
	EncodingDoubleDeltaVarint
	EncodingNullableInt64
	EncodingBoolBitpackRLE
	EncodingLowCardinalityUint32
	EncodingRawFloat32Vector
	EncodingRawUint32Dense
	EncodingRawFloat32
	EncodingRawFloat64
	// EncodingRawUint32OffsetsList is the v1 variable-length uint32 list
	// primitive: little-endian uint64 offsets plus little-endian uint32 values.
	EncodingRawUint32OffsetsList
	// EncodingRawBytesOffsets is the v1 non-null opaque bytes primitive:
	// little-endian uint64 offsets plus exact concatenated byte payloads.
	EncodingRawBytesOffsets
	// Raw fixed-width scalar primitives used by quantized side arrays. Multi-byte
	// values are little-endian; float16/bfloat16 are raw uint16 bit payloads.
	EncodingRawInt8
	EncodingRawUint8
	EncodingRawInt16
	EncodingRawUint16
	EncodingRawInt32
	EncodingRawUint32
	EncodingRawUint64
	EncodingRawFloat16
	EncodingRawBFloat16

	// Dense numeric vector encodings. The legacy raw_float32_vector keeps its
	// original value; this block follows #1929's scalar encodings.
	EncodingRawUint8Vector
	EncodingRawInt8Vector
	EncodingRawUint16Vector
	EncodingRawInt16Vector
	EncodingRawUint32Vector
	EncodingRawInt32Vector
	EncodingRawUint64Vector
	EncodingRawInt64Vector
	EncodingRawFloat16Vector
	EncodingRawBFloat16Vector
	EncodingRawFloat64Vector

	// Fixed-byte and packed sub-byte row-major vector encodings. Packed elements
	// are unsigned and LSB-first within each byte; unused high bits in the final
	// byte of each row must be zero.
	EncodingRawFixedBytes
	EncodingRawPackedBitVector
	EncodingRawPackedUint2Vector
	EncodingRawPackedUint4Vector

	// Section-level row locator encoding for contiguous primary IDs. This is
	// not a column granule codec.
	EncodingRowLocatorContiguous
	// Section-level dictionary encoding for dense 0..n-1 dictionary codes. This
	// is not a column granule codec.
	EncodingDictionaryDense
)

func DenseFixedWidthVectorEncoding

func DenseFixedWidthVectorEncoding(t ColumnType) (Encoding, bool)

func FixedBytesEncoding

func FixedBytesEncoding(t ColumnType) (Encoding, bool)

func PackedUintVectorEncoding

func PackedUintVectorEncoding(t ColumnType) (Encoding, bool)

func PackedUintVectorEncodingForBits

func PackedUintVectorEncodingForBits(bitsPerElement int) (Encoding, bool)

func (Encoding) String

func (e Encoding) String() string

type FixedBytesRows

type FixedBytesRows struct {
	Rows        int
	BytesPerRow int
	Values      []byte
}

FixedBytesRows is a row-major fixed-byte payload. Row slices alias Values and therefore inherit the caller-owned/mapped-resource lifetime of Values.

func FixedBytesViewFromBytes

func FixedBytesViewFromBytes(raw []byte, rows int, bytesPerRow int) (FixedBytesRows, error)

func NewFixedBytesRows

func NewFixedBytesRows(rows int, bytesPerRow int, values []byte) (FixedBytesRows, error)

NewFixedBytesRows validates a fixed-byte row payload and returns the view.

func (FixedBytesRows) Row

func (c FixedBytesRows) Row(row int) ([]byte, error)

Row returns Values for row without allocating. The returned slice aliases Values until the backing resource is released or mutated by the caller.

func (FixedBytesRows) RowWords

func (c FixedBytesRows) RowWords(row int, scratch []uint64) (words []uint64, direct bool, err error)

RowWords returns the row as little-endian uint64 words. If the row is 8-byte aligned, host-endian-compatible, and its length is a multiple of 8, the returned words alias Values and direct is true. Otherwise words aliases or allocates caller scratch and the final partial word is zero-padded.

func (FixedBytesRows) Validate

func (c FixedBytesRows) Validate() error

Validate checks the fixed-byte payload shape.

func (FixedBytesRows) WordCount

func (c FixedBytesRows) WordCount() int

WordCount returns ceil(BytesPerRow/8), the scratch-word count for RowWords.

type GranuleBuilder

type GranuleBuilder struct {
	// contains filtered or unexported fields
}

func NewGranuleBuilder

func NewGranuleBuilder(cfg Config) *GranuleBuilder

func (*GranuleBuilder) BuildBFloat16Bits

func (b *GranuleBuilder) BuildBFloat16Bits(values []uint16) (EncodedGranule, error)

BuildBFloat16Bits stores bfloat16 raw uint16 bits without arithmetic interpretation. NaN payloads, infinities, and signed zero bits are preserved.

func (*GranuleBuilder) BuildBool

func (b *GranuleBuilder) BuildBool(values []bool) (EncodedGranule, error)

BuildBool returns a granule whose payload aliases builder-owned scratch until the next builder Build* or Reset call.

func (*GranuleBuilder) BuildBytes

func (b *GranuleBuilder) BuildBytes(rows int, offsets []uint64, values []byte) (EncodedGranule, error)

BuildBytes builds an uncompressed raw_bytes_offsets granule. The granule payload is the canonical combined block payload: offsets bytes followed by values bytes. Image serialization splits that payload into independently checksummed offsets and values sections.

func (*GranuleBuilder) BuildDenseFixedWidth

func (b *GranuleBuilder) BuildDenseFixedWidth(values []byte, rows int, elementsPerRow int, elementBytes int) (EncodedGranule, error)

func (*GranuleBuilder) BuildFixedBytes

func (b *GranuleBuilder) BuildFixedBytes(values []byte, rows int, bytesPerRow int) (EncodedGranule, error)

func (*GranuleBuilder) BuildFloat16Bits

func (b *GranuleBuilder) BuildFloat16Bits(values []uint16) (EncodedGranule, error)

BuildFloat16Bits stores IEEE binary16 raw uint16 bits without arithmetic interpretation. NaN payloads, infinities, and signed zero bits are preserved.

func (*GranuleBuilder) BuildFloat32

func (b *GranuleBuilder) BuildFloat32(values []float32) (EncodedGranule, error)

BuildFloat32 returns a granule whose payload aliases builder-owned scratch until the next builder Build* or Reset call. Payload values are raw little-endian IEEE-754 float32 bits; NaN payloads and signed zeroes are preserved exactly.

func (*GranuleBuilder) BuildFloat32Vector

func (b *GranuleBuilder) BuildFloat32Vector(values []float32, rows int, elementsPerRow int) (EncodedGranule, error)

func (*GranuleBuilder) BuildFloat64

func (b *GranuleBuilder) BuildFloat64(values []float64) (EncodedGranule, error)

BuildFloat64 returns a granule whose payload aliases builder-owned scratch until the next builder Build* or Reset call. Payload values are raw little-endian IEEE-754 float64 bits; NaN payloads and signed zeroes are preserved exactly.

func (*GranuleBuilder) BuildInt8

func (b *GranuleBuilder) BuildInt8(values []int8) (EncodedGranule, error)

func (*GranuleBuilder) BuildInt16

func (b *GranuleBuilder) BuildInt16(values []int16) (EncodedGranule, error)

func (*GranuleBuilder) BuildInt32

func (b *GranuleBuilder) BuildInt32(values []int32) (EncodedGranule, error)

func (*GranuleBuilder) BuildInt64

func (b *GranuleBuilder) BuildInt64(values []int64) (EncodedGranule, error)

BuildInt64 returns a granule whose payload aliases builder-owned scratch until the next BuildInt64 or Reset call.

func (*GranuleBuilder) BuildNullableInt64

func (b *GranuleBuilder) BuildNullableInt64(values []int64, nulls []bool, defaults []bool, defaultValue int64) (EncodedGranule, error)

BuildNullableInt64 returns a granule whose payload aliases builder-owned scratch until the next builder Build* or Reset call.

func (*GranuleBuilder) BuildPackedUint

func (b *GranuleBuilder) BuildPackedUint(values []byte, rows int, elementsPerRow int, bitsPerElement int) (EncodedGranule, error)

func (*GranuleBuilder) BuildUint8

func (b *GranuleBuilder) BuildUint8(values []uint8) (EncodedGranule, error)

func (*GranuleBuilder) BuildUint16

func (b *GranuleBuilder) BuildUint16(values []uint16) (EncodedGranule, error)

func (*GranuleBuilder) BuildUint32

func (b *GranuleBuilder) BuildUint32(values []uint32) (EncodedGranule, error)

func (*GranuleBuilder) BuildUint32Codes

func (b *GranuleBuilder) BuildUint32Codes(codes []uint32, cardinality uint32) (EncodedGranule, error)

BuildUint32Codes returns a granule whose payload aliases builder-owned scratch until the next builder Build* or Reset call.

func (*GranuleBuilder) BuildUint32Dense

func (b *GranuleBuilder) BuildUint32Dense(values []uint32, rows int, elementsPerRow int) (EncodedGranule, error)

func (*GranuleBuilder) BuildUint32OffsetsList

func (b *GranuleBuilder) BuildUint32OffsetsList(rows int, offsets []uint64, values []uint32) (EncodedGranule, error)

BuildUint32OffsetsList builds an uncompressed raw_uint32_offsets_list granule. The granule payload is the canonical combined block payload: offsets bytes followed by values bytes. Image serialization splits that payload into independently checksummed offsets and values sections.

func (*GranuleBuilder) BuildUint64

func (b *GranuleBuilder) BuildUint64(values []uint64) (EncodedGranule, error)

func (*GranuleBuilder) Reset

func (b *GranuleBuilder) Reset(cfg Config)

type GranuleDescriptor

type GranuleDescriptor struct {
	Ordinal          int
	FirstRow         int
	RowCount         int
	VisibleRows      int
	DeletedRows      int
	IDLower          int64
	IDUpperExclusive int64
	MarkOrdinal      int
}

type GranuleReader

type GranuleReader struct {
	// contains filtered or unexported fields
}

func (*GranuleReader) CountBool

func (r *GranuleReader) CountBool(g EncodedGranule, selection RowSelection, value bool) (int, error)

CountBool counts rows in selection whose bool payload equals value. Bitpack payloads use popcount over selected bit ranges/words; RLE payloads aggregate matching runs without materializing []bool.

func (*GranuleReader) CountInt64RangeWithDiagnostics

func (r *GranuleReader) CountInt64RangeWithDiagnostics(granules []EncodedGranule, marks []SortKeyMark, plan PredicatePlan) (int, PredicateDiagnostics, error)

func (*GranuleReader) CountSumInt64

func (r *GranuleReader) CountSumInt64(g EncodedGranule) (int, int64, error)

CountSumInt64 streams all int64 values in the granule and returns count+sum without materializing a []int64. It is intended for full-block reducers on certified prepared paths where predicate/visibility/selection do not require per-row branching in the caller.

func (*GranuleReader) CountTrueBool

func (r *GranuleReader) CountTrueBool(g EncodedGranule) (int, error)

func (*GranuleReader) CountUint32Code

func (r *GranuleReader) CountUint32Code(g EncodedGranule, selection RowSelection, code uint32) (int, error)

CountUint32Code counts rows in selection whose low-cardinality uint32 code equals code. It scans encoded codes directly and does not decode dictionary strings.

func (*GranuleReader) CountUint32Codes

func (r *GranuleReader) CountUint32Codes(g EncodedGranule, counts []int) ([]int, error)

func (*GranuleReader) CountUint32CodesIn

func (r *GranuleReader) CountUint32CodesIn(g EncodedGranule, selection RowSelection, codes []uint32, scratch *Uint32CodeSelectionScratch) (int, error)

CountUint32CodesIn counts rows in selection whose low-cardinality uint32 code is present in codes. Out-of-cardinality query codes are ignored.

func (*GranuleReader) DecodeBFloat16Bits

func (r *GranuleReader) DecodeBFloat16Bits(g EncodedGranule) ([]uint16, error)

func (*GranuleReader) DecodeBFloat16BitsInto

func (r *GranuleReader) DecodeBFloat16BitsInto(dst []uint16, g EncodedGranule) ([]uint16, error)

func (*GranuleReader) DecodeBool

func (r *GranuleReader) DecodeBool(g EncodedGranule) ([]bool, error)

func (*GranuleReader) DecodeBoolInto

func (r *GranuleReader) DecodeBoolInto(dst []bool, g EncodedGranule) ([]bool, error)

func (*GranuleReader) DecodeBytesInto

func (r *GranuleReader) DecodeBytesInto(offsetsDst []uint64, valuesDst []byte, g EncodedGranule) (RawBytesOffsets, error)

DecodeBytesInto decodes a raw_bytes_offsets granule into owned offsets and value byte slices. It does not expose direct/mmap views.

func (*GranuleReader) DecodeDenseFixedWidthInto

func (r *GranuleReader) DecodeDenseFixedWidthInto(dst []byte, g EncodedGranule, elementsPerRow int, elementBytes int) ([]byte, error)

func (*GranuleReader) DecodeFixedBytesInto

func (r *GranuleReader) DecodeFixedBytesInto(dst []byte, g EncodedGranule, bytesPerRow int) ([]byte, error)

func (*GranuleReader) DecodeFloat16Bits

func (r *GranuleReader) DecodeFloat16Bits(g EncodedGranule) ([]uint16, error)

func (*GranuleReader) DecodeFloat16BitsInto

func (r *GranuleReader) DecodeFloat16BitsInto(dst []uint16, g EncodedGranule) ([]uint16, error)

func (*GranuleReader) DecodeFloat32

func (r *GranuleReader) DecodeFloat32(g EncodedGranule) ([]float32, error)

func (*GranuleReader) DecodeFloat32Into

func (r *GranuleReader) DecodeFloat32Into(dst []float32, g EncodedGranule) ([]float32, error)

func (*GranuleReader) DecodeFloat32VectorInto

func (r *GranuleReader) DecodeFloat32VectorInto(dst []float32, g EncodedGranule, elementsPerRow int) ([]float32, error)

func (*GranuleReader) DecodeFloat64

func (r *GranuleReader) DecodeFloat64(g EncodedGranule) ([]float64, error)

func (*GranuleReader) DecodeFloat64Into

func (r *GranuleReader) DecodeFloat64Into(dst []float64, g EncodedGranule) ([]float64, error)

func (*GranuleReader) DecodeInt8

func (r *GranuleReader) DecodeInt8(g EncodedGranule) ([]int8, error)

func (*GranuleReader) DecodeInt8Into

func (r *GranuleReader) DecodeInt8Into(dst []int8, g EncodedGranule) ([]int8, error)

func (*GranuleReader) DecodeInt16

func (r *GranuleReader) DecodeInt16(g EncodedGranule) ([]int16, error)

func (*GranuleReader) DecodeInt16Into

func (r *GranuleReader) DecodeInt16Into(dst []int16, g EncodedGranule) ([]int16, error)

func (*GranuleReader) DecodeInt32

func (r *GranuleReader) DecodeInt32(g EncodedGranule) ([]int32, error)

func (*GranuleReader) DecodeInt32Into

func (r *GranuleReader) DecodeInt32Into(dst []int32, g EncodedGranule) ([]int32, error)

func (*GranuleReader) DecodeInt64

func (r *GranuleReader) DecodeInt64(g EncodedGranule) ([]int64, error)

func (*GranuleReader) DecodeInt64Into

func (r *GranuleReader) DecodeInt64Into(dst []int64, g EncodedGranule) ([]int64, error)

func (*GranuleReader) DecodeIntegerAsInt64Into

func (r *GranuleReader) DecodeIntegerAsInt64Into(dst []int64, columnType ColumnType, g EncodedGranule) ([]int64, error)

func (*GranuleReader) DecodeNullableInt64

func (r *GranuleReader) DecodeNullableInt64(g EncodedGranule) ([]int64, []bool, []bool, error)

func (*GranuleReader) DecodeNullableInt64Into

func (r *GranuleReader) DecodeNullableInt64Into(dst []int64, nulls []bool, defaults []bool, g EncodedGranule) ([]int64, []bool, []bool, error)

func (*GranuleReader) DecodePackedUintInto

func (r *GranuleReader) DecodePackedUintInto(dst []byte, g EncodedGranule, elementsPerRow int, bitsPerElement int) ([]byte, error)

func (*GranuleReader) DecodeUint8

func (r *GranuleReader) DecodeUint8(g EncodedGranule) ([]uint8, error)

func (*GranuleReader) DecodeUint8Into

func (r *GranuleReader) DecodeUint8Into(dst []uint8, g EncodedGranule) ([]uint8, error)

func (*GranuleReader) DecodeUint16

func (r *GranuleReader) DecodeUint16(g EncodedGranule) ([]uint16, error)

func (*GranuleReader) DecodeUint16Into

func (r *GranuleReader) DecodeUint16Into(dst []uint16, g EncodedGranule) ([]uint16, error)

func (*GranuleReader) DecodeUint32

func (r *GranuleReader) DecodeUint32(g EncodedGranule) ([]uint32, error)

func (*GranuleReader) DecodeUint32Codes

func (r *GranuleReader) DecodeUint32Codes(g EncodedGranule) ([]uint32, error)

func (*GranuleReader) DecodeUint32CodesInto

func (r *GranuleReader) DecodeUint32CodesInto(dst []uint32, g EncodedGranule) ([]uint32, error)

func (*GranuleReader) DecodeUint32DenseInto

func (r *GranuleReader) DecodeUint32DenseInto(dst []uint32, g EncodedGranule, elementsPerRow int) ([]uint32, error)

func (*GranuleReader) DecodeUint32Into

func (r *GranuleReader) DecodeUint32Into(dst []uint32, g EncodedGranule) ([]uint32, error)

func (*GranuleReader) DecodeUint32OffsetsListInto

func (r *GranuleReader) DecodeUint32OffsetsListInto(offsetsDst []uint64, valuesDst []uint32, g EncodedGranule) (RawUint32OffsetsList, error)

DecodeUint32OffsetsListInto decodes a raw_uint32_offsets_list granule into owned offsets and values slices. It does not expose direct/mmap views.

func (*GranuleReader) DecodeUint64

func (r *GranuleReader) DecodeUint64(g EncodedGranule) ([]uint64, error)

func (*GranuleReader) DecodeUint64Into

func (r *GranuleReader) DecodeUint64Into(dst []uint64, g EncodedGranule) ([]uint64, error)

func (*GranuleReader) Int64Cursor

func (r *GranuleReader) Int64Cursor(g EncodedGranule) (Int64Cursor, error)

Int64Cursor returns a cursor over raw, delta-varint, or double-delta int64 payloads after payload decompression/validation. The cursor aliases reader scratch for compressed payloads and is valid until the next GranuleReader operation.

func (*GranuleReader) RangeScanCountInt64

func (r *GranuleReader) RangeScanCountInt64(g EncodedGranule, low, high int64) (int, error)

func (*GranuleReader) SelectBool

func (r *GranuleReader) SelectBool(g EncodedGranule, selection RowSelection, value bool, scratch *BoolSelectionScratch) (RowSelection, error)

SelectBool returns the subset of selection whose bool payload equals value. The returned selection may alias scratch when scratch is non-nil.

func (*GranuleReader) SelectUint32Code

func (r *GranuleReader) SelectUint32Code(g EncodedGranule, selection RowSelection, code uint32, scratch *Uint32CodeSelectionScratch) (RowSelection, error)

SelectUint32Code returns the subset of selection whose low-cardinality uint32 code equals code. The returned selection may alias scratch when scratch is non-nil.

func (*GranuleReader) SelectUint32CodesIn

func (r *GranuleReader) SelectUint32CodesIn(g EncodedGranule, selection RowSelection, codes []uint32, scratch *Uint32CodeSelectionScratch) (RowSelection, error)

SelectUint32CodesIn returns the subset of selection whose low-cardinality uint32 code is present in codes. The returned selection may alias scratch when scratch is non-nil. Out-of-cardinality query codes are ignored.

type ImageSectionReader

type ImageSectionReader struct {
	Image ColumnPartImage
}

ImageSectionReader reads sections from an in-memory part image.

func NewImageSectionReader

func NewImageSectionReader(image ColumnPartImage) ImageSectionReader

func (ImageSectionReader) ReadSection

func (r ImageSectionReader) ReadSection(section ColumnPartImageSection) ([]byte, error)

type Int64BlockStats

type Int64BlockStats struct {
	Index        int
	FirstRow     int
	RowCount     int
	NullCount    int
	DefaultCount int
	VisibleCount int
	ValueCount   int
	Sum          int64
	SumValid     bool
	HasMinMax    bool
	Min          int64
	Max          int64
}

func (Int64BlockStats) CanAnswer

func (b Int64BlockStats) CanAnswer(op ColumnStatsOperation) (bool, string)

type Int64ColumnStats

type Int64ColumnStats struct {
	Envelope     ColumnStatsEnvelope
	Count        int
	NullCount    int
	DefaultCount int
	VisibleCount int
	ValueCount   int
	Sum          int64
	SumValid     bool
	HasMinMax    bool
	Min          int64
	Max          int64
	Blocks       []Int64BlockStats
}

func (Int64ColumnStats) Block

func (s Int64ColumnStats) Block(index int) (Int64BlockStats, bool)

func (Int64ColumnStats) CanAnswer

type Int64Cursor

type Int64Cursor struct {
	// contains filtered or unexported fields
}

Int64Cursor streams int64 values from a granule without materializing a full []int64. It is intended for reducers and predicate cursors that have already selected a concrete typed-column path during prepare.

func (*Int64Cursor) Finish

func (c *Int64Cursor) Finish() error

Finish validates that exactly the declared rows were consumed and that variable-width encodings have no trailing bytes.

func (*Int64Cursor) Next

func (c *Int64Cursor) Next() (int64, error)

Next returns the next value. Call Finish after consuming Rows values to catch truncated or trailing variable-width payload bytes.

func (*Int64Cursor) RawBytesRead

func (c *Int64Cursor) RawBytesRead() int

RawBytesRead returns the payload byte offset consumed by the cursor.

func (*Int64Cursor) Row

func (c *Int64Cursor) Row() int

Row returns the next row index to be decoded.

func (*Int64Cursor) Rows

func (c *Int64Cursor) Rows() int

Rows returns the declared row count.

type Int64PruningBlock

type Int64PruningBlock struct {
	Index     int
	FirstRow  int
	RowCount  int
	HasMinMax bool
	Min       int64
	Max       int64
}

type Int64PruningEntry

type Int64PruningEntry struct {
	Value int64
	Row   int
}

type Int64PruningPredicate

type Int64PruningPredicate struct {
	Kind  Int64PruningPredicateKind
	Value int64
	Low   int64
	High  int64
}

type Int64PruningPredicateKind

type Int64PruningPredicateKind string
const (
	Int64PruningPredicateAll   Int64PruningPredicateKind = "all"
	Int64PruningPredicateEqual Int64PruningPredicateKind = "equal"
	Int64PruningPredicateRange Int64PruningPredicateKind = "range"
)

type Int64RangePredicate

type Int64RangePredicate struct {
	Column string
	Low    int64
	High   int64
}

func (Int64RangePredicate) Empty

func (p Int64RangePredicate) Empty() bool

type Int64ValueRowIndex

type Int64ValueRowIndex struct {
	Envelope     ColumnPruningEnvelope
	Rows         int
	NullCount    int
	DefaultCount int
	Blocks       []Int64PruningBlock
	Entries      []Int64PruningEntry
}

func (Int64ValueRowIndex) CanPlan

func (Int64ValueRowIndex) PlanInt64Predicate

PlanInt64Predicate turns a durable int64 value-row index into block-local RowSelection candidates. The returned selections are immutable and own their slice storage; callers may retain them for a prepared/session lifetime.

type LogicalPrimaryKey

type LogicalPrimaryKey struct {
	Columns []string
}

type Options

type Options struct {
	SchemaVersion     uint32
	SchemaMode        ColumnSchemaMode
	Columns           []ColumnDefinition
	LogicalPrimaryKey LogicalPrimaryKey
	SortKey           SortKey
	PartPolicy        ColumnPartPolicy
	Compression       ColumnCompressionPolicy
	AggregateMetadata []AggregateMetadataDefinition
}

type PackedUintRows

type PackedUintRows struct {
	Rows           int
	ElementsPerRow int
	BitsPerElement int
	BytesPerRow    int
	Values         []byte
}

PackedUintRows is a row-major sub-byte unsigned integer payload. Elements are packed LSB-first within each byte. Multi-byte word views are little-endian.

func NewPackedUintRows

func NewPackedUintRows(rows int, elementsPerRow int, bitsPerElement int, values []byte) (PackedUintRows, error)

NewPackedUintRows validates a packed uint payload and returns the view.

func PackedUintViewFromBytes

func PackedUintViewFromBytes(raw []byte, rows int, elementsPerRow int, bitsPerElement int) (PackedUintRows, error)

func (PackedUintRows) Element

func (p PackedUintRows) Element(row int, element int) (uint8, error)

Element returns one unpacked element from row/element without allocation.

func (PackedUintRows) RowBytes

func (p PackedUintRows) RowBytes(row int) ([]byte, error)

RowBytes returns packed bytes for row without allocating. The returned slice aliases Values and uses LSB-first element order within each byte.

func (PackedUintRows) RowWords

func (p PackedUintRows) RowWords(row int, scratch []uint64) (words []uint64, direct bool, err error)

RowWords returns the packed row as little-endian uint64 words. If the row is 8-byte aligned, host-endian-compatible, and its physical length is a multiple of 8, the returned words alias Values and direct is true. Otherwise words aliases or allocates caller scratch and the final partial word is zero-padded.

func (PackedUintRows) Validate

func (p PackedUintRows) Validate() error

Validate checks shape and zero padding bits.

func (PackedUintRows) ValidatePadding

func (p PackedUintRows) ValidatePadding() error

ValidatePadding verifies that unused high bits in the final byte of every row are zero. It is intended for writer/open integrity validation; hot row access can rely on this once the immutable payload is certified.

func (PackedUintRows) WordCount

func (p PackedUintRows) WordCount() int

WordCount returns ceil(BytesPerRow/8), the scratch-word count for RowWords.

type PartRef

type PartRef struct {
	Role         PartRole
	GenerationID uint64
	Part         *ColumnPart
}

PartRef is the data-plane identity needed to evaluate latest-visible rows across base and delta parts. It deliberately excludes collection manifest, WAL, recovery, and asset publication references.

type PartRole

type PartRole string

PartRole records whether a typed-column part participates as a base or delta in a logical part set. Publication/control-plane ownership is intentionally outside this package; callers provide already-open non-authoritative parts.

const (
	PartRoleBase  PartRole = "base"
	PartRoleDelta PartRole = "delta"
)

type PartScanDiagnostics

type PartScanDiagnostics struct {
	RowsScanned        int
	ColumnsProjected   int
	GranulesConsidered int
	GranulesDecoded    int
	BlocksDecoded      int
	BytesDecoded       int
}

type PartSetReader

type PartSetReader struct {
	// contains filtered or unexported fields
}

func NewPartSetReader

func NewPartSetReader(refs []PartRef, tombstones []Tombstone) (*PartSetReader, error)

NewPartSetReader builds a non-authoritative latest-visible row index over caller-owned parts. The algorithm is copied/adapted from experiments/colgranule ColumnPartSetReader.buildVisibility without workspace or collection-manifest publication plumbing.

func (*PartSetReader) LatestLocator

func (r *PartSetReader) LatestLocator(primaryID int64) (RowLocator, bool)

func (*PartSetReader) ScanLatestLocator

func (r *PartSetReader) ScanLatestLocator(primaryID int64) (RowLocator, bool)

ScanLatestLocator recomputes latest-visible selection from the caller-owned parts instead of reading the cached index. It is retained as a data-plane cross-check and adapter seam for future lazy/mappedresource-backed readers; immutable in-memory parts should match LatestLocator.

func (*PartSetReader) ValueAtLatest

func (r *PartSetReader) ValueAtLatest(primaryID int64, columnName string) (int64, bool, error)

func (*PartSetReader) VisibilityStats

func (r *PartSetReader) VisibilityStats() PartSetVisibilityStats

func (*PartSetReader) VisibleRowsForPart

func (r *PartSetReader) VisibleRowsForPart(partIndex int) ([]int, bool)

type PartSetVisibilityStats

type PartSetVisibilityStats struct {
	Parts          int `json:"parts"`
	BaseParts      int `json:"base_parts"`
	DeltaParts     int `json:"delta_parts"`
	InputRows      int `json:"input_rows"`
	VisibleRows    int `json:"visible_rows"`
	SupersededRows int `json:"superseded_rows"`
	DeletedRows    int `json:"deleted_rows"`
	Tombstones     int `json:"tombstones"`
}

type PayloadRef

type PayloadRef struct {
	Kind   PayloadRefKind
	Offset int64
	Length int
}

type PayloadRefKind

type PayloadRefKind uint8
const (
	PayloadRefInline PayloadRefKind = iota + 1
)

func (PayloadRefKind) String

func (k PayloadRefKind) String() string

type PredicateDiagnostics

type PredicateDiagnostics struct {
	Considered      int
	SkippedByMark   int
	SkippedByMinMax int
	Decoded         int
	Matched         int
}

type PredicatePlan

type PredicatePlan struct {
	Filter        Int64RangePredicate
	SortKeyRanges []Int64RangePredicate
}

type ProjectedScanResult

type ProjectedScanResult struct {
	Rows        int
	Columns     map[string][]int64
	Diagnostics PartScanDiagnostics
}

type RawBytesOffsets

type RawBytesOffsets = BytesColumn

RawBytesOffsets is the physical-encoding compatibility name for the raw_bytes_offsets layout.

func DecodeRawBytesOffsetsFallback

func DecodeRawBytesOffsetsFallback(offsetsDst []uint64, valuesDst []byte, offsetsRaw []byte, valuesRaw []byte, rows int) (RawBytesOffsets, error)

DecodeRawBytesOffsetsFallback decodes the split offsets and values sections into owned Go slices. The returned slices never alias offsetsRaw or valuesRaw, but they may reuse caller-provided destination slices.

func DecodeRawBytesOffsetsPayload

func DecodeRawBytesOffsetsPayload(offsetsDst []uint64, valuesDst []byte, raw []byte, rows int) (RawBytesOffsets, error)

DecodeRawBytesOffsetsPayload decodes the combined block payload produced by EncodeRawBytesOffsetsPayload into owned Go slices.

type RawDenseFixedWidth

type RawDenseFixedWidth struct {
	Rows              int
	ElementsPerRow    int
	ElementWidthBytes int
	Values            []byte
}

RawDenseFixedWidth is a row-major fixed-width dense vector/matrix payload. Values are already encoded in the column type's raw storage byte order: one byte for 1-byte elements and little-endian for 2/4/8-byte elements.

type RawUint32OffsetsList

type RawUint32OffsetsList = Uint32List

RawUint32OffsetsList is retained as the physical-encoding compatibility name for existing raw_uint32_offsets_list code. New logical APIs should prefer Uint32List.

func DecodeRawUint32OffsetsListFallback

func DecodeRawUint32OffsetsListFallback(offsetsDst []uint64, valuesDst []uint32, offsetsRaw []byte, valuesRaw []byte, rows int) (RawUint32OffsetsList, error)

DecodeRawUint32OffsetsListFallback decodes the split offsets and values sections into owned Go slices. The returned slices never alias offsetsRaw or valuesRaw, but they may reuse caller-provided destination slices.

func DecodeRawUint32OffsetsListPayload

func DecodeRawUint32OffsetsListPayload(offsetsDst []uint64, valuesDst []uint32, raw []byte, rows int) (RawUint32OffsetsList, error)

DecodeRawUint32OffsetsListPayload decodes the combined block payload produced by EncodeRawUint32OffsetsListPayload into owned Go slices.

type RowLocator

type RowLocator struct {
	PrimaryID      int64
	PartID         uint64
	PartRow        int
	GranuleOrdinal int
	RowInGranule   int
}

type RowMaskRole

type RowMaskRole uint8

RowMaskRole identifies how a mask participates in selection composition.

const (
	RowMaskPredicate  RowMaskRole = rowMaskPredicate
	RowMaskVisibility RowMaskRole = rowMaskVisibility
	RowMaskDelete     RowMaskRole = rowMaskDelete
	RowMaskNull       RowMaskRole = rowMaskNull
	RowMaskDefault    RowMaskRole = rowMaskDefault
)

func (RowMaskRole) String

func (r RowMaskRole) String() string

type RowRange

type RowRange struct {
	Start int
	End   int
}

RowRange is a half-open [Start, End) range over a block-local row domain.

type RowSelection

type RowSelection struct {
	// contains filtered or unexported fields
}

RowSelection is an immutable block-local selection over rows [0, Rows()). Slice-backed selections may alias caller-owned scratch when constructed by a NoCopy/Into helper; callers must not mutate those slices until the selection is no longer used. The type deliberately has no package-global cache.

func AndRowSelections

func AndRowSelections(a RowSelection, b RowSelection) (RowSelection, error)

AndRowSelections returns the intersection of a and b. Mixed-shape results are range-normalized and may allocate; use AndRowSelectionsInto with caller scratch on prepared/hot paths.

func AndRowSelectionsInto

func AndRowSelectionsInto(a RowSelection, b RowSelection, scratch *RowSelectionScratch) (RowSelection, error)

AndRowSelectionsInto intersects a and b using caller-owned scratch for mixed range results. The returned selection may alias scratch until the next scratch use.

func ComposeRowSelections

func ComposeRowSelections(rows int, components RowSelectionComponents) (RowSelection, error)

ComposeRowSelections composes predicate, visibility/delete, and null/default masks over a block-local row domain. Mismatched domains return an empty fail-closed selection plus an error.

func ComposeRowSelectionsInto

func ComposeRowSelectionsInto(rows int, components RowSelectionComponents, scratch *RowSelectionScratch) (RowSelection, error)

ComposeRowSelectionsInto is the scratch-backed form of ComposeRowSelections. Returned selections may alias scratch until scratch is reused.

func NewAllRowSelection

func NewAllRowSelection(rows int) (RowSelection, error)

NewAllRowSelection returns a compact all-rows selection over rows.

func NewBitmapRowSelection

func NewBitmapRowSelection(rows int, bitmap []uint64) (RowSelection, error)

NewBitmapRowSelection returns a bitmap selection. The bitmap slice is copied and must have exactly ceil(rows/64) words with zero padding bits.

func NewBitmapRowSelectionNoCopy

func NewBitmapRowSelectionNoCopy(rows int, bitmap []uint64) (RowSelection, error)

NewBitmapRowSelectionNoCopy returns a bitmap selection that may alias bitmap. The bitmap must have exactly ceil(rows/64) words with zero padding bits and must not be mutated while the selection is in use.

func NewEmptyRowSelection

func NewEmptyRowSelection(rows int) (RowSelection, error)

NewEmptyRowSelection returns an empty selection over rows.

func NewRangeRowSelection

func NewRangeRowSelection(rows int, start int, end int) (RowSelection, error)

NewRangeRowSelection returns a compact half-open range selection.

func NewRangesRowSelection

func NewRangesRowSelection(rows int, ranges []RowRange) (RowSelection, error)

NewRangesRowSelection returns a compact multi-range selection. Ranges may be unsorted; overlapping and adjacent ranges are coalesced, and fully-covered domains collapse to all.

func NewRangesRowSelectionNoCopy

func NewRangesRowSelectionNoCopy(rows int, ranges []RowRange) (RowSelection, error)

NewRangesRowSelectionNoCopy returns a compact multi-range selection that may alias ranges. Ranges must already be sorted, non-overlapping, and must not be mutated while the selection is in use.

func NewSparseRowSelection

func NewSparseRowSelection(rows int, sparse []int) (RowSelection, error)

NewSparseRowSelection returns a sparse selection from strictly increasing row indexes. The input slice is copied.

func NewSparseRowSelectionNoCopy

func NewSparseRowSelectionNoCopy(rows int, sparse []int) (RowSelection, error)

NewSparseRowSelectionNoCopy returns a sparse selection that may alias sparse. The input must be strictly increasing and must not be mutated while the selection is in use.

func NotRowSelection

func NotRowSelection(s RowSelection) (RowSelection, error)

NotRowSelection returns the complement of s over its row domain.

func NotRowSelectionInto

func NotRowSelectionInto(s RowSelection, scratch *RowSelectionScratch) (RowSelection, error)

NotRowSelectionInto returns the complement of s using caller scratch for range decomposition. Returned selections may alias scratch.

func OrRowSelections

func OrRowSelections(a RowSelection, b RowSelection) (RowSelection, error)

OrRowSelections returns the union of a and b.

func SubtractRowSelectionsInto

func SubtractRowSelectionsInto(a RowSelection, b RowSelection, scratch *RowSelectionScratch) (RowSelection, error)

SubtractRowSelectionsInto returns a-b using caller-owned scratch. The returned selection may alias scratch until scratch is reused.

func (RowSelection) AppendRanges

func (s RowSelection) AppendRanges(dst []RowRange) []RowRange

AppendRanges appends a range decomposition to dst without materializing rows.

func (RowSelection) AppendRows

func (s RowSelection) AppendRows(dst []int) []int

AppendRows appends selected rows in ascending order to dst.

func (RowSelection) BitmapWords

func (s RowSelection) BitmapWords() []uint64

BitmapWords returns the internal bitmap words for bitmap selections. Treat the returned slice as read-only.

func (RowSelection) Contains

func (s RowSelection) Contains(row int) bool

Contains reports whether row is selected.

func (RowSelection) Count

func (s RowSelection) Count() int

Count returns the selected row count without materializing row IDs.

func (RowSelection) ForEach

func (s RowSelection) ForEach(fn func(row int))

ForEach calls fn once for each selected row in ascending order. It does not allocate, but performance-sensitive kernels should switch on Kind and use the concrete accessors to avoid callback overhead in inner loops.

func (RowSelection) IsAll

func (s RowSelection) IsAll() bool

IsAll reports whether the selection covers the whole row domain.

func (RowSelection) IsEmpty

func (s RowSelection) IsEmpty() bool

IsEmpty reports whether no rows are selected.

func (RowSelection) Kind

func (s RowSelection) Kind() RowSelectionKind

Kind returns the compact representation shape.

func (RowSelection) Ranges

func (s RowSelection) Ranges() []RowRange

Ranges returns a range decomposition for range/ranges/all selections. Treat the returned slice as read-only and copy it if it must outlive caller scratch. For all/range shapes this returns a fresh one-element slice; hot paths should prefer SingleRange or AppendRanges with caller-owned scratch.

func (RowSelection) Rows

func (s RowSelection) Rows() int

Rows returns the row-domain size.

func (RowSelection) Shape

func (s RowSelection) Shape() RowSelectionShape

Shape returns diagnostic metadata for the selection representation.

func (RowSelection) SingleRange

func (s RowSelection) SingleRange() (start int, end int, ok bool)

SingleRange exposes the half-open range for range selections.

func (RowSelection) SparseRows

func (s RowSelection) SparseRows() []int

SparseRows returns the internal sparse row slice for sparse selections. Treat the returned slice as read-only.

type RowSelectionComponents

type RowSelectionComponents struct {
	Predicate  *RowSelection
	Visibility *RowSelection
	Deletes    *RowSelection
	Nulls      *RowSelection
	Defaults   *RowSelection
}

RowSelectionComponents centralizes block-local composition. Predicate and visibility selections are included; delete/null/default selections name rows that must be excluded from value semantics.

type RowSelectionKind

type RowSelectionKind uint8

RowSelectionKind names the concrete representation used for a block-local row selection. Callers should prefer all/range/ranges forms when possible; bitmap and sparse forms are available for genuinely irregular selections.

const (
	RowSelectionEmpty  RowSelectionKind = rowSelectionEmpty
	RowSelectionAll    RowSelectionKind = rowSelectionAll
	RowSelectionRange  RowSelectionKind = rowSelectionRange
	RowSelectionRanges RowSelectionKind = rowSelectionRanges
	RowSelectionBitmap RowSelectionKind = rowSelectionBitmap
	RowSelectionSparse RowSelectionKind = rowSelectionSparse
)

type RowSelectionScratch

type RowSelectionScratch struct {
	// contains filtered or unexported fields
}

RowSelectionScratch owns caller-scoped temporary storage for composition and no-copy constructors. Returned selections may alias these slices until the next operation using the same scratch.

type RowSelectionShape

type RowSelectionShape struct {
	Kind        string
	Rows        int
	Count       int
	Ranges      int
	BitmapWords int
	SparseRows  int
}

RowSelectionShape is a cheap diagnostic summary that does not materialize row IDs.

type RowSpan

type RowSpan struct {
	FirstRow int
	RowCount int
}

RowSpan is a half-open row span over a column section or block.

func AlignColumnRowSpans

func AlignColumnRowSpans(columns []ColumnRowDescriptor) (RowSpan, bool, error)

AlignColumnRowSpans validates that all columns share the same row span and optional snapshot/asset/schema identity. It fails closed on any mismatch.

func NewRowSpan

func NewRowSpan(firstRow int, rowCount int) (RowSpan, error)

func ValidateSectionDependencies

func ValidateSectionDependencies(deps []SectionDependencyDescriptor) (RowSpan, bool, error)

ValidateSectionDependencies validates dependency descriptors and their row alignment. It does not read any payloads.

func (RowSpan) EndRow

func (s RowSpan) EndRow() int

EndRow returns the exclusive row end.

type SectionDependencyDescriptor

type SectionDependencyDescriptor struct {
	Role        ColumnExecutionRole
	Column      string
	Type        ColumnType
	Kind        SectionDependencyKind
	SectionKind ColumnPartImageSectionKind
	Span        RowSpan
	Required    bool
}

SectionDependencyDescriptor describes a section read dependency for a column role and row span. SectionKind identifies the tcs1 section class, while Kind identifies the logical payload/metadata need.

func MeasureColumnDependencies

func MeasureColumnDependencies(column string, columnType ColumnType, span RowSpan, nullable bool) ([]SectionDependencyDescriptor, error)

func NewSectionDependency

func NewSectionDependency(role ColumnExecutionRole, column string, columnType ColumnType, kind SectionDependencyKind, sectionKind ColumnPartImageSectionKind, span RowSpan, required bool) (SectionDependencyDescriptor, error)

func PredicateColumnDependencies

func PredicateColumnDependencies(column string, columnType ColumnType, span RowSpan, nullable bool) ([]SectionDependencyDescriptor, error)

PredicateColumnDependencies describes the ordinary sections needed to produce a predicate selection for one column. Callers may add dictionary/vector/etc. dependencies when the operation requires them.

func ProjectionColumnDependencies

func ProjectionColumnDependencies(column string, columnType ColumnType, span RowSpan, nullable bool) ([]SectionDependencyDescriptor, error)

type SectionDependencyKind

type SectionDependencyKind string

SectionDependencyKind names the logical section/payload class needed by an operation. It intentionally covers future codecs without requiring a planner.

const (
	SectionDependencyValues           SectionDependencyKind = "values"
	SectionDependencyDictionaries     SectionDependencyKind = "dictionaries"
	SectionDependencyOffsets          SectionDependencyKind = "offsets"
	SectionDependencyNullMask         SectionDependencyKind = "null_mask"
	SectionDependencyDefaultMask      SectionDependencyKind = "default_mask"
	SectionDependencyVisibility       SectionDependencyKind = "visibility"
	SectionDependencyStats            SectionDependencyKind = "stats"
	SectionDependencyPruningMetadata  SectionDependencyKind = "pruning_metadata"
	SectionDependencyVectorPayload    SectionDependencyKind = "vector_payload"
	SectionDependencyAdjacencyPayload SectionDependencyKind = "adjacency_payload"
)

type SectionReader

type SectionReader interface {
	ReadSection(section ColumnPartImageSection) ([]byte, error)
}

SectionReader is the narrow byte-access seam that #1754 can adapt to #1736 mappedresource-backed handles without changing the typed-column data-plane section directory or codecs.

type SortKey

type SortKey struct {
	Columns []SortKeyColumn
}

type SortKeyBound

type SortKeyBound struct {
	Values    []int64
	Exclusive bool
	Unbounded bool
}

type SortKeyColumn

type SortKeyColumn struct {
	Column    string
	Direction SortKeyDirection
	Nulls     SortKeyNullOrder
}

func DecodeColumnPartSortKeyMetadataSectionPayload

func DecodeColumnPartSortKeyMetadataSectionPayload(data []byte) ([]SortKeyColumn, error)

type SortKeyColumnValues

type SortKeyColumnValues struct {
	Name   string
	Values []int64
}

type SortKeyDirection

type SortKeyDirection string
const (
	SortKeyAsc  SortKeyDirection = "asc"
	SortKeyDesc SortKeyDirection = "desc"
)

type SortKeyMark

type SortKeyMark struct {
	Rows         int
	Columns      []string
	ColumnValues [][]int64
	Prefixes     []SortKeyPrefixSummary
}

func BuildSortKeyMark

func BuildSortKeyMark(columns []SortKeyColumnValues) (SortKeyMark, error)

func DecodeColumnPartSortKeyMarksSectionPayload

func DecodeColumnPartSortKeyMarksSectionPayload(data []byte) ([]SortKeyMark, error)

func (SortKeyMark) MayContainRanges

func (m SortKeyMark) MayContainRanges(ranges []Int64RangePredicate) (bool, bool, error)

type SortKeyNullOrder

type SortKeyNullOrder string
const (
	SortKeyNullsDefault SortKeyNullOrder = ""
	SortKeyNullsFirst   SortKeyNullOrder = "first"
	SortKeyNullsLast    SortKeyNullOrder = "last"
)

type SortKeyPrefixSummary

type SortKeyPrefixSummary struct {
	Columns        []string
	Lower          SortKeyBound
	UpperExclusive SortKeyBound
}

type TCS1PartRecord

type TCS1PartRecord struct {
	Version      uint16 `json:"version"`
	Kind         uint16 `json:"kind"`
	Flags        uint32 `json:"flags"`
	PartID       uint64 `json:"part_id"`
	Rows         int    `json:"rows"`
	ImageVersion uint16 `json:"image_version"`
	PayloadBytes int    `json:"payload_bytes"`
	TotalBytes   int    `json:"total_bytes"`
	PayloadCRC32 uint32 `json:"payload_crc32"`
}

func DecodeTCS1ColumnPartHeader

func DecodeTCS1ColumnPartHeader(header []byte, totalBytes int64) (TCS1PartRecord, error)

func EncodeTCS1ColumnPartImage

func EncodeTCS1ColumnPartImage(image ColumnPartImage) ([]byte, TCS1PartRecord, error)

type TimeBucketedCounts

type TimeBucketedCounts struct {
	BucketWidth int64
	MinBucket   int64
	Buckets     int
	Cardinality uint32
	Counts      []uint64
}

func (TimeBucketedCounts) Count

func (c TimeBucketedCounts) Count(bucket int64, code uint32) uint64

type Tombstone

type Tombstone struct {
	PrimaryID    int64
	GenerationID uint64
}

Tombstone removes a primary ID at or below the tombstone generation.

type Uint32CodeSelectionScratch

type Uint32CodeSelectionScratch struct {
	Rows   []int
	Ranges []RowRange
	Bitmap []uint64
	Bits   []uint64
}

Uint32CodeSelectionScratch owns caller/session-scoped temporary storage for low-cardinality dictionary-code predicate selections. Returned selections may alias this scratch until the next use.

type Uint32List

type Uint32List struct {
	Rows    int
	Offsets []uint64
	Values  []uint32
}

Uint32List is the generic owned representation for the v1 non-null variable-length uint32_list primitive. It is consumer-neutral storage machinery: HNSW adjacency may consume it, but graph-specific semantics belong above this layer. Offsets has Rows+1 entries and Values stores all row values concatenated; row i spans Values[Offsets[i]:Offsets[i+1]].

func (Uint32List) Row

func (l Uint32List) Row(row int) ([]uint32, error)

Row returns the row slice Values[Offsets[row]:Offsets[row+1]] after validating row bounds and host-int conversions. The returned slice aliases Values.

func (Uint32List) Validate

func (l Uint32List) Validate() error

Validate checks the complete offsets/value shape for this uint32_list value.

type Uint32ListView

type Uint32ListView = Uint32List

Uint32ListView is the same row-slicing contract used by direct views: Offsets and Values may alias a mmap/heap resource whose lifetime is owned elsewhere.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL