Documentation
¶
Index ¶
- Constants
- func BatchBytes(b *dataplane.Batch) int64
- func EncodeKey(record arrow.RecordBatch, row int, pkIdxs []int, pkCols []string) ([]byte, error)
- type Batch
- func AdversarialCompositeKey(alloc memory.Allocator) *Batch
- func AdversarialDeleteLast(alloc memory.Allocator) *Batch
- func AdversarialInsertAfterDelete(alloc memory.Allocator) *Batch
- func AdversarialInt64Overflow(alloc memory.Allocator) *Batch
- func AdversarialNullBefore(alloc memory.Allocator) *Batch
- func Collapse(ctx context.Context, alloc memory.Allocator, batch *Batch, pkCols []string) (upserts, deletes *Batch, err error)
- func GenerateBatch(seed int64, opts GeneratorOpts) *Batch
- type GeneratorOpts
Constants ¶
const ( OpInsert = 0 OpUpdate = 1 OpDelete = 2 )
Op values matching the wire schema __op column (CR-021).
Variables ¶
This section is empty.
Functions ¶
func BatchBytes ¶ added in v0.4.0
BatchBytes is a batch's in-memory size: the length of every buffer of every column, child arrays included. Rows carry payloads, so a row count alone is no bound on memory: batching limits check this as well (#437).
func EncodeKey ¶
EncodeKey encodes a composite key as type-tagged binary payload concat — used by collapse to avoid hash collisions (CR-069 §3.2).
pkIdxs are the pre-resolved data-column indices of the PK columns (resolved ONCE by the caller — M-9); pkCols ride along for error messages only. Null in any PK column is an error here — EncodeKey is the single authority for key validity (M-9).
Each field is [type-byte][payload…]: fixed-width payloads for numerics/bool/date/time/timestamp, length- prefixed for string/binary/decimal, raw fixed width for FixedSizeBinary. Different types cannot collide (distinct tags); equal values of the same type always produce equal keys (NaN/±0 compare by bit pattern — M-11).
INVARIANT: keys are EPHEMERAL — they live for the duration of one Collapse call and never persist or cross batches. If keys ever need to persist or be compared across batches, this design must be revisited (type widening across batches, e.g. int32→int64, would change keys). See website/docs/architecture/encode-key.md.
Types ¶
type Batch ¶
Batch is the seam type. It lives in the public dataplane package so that source, sink, and plugin contracts can reference it without violating the internal/ boundary. The alias means internal/dataplane and dataplane.Batch are the same type — zero conversion cost.
func AdversarialCompositeKey ¶
AdversarialCompositeKey produces a batch where naive key concatenation would collide: ("ab","c") vs ("a","bc").
func AdversarialDeleteLast ¶
AdversarialDeleteLast produces a batch whose last row is a DELETE.
func AdversarialInsertAfterDelete ¶
AdversarialInsertAfterDelete produces a batch where a row is inserted after being deleted in the same batch.
func AdversarialInt64Overflow ¶
AdversarialInt64Overflow produces a batch with int64 values > 2^53 that would be corrupted by JSON float64 (CR-021 regression test).
func AdversarialNullBefore ¶
AdversarialNullBefore produces a batch with null val/__before_val cells. Predicate evaluation over these nulls COALESCES to false — the point is the coalesce rule, not Kleene logic: a null never passes a predicate.
func Collapse ¶
func Collapse(ctx context.Context, alloc memory.Allocator, batch *Batch, pkCols []string) (upserts, deletes *Batch, err error)
Collapse deduplicates by PK, keeping last-occurrence per group in first-appearance emission order (CR-069 §3.2).
The encoding MUST be exact: length-prefixed binary concat of encoded key columns. Naive concat ("ab"+"c" == "a"+"bc") collides.
Null in a PK column is a source contract violation → error.
func GenerateBatch ¶
func GenerateBatch(seed int64, opts GeneratorOpts) *Batch
GenerateBatch creates a batch with controlled data for property tests.
type GeneratorOpts ¶
type GeneratorOpts struct {
NumRows int
PKDomain int // distinct PKs; 0 = max(1, NumRows/3) — duplicates by default
DeletesOnlyAtEnd bool // true = deletes only at last row; false (default) = deletes at any position
IncludeNullPK bool // true = one row with null PK
Allocator memory.Allocator // if nil, GoAllocator is used
}
GeneratorOpts configures batch generation.
Source Files