typeddecode

package
v0.6.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 1, 2026 License: MIT Imports: 6 Imported by: 0

Documentation

Overview

Package typeddecode contains shared typed-column fast-decode planning and validated direct-view helpers. It deliberately keeps unsafe byte reinterpretation inside TreeDB/internal/mappedresource and only returns direct views after semantic, layout, writer-certification, row-count, endian, alignment, and lifetime checks have all succeeded.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

Types

type BytesDirectViewRequest

type BytesDirectViewRequest struct {
	Plan          Plan
	Certification typedcolumn.ColumnPartLayoutContractColumn
	Rows          int
	OffsetsBytes  int
	ValuesBytes   int
	// AssetOffset is the absolute byte offset of the containing asset in its
	// mapped storage segment. Direct-view eligibility checks
	// AssetOffset+OffsetsSection.Offset against 8-byte alignment. Byte values are
	// one-byte aligned but still require a known absolute storage identity.
	AssetOffset    int64
	HasAssetOffset bool
}

BytesDirectViewRequest validates split offsets/value sections for raw_bytes_offsets. OffsetsBytes is the uint64 offsets-section byte length; ValuesBytes is the exact concatenated opaque byte payload length.

type BytesShapeRequest

type BytesShapeRequest struct {
	Rows    int
	Offsets []uint64
	Values  uint64
}

BytesShapeRequest validates the bytes/raw_bytes_offsets primitive shape. Values is the number of payload bytes.

type Counter

type Counter string

Counter identifies a stable direct-view/fallback accounting bucket.

const (
	CounterMmapDirectView           Counter = "mmap_direct_view"
	CounterOffsetsMmapDirectView    Counter = "offsets_mmap_direct_view"
	CounterValuesMmapDirectView     Counter = "values_mmap_direct_view"
	CounterHeapCopyTypedView        Counter = "heap_copy_typed_view"
	CounterOffsetsHeapCopyTypedView Counter = "offsets_heap_copy_typed_view"
	CounterValuesHeapCopyTypedView  Counter = "values_heap_copy_typed_view"
	CounterScratchDecode            Counter = "scratch_decode"
	CounterStreamingFallback        Counter = "streaming_fallback"
	CounterSourceUnsupported        Counter = "source_unsupported"
	CounterCertificationFailure     Counter = "certification_failure"
	CounterAbsoluteOffsetUnaligned  Counter = "absolute_offset_unaligned"
	CounterActualPointerUnaligned   Counter = "actual_pointer_unaligned"
	CounterStaleHandle              Counter = "stale_handle"
	CounterOffsetsListValidation    Counter = "offsets_list_validation_failure"
)

func CounterVocabulary

func CounterVocabulary() []Counter

CounterVocabulary returns the stable counter names that benchmark/reporting code should use when distinguishing zero-copy mmap views from safe fallbacks.

type Counters

type Counters struct {
	DirectViewPlans     uint64
	StreamingPlans      uint64
	MaterializePlans    uint64
	UnsupportedPlans    uint64
	DirectViewSuccesses uint64
	DirectViewFailures  uint64
	FallbackReasons     map[Reason]uint64
}

Counters is a small shared accounting shape for prepared scans and future kernels. It is caller-owned; there is no package-global cache.

func (*Counters) ObserveDirectViewStatus

func (c *Counters) ObserveDirectViewStatus(s Status)

func (*Counters) ObservePlan

func (c *Counters) ObservePlan(p Plan)

func (*Counters) ObserveStatus

func (c *Counters) ObserveStatus(s Status)

type DirectViewBlockRequest

type DirectViewBlockRequest struct {
	Plan          Plan
	Certification typedcolumn.ColumnPartLayoutContractColumn
	Block         typedcolumn.ColumnPartLayoutContractBlock
	Rows          int
	PayloadBytes  int
	// AssetOffset is the absolute byte offset of the containing asset in its
	// mapped storage segment. Direct-view eligibility requires
	// AssetOffset+Block.PayloadOffset to satisfy Certification.Alignment; relative
	// image-local alignment alone is not sufficient.
	AssetOffset    int64
	HasAssetOffset bool
}

DirectViewBlockRequest validates one payload/block against a direct-view plan.

type DirectViewColumnRequest

type DirectViewColumnRequest struct {
	Plan          Plan
	Certification typedcolumn.ColumnPartLayoutContractColumn
	Rows          int
	PayloadBytes  int
	// AssetOffset is the absolute byte offset of the containing asset in its
	// mapped storage segment. Direct-view eligibility requires
	// AssetOffset+Certification.Section.Offset and each block payload offset to
	// satisfy Certification.Alignment.
	AssetOffset    int64
	HasAssetOffset bool
}

DirectViewColumnRequest validates a complete fixed-width column-data section before callers expose a section-wide direct view.

type GraphBytesDirectViewRequest

type GraphBytesDirectViewRequest struct {
	Expectation        GraphDirectViewExpectation
	Certification      typedcolumn.ColumnPartLayoutContractColumn
	OffsetsSection     typedcolumn.ColumnPartImageSection
	ValuesSection      typedcolumn.ColumnPartImageSection
	ExpectedOffsetsKey mappedresource.Key
	ExpectedValuesKey  mappedresource.Key
	OffsetsHandle      *mappedresource.Handle
	ValuesHandle       *mappedresource.Handle
	Manager            *mappedresource.Manager
}

type GraphDirectViewExpectation

type GraphDirectViewExpectation struct {
	ExpectedOwner  string
	ActualOwner    string
	ExpectedRole   string
	ActualRole     string
	Column         string
	Rows           int
	AssetOffset    int64
	HasAssetOffset bool
}

GraphDirectViewExpectation is the caller-supplied identity and shape contract for graph-search typed-column direct-view certification. The reusable helper validates these fields against the writer-certified typed-column layout and the live mappedresource handles before returning a prepared direct view.

ExpectedOwner/ExpectedRole identify the graph/typed-column state the caller is preparing, while ActualOwner/ActualRole must come from the caller's manifest or asset context. The primitive certifier intentionally does not replace graph-owned TVIS/manifest validation; it fails closed when those supplied identities do not match.

type GraphFloat32DirectViewRequest

type GraphFloat32DirectViewRequest struct {
	Expectation   GraphDirectViewExpectation
	Certification typedcolumn.ColumnPartLayoutContractColumn
	Section       typedcolumn.ColumnPartImageSection
	ExpectedKey   mappedresource.Key
	Handle        *mappedresource.Handle
	Manager       *mappedresource.Manager
}

type GraphFloat32VectorDirectViewRequest

type GraphFloat32VectorDirectViewRequest struct {
	Expectation   GraphDirectViewExpectation
	Dims          int
	Certification typedcolumn.ColumnPartLayoutContractColumn
	Section       typedcolumn.ColumnPartImageSection
	ExpectedKey   mappedresource.Key
	Handle        *mappedresource.Handle
	Manager       *mappedresource.Manager
}

type GraphInt64DirectViewRequest

type GraphInt64DirectViewRequest struct {
	Expectation   GraphDirectViewExpectation
	Certification typedcolumn.ColumnPartLayoutContractColumn
	Section       typedcolumn.ColumnPartImageSection
	ExpectedKey   mappedresource.Key
	Handle        *mappedresource.Handle
	Manager       *mappedresource.Manager
}

type GraphUint32ListDirectViewRequest

type GraphUint32ListDirectViewRequest struct {
	Expectation        GraphDirectViewExpectation
	Certification      typedcolumn.ColumnPartLayoutContractColumn
	OffsetsSection     typedcolumn.ColumnPartImageSection
	ValuesSection      typedcolumn.ColumnPartImageSection
	ExpectedOffsetsKey mappedresource.Key
	ExpectedValuesKey  mappedresource.Key
	OffsetsHandle      *mappedresource.Handle
	ValuesHandle       *mappedresource.Handle
	Manager            *mappedresource.Manager
}

type Path

type Path string

Path is the explicit fast-decode path selected for one column/operator/block.

const (
	PathDirectView  Path = "direct_view"
	PathStreaming   Path = "streaming"
	PathMaterialize Path = "materialize"
	PathUnsupported Path = "unsupported"
)

type Plan

type Plan struct {
	Path              Path
	Reason            Reason
	Message           string
	ElementSize       int
	ElementsPerRow    int
	BytesPerRow       int
	BitsPerElement    int
	LogicalBitsPerRow int
	Alignment         int
	Rows              int
}

Plan is a reusable per-column/operator decision. Per-block and per-handle validation still has to run before exposing a direct-view slice.

func AdjacencyListPlan

func AdjacencyListPlan(cert typedcolumn.ColumnPartLayoutContractColumn, degree int) Plan

func AdjacencyOffsetsListPlan

func AdjacencyOffsetsListPlan(cert typedcolumn.ColumnPartLayoutContractColumn) Plan

AdjacencyOffsetsListPlan selects a direct-view candidate for the legacy adjacency_list/raw_uint32_offsets_list compatibility selector. Graph-specific naming is quarantined by #1989; prefer Uint32ListPlan for generic datastore primitives.

func BytesPlan

BytesPlan selects a direct-view candidate only for writer-certified generic bytes/raw_bytes_offsets sections: little-endian uint64 offsets plus exact uninterpreted value bytes.

func DenseFixedWidthVectorBytesPlan

func DenseFixedWidthVectorBytesPlan(cert typedcolumn.ColumnPartLayoutContractColumn, logical columnsemantics.LogicalType, physical typedcolumn.ColumnType, encoding typedcolumn.Encoding, elementSize int, elementsPerRow int) Plan

AdjacencyListPlan intentionally keeps the legacy dense fixed-degree adjacency layout as deferred/fallback-only. The #1901 offsets-list path is quarantined compatibility; generic uint32_list direct-view planning owns the reusable raw_uint32_offsets_list mechanics.

func DenseFloat32VectorPlan

func DenseFloat32VectorPlan(cert typedcolumn.ColumnPartLayoutContractColumn, dims int) Plan

DenseFloat32VectorPlan selects a direct-view candidate only for writer- certified raw little-endian float32_vector sections with the requested fixed dimension. Callers must still validate each column/block payload and handle lifetime before exposing the returned []float32.

func FixedBytesPlan

func FixedBytesPlan(cert typedcolumn.ColumnPartLayoutContractColumn, bytesPerRow int) Plan

FixedBytesPlan selects a direct-view candidate only for writer-certified fixed_bytes/raw_fixed_bytes sections with the requested fixed bytes per row.

func Float32ScalarPlan

func Float32ScalarPlan(cert typedcolumn.ColumnPartLayoutContractColumn) Plan

Float32ScalarPlan selects a direct-view candidate only for writer-certified native raw little-endian float32 scalar sections. Raw-int64 compatibility carriers for logical float32 remain non-native fallback layouts.

func Float64ScalarPlan

func Float64ScalarPlan(cert typedcolumn.ColumnPartLayoutContractColumn) Plan

Float64ScalarPlan selects a direct-view candidate only for writer-certified native raw little-endian float64/double scalar sections. Raw-int64 compatibility carriers for logical double remain non-native fallback layouts.

func Int64ReducerPlan

Int64ReducerPlan chooses direct_view for certified raw int64 layouts and streaming for certified delta/double-delta or raw layouts that cannot direct view. Materialization is intentionally not selected by the shared int64 aggregate planner.

func PackedUintVectorPlan

func PackedUintVectorPlan(cert typedcolumn.ColumnPartLayoutContractColumn, logical columnsemantics.LogicalType, elementsPerRow int, bitsPerElement int) Plan

PackedUintVectorPlan selects a direct-view candidate only for writer-certified packed_bit_vector / packed_uint{2,4}_vector sections. The returned direct view is a byte payload; BitsPerElement and LogicalBitsPerRow describe packed-code interpretation.

func Uint32ListPlan

func Uint32ListPlan(cert typedcolumn.ColumnPartLayoutContractColumn) Plan

Uint32ListPlan selects a direct-view candidate only for writer-certified generic uint32_list/raw_uint32_offsets_list sections: little-endian uint64 offsets plus little-endian uint32 values.

func (Plan) DirectCandidate

func (p Plan) DirectCandidate() bool

func (Plan) Status

func (p Plan) Status() Status

type PreparedBytesDirectView

type PreparedBytesDirectView struct {
	Expectation   GraphDirectViewExpectation
	Rows          int
	Offsets       []uint64
	Values        []byte
	OffsetsHandle *mappedresource.Handle
	ValuesHandle  *mappedresource.Handle
}

PreparedBytesDirectView is a certified raw_bytes_offsets view.

func (PreparedBytesDirectView) Alive

func (v PreparedBytesDirectView) Alive() bool

func (*PreparedBytesDirectView) Close

func (v *PreparedBytesDirectView) Close() error

func (PreparedBytesDirectView) Row

func (v PreparedBytesDirectView) Row(row int) []byte

type PreparedFloat32DirectView

type PreparedFloat32DirectView struct {
	Expectation GraphDirectViewExpectation
	Rows        int
	Values      []float32
	Handle      *mappedresource.Handle
}

PreparedFloat32DirectView is a certified fixed-width float32 scalar view.

func (PreparedFloat32DirectView) Alive

func (v PreparedFloat32DirectView) Alive() bool

func (*PreparedFloat32DirectView) Close

func (v *PreparedFloat32DirectView) Close() error

func (PreparedFloat32DirectView) Value

func (v PreparedFloat32DirectView) Value(row int) (float32, bool)

type PreparedFloat32VectorDirectView

type PreparedFloat32VectorDirectView struct {
	Expectation GraphDirectViewExpectation
	Rows        int
	Dims        int
	Values      []float32
	Handle      *mappedresource.Handle
}

PreparedFloat32VectorDirectView is a certified row-major graph-search vector view. Values aliases Handle and is valid only while Handle remains live.

func (PreparedFloat32VectorDirectView) Alive

func (*PreparedFloat32VectorDirectView) Close

func (PreparedFloat32VectorDirectView) Row

type PreparedInt64DirectView

type PreparedInt64DirectView struct {
	Expectation GraphDirectViewExpectation
	Rows        int
	Values      []int64
	Handle      *mappedresource.Handle
}

PreparedInt64DirectView is a certified fixed-width int64 scalar view.

func (PreparedInt64DirectView) Alive

func (v PreparedInt64DirectView) Alive() bool

func (*PreparedInt64DirectView) Close

func (v *PreparedInt64DirectView) Close() error

func (PreparedInt64DirectView) Value

func (v PreparedInt64DirectView) Value(row int) (int64, bool)

type PreparedUint32ListDirectView

type PreparedUint32ListDirectView struct {
	Expectation   GraphDirectViewExpectation
	Rows          int
	Offsets       []uint64
	Values        []uint32
	OffsetsHandle *mappedresource.Handle
	ValuesHandle  *mappedresource.Handle
}

PreparedUint32ListDirectView is a certified raw_uint32_offsets_list view.

func (PreparedUint32ListDirectView) Alive

func (*PreparedUint32ListDirectView) Close

func (PreparedUint32ListDirectView) Row

type Reason

type Reason string

Reason is a stable planning/fallback diagnostic token. Add new values rather than changing strings; tests and PR benchmark output may key on them.

const (
	ReasonSupported               Reason = "supported"
	ReasonUnsupportedOperation    Reason = "unsupported_operation"
	ReasonLayoutCapability        Reason = "layout_capability"
	ReasonNotWriterCertified      Reason = "writer_certification_missing"
	ReasonCompressed              Reason = "compressed"
	ReasonVariableWidth           Reason = "variable_width"
	ReasonNullableWrapper         Reason = "nullable_default_wrapper"
	ReasonWrongEndian             Reason = "wrong_endian"
	ReasonLengthMultipleMismatch  Reason = "length_multiple_mismatch"
	ReasonPayloadLengthMismatch   Reason = "payload_length_mismatch"
	ReasonRowCountMismatch        Reason = "row_count_mismatch"
	ReasonDimensionMismatch       Reason = "dimension_mismatch"
	ReasonOffsetsCountMismatch    Reason = "offsets_count_mismatch"
	ReasonOffsetsStartMismatch    Reason = "offsets_start_mismatch"
	ReasonOffsetsNonMonotonic     Reason = "offsets_non_monotonic"
	ReasonOffsetsGoIntRange       Reason = "offsets_go_int_range"
	ReasonValuesLengthMismatch    Reason = "values_length_mismatch"
	ReasonAbsoluteOffsetUnaligned Reason = "absolute_offset_unaligned"
	ReasonActualPointerUnaligned  Reason = "actual_pointer_unaligned"
	ReasonDirectViewDeferred      Reason = "direct_view_deferred"
	// ReasonUnaligned is retained for compatibility with older diagnostics;
	// new direct-view validation distinguishes absolute storage offset alignment
	// from actual Go pointer alignment.
	ReasonUnaligned                  Reason = "unaligned"
	ReasonNilHandle                  Reason = "nil_handle"
	ReasonStaleHandle                Reason = "stale_handle"
	ReasonHandleSourceUnsupported    Reason = "handle_source_unsupported"
	ReasonDictionarySemanticsMissing Reason = "dictionary_semantics_missing"
	ReasonMaterializationRequired    Reason = "materialization_required"
	ReasonValidationFailed           Reason = "validation_failed"
	ReasonOwnerMismatch              Reason = "owner_mismatch"
	ReasonRoleMismatch               Reason = "role_mismatch"
	ReasonColumnMismatch             Reason = "column_mismatch"
	ReasonTypeEncodingMismatch       Reason = "type_encoding_mismatch"
	ReasonResourceMismatch           Reason = "resource_mismatch"
)

func ReasonVocabulary

func ReasonVocabulary() []Reason

ReasonVocabulary returns stable direct-view/fallback reason names that tests, diagnostics, and benchmark summaries may key on.

type ResourceViewOptions

type ResourceViewOptions struct {
	// ExpectedElements validates the view length. Use a negative value to skip
	// length validation; zero intentionally means an empty view is expected.
	ExpectedElements int
	RequireMapped    bool
}

ResourceViewOptions controls handle-level direct-view validation.

type Status

type Status struct {
	Path    Path
	Reason  Reason
	Message string
	Err     error
}

Status describes the outcome of planning or validating a fast-decode path.

func BytesView

func BytesView(mgr *mappedresource.Manager, offsetsHandle *mappedresource.Handle, valuesHandle *mappedresource.Handle, req BytesDirectViewRequest, opts ResourceViewOptions) ([]uint64, []byte, Status)

func DirectStatus

func DirectStatus() Status

func Float32ByteView

func Float32ByteView(raw []byte, opts ResourceViewOptions) ([]float32, Status)

Float32ByteView validates and exposes immutable bytes as []float32 without a mappedresource handle. Callers are responsible for tying the byte slice to an explicit lifetime; handle-backed optimized paths should prefer Float32View, Float32ScalarView, or DenseFloat32VectorView.

func MaterializeStatus

func MaterializeStatus(reason Reason, msg string) Status

func StreamingStatus

func StreamingStatus(reason Reason, msg string) Status

func Uint32ByteView

func Uint32ByteView(raw []byte, opts ResourceViewOptions) ([]uint32, Status)

Uint32ByteView validates and exposes immutable bytes as []uint32 without a mappedresource handle. Callers are responsible for tying the byte slice to an explicit lifetime; handle-backed optimized paths should prefer Uint32View or AdjacencyListView.

func Uint64ByteView

func Uint64ByteView(raw []byte, opts ResourceViewOptions) ([]uint64, Status)

Uint64ByteView validates and exposes immutable bytes as []uint64 without a mappedresource handle. Callers are responsible for tying the byte slice to an explicit lifetime; handle-backed optimized paths should prefer Uint64View.

func UnsupportedStatus

func UnsupportedStatus(reason Reason, msg string) Status

func ValidateBytesDirectView

func ValidateBytesDirectView(req BytesDirectViewRequest, offsets []uint64, values []byte) Status

func ValidateBytesDirectViewSections

func ValidateBytesDirectViewSections(req BytesDirectViewRequest) Status

func ValidateBytesShape

func ValidateBytesShape(req BytesShapeRequest) Status

func ValidateDirectViewBlock

func ValidateDirectViewBlock(req DirectViewBlockRequest) Status

func ValidateDirectViewColumn

func ValidateDirectViewColumn(req DirectViewColumnRequest) Status

func ValidateUint32OffsetsListDirectView

func ValidateUint32OffsetsListDirectView(req Uint32OffsetsListDirectViewRequest, offsets []uint64, values []uint32) Status

func ValidateUint32OffsetsListDirectViewSections

func ValidateUint32OffsetsListDirectViewSections(req Uint32OffsetsListDirectViewRequest) Status

func ValidateUint32OffsetsListShape

func ValidateUint32OffsetsListShape(req Uint32OffsetsListShapeRequest) Status

func (Status) Direct

func (s Status) Direct() bool

func (Status) Streaming

func (s Status) Streaming() bool

func (Status) String

func (s Status) String() string

func (Status) Unsupported

func (s Status) Unsupported() bool

type Uint32OffsetsListDirectViewRequest

type Uint32OffsetsListDirectViewRequest struct {
	Plan          Plan
	Certification typedcolumn.ColumnPartLayoutContractColumn
	Rows          int
	OffsetsBytes  int
	ValuesBytes   int
	// AssetOffset is the absolute byte offset of the containing asset in its
	// mapped storage segment. Direct-view eligibility checks
	// AssetOffset+OffsetsSection.Offset against 8-byte alignment and
	// AssetOffset+ValuesSection.Offset against 4-byte alignment.
	AssetOffset    int64
	HasAssetOffset bool
}

Uint32OffsetsListDirectViewRequest validates split offsets/value sections for raw_uint32_offsets_list. OffsetsBytes is the uint64 offsets-section byte length; ValuesBytes is the uint32 flattened values-section byte length.

type Uint32OffsetsListShapeRequest

type Uint32OffsetsListShapeRequest struct {
	Rows    int
	Offsets []uint64
	Values  uint64
}

Uint32OffsetsListShapeRequest validates the #1914 primitive shape without constructing an unsafe direct view. Values is the number of uint32 values, not the values byte length.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL