Documentation
¶
Overview ¶
Package quantizedasset validates quantized vector-index asset schemas and prepares allocation-conscious ordinal readers over typed-column part images.
The package deliberately stops at schema/identity validation and random ordinal access. It does not implement quantized scoring, search-mode selection, exact rerank, training, rebuild, or document reconstruction.
Index ¶
- func KnownRole(role Role) bool
- func Validate(schema SchemaDescriptor, expected ExpectedSchema) error
- type AssetRefIdentity
- type BaseGraphIdentity
- type CodeRowView
- type CodecDescriptor
- type ColumnDescriptor
- type ColumnFootprint
- type ExpectedSchema
- type Footprint
- type GraphOrdinalOrder
- type PartImageSource
- type PrepareRequest
- type Prepared
- func (p *Prepared) BytesPerRow(role Role) (int, bool)
- func (p *Prepared) CodeRowBytes(role Role, ordinal int) ([]byte, bool)
- func (p *Prepared) CodeRowView(role Role) (CodeRowView, bool)
- func (p *Prepared) DenseUint32Row(role Role, ordinal int, scratch []uint32) ([]uint32, bool)
- func (p *Prepared) ElementsPerRow(role Role) (int, bool)
- func (p *Prepared) Float32(role Role, ordinal int) (float32, bool)
- func (p *Prepared) Float32Payload(role Role) ([]byte, bool)
- func (p *Prepared) Footprint() Footprint
- func (p *Prepared) HasRole(role Role) bool
- func (p *Prepared) PackedElements(role Role, ordinal int, scratch []uint8) ([]uint8, bool)
- func (p *Prepared) RoleRows(role Role) (int, bool)
- func (p *Prepared) RowWords(role Role, ordinal int, scratch []uint64) (words []uint64, direct bool, ok bool)
- func (p *Prepared) Rows() int
- func (p *Prepared) Uint32(role Role, ordinal int) (uint32, bool)
- func (p *Prepared) Uint32Payload(role Role) ([]byte, bool)
- func (p *Prepared) Uint64(role Role, ordinal int) (uint64, bool)
- type Role
- type SchemaDescriptor
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Validate ¶
func Validate(schema SchemaDescriptor, expected ExpectedSchema) error
Validate checks only descriptor/expected schema metadata, without opening any typed-column images.
Types ¶
type AssetRefIdentity ¶
type AssetRefIdentity struct {
Present bool
Kind string
Namespace string
Generation uint64
PartID uint64
FileID uint32
Offset int64
Length int64
Checksum uint32
}
AssetRefIdentity is a typed-column asset identity independent of collections' concrete ColumnAssetRef type. Present=false skips ref checks for in-memory tests/fixtures; persisted callers should set it.
type BaseGraphIdentity ¶
type BaseGraphIdentity struct {
IndexName string
Field string
Metric string
Dimensions int
RowCount int
BaseManifestGeneration uint64
BaseManifestChecksum uint64
BaseSchemaHash uint64
GraphSchemaHash uint64
}
BaseGraphIdentity ties quantized state to the graph/base manifest whose row order it was built for.
type CodeRowView ¶
type CodeRowView struct {
// contains filtered or unexported fields
}
CodeRowView is an immutable, zero-copy fixed-width row view over a prepared code role. RowBytes slices the validated payload directly, avoiding per-row role-map lookups. The returned row slices alias prepared image bytes and must be treated as read-only.
func (CodeRowView) BytesPerRow ¶
func (v CodeRowView) BytesPerRow() int
BytesPerRow returns the validated physical bytes per code row.
func (CodeRowView) ElementsPerRow ¶
func (v CodeRowView) ElementsPerRow() int
ElementsPerRow returns the validated code elements per row.
func (CodeRowView) PayloadBytes ¶
func (v CodeRowView) PayloadBytes() ([]byte, bool)
PayloadBytes returns the validated row-major payload backing this view. The slice aliases immutable prepared image bytes and must be treated as read-only.
func (CodeRowView) Role ¶
func (v CodeRowView) Role() Role
Role returns the code role validated for this view.
func (CodeRowView) RowBytes ¶
func (v CodeRowView) RowBytes(ordinal int) ([]byte, bool)
RowBytes returns a zero-copy row byte slice by graph/vector ordinal. The slice aliases immutable prepared image bytes and must be treated as read-only.
func (CodeRowView) Rows ¶
func (v CodeRowView) Rows() int
Rows returns the number of graph/vector ordinal rows covered by the view.
func (CodeRowView) Valid ¶
func (v CodeRowView) Valid() bool
Valid reports whether the view was produced by Prepared.CodeRowView.
type CodecDescriptor ¶
CodecDescriptor is the fail-closed codec identity recorded by a quantized asset manifest. Config may be empty for codecs with no serialized config; ConfigHash is caller-defined manifest identity and is compared when non-zero.
type ColumnDescriptor ¶
type ColumnDescriptor struct {
Role Role
Column string
AssetID string
Required bool
LogicalType string
Type typedcolumn.ColumnType
Encoding typedcolumn.Encoding
RowCount int
ElementsPerRow int
BytesPerRow int
BitsPerElement int
SourceSchemaHash uint64
AssetBytes int64
Ref AssetRefIdentity
}
ColumnDescriptor binds one logical role to one typed-column column.
type ColumnFootprint ¶
type ColumnFootprint struct {
Role Role
Column string
AssetID string
LogicalType string
Type typedcolumn.ColumnType
Encoding typedcolumn.Encoding
Rows int
ElementsPerRow int
BytesPerRow int
BitsPerElement int
ElementWidthBytes int
SectionBytes int64
AssetBytes int64
BytesPerVector float64
}
ColumnFootprint records the typed-column section footprint for one role.
type ExpectedSchema ¶
type ExpectedSchema struct {
Metric string
VectorDimensions int
CodeDimensions int
CodeWidthBits int
RowCount int
GranuleCount int
OrdinalOrder GraphOrdinalOrder
Codec CodecDescriptor
BaseGraph BaseGraphIdentity
RequiredRoles []Role
}
ExpectedSchema is the scorer/index definition expected by the caller. Prepare compares SchemaDescriptor against this before any row access is returned.
type Footprint ¶
type Footprint struct {
Rows int
AssetBytes int64
SectionBytes int64
PaddingBytes int64
Columns []ColumnFootprint
BytesPerVector float64
}
Footprint records whole-asset and per-role storage accounting captured during prepare. BytesPerVector values are zero when Rows is zero.
type GraphOrdinalOrder ¶
type GraphOrdinalOrder string
GraphOrdinalOrder describes how typed-column rows line up with scorer input ordinals. The current contract is row i == graph/vector ordinal i; callers that need remapping must prepare that side state before using this package.
const GraphOrdinalOrderVectorOrdinal GraphOrdinalOrder = "vector_ordinal"
type PartImageSource ¶
type PartImageSource struct {
AssetID string
Image typedcolumn.ColumnPartImage
Ref AssetRefIdentity
AssetBytes int64
SourceSchemaHash uint64
}
PartImageSource supplies one typed-column part image referenced by descriptor columns. Image.Bytes must remain immutable for the lifetime of Prepared.
type PrepareRequest ¶
type PrepareRequest struct {
Schema SchemaDescriptor
Expected ExpectedSchema
Parts []PartImageSource
}
PrepareRequest validates a manifest schema against caller expectations and prepares allocation-free ordinal readers over typed-column section bytes.
type Prepared ¶
type Prepared struct {
// contains filtered or unexported fields
}
Prepared is immutable and safe for concurrent read-only ordinal access. Caller scratch passed to RowWords, PackedElements, or DenseUint32Row is caller-owned and must not be shared concurrently without synchronization.
func Prepare ¶
func Prepare(req PrepareRequest) (*Prepared, error)
Prepare validates req and returns prepared ordinal readers. It fails closed on schema, identity, typed-column layout, row-count, role, or ref mismatches.
func (*Prepared) BytesPerRow ¶
BytesPerRow returns the physical bytes per row for role.
func (*Prepared) CodeRowBytes ¶
CodeRowBytes returns a zero-copy row byte view for fixed-byte, packed-code, and dense unsigned code-vector roles. The slice aliases the prepared image.
func (*Prepared) CodeRowView ¶
func (p *Prepared) CodeRowView(role Role) (CodeRowView, bool)
CodeRowView returns a zero-copy fixed-width row view for fixed-byte, packed-code, and dense unsigned code-vector roles. Role/kind/shape/payload checks happen once when the view is created; per-row access only validates the ordinal before slicing the immutable prepared payload.
func (*Prepared) DenseUint32Row ¶
DenseUint32Row decodes a uint32_vector code row into caller scratch. If scratch is too small the method allocates.
func (*Prepared) ElementsPerRow ¶
ElementsPerRow returns the fixed vector/code elements per row for role.
func (*Prepared) Float32Payload ¶
Float32Payload returns the immutable little-endian scalar float32 payload for role. The slice aliases prepared image bytes and must be treated as read-only.
func (*Prepared) PackedElements ¶
PackedElements unpacks a packed-code row into caller scratch. If scratch is too small the method allocates; pass at least ElementsPerRow(role) bytes for steady-state allocation-free access.
func (*Prepared) RoleRows ¶
RoleRows returns the validated row count for a prepared role. Most code roles use the vector row count; side metadata roles may use a smaller row count such as one row per storage granule.
func (*Prepared) RowWords ¶
func (p *Prepared) RowWords(role Role, ordinal int, scratch []uint64) (words []uint64, direct bool, ok bool)
RowWords returns the code row as little-endian uint64 words. It uses caller scratch unless a future implementation can prove a direct word view; direct is currently false. If scratch is too small the method allocates.
func (*Prepared) Uint32Payload ¶
Uint32Payload returns the immutable little-endian scalar uint32 payload for role. The slice aliases prepared image bytes and must be treated as read-only.
type Role ¶
type Role string
Role names a logical quantized-vector asset column. These roles are schema metadata above typed-column storage; scoring/query semantics are intentionally not implemented here.
const ( RoleCodes Role = "codes" RolePackedCodes Role = "packed_codes" RoleNorm Role = "norm" RoleStep Role = "step" RoleLower Role = "lower" RoleCodeSum Role = "code_sum" RoleNorm2 Role = "norm2" RoleCodeCount Role = "code_count" RoleCentroidDistance Role = "centroid_distance" RoleQuantizedDotProductInv Role = "quantized_dot_product_inv" RoleCentroidDotProduct Role = "centroid_dot_product" RoleCentroidID Role = "centroid_id" RoleListID Role = "list_id" RoleScalarU8Alpha Role = "scalar_u8_alpha" RoleGranuleRowCount Role = "granule_row_count" )
type SchemaDescriptor ¶
type SchemaDescriptor struct {
Name string
Metric string
VectorDimensions int
CodeDimensions int
CodeWidthBits int
RowCount int
GranuleCount int
OrdinalOrder GraphOrdinalOrder
Codec CodecDescriptor
BaseGraph BaseGraphIdentity
Columns []ColumnDescriptor
}
SchemaDescriptor is the manifest-declared quantized asset schema. It maps logical roles to columns in one or more typed-column part images.
Source Files