Documentation
¶
Overview ¶
Package compile implements the IVF-PQ plugin's compile-layer (DDL) hooks.
Scope: anything that runs during DDL execution — CREATE INDEX, ALTER REINDEX, DROP INDEX, ALTER TABLE column changes that touch an indexed column. The four Hooks methods cover those four operations.
The lifted logic operates through a CompileContext provided by the SQL layer (pkg/sql/compile/plugin_context.go), so this package does not import pkg/sql/compile — that would create a cycle.
What CompileContext exposes (see pkg/indexplugin/compile/hooks.go for the contract):
Ctx() — request context.Context
Database() — engine.Database for the indexed table's db
QryDatabase() — database name from the parsed query
OriginalTableDef() — plan.TableDef of the parent (indexed) table
IndexInfo() — plan.CreateTable carrying the hidden-table
DDL during CREATE; nil during ALTER REINDEX
MainTableID() — parent table ID
MainExtra() — parent table SchemaExtra (gets mutated to
record new index-table IDs)
RunSql(sql) — execute a SQL statement in the current txn
BuildIndexTable(def) — create one hidden table from its TableDef
ResolveVariable(...) — system-variable lookup (proc.GetResolveVar)
Lifted from:
- pkg/sql/compile/ddl_index_algo.go:802 (handleVectorIvfpqIndex)
- pkg/sql/compile/util.go:740,757 (gen{Delete,Build}IvfpqIndex)
Index ¶
- type Hooks
- func (h Hooks) HandleCreateIndex(ctx compileplugin.CompileContext, indexDefs map[string]*plan.IndexDef) error
- func (Hooks) HandleDropIndex(_ compileplugin.CompileContext, defs map[string]*plan.IndexDef) error
- func (h Hooks) HandleReindex(ctx compileplugin.CompileContext, indexDefs map[string]*plan.IndexDef, ...) error
- func (Hooks) IdxcronMetadata(ctx compileplugin.CompileContext) ([]byte, error)
- func (Hooks) RestoreInitSQL(ctx compileplugin.CompileContext, indexDefs map[string]*plan.IndexDef) (bool, string, error)
- func (Hooks) ValidateReindexParams(old map[string]string, alter compileplugin.ReindexParamUpdate) (map[string]string, error)
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Hooks ¶
type Hooks struct{}
Hooks implements plugin/compile.Hooks for IVF-PQ.
All four methods below are required by the framework. If you add a hook to plugin/compile/hooks.go and don't implement it here, the `var _ compileplugin.Hooks = Hooks{}` interface check (at the bottom of this file) breaks the build.
func (Hooks) HandleCreateIndex ¶
func (h Hooks) HandleCreateIndex(ctx compileplugin.CompileContext, indexDefs map[string]*plan.IndexDef) error
HandleCreateIndex runs during CREATE INDEX (and as the worker for HandleReindex). Called once per multi-table index; indexDefs is keyed by IndexAlgoTableType (the same strings CatalogHooks.HiddenTableTypes() returns). For IVF-PQ that's {"ivfpq_meta", "ivfpq_index"}.
Responsibilities:
- Validate the indexDefs shape (number of tables, key parts).
- Create the hidden tables via ctx.BuildIndexTable.
- Clear stale runtime cache entries (vectorindex/cache).
- Wipe any pre-existing rows in the hidden tables (DELETE FROM ...).
- Populate the storage table from the source table — IVF-PQ uses CROSS APPLY ivfpq_create(...) which the engine routes to the ivfpq_create table-function builder in pkg/sql/plan/ivfpq.go.
The sync-vs-async branch is driven by the index's `async` IndexAlgoParam (catalog.IsIndexAsync). Default (key missing or "false"): forceSync=true — ivfpq_create runs inline before the CDC task is registered. Explicit async="true": forceSync=false — the build SQL is stashed as ConsumerInfo.InitSQL and runs at the first CDC iteration.
Lifted from Scope.handleVectorIvfpqIndex (pkg/sql/compile/ddl_index_algo.go:802).
func (Hooks) HandleDropIndex ¶
func (Hooks) HandleDropIndex(_ compileplugin.CompileContext, defs map[string]*plan.IndexDef) error
HandleDropIndex runs algorithm-specific cleanup beyond the generic hidden-table deletion the SQL layer already performs.
Implementations typically: unregister CDC tasks (DropIndexCdcTask), remove idxcron schedules, clear runtime caches.
IVF-PQ does none of those — generic hidden-table deletion is enough — so this is a no-op. Compare HNSW, which does maintain CDC tasks.
func (Hooks) HandleReindex ¶
func (h Hooks) HandleReindex(ctx compileplugin.CompileContext, indexDefs map[string]*plan.IndexDef, forceSync bool) error
HandleReindex runs during ALTER … REINDEX (foreground forceSync=false) and during idxcron background reindex (forceSync=true). The forceSync branch builds ivfpq_create synchronously inside the txn so the new tag=0 model lands before subsequent steps observe the index — mirrors IVF-FLAT and CAGRA.
func (Hooks) IdxcronMetadata ¶
func (Hooks) IdxcronMetadata(ctx compileplugin.CompileContext) ([]byte, error)
IdxcronMetadata pins IVF-PQ's build-time params into the cron task's metadata blob — see CAGRA's compile.go for the rationale. kmeans_train_percent is consumed at rebuild time by the cuvs ivfpq_create table function (pkg/sql/colexec/table_function/ ivfpq_create_gpu.go:310 — read via proc.GetResolveVariableFunc), so it must be pinned here for the value the user picked at CREATE INDEX to survive into the cron-triggered rebuild. kmeans_max_iteration is captured for parity with kmeans_train_percent and to future-proof against a downstream cuvs ivfpq_create that consumes it.
func (Hooks) RestoreInitSQL ¶
func (Hooks) RestoreInitSQL(ctx compileplugin.CompileContext, indexDefs map[string]*plan.IndexDef) (bool, string, error)
RestoreInitSQL — see CAGRA. Rebuilds the IVF-PQ index post-commit during restore.
func (Hooks) ValidateReindexParams ¶
func (Hooks) ValidateReindexParams(old map[string]string, alter compileplugin.ReindexParamUpdate) (map[string]string, error)
ValidateReindexParams is the per-algo arm of the ALTER … REINDEX parameter-change validator (originally pkg/sql/compile/ddl.go:929 switch). Receive `old` (the current params map for the index) and a ReindexParamUpdate carrying the user's new values; return the merged params or an error.
IVF-PQ supports updating `lists` at REINDEX time — mirrors IVF-FLAT since both algorithms key on the inverted-list count for their build.
Source Files