Documentation
¶
Overview ¶
Package pgvector implements the second external VectorIndex adapter (ADR-05, REQ-VEC-001/002). It uses the pgx pure-Go driver (zero-CGO) to talk to a PostgreSQL database with the pgvector extension.
SERVER-TRACK BOUNDARY: this package is SERVER-track (arch_test.go bans it from local composition). The local composition path (internal/app) wires the sqlite_blob zero-CGO default and MUST NOT import this package. Provider selection (pgvector.New) happens ONLY in the server/external composition path. This preserves REQ-FOUND-001: CGO_ENABLED=0 local build with zero external vector dependencies.
ISOLATED REPLICA: SQLite remains authoritative for observation data. The pgvector table is a vector REPLICA keyed by observation ID (BIGINT PK from SQLite). This is NOT a full Postgres source-of-truth (W11) — it is a read-optional dense candidate source for vector similarity search. No observation metadata (content, timestamps, lifecycle) is replicated; only the vector embedding and the filter columns needed for PreFilter/PostFilter search.
Schema/table isolation: the adapter creates its own schema (default cortex_vector) and table (default embeddings). These identifiers are validated against a safe regex before interpolation into SQL (schema/table names cannot be parameterized in PostgreSQL DDL). All data values use parameterized queries ($N) — no string interpolation of user data.
Dimension/model mismatch are rejected FAIL-CLOSED before any DB call (REQ-VEC-001 dim-mismatch corruption pin; model-namespace mismatch).
No plaintext secrets: the DSN password is extracted at construction time and scrubbed from every error message via redactWith (REQ-CP-002). The DSN is NEVER logged or surfaced in error strings.
Statement/query timeouts: every operation runs under a context with a configurable deadline, AND each transaction sets statement_timeout via set_config (server-side kill on slow queries).
Index ¶
- Constants
- type Adapter
- func (a *Adapter) Capabilities(_ context.Context) (domain.Capabilities, error)
- func (a *Adapter) Close() error
- func (a *Adapter) Delete(ctx context.Context, ids []int64) error
- func (a *Adapter) Health(ctx context.Context) domain.Health
- func (a *Adapter) ID() string
- func (a *Adapter) Search(ctx context.Context, q domain.VectorQuery) ([]domain.VectorCandidate, error)
- func (a *Adapter) Upsert(ctx context.Context, points []domain.VectorPoint) error
- type AdapterConfig
- type SearchTuning
Constants ¶
const ( // EnvMaxParallelWorkersPerGather enables SET LOCAL // max_parallel_workers_per_gather = N inside the search transaction. // A positive integer enables the tuning; unset/invalid/<= 0 leaves the // default (no SET LOCAL, pool-level query, no transaction wrapper). EnvMaxParallelWorkersPerGather = "CORTEX_VECTOR_PGVECTOR_MAX_PARALLEL_WORKERS_PER_GATHER" // EnvDistanceMode selects the search distance operator. "cosine" (default, // also on unset) keeps the exact '<=>' cosine scan; "ip" switches to the // normalized inner-product '<#>' distance for pre-normalized corpora. EnvDistanceMode = "CORTEX_VECTOR_PGVECTOR_DISTANCE_MODE" // DistanceModeCosine / DistanceModeIP are the recognized mode values. DistanceModeCosine = "cosine" DistanceModeIP = "ip" )
Exact-scan tuning knobs for the pgvector adapter (REQ-RET-102). Both are config-gated: with the env unset the adapter behavior is byte-identical in semantics to the pre-tuning exact '<=>' cosine scan.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Adapter ¶
type Adapter struct {
// contains filtered or unexported fields
}
Adapter implements domain.VectorIndex over a PostgreSQL database with the pgvector extension via the pgx pure-Go driver.
func New ¶
func New(ctx context.Context, cfg AdapterConfig) (*Adapter, error)
New constructs a pgvector Adapter with a real pgxpool connection pool. The caller MUST be in the server/external composition path (this package is server-track). The returned Adapter owns its pool; Close closes it.
Bootstrap: New connects with a raw pgx.Conn first to create the extension, schema, table, and index, then creates the pool. This ensures the vector type exists before any pooled connection uses it.
func NewWithDB ¶
func NewWithDB(db pgvectorDB, cfg AdapterConfig) (*Adapter, error)
NewWithDB constructs an Adapter over a pre-built (or fake) DB. Used by tests and by compositions that manage the pool lifecycle externally. The caller controls whether Close closes the pool (see ownDB; default false).
func (*Adapter) Capabilities ¶
Capabilities declares the adapter's supported features for capability-driven strategy selection (ADR-05).
func (*Adapter) Close ¶
Close releases resources. If the adapter owns its pool (factory-built via New), the underlying pool is closed.
func (*Adapter) Delete ¶
Delete removes vectors by observation ID using a parameterized ANY($1) clause. Missing IDs are tolerated (idempotent delete).
func (*Adapter) Health ¶
Health reports the adapter's current health by pinging the PostgreSQL pool. The DSN/password is never included in the health message.
func (*Adapter) Search ¶
func (a *Adapter) Search(ctx context.Context, q domain.VectorQuery) ([]domain.VectorCandidate, error)
Search translates a domain.VectorQuery into a parameterized SELECT against the pgvector table. Filters are mapped to WHERE conditions (PostFilter). The score threshold is applied client-side after receiving results.
func (*Adapter) Upsert ¶
Upsert stores a batch of vectors. Dimension and model-namespace mismatch are rejected FAIL-CLOSED before any DB call (REQ-VEC-001 dim-mismatch pin; model mismatch). The batch is chunked at maxBatchSize within a transaction, and each chunk is written as ONE multi-row INSERT ... ON CONFLICT (id) DO UPDATE statement (pgvector batch pattern), removing the per-point round-trips. Duplicate ids inside a chunk are collapsed to the last occurrence (latest write wins) because ON CONFLICT DO UPDATE cannot affect the same row twice within a single statement.
type AdapterConfig ¶
type AdapterConfig struct {
DSN string // PostgreSQL runtime connection string
BootstrapDSN string // optional privileged DDL connection; empty falls back to DSN
Schema string // schema name (default cortex_vector)
Table string // table name (default embeddings)
Dimension int // expected vector dimension
ModelName string // model name for namespace enforcement (empty = skip)
IndexType string // hnsw or ivfflat (default hnsw)
HNSWM int // HNSW max connections per node (default 16, range 2-100)
HNSWEfConstruction int // HNSW dynamic candidate list for build (default 64, range 1-1000)
IVFFlatLists int // IVFFlat number of inverted lists (default 100, range 1-50000)
MaxBatchSize int // upsert batch ceiling (default 256)
Timeout time.Duration // per-operation timeout (default 30s)
MaxConns int32 // pool max connections (default 10)
StatementTimeoutMs int // PostgreSQL statement_timeout in ms (default 5000)
// GrantRoles names additional PostgreSQL roles to receive USAGE on the
// schema and SELECT on the table during bootstrap. The server composition
// passes the storage runtime role so the background embedding worker's
// RLS-bound observation query can check embedding existence through
// NOT EXISTS on this table (issue #115). Best-effort: a vector-dedicated
// database may legitimately not contain the storage role.
GrantRoles []string
// Tuning carries the config-gated exact-scan tuning knobs (REQ-RET-102).
// The zero value is fully disabled (byte-identical default cosine exact
// scan). The factory constructor New() populates it from the
// CORTEX_VECTOR_PGVECTOR_* environment pattern; NewWithDB callers set it
// explicitly (tests and external compositions stay deterministic).
Tuning SearchTuning
}
AdapterConfig holds the parameters to construct a pgvector Adapter. It mirrors config.PGVectorConfig but is declared here so the adapter package is self-contained (the server composition path maps config.PGVectorConfig → AdapterConfig). Index tuning uses typed validated integers — there is NO raw SQL string surface for index options (prevents injection via DDL).
type SearchTuning ¶ added in v2.4.14
type SearchTuning struct {
// MaxParallelWorkersPerGather > 0 enables SET LOCAL
// max_parallel_workers_per_gather scoped to the search transaction only.
MaxParallelWorkersPerGather int
// InnerProduct enables the normalized inner-product distance mode
// ('<#>') for pre-normalized corpora. Scores stay 1 - distance per the
// retrieval-engine contract; on unit vectors the ordering is identical to
// cosine ordering.
InnerProduct bool
}
SearchTuning holds the config-gated exact-scan tuning knobs. The zero value is fully disabled: default cosine operator, no SET LOCAL, no transaction wrapper (REQ-RET-102 "default behavior unchanged").
func SearchTuningFromEnv ¶ added in v2.4.14
func SearchTuningFromEnv(getenv func(string) string) SearchTuning
SearchTuningFromEnv builds SearchTuning from the CORTEX_VECTOR_PGVECTOR_* environment pattern (same style as internal/store/postgres CORTEX_VECTOR_PGVECTOR_DIMENSION handling). getenv is injected for deterministic unit testing; SearchTuningFromOS reads the real environment.
func SearchTuningFromOS ¶ added in v2.4.14
func SearchTuningFromOS() SearchTuning
SearchTuningFromOS reads the tuning env vars from the process environment.
func (SearchTuning) Enabled ¶ added in v2.4.14
func (t SearchTuning) Enabled() bool
Enabled reports whether any tuning knob is active. When false the search path is untouched (no transaction, default SQL).