Documentation
¶
Overview ¶
Package tidb exposes TiDB's native VECTOR column type through the Core vector-store capability interfaces. Documents live in a regular TiDB table (id / content / metadata JSON / embedding VECTOR) reached over the MySQL wire protocol via `database/sql` + go-sql-driver/mysql. Documents containing media are rejected before indexing I/O because this adapter persists document text and metadata only.
Requirements: TiDB v8.4.0+, which is where PingCAP sets the floor for self-managed and Dedicated clusters while recommending v8.5.0 or later. The vector data type still carries a beta notice, so it "might be changed without prior notice". The HNSW vector index needs the function-expression form `((VEC_<metric>_DISTANCE(embedding))) USING HNSW` and is only available on TiKV-backed columnar storage in some deployments; the store creates it under StoreConfig.InitializeSchema = true and propagates any backend error so callers can react.
Distance metrics — they map to TiDB's built-in functions:
- DistanceCosine → `VEC_COSINE_DISTANCE`
- DistanceL2 → `VEC_L2_DISTANCE`
- DistanceNegativeIP → `VEC_NEGATIVE_INNER_PRODUCT`
Vector binding. TiDB accepts `'[v1,v2,...]'` text literals directly — the store renders them and binds as a regular `?` parameter, so no special vector codec is needed.
Filter visitor reaches into the JSON metadata column with `JSON_VALUE(metadata, '$.k')`, wrapping numeric / ordering comparisons in `CAST(... AS DECIMAL(65,30))`.
Partial writes. Index prepares one upsert and runs it per document without wrapping the batch in a transaction, so a failure leaves the rows already written in place. The returned error names the id that failed, and repeating the call is safe because the statement is idempotent per row.
Numeric comparisons cast to DECIMAL, not DOUBLE. DOUBLE is an approximate type whose 53-bit mantissa cannot hold every int64, so an id or timestamp past 2^53 would compare equal to its neighbor and match the wrong row. DECIMAL stores exact values up to the documented 65 digits, which covers every integer the filter AST can carry — and the AST compares as a rational precisely so an integer is never rounded to a float's precision.
See https://docs.pingcap.com/tidb/stable/vector-search-overview/ for the official reference.
Index ¶
- Constants
- type DistanceMetric
- type Store
- func (s *Store) DeleteIDs(ctx context.Context, ids []string) (err error)
- func (s *Store) DeleteWhere(ctx context.Context, expr filter.Predicate) (err error)
- func (s *Store) Index(ctx context.Context, request *vectorstore.IndexRequest) (err error)
- func (s *Store) Search(ctx context.Context, req *vectorstore.SearchRequest) (response *vectorstore.SearchResponse, err error)
- type StoreConfig
Constants ¶
const ( DefaultTableName = "vector_store" DefaultIDColumn = "id" DefaultContentColumn = "content" DefaultMetadataColumn = "metadata" DefaultEmbeddingColumn = "embedding" DefaultDistanceMetric = DistanceCosine )
Exported defaults keep constructor behavior visible and overridable.
const Provider = "TiDB"
Provider is the stable backend name for host-side attribution.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type DistanceMetric ¶
type DistanceMetric string
DistanceMetric selects the VEC_*_DISTANCE function used at query time.
const ( DistanceCosine DistanceMetric = "COSINE" DistanceL2 DistanceMetric = "L2" DistanceNegativeIP DistanceMetric = "NEGATIVE_INNER_PRODUCT" )
The metric is a closed vocabulary because score direction and threshold semantics depend on it: the same raw number means "near" under one metric and "far" under another, so an unrecognized value must be rejected rather than guessed.
func (DistanceMetric) String ¶
func (d DistanceMetric) String() string
func (DistanceMetric) Valid ¶
func (d DistanceMetric) Valid() bool
type Store ¶
type Store struct {
// contains filtered or unexported fields
}
Store implements vector-store capabilities with TiDB's native VECTOR column type and VEC_*_DISTANCE functions.
func NewStore ¶
func NewStore(ctx context.Context, config StoreConfig) (*Store, error)
NewStore performs schema setup during construction, which is why it takes a context: a store returned before its table and vector index exist would fail on the first index rather than at wiring, where the misconfiguration actually is.
func (*Store) DeleteIDs ¶
DeleteIDs removes rows by primary key — `DELETE ... WHERE <idCol> IN (?, ...)` with one placeholder per id. An empty slice is a no-op; unknown ids are silently ignored (idempotent). Implements vectorstore.IDDeleter.
func (*Store) DeleteWhere ¶
func (*Store) Index ¶
func (s *Store) Index(ctx context.Context, request *vectorstore.IndexRequest) (err error)
Index embeds documents and upserts them.
func (*Store) Search ¶
func (s *Store) Search(ctx context.Context, req *vectorstore.SearchRequest) (response *vectorstore.SearchResponse, err error)
Search runs an ANN search ordered by the configured distance function.
type StoreConfig ¶
type StoreConfig struct {
// DB is the database handle. Required. Use a *sql.DB built from
// github.com/go-sql-driver/mysql pointed at a TiDB cluster.
DB *sql.DB
SchemaName string
TableName string
IDColumn string
ContentColumn string
MetadataColumn string
EmbeddingColumn string
EmbeddingModel embedding.Model
DocumentBatcher vectorstore.Batcher
Dimensions int
DistanceMetric DistanceMetric
InitializeSchema bool
}
StoreConfig contains configuration options for the TiDB Vector store (TiDB 7.4+ with vector support enabled).
func (StoreConfig) Validate ¶
func (s StoreConfig) Validate() error
Source Files