Documentation
¶
Overview ¶
Package index is fast full-text search over your own data, typos forgiven.
A native-Go, multi-tenant full-text index on Base/SQLite that speaks the Meilisearch REST dialect.
It is the in-binary replacement for the standalone Meilisearch containers. Search is a subsystem of the one cloud binary like every other client, so it inherits per-org tenancy, encryption at rest (cek), and the platform's auth and o11y instead of running its own process with its own master key and its own RWO volume.
Why the Meilisearch dialect ¶
Hanzo Chat drives search through the `meilisearch@0.38` JS client and its mongoMeili Mongoose plugin. Speaking that dialect means chat points MEILI_HOST at this surface and needs no client change:
GET /v1/index/health {"status":"available"}
GET /v1/index/version
POST /v1/index/indexes {uid, primaryKey}
GET /v1/index/indexes/:uid
GET /v1/index/indexes/:uid/settings
PATCH /v1/index/indexes/:uid/settings {filterableAttributes}
POST /v1/index/indexes/:uid/documents [doc,…] add/replace
PUT /v1/index/indexes/:uid/documents [doc,…] update/upsert
GET /v1/index/indexes/:uid/documents ?limit&offset
GET /v1/index/indexes/:uid/documents/:id
DELETE /v1/index/indexes/:uid/documents/:id
POST /v1/index/indexes/:uid/documents/delete-batch [id,…]
POST /v1/index/indexes/:uid/search {q, filter, limit, offset}
GET /v1/index/tasks/:uid
Error bodies use Meilisearch's {message, code, type, link} shape rather than cloud's, because the JS client branches on those codes — index_not_found is how mongoMeili decides to create an index.
Tenancy ¶
A standalone Meilisearch has one global keyspace guarded by a master key, so every consumer sharing an instance shares its indexes. Here the tenant is principal.Org(c) — the value SanitizeIdentity minted from the VALIDATED bearer owner claim (HIP-0026), never a client-supplied header — and every query filters WHERE org=?. Two orgs may both hold an index named "messages" without ever seeing each other's documents. The bearer token is the org's cloud API key; the JS client already sends `Authorization: Bearer …`, so the wire shape is unchanged.
Within an org, chat scopes results to the end user with a `user = "<id>"` filter, which this surface honours as Meilisearch does.
Writes are synchronous ¶
Meilisearch queues writes and returns an EnqueuedTask. SQLite applies them before the response, so the task ids reported here are already complete and GET /tasks/:uid always reports `succeeded` — a client polling waitForTask resolves immediately rather than never.
Index ¶
- Constants
- Variables
- func Mount(app cloud.Router, deps cloud.Deps) error
- func ParseUserFilter(f any) []string
- func Query(ctx context.Context, org, uid, q string, limit, offset int) ([]json.RawMessage, error)
- func Ready() bool
- func Reconcile(ctx context.Context, org, uid, primaryKey string, docs []map[string]any) (kept, removed int, err error)
- func Shutdown() error
- type Index
- type Store
- func (s *Store) Close() error
- func (s *Store) Delete(ctx context.Context, org, uid string, pks []string) error
- func (s *Store) Document(ctx context.Context, org, uid, pk string) (json.RawMessage, error)
- func (s *Store) Documents(ctx context.Context, org, uid string, limit, offset int) ([]json.RawMessage, int, error)
- func (s *Store) DropIndex(ctx context.Context, org, uid string) error
- func (s *Store) EnsureIndex(ctx context.Context, org, uid, primaryKey string) (Index, error)
- func (s *Store) Index(ctx context.Context, org, uid string) (Index, error)
- func (s *Store) Indexes(ctx context.Context, org string) ([]Index, []int, error)
- func (s *Store) PKs(ctx context.Context, org, uid string) ([]string, error)
- func (s *Store) Ping(ctx context.Context) error
- func (s *Store) Search(ctx context.Context, org, uid, q string, users []string, limit, offset int) ([]json.RawMessage, error)
- func (s *Store) SetFilterable(ctx context.Context, org, uid string, attrs []string) error
- func (s *Store) Upsert(ctx context.Context, org, uid, primaryKey string, docs []map[string]any) error
Constants ¶
const Version = "1.0.0"
Version is the pkgVersion this surface reports to a Meilisearch client. It names the dialect implementation, not the Meilisearch release it emulates.
Variables ¶
var ErrNotMounted = errors.New("index: not mounted")
ErrNotMounted reports that the index subsystem is not mounted in this binary. The surface maps it to a DISABLED backend, never to a failed query — a deployment that does not run the index simply has no lexical leg.
Functions ¶
func ParseUserFilter ¶
ParseUserFilter extracts the user id(s) from a Meilisearch filter expression. Chat only ever sends `user = "<id>"`, but the array form and `user IN [...]` are accepted too. An expression naming no user yields nil, which callers read as "no user filter" — never as "match nothing".
func Query ¶
Query runs the org-scoped lexical search over one index. org MUST come from a validated principal; the store pins every row to it, so a caller can never read another tenant's documents. An index that does not exist yields no rows rather than an error: "nothing indexed yet" is an empty result, not a failure.
func Ready ¶
func Ready() bool
Ready reports whether the lexical leg can serve a query in this binary.
func Reconcile ¶
func Reconcile(ctx context.Context, org, uid, primaryKey string, docs []map[string]any) (kept, removed int, err error)
Reconcile REPLACES one index's whole corpus in a single idempotent call: every document is upserted and every key no longer present is deleted. It is Query's mirror — the write a subsystem that OWNS a corpus uses instead of POSTing its own documents back to itself through the Meilisearch dialect.
A full swap rather than incremental writes because the corpus's truth lives UPSTREAM (a git forge, a sites table): re-running a sync must converge, and a repo deleted upstream must leave the index. Same prune-on-index contract the code index already keeps.
org is the corpus's owner and is pinned into every row exactly as it is for a dialect write, so a corpus published under an org no principal can mint is readable by anyone allowed to query it and writable by nothing else.
Types ¶
type Index ¶
type Index struct {
UID string `json:"uid"`
PrimaryKey string `json:"primaryKey"`
FilterableAttributes []string `json:"filterableAttributes,omitempty"`
CreatedAt string `json:"createdAt"`
UpdatedAt string `json:"updatedAt"`
}
Index is the per-org index descriptor — Meilisearch's index object.
type Store ¶
type Store struct {
// contains filtered or unexported fields
}
Store is the index database. ONE SQLite file ({DataDir}/index.db) holds every org's indexes and documents; tenant isolation is the `org` column, enforced on EVERY query — the same storage shape clients/ads and clients/crm use. MaxOpenConns(1) serializes writes against the single-writer file.
Three tables, not one pair per index: a standalone Meilisearch mints a directory per index, but SQLite has no reason to. `org` and `uid` are ordinary columns, so a new index is a row, never DDL — which is what lets an untrusted uid be stored verbatim instead of sanitized into a table name.
Why the term table and not FTS5 ¶
FTS5 is a compile-time module, and cloud links the SYSTEM SQLite so the SQLCipher codec is real: that library is built by the distro, ships ENABLE_FTS3 and HAS_CODEC, and has no fts5 module — the `sqlite_fts5` build tag only affects the vendored amalgamation, so it is inert here. An index that depended on FTS5 would open on a pure-Go build, pass its tests, and then fail to create a single table in the shipped binary.
terms is an ordinary inverted index instead: one row per (document, term), keyed so a prefix query is an index range scan. It behaves identically in every build lane and costs nothing but rows.
func (*Store) Delete ¶
Delete removes documents by primary key from both the row store and the FTS index. Unknown keys are a no-op, as they are in Meilisearch.
func (*Store) Documents ¶
func (s *Store) Documents(ctx context.Context, org, uid string, limit, offset int) ([]json.RawMessage, int, error)
Documents pages an index in primary-key order and reports the total.
func (*Store) DropIndex ¶
DropIndex removes an index and every document and term in it, in ONE transaction so an index can never survive as a registry row with no documents or as orphaned documents with no registry row.
func (*Store) EnsureIndex ¶
EnsureIndex creates the index for (org, uid) if absent and returns it. An empty primaryKey defaults to "id", matching Meilisearch. Idempotent: an existing index keeps its original primary key, because changing it would silently orphan every document already stored under the old one.
func (*Store) Indexes ¶
Indexes lists an org's indexes in creation order, newest last, with the document count each one holds. It is the org's whole search surface in one answer — and the only way to see an index whose uid you do not already know.
func (*Store) PKs ¶
PKs lists an index's primary keys. Reconcile needs the KEYS and not the documents to work out what a full-corpus swap must prune; reading the doc column to throw it away would pull a whole corpus into memory for a set difference.
func (*Store) Search ¶
func (s *Store) Search(ctx context.Context, org, uid, q string, users []string, limit, offset int) ([]json.RawMessage, error)
Search returns the documents matching q, optionally narrowed to a set of users. Query terms are OR-joined prefixes, and a document matching MORE of them ranks higher — the only ranking signal an inverted index gives for free, and enough to put "kubernetes migration" above "kubernetes" for that query. Ties break on primary key so paging is stable. An empty q with no users lists the index, matching Meilisearch's placeholder search.
func (*Store) SetFilterable ¶
SetFilterable replaces the filterable attribute list for an index.
func (*Store) Upsert ¶
func (s *Store) Upsert(ctx context.Context, org, uid, primaryKey string, docs []map[string]any) error
Upsert adds or replaces documents in an index. Documents whose primary key is missing or empty are skipped, as Meilisearch skips documents it cannot key. The whole batch is one transaction, so a failure mid-batch leaves the index exactly as it was.
Source Files