search

package
v0.2.46 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 8, 2026 License: MPL-2.0 Imports: 6 Imported by: 0

Documentation

Overview

Package search is the chassis-owned lexical search store: a durable, tenant-scoped place to index text records and find them again by the words they contain, exposed to txcl as txco://search/*.

**Separation of concerns.** This package does lexical retrieval and nothing else. It stores no embeddings and never talks to a model. Semantic retrieval is chassis/vector's job; fusing the two lanes belongs to the stack, above both primitives.

**The collection is the ranking unit.** Term statistics are local to one (tenant, collection), so one collection's documents never move another's ranking. A query addresses exactly one collection.

**One filter grammar.** Filter is the vector store's Filter, by alias, so a stack narrows both retrieval lanes with the same object and the two cannot drift.

Index

Constants

View Source
const (
	CodeDisabled = "txco_search_disabled"
	CodeNoTenant = "txco_search_no_tenant"
	// CodeStore is the code of any backend error that carries none.
	CodeStore = "txco_search_store"
)

Codes the op layer raises itself, for conditions no backend sees.

View Source
const (
	MaxTextBytes       = 64 << 10
	MaxShortFieldBytes = 1 << 10 // title, heading, name
	MaxEntities        = 64
	MaxEntityBytes     = 256
	MaxMetadataBytes   = 16 << 10
	MaxMetadataKeys    = 64
	MaxIDBytes         = 512
	MaxItemsPerUpsert  = 500
	MaxIDsPerDelete    = 500
	MaxQueryBytes      = 8 << 10
	MaxNameBytes       = 256 // a collection name
	DefaultLimit       = 10
	MaxLimit           = 100
)

Limits of the stack-facing API. They are public limits, not engine limits: every backend enforces the same ones by calling the Validate functions, so a request that works on a laptop works hosted.

View Source
const CountPending = -1

CountPending is returned in place of a count by a backend that accepted a Delete or an Update but has not applied it yet. The bundled backend applies synchronously and never returns it.

View Source
const ManifestFile = "manifest.json"

ManifestFile is the manifest's name inside a snapshot directory.

View Source
const StoreNone = "none"

StoreNone is the --search-store value that turns lexical search off. Open returns a nil Store for it, and the ops answer txco_search_disabled.

Variables

This section is empty.

Functions

func Register

func Register(name string, c Constructor)

Register adds a backend constructor. Called from a backend package's init().

func ValidateCollectionName

func ValidateCollectionName(name string) error

ValidateCollectionName rejects a name a backend could not address.

func ValidateItems

func ValidateItems(items []Item) error

ValidateItems checks one upsert: its size, and each item's id and fields. A repeated id inside one upsert is refused, because "which one wins" would differ between backends.

func ValidateQuery

func ValidateQuery(q string, limit int) (int, error)

ValidateQuery checks a query string and clamps its limit: zero or less takes DefaultLimit, and anything above MaxLimit is MaxLimit.

func ValidateSelector

func ValidateSelector(sel Selector) error

ValidateSelector checks that a Delete names records exactly one way.

func ValidateUpdate

func ValidateUpdate(filter Filter, ch Change) error

ValidateUpdate checks that an Update is conditional and changes something.

Types

type AnalyzerMismatchError

type AnalyzerMismatchError struct {
	Collection string
	Field      string // "analyzer_version" | "scoring_model"
	Existing   string
	Requested  string
}

AnalyzerMismatchError is returned when a collection is pinned to an analyzer or scoring model other than the one asked for, or than this build provides. The fix is a re-index of the collection, never an edit of the pin.

func (*AnalyzerMismatchError) Code

func (e *AnalyzerMismatchError) Code() string

func (*AnalyzerMismatchError) Error

func (e *AnalyzerMismatchError) Error() string

type Change

type Change struct {
	Merge  map[string]any `json:"merge,omitempty"`
	Fields *FieldSet      `json:"fields,omitempty"`
}

Change is what an Update does to each record its filter admits. Merge is a shallow metadata merge, RFC 7396 style: a key replaces the record's key, and a nil value removes it. Fields replaces short searched fields. Text is never changed by an update: new text is a new upsert.

func (Change) Empty

func (c Change) Empty() bool

Empty reports whether the change would alter nothing.

type CodedError

type CodedError interface {
	error
	Code() string
}

CodedError carries a stable txco_search_* code surfaced on the op envelope's `search.error.code` so rule authors dispatch uniformly with `WHEN @search.error EXEC ...`.

type Collection

type Collection struct {
	Name            string `json:"name"`
	AnalyzerVersion string `json:"analyzer_version"`
	ScoringModel    string `json:"scoring_model"`
	// Records is filled by DescribeCollection and ListCollections.
	Records uint64 `json:"records"`
}

Collection is one lexical corpus: the ranking unit, the physical index unit, and the unit a query addresses. AnalyzerVersion and ScoringModel are pinned when the collection is created. Changing either means re-indexing it.

type CollectionNotFoundError

type CollectionNotFoundError struct {
	Tenant     string
	Collection string
}

CollectionNotFoundError is returned by Upsert/Query/Delete/Update for an unknown (tenant, collection).

func (*CollectionNotFoundError) Code

func (e *CollectionNotFoundError) Code() string

func (*CollectionNotFoundError) Error

func (e *CollectionNotFoundError) Error() string

type Condition

type Condition = vector.Condition

Condition is one Filter clause.

type Config

type Config struct {
	// Path is the bundled backend's root directory (--search-path).
	Path string
	// MaxOpenIndexes bounds how many collection indexes the bundled backend
	// holds open at once (--search-max-open-indexes). Zero takes its default.
	MaxOpenIndexes int
}

Config carries backend-selecting options resolved from chassis config. A backend reads only the fields it needs. Adding a field here doesn't affect existing backends (same posture as vector.Config).

type Constructor

type Constructor func(Config) (Store, error)

Constructor builds a Store from resolved config.

type FieldSet

type FieldSet struct {
	Name    *string `json:"name,omitempty"`
	Title   *string `json:"title,omitempty"`
	Heading *string `json:"heading,omitempty"`
}

FieldSet replaces the short searched fields it names. A nil field is left alone; a pointer to "" clears it.

type Filter

type Filter = vector.Filter

Filter narrows a query, a delete or an update by metadata. It is the vector store's grammar (eq, in, not_in, gte, lte, gt, lt; conjunction), including its two rules: the special field "id" addresses the record id, and an absent metadata field passes not_in.

type Hit

type Hit struct {
	ID       string         `json:"id"`
	Rank     int            `json:"rank"`
	Text     string         `json:"text"`
	Metadata map[string]any `json:"metadata,omitempty"`
}

Hit is one query result. Rank is 1 for the best hit. No score is exposed: the contract is "best first", and a raw engine score is not portable across analyzer changes, backends or shards. Text is returned because a lexical hit may be absent from the vector lane's results, and the stack needs its body.

type InvalidArgError

type InvalidArgError struct{ Reason string }

InvalidArgError is a malformed request: a missing id, an unknown filter op, an update with no filter.

func (*InvalidArgError) Code

func (e *InvalidArgError) Code() string

func (*InvalidArgError) Error

func (e *InvalidArgError) Error() string

type Item

type Item struct {
	ID       string         `json:"id"`
	Text     string         `json:"text,omitempty"`
	Title    string         `json:"title,omitempty"`
	Heading  string         `json:"heading,omitempty"`
	Name     string         `json:"name,omitempty"`
	Entities []string       `json:"entities,omitempty"`
	Metadata map[string]any `json:"metadata,omitempty"`
}

Item is one searchable record. ID is unique within a collection; an upsert of an existing ID replaces the record. Text, Title, Heading, Name and Entities are searched, and a backend may weight them differently. Metadata is filtered on, never searched.

type Manifest

type Manifest struct {
	Tenant          string `json:"tenant"`
	Collection      string `json:"collection"`
	AppliedSeq      uint64 `json:"applied_seq"`
	Records         uint64 `json:"records"`
	AnalyzerVersion string `json:"analyzer_version"`
	ScoringModel    string `json:"scoring_model"`
	Engine          string `json:"engine"`
	CreatedAt       string `json:"created_at"`
}

Manifest describes one snapshot of one collection.

type Mutation

type Mutation struct {
	Op         MutationOp `json:"op"`
	Collection Collection `json:"collection,omitempty"` // ensure
	Items      []Item     `json:"items,omitempty"`      // upsert
	Select     Selector   `json:"select,omitempty"`     // delete
	Filter     Filter     `json:"filter,omitempty"`     // update
	Change     Change     `json:"change,omitempty"`     // update
}

Mutation is one write to one collection, in a form that does not depend on how it arrived. Every write a Store accepts reduces to one, so a backend has a single apply path, and a backend that ships writes elsewhere ships exactly what the far side applies.

type MutationOp

type MutationOp string

MutationOp names a write.

const (
	OpEnsure MutationOp = "ensure"
	OpUpsert MutationOp = "upsert"
	OpDelete MutationOp = "delete"
	OpUpdate MutationOp = "update"
	OpDrop   MutationOp = "drop"
)

type RecoveringError

type RecoveringError struct{ Reason string }

RecoveringError is returned by a backend whose engine is up but has not yet caught up, so an answer now would be silently incomplete.

func (*RecoveringError) Code

func (e *RecoveringError) Code() string

func (*RecoveringError) Error

func (e *RecoveringError) Error() string

type Selector

type Selector struct {
	IDs    []string `json:"ids,omitempty"`
	Filter Filter   `json:"filter,omitempty"`
}

Selector names the records a Delete removes: IDs, or a Filter with at least one condition. Exactly one of the two is set.

type Sequenced

type Sequenced interface {
	// ApplyAt applies m to the collection unless the collection has already
	// applied a mutation at seq or later, in which case it does nothing and
	// reports applied == false. The position is stored in the same write as
	// the mutation's last change, so after a crash a collection is never
	// ahead of, or behind, the position it reports. A replay of the same log
	// from any earlier point therefore converges on the same records.
	//
	// OpEnsure creates the collection, OpDrop removes it, and the other ops
	// need it to exist (CollectionNotFoundError otherwise). count is what the
	// matching Store method would have returned.
	ApplyAt(ctx context.Context, tenant, collection string, seq uint64, m Mutation) (count int, applied bool, err error)

	// AppliedSeq reports the position of the last mutation the collection
	// applied. found is false when the collection is not on this node's disk.
	AppliedSeq(ctx context.Context, tenant, collection string) (seq uint64, found bool, err error)
}

Sequenced applies mutations that carry a position in some ordered log.

type Snapshotter

type Snapshotter interface {
	// StagingDir is a scratch directory on the same filesystem as the store,
	// so that Install can move a snapshot into place instead of copying it.
	StagingDir() (string, error)

	// Snapshot writes a complete, self-contained copy of the collection into
	// dir, which must not exist. The collection stays open and writable
	// meanwhile. The copy is opened and checked before Snapshot returns, and
	// the manifest records what it actually holds, not what was asked for.
	Snapshot(ctx context.Context, tenant, collection, dir string) (Manifest, error)

	// Install makes the snapshot at dir (as written by Snapshot) this store's
	// copy of the collection, replacing any it has. dir is consumed.
	Install(ctx context.Context, tenant, collection, dir string) (Manifest, error)
}

Snapshotter copies a collection out of a live store, and back in.

type Store

type Store interface {
	// EnsureCollection creates the collection if absent and is a no-op if it
	// exists. c.AnalyzerVersion and c.ScoringModel may be left empty to take
	// the backend's; a value that differs from an existing collection's pin,
	// or that the backend cannot provide, returns AnalyzerMismatchError.
	EnsureCollection(ctx context.Context, tenant string, c Collection) error

	// DescribeCollection returns the pinned collection and its record count.
	// found is false (nil error) when it doesn't exist.
	DescribeCollection(ctx context.Context, tenant, name string) (c Collection, found bool, err error)

	// Upsert inserts or replaces items by ID, all or nothing. Returns the
	// number of items written.
	Upsert(ctx context.Context, tenant, collection string, items []Item) (int, error)

	// Query returns up to limit hits for q among the records filter admits,
	// best first. q is plain language: a quoted string is a phrase, an
	// identifier is matched whole, and the rest is OR-ed. No engine syntax is
	// recognised. An empty result is not an error.
	Query(ctx context.Context, tenant, collection, q string, limit int, filter Filter) ([]Hit, error)

	// Delete removes the records sel names. Returns the number removed, or
	// CountPending.
	Delete(ctx context.Context, tenant, collection string, sel Selector) (int, error)

	// Update applies ch to every record filter admits, leaving text alone.
	// filter must carry at least one condition: an unconditional rewrite is
	// refused with InvalidArgError. The filter is evaluated when the update is
	// applied. Returns the number of records matched, or CountPending.
	Update(ctx context.Context, tenant, collection string, filter Filter, ch Change) (int, error)

	// ListCollections returns the tenant's collections, sorted by name. Not a
	// hot path.
	ListCollections(ctx context.Context, tenant string) ([]Collection, error)

	// DropCollection removes a collection and all its records, returning the
	// number of records removed. A missing collection is not an error
	// (returns 0).
	DropCollection(ctx context.Context, tenant, name string) (int, error)

	// Shared reports whether this backend is shared across nodes, versus
	// per-node. The bundled backend is per-node (returns false).
	Shared() bool

	// Close flushes and releases the backend. A search index must be closed.
	Close() error
}

Store is the backend-agnostic lexical search interface. Implementations must be safe for concurrent use. tenant scopes every call (collections are namespaced by (tenant, name)); the op layer pins it from TenantScope, never from the envelope.

func Open

func Open(name string, cfg Config) (Store, error)

Open constructs the named backend. StoreNone returns (nil, nil). An unknown name is a startup error listing what is available (so a misconfigured --search-store fails loudly).

type TooLargeError

type TooLargeError struct {
	What  string
	Got   int
	Limit int
}

TooLargeError is a request past one of the limits in limits.go.

func (*TooLargeError) Code

func (e *TooLargeError) Code() string

func (*TooLargeError) Error

func (e *TooLargeError) Error() string

type UnavailableError

type UnavailableError struct{ Reason string }

UnavailableError is returned by a backend that cannot reach its engine right now. A stack should treat it as "no lexical results", not as a failure of the request.

func (*UnavailableError) Code

func (e *UnavailableError) Code() string

func (*UnavailableError) Error

func (e *UnavailableError) Error() string

Directories

Path Synopsis
Package blevestore is the bundled lexical search backend: one Bleve index per (tenant, collection) on local disk.
Package blevestore is the bundled lexical search backend: one Bleve index per (tenant, collection) on local disk.
searchbench command
Command searchbench indexes a file of records into one blevestore collection and runs a file of queries against it.
Command searchbench indexes a file of records into one blevestore collection and runs a file of queries against it.
Package searchtest holds the backend-agnostic conformance suite for search.Store implementations.
Package searchtest holds the backend-agnostic conformance suite for search.Store implementations.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL