Documentation
¶
Overview ¶
Package embedding provides text embedding capabilities.
Index ¶
- func ProbeV1(ctx context.Context, proxyURL string, tokenFn func() string) bool
- func ProbeV2(ctx context.Context, proxyURL string, tokenFn func() string) (string, int, bool)
- func ProbeV3(ctx context.Context, proxyURL string, tokenFn func() string) (string, int, bool)
- type Embedder
- type Protocol
- type RemoteEmbedder
- func (e *RemoteEmbedder) Close() error
- func (e *RemoteEmbedder) Embed(text string) ([]float32, error)
- func (e *RemoteEmbedder) EmbedBatch(texts []string) ([][]float32, error)
- func (e *RemoteEmbedder) EmbedQueryBatch(texts []string) ([][]float32, error)
- func (e *RemoteEmbedder) Model() string
- func (e *RemoteEmbedder) OnProgress(fn func(completed, total int))
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func ProbeV1 ¶ added in v0.38.0
ProbeV1 confirms the legacy /embed/check endpoint exists. The v1 embedding space comes from datasource discovery, so the response body is not decoded.
Types ¶
type Embedder ¶
type Embedder interface {
// Embed returns the L2-normalized QUERY embedding vector for a single
// search-query string.
Embed(text string) ([]float32, error)
// EmbedQueryBatch returns L2-normalized QUERY embedding vectors for
// query-shaped index texts (e.g. runbook triggers — authored hypothetical
// queries that incoming queries should match near-verbatim, which only
// works when both sit in the same query space).
EmbedQueryBatch(texts []string) ([][]float32, error)
// EmbedBatch returns L2-normalized DOCUMENT embedding vectors for the
// texts of an index build.
EmbedBatch(texts []string) ([][]float32, error)
// Close releases resources held by the embedder.
Close() error
}
Embedder provides text embedding capabilities for retrieval. The methods select which side of asymmetric retrieval a text embeds on: Embed and EmbedQueryBatch produce QUERY-space vectors, EmbedBatch produces DOCUMENT-space vectors for the indexed texts queries are matched against.
type Protocol ¶ added in v0.38.0
type Protocol string
Protocol identifies the proxy embedding protocol negotiated by the server.
func Probe ¶ added in v0.38.0
func Probe(ctx context.Context, proxyURL string, tokenFn func() string, fallbackModel string) (string, int, Protocol)
Probe negotiates the newest embedding protocol the proxy supports. v3 and v2 are discovered by their check endpoints. v1 is the legacy datasources advertisement and has no dimensionality echo.
type RemoteEmbedder ¶ added in v0.14.0
type RemoteEmbedder struct {
// contains filtered or unexported fields
}
RemoteEmbedder implements Embedder by calling the negotiated proxy embedding routes. v3 is task-typed; v2 and v1 are symmetric. An optional local cache avoids round-trips to the proxy on warm restarts.
func NewRemote ¶ added in v0.14.0
func NewRemote( log logrus.FieldLogger, proxyURL string, tokenFn func() string, invalidateFn func(), localCache cache.Cache, model string, dimensions int, protocol Protocol, ) *RemoteEmbedder
NewRemote creates a RemoteEmbedder for the negotiated embedding space. tokenFn is called on each request to get the current auth token, and invalidateFn drops the cached token so a 401/403 can be retried with a fresh one (it may be nil to disable the retry). localCache is optional — when set, embedding vectors are cached locally using protocol-specific keys to avoid proxy round-trips.
func (*RemoteEmbedder) Close ¶ added in v0.14.0
func (e *RemoteEmbedder) Close() error
Close releases resources held by the embedder.
func (*RemoteEmbedder) Embed ¶ added in v0.14.0
func (e *RemoteEmbedder) Embed(text string) ([]float32, error)
Embed returns the L2-normalized QUERY embedding vector for a single search-query string.
func (*RemoteEmbedder) EmbedBatch ¶ added in v0.14.0
func (e *RemoteEmbedder) EmbedBatch(texts []string) ([][]float32, error)
EmbedBatch returns L2-normalized DOCUMENT embedding vectors for multiple texts.
func (*RemoteEmbedder) EmbedQueryBatch ¶ added in v0.38.0
func (e *RemoteEmbedder) EmbedQueryBatch(texts []string) ([][]float32, error)
EmbedQueryBatch returns L2-normalized QUERY embedding vectors for multiple query-shaped texts. v1/v2 are symmetric, so this is equivalent to EmbedBatch.
func (*RemoteEmbedder) Model ¶ added in v0.35.0
func (e *RemoteEmbedder) Model() string
Model returns the embedding model this embedder is keyed to. Index builds use it to tag the embedding space they were built in, so a later change to the proxy's served model can be detected and trigger a re-index.
func (*RemoteEmbedder) OnProgress ¶ added in v0.34.0
func (e *RemoteEmbedder) OnProgress(fn func(completed, total int))
OnProgress registers a callback invoked during EmbedBatch with the number of documents embedded so far and the total in the batch. It enables document-level progress reporting for index builds.
Source Files