Documentation
¶
Overview ¶
Package embeddings provides interfaces and utilities for text embedding. Embeddings are vector representations of text that capture semantic meaning for use in similarity search and RAG applications.
Index ¶
- Variables
- type Embedder
- type EmbedderImpl
- func (e *EmbedderImpl) EmbedDocuments(ctx context.Context, texts []string) ([][]float32, error)
- func (e *EmbedderImpl) EmbedQueries(ctx context.Context, texts []string) ([][]float32, error)
- func (e *EmbedderImpl) EmbedQuery(ctx context.Context, text string) ([]float32, error)
- func (e *EmbedderImpl) GetDimension(ctx context.Context) (int, error)
- type EmbedderWithOptions
- type EmbeddingOptions
- type Option
Constants ¶
This section is empty.
Variables ¶
var ErrEmptyText = errors.New("text cannot be empty")
ErrEmptyText is returned when an empty text is provided for embedding.
Functions ¶
This section is empty.
Types ¶
type Embedder ¶
type Embedder interface {
// EmbedDocuments generates embeddings for multiple documents.
EmbedDocuments(ctx context.Context, texts []string) ([][]float32, error)
// EmbedQuery generates an embedding for a single query.
EmbedQuery(ctx context.Context, text string) ([]float32, error)
// EmbedQueries generates embeddings for multiple queries.
EmbedQueries(ctx context.Context, texts []string) ([][]float32, error)
// GetDimension returns the dimension of the embeddings.
GetDimension(ctx context.Context) (int, error)
}
Embedder is the interface for embedding providers. Implementations convert text into dense vector representations.
type EmbedderImpl ¶
type EmbedderImpl struct {
// contains filtered or unexported fields
}
EmbedderImpl wraps an Embedder with preprocessing and batching capabilities. It adds support for query/document prefixes and batch processing.
func (*EmbedderImpl) EmbedDocuments ¶
EmbedDocuments generates embeddings for multiple documents with batching.
func (*EmbedderImpl) EmbedQueries ¶ added in v0.7.0
EmbedQueries generates embeddings for multiple queries with batching.
func (*EmbedderImpl) EmbedQuery ¶
EmbedQuery generates an embedding for a single query with preprocessing.
func (*EmbedderImpl) GetDimension ¶
func (e *EmbedderImpl) GetDimension(ctx context.Context) (int, error)
GetDimension returns the dimension of the embeddings.
type EmbedderWithOptions ¶ added in v0.15.0
type EmbedderWithOptions interface {
Embedder
// EmbedDocumentsWithOpts generates embeddings with additional options.
EmbedDocumentsWithOpts(ctx context.Context, texts []string, opts EmbeddingOptions) ([][]float32, error)
// EmbedQueryWithOpts generates an embedding with additional options.
EmbedQueryWithOpts(ctx context.Context, text string, opts EmbeddingOptions) ([]float32, error)
}
EmbedderWithOptions is an optional interface that embedders can implement to support additional embedding options like truncate and dimensions.
type EmbeddingOptions ¶ added in v0.15.0
type EmbeddingOptions struct {
// Truncate truncates the input to fit the model's max sequence length.
Truncate bool
// Dimensions truncates the output embedding to the specified dimension.
Dimensions int
}
EmbeddingOptions contains options for embedding requests.
type Option ¶
type Option func(*options)
Option configures an Embedder.
func WithBatchSize ¶
WithBatchSize sets the batch size for embedding operations.
func WithDocumentPrefix ¶ added in v0.11.0
WithDocumentPrefix sets the prefix prepended to document texts. Some embedding models perform better with prefixed documents.
func WithQueryPrefix ¶ added in v0.11.0
WithQueryPrefix sets the prefix prepended to query texts. Some embedding models perform better with prefixed queries.
func WithStripNewLines ¶
WithStripNewLines sets whether to strip newlines from text before embedding.
Source Files
Directories