embed

package
v0.1.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 11, 2026 License: MIT Imports: 20 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func EmbedQueries

func EmbedQueries(ctx context.Context, e interface {
	Embed(ctx context.Context, texts []string) ([][]float32, error)
}, texts []string) ([][]float32, error)

EmbedQueries embeds query-side texts through e, using the model's query instruction prefix when the embedder supports it and falling back to plain Embed for symmetric models, fakes and the zero embedder. The parameter is the minimal Embed shape so callers' narrower embedder interfaces fit.

func EnsureModelFiles

func EnsureModelFiles(cacheDir string, spec ModelSpec, log *slog.Logger) (modelPath, tokenizerPath string, err error)

func EnsureONNXEnvironment

func EnsureONNXEnvironment(cacheDir string, log *slog.Logger) (string, error)

func NewSessionOptionsForProvider

func NewSessionOptionsForProvider(log *slog.Logger) (*ort.SessionOptions, error)

NewSessionOptionsForProvider returns session options configured for the provider selected via CONTEXTMAXXER_ORT_PROVIDER, or nil for plain CPU. Callers own the returned options and must Destroy them after session creation. With the default "auto" provider a failed GPU attach degrades to CPU with a warning; an explicitly requested provider fails hard.

func OrtProvider

func OrtProvider() string

OrtProvider resolves the ONNX Runtime execution provider from the CONTEXTMAXXER_ORT_PROVIDER env (cpu|cuda|directml|auto, default auto). DECISION(2026-06): auto = DirectML on windows/amd64 (works on any DX12 GPU with no extra installs; the DML runtime build still contains the CPU EP, so a failed DML attach falls back to CPU at session creation), plain CPU elsewhere. CUDA stays explicit-only: it needs system cuBLAS/cuDNN and a 250MB runtime download, too heavy to trigger silently.

Types

type Config

type Config struct {
	ModelName string
	CacheDir  string
	BatchSize int
	Log       *slog.Logger
}

type Embedder

type Embedder interface {
	Dimension() int
	Embed(ctx context.Context, texts []string) ([][]float32, error)
	Close() error
}

func Synchronized

func Synchronized(e Embedder) Embedder

Synchronized wraps e so concurrent Embed calls are serialized. Dimension and Close delegate directly (Dimension is an immutable read; Close is called once at shutdown).

type ModelSpec

type ModelSpec struct {
	Name         string
	Dim          int
	MaxTokens    int
	ModelURL     string
	TokenizerURL string
	// ModelSHA256/TokenizerSHA256 pin the expected content of the downloads.
	// DECISION(2026-07): URLs point at immutable HF revision commits and the
	// hash is verified after download — /main/ URLs plus a merely-logged
	// checksum meant a silent upstream change (or truncated download) would
	// be trusted. Empty is now REFUSED at download time unless the operator sets
	// CONTEXTMAXXER_ALLOW_UNVERIFIED_DOWNLOAD; the locally-trained ft specs carry
	// no URLs at all and never reach that path.
	ModelSHA256     string
	TokenizerSHA256 string
	InputNames      []string
	// Pooling collapses per-token states to one vector: "mean" (default,
	// encoder models) or "last" (decoder-based embedding models).
	Pooling string
	// QueryPrefix/DocPrefix are prepended to query/passage texts for
	// instruction-tuned models; empty for symmetric encoders.
	QueryPrefix string
	DocPrefix   string
	// EOSTokenID, when >0, is enforced as the final token of every sequence
	// (required for last-token pooling to land on the trained position, and
	// must survive truncation).
	EOSTokenID int64
}

func GetModel

func GetModel(name string) (ModelSpec, error)

type OnnxEmbedder

type OnnxEmbedder struct {
	// contains filtered or unexported fields
}

func NewOnnxEmbedder

func NewOnnxEmbedder(ctx context.Context, cfg Config) (*OnnxEmbedder, error)

func (*OnnxEmbedder) Close

func (e *OnnxEmbedder) Close() error

func (*OnnxEmbedder) Dimension

func (e *OnnxEmbedder) Dimension() int

func (*OnnxEmbedder) Embed

func (e *OnnxEmbedder) Embed(ctx context.Context, texts []string) ([][]float32, error)

Embed embeds passage/document-side texts (the model's DocPrefix, if any, is applied). Query-side texts must go through EmbedQueries so instruction- tuned models get their query prefix.

func (*OnnxEmbedder) EmbedQueries

func (e *OnnxEmbedder) EmbedQueries(ctx context.Context, texts []string) ([][]float32, error)

EmbedQueries embeds query-side texts, applying the model's query instruction prefix (no-op for symmetric encoders like jina v2).

func (*OnnxEmbedder) EmbedTokens

func (e *OnnxEmbedder) EmbedTokens(ctx context.Context, texts []string) ([][][]float32, error)

EmbedTokens returns the per-token, L2-normalized hidden states for each input text (padding tokens excluded). This is the multi-vector representation used by late-interaction (ColBERT-style MaxSim) experiments; the production retrieval path uses Embed, which mean-pools to a single vector. Kept separate so the index format and hot path are unaffected.

type ZeroEmbedder

type ZeroEmbedder struct {
	// contains filtered or unexported fields
}

DECISION: ZeroEmbedder is kept for --no-embeddings flag and unit tests that don't need the model.

func NewZeroEmbedder

func NewZeroEmbedder() *ZeroEmbedder

func NewZeroEmbedderWithDim

func NewZeroEmbedderWithDim(dim int) *ZeroEmbedder

func (*ZeroEmbedder) Close

func (z *ZeroEmbedder) Close() error

func (*ZeroEmbedder) Dimension

func (z *ZeroEmbedder) Dimension() int

func (*ZeroEmbedder) Embed

func (z *ZeroEmbedder) Embed(_ context.Context, texts []string) ([][]float32, error)

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL