Documentation
¶
Index ¶
- func EmbedQueries(ctx context.Context, e interface{ ... }, texts []string) ([][]float32, error)
- func EnsureModelFiles(cacheDir string, spec ModelSpec, log *slog.Logger) (modelPath, tokenizerPath string, err error)
- func EnsureONNXEnvironment(cacheDir string, log *slog.Logger) (string, error)
- func NewSessionOptionsForProvider(log *slog.Logger) (*ort.SessionOptions, error)
- func OrtProvider() string
- type Config
- type Embedder
- type ModelSpec
- type OnnxEmbedder
- func (e *OnnxEmbedder) Close() error
- func (e *OnnxEmbedder) Dimension() int
- func (e *OnnxEmbedder) Embed(ctx context.Context, texts []string) ([][]float32, error)
- func (e *OnnxEmbedder) EmbedQueries(ctx context.Context, texts []string) ([][]float32, error)
- func (e *OnnxEmbedder) EmbedTokens(ctx context.Context, texts []string) ([][][]float32, error)
- type ZeroEmbedder
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func EmbedQueries ¶
func EmbedQueries(ctx context.Context, e interface { Embed(ctx context.Context, texts []string) ([][]float32, error) }, texts []string) ([][]float32, error)
EmbedQueries embeds query-side texts through e, using the model's query instruction prefix when the embedder supports it and falling back to plain Embed for symmetric models, fakes and the zero embedder. The parameter is the minimal Embed shape so callers' narrower embedder interfaces fit.
func EnsureModelFiles ¶
func EnsureONNXEnvironment ¶
func NewSessionOptionsForProvider ¶
func NewSessionOptionsForProvider(log *slog.Logger) (*ort.SessionOptions, error)
NewSessionOptionsForProvider returns session options configured for the provider selected via CONTEXTMAXXER_ORT_PROVIDER, or nil for plain CPU. Callers own the returned options and must Destroy them after session creation. With the default "auto" provider a failed GPU attach degrades to CPU with a warning; an explicitly requested provider fails hard.
func OrtProvider ¶
func OrtProvider() string
OrtProvider resolves the ONNX Runtime execution provider from the CONTEXTMAXXER_ORT_PROVIDER env (cpu|cuda|directml|auto, default auto). DECISION(2026-06): auto = DirectML on windows/amd64 (works on any DX12 GPU with no extra installs; the DML runtime build still contains the CPU EP, so a failed DML attach falls back to CPU at session creation), plain CPU elsewhere. CUDA stays explicit-only: it needs system cuBLAS/cuDNN and a 250MB runtime download, too heavy to trigger silently.
Types ¶
type Embedder ¶
type Embedder interface {
Dimension() int
Embed(ctx context.Context, texts []string) ([][]float32, error)
Close() error
}
func Synchronized ¶
Synchronized wraps e so concurrent Embed calls are serialized. Dimension and Close delegate directly (Dimension is an immutable read; Close is called once at shutdown).
type ModelSpec ¶
type ModelSpec struct {
Name string
Dim int
MaxTokens int
ModelURL string
TokenizerURL string
// ModelSHA256/TokenizerSHA256 pin the expected content of the downloads.
// DECISION(2026-07): URLs point at immutable HF revision commits and the
// hash is verified after download — /main/ URLs plus a merely-logged
// checksum meant a silent upstream change (or truncated download) would
// be trusted. Empty is now REFUSED at download time unless the operator sets
// CONTEXTMAXXER_ALLOW_UNVERIFIED_DOWNLOAD; the locally-trained ft specs carry
// no URLs at all and never reach that path.
ModelSHA256 string
TokenizerSHA256 string
InputNames []string
// Pooling collapses per-token states to one vector: "mean" (default,
// encoder models) or "last" (decoder-based embedding models).
Pooling string
// QueryPrefix/DocPrefix are prepended to query/passage texts for
// instruction-tuned models; empty for symmetric encoders.
QueryPrefix string
DocPrefix string
// EOSTokenID, when >0, is enforced as the final token of every sequence
// (required for last-token pooling to land on the trained position, and
// must survive truncation).
EOSTokenID int64
}
type OnnxEmbedder ¶
type OnnxEmbedder struct {
// contains filtered or unexported fields
}
func NewOnnxEmbedder ¶
func NewOnnxEmbedder(ctx context.Context, cfg Config) (*OnnxEmbedder, error)
func (*OnnxEmbedder) Close ¶
func (e *OnnxEmbedder) Close() error
func (*OnnxEmbedder) Dimension ¶
func (e *OnnxEmbedder) Dimension() int
func (*OnnxEmbedder) Embed ¶
Embed embeds passage/document-side texts (the model's DocPrefix, if any, is applied). Query-side texts must go through EmbedQueries so instruction- tuned models get their query prefix.
func (*OnnxEmbedder) EmbedQueries ¶
EmbedQueries embeds query-side texts, applying the model's query instruction prefix (no-op for symmetric encoders like jina v2).
func (*OnnxEmbedder) EmbedTokens ¶
EmbedTokens returns the per-token, L2-normalized hidden states for each input text (padding tokens excluded). This is the multi-vector representation used by late-interaction (ColBERT-style MaxSim) experiments; the production retrieval path uses Embed, which mean-pools to a single vector. Kept separate so the index format and hot path are unaffected.
type ZeroEmbedder ¶
type ZeroEmbedder struct {
// contains filtered or unexported fields
}
DECISION: ZeroEmbedder is kept for --no-embeddings flag and unit tests that don't need the model.
func NewZeroEmbedder ¶
func NewZeroEmbedder() *ZeroEmbedder
func NewZeroEmbedderWithDim ¶
func NewZeroEmbedderWithDim(dim int) *ZeroEmbedder
func (*ZeroEmbedder) Close ¶
func (z *ZeroEmbedder) Close() error
func (*ZeroEmbedder) Dimension ¶
func (z *ZeroEmbedder) Dimension() int