gomlx

type Embedder ¶

type Embedder struct {
	// contains filtered or unexported fields
}

Embedder runs sentence-transformers BERT-style embedding models in-process via the pure-Go gomlx/compute/gobackend backend. No subprocess, no external runtime (no .so, no ONNX Runtime), no CGO — talon's single-binary cross-compile story stays intact.

Construction downloads the model + vocab from HuggingFace on first use (cached under HF's standard location: $HF_HOME or ~/.cache/huggingface). Subsequent constructions are warm.

Concurrency: Embed is safe to call from multiple goroutines. The internal exec is shared and protected by a mutex — running embeddings serially for now since gobackend's reentrancy isn't well-characterized and memory writes don't need parallelism.

func NewEmbedder(opts Options) (*Embedder, error)

func (e *Embedder) Dim() int

func (e *Embedder) Embed(ctx context.Context, text string) ([]float32, error)

func (e *Embedder) EmbedBatch(_ context.Context, sentences []string) ([][]float32, error)

func (e *Embedder) Name() string

type Model ¶

type Model struct {
	ID     string
	Dim    int
	SeqLen int
}

Model bundles the three things callers need to use an embedding model: the HuggingFace repo ID, the output dimensionality, and the model's positional-embedding limit. Bundling avoids the footgun of passing ModelID alone and forgetting to update Dim.

Pass to Options.Model to construct an Embedder against a known-good model:

emb, _ := gomlx.NewEmbedder(gomlx.Options{Model: gomlx.ModelNomicEmbedText})

Hosts that want a model not on the short list pass ModelID directly and let NewEmbedder auto-detect Dim/SeqLen from the repo's config.json. Pass Dim/SeqLen explicitly when you want to override the auto-detected values.

type Options ¶

type Options struct {
	// Model bundles ID/Dim/SeqLen as one value. Highest-precedence
	// way to specify the model. Use the canonical constants
	// (gomlx.ModelMiniLM_L6_V2, gomlx.ModelNomicEmbedText_V1_5,
	// etc.) for known-good targets, or build your own Model
	// literal for a HF repo not on the list.
	Model Model

	// ModelID is the HuggingFace repo (e.g.
	// "sentence-transformers/all-MiniLM-L6-v2"). Used when Model
	// is the zero value. When Dim/SeqLen below are zero,
	// NewEmbedder auto-detects them from the repo's config.json.
	ModelID string
	// Dim is the embedding dimensionality the model produces.
	// When zero (and Model is unset), auto-detected from
	// config.json's hidden_size.
	Dim int
	// SeqLen is the fixed token length the model graph is built
	// for. When zero (and Model is unset), auto-detected from
	// config.json's max_position_embeddings. Longer texts are
	// truncated; shorter are padded.
	SeqLen int
	// AuthToken is the HuggingFace token (HF_TOKEN env). Required
	// only for gated repos; public models load without it.
	AuthToken string
}

Options configures NewEmbedder. Resolution order, highest precedence first:

1. Model (a bundled ID/Dim/SeqLen triple) — use one of the known-good constants (ModelNomicEmbedText, etc.).
2. ModelID + Dim + SeqLen all set — explicit override.
3. ModelID set with Dim/SeqLen zero — auto-detect Dim/SeqLen from the model's HuggingFace config.json (one extra ~5KB download on first run; cached after).
4. Everything empty — falls back to DefaultModel (MiniLM-L6-v2).

Resolution happens BEFORE the model download — invalid combinations surface as construction errors, not silent wrong-vector bugs.