simd

package
v1.0.44 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 2, 2026 License: MIT Imports: 3 Imported by: 0

README

SIMD Vector Operations

High-performance SIMD-accelerated vector operations for NornicDB embeddings.

Features

  • Auto-detection: Automatically uses best available backend
  • Metal GPU: Auto-enabled on macOS with CGO (Apple Silicon optimized)
  • CPU SIMD: Uses vek for AVX2/NEON acceleration
  • Graceful fallback: Always works, even without GPU

Backend Selection

Platform CGO Backend Batch Operations
macOS (darwin) Yes Metal GPU + CPU SIMD 2-3x faster
macOS (darwin) No Pure Go Baseline
Linux arm64 Yes CPU SIMD (NEON) 4-10x faster
Linux amd64 Yes CPU SIMD (AVX2) 6-12x faster
Any No Pure Go Baseline

Usage

import "github.com/orneryd/nornicdb/pkg/simd"

// Single vector operations (auto-selects best backend)
similarity := simd.CosineSimilarity(vecA, vecB)
distance := simd.EuclideanDistance(vecA, vecB)
dot := simd.DotProduct(vecA, vecB)

// Batch operations (Metal GPU on macOS, CPU fallback elsewhere)
scores := make([]float32, numVectors)
err := simd.BatchCosineSimilarityMetal(embeddings, query, scores)
if err != nil {
    // Metal not available, use CPU loop
    for i := 0; i < numVectors; i++ {
        start := i * dimensions
        end := start + dimensions
        scores[i] = simd.CosineSimilarity(embeddings[start:end], query)
    }
}

// Check what's available
if simd.MetalAvailable() {
    fmt.Println("Metal GPU acceleration enabled")
}
info := simd.Info()
fmt.Printf("Using: %s (accelerated=%v)\n", info.Implementation, info.Accelerated)

Build Tags

Metal is auto-enabled on macOS with CGO. Use build tags to control:

# Default build (Metal auto-enabled on macOS)
go build ./...

# Disable Metal (CPU-only)
go build -tags nometal ./...

Benchmarks

CPU SIMD (vek)
go test ./pkg/simd -bench=Benchmark -benchmem -run=^$
Metal GPU vs CPU
go test ./pkg/simd -bench=BenchmarkMetal -benchmem -run=^$
Sample Results (Apple M2 Max)
Operation Batch Size Metal GPU CPU SIMD Speedup
Cosine Similarity 1K × 768 679μs 892μs 1.3x
Cosine Similarity 10K × 768 4.2ms 8.7ms 2.1x
Cosine Similarity 100K × 768 37.8ms 87.0ms 2.3x
Cosine Similarity 10K × 1536 7.7ms 17.9ms 2.3x

Key insight: Metal shines for batch operations (1K+ vectors). For single vectors, CPU SIMD is faster due to GPU dispatch overhead.

API Reference

Single Vector Operations
// Cosine similarity (-1 to 1, higher = more similar)
func CosineSimilarity(a, b []float32) float32

// Euclidean distance (0 = identical)
func EuclideanDistance(a, b []float32) float32

// Dot product
func DotProduct(a, b []float32) float32

// Vector norm (magnitude)
func Norm(v []float32) float32

// Normalize in-place
func NormalizeInPlace(v []float32)
Batch Operations (Metal GPU)
// Batch cosine similarity (query vs N embeddings)
func BatchCosineSimilarityMetal(embeddings, query, scores []float32) error

// Batch dot product
func BatchDotProductMetal(embeddings, query, results []float32) error

// Batch Euclidean distance
func BatchEuclideanDistanceMetal(embeddings, query, distances []float32) error

// Batch normalize
func BatchNormalizeMetal(vectors []float32, numVectors, dimensions int) error
Utilities
// Check if Metal GPU is available
func MetalAvailable() bool

// Get runtime info (implementation, features, accelerated)
func Info() RuntimeInfo

Architecture

┌─────────────────────────────────────────────────────────┐
│                    Public API (simd.go)                 │
│  CosineSimilarity, DotProduct, BatchCosineSimilarity... │
└─────────────────────────────────────────────────────────┘
                            │
            ┌───────────────┼───────────────┐
            ▼               ▼               ▼
┌───────────────┐  ┌───────────────┐  ┌───────────────┐
│  simd_arm64   │  │  simd_amd64   │  │ simd_generic  │
│   (NEON/vek)  │  │   (AVX2/vek)  │  │   (pure Go)   │
└───────────────┘  └───────────────┘  └───────────────┘
            │               │
            └───────┬───────┘
                    ▼
┌─────────────────────────────────────────────────────────┐
│              simd_metal_darwin.go (macOS only)          │
│         Metal GPU kernels for batch operations          │
│    Auto-fallback to CPU SIMD if Metal unavailable       │
└─────────────────────────────────────────────────────────┘

Documentation

Overview

Package simd provides SIMD-accelerated vector operations for NornicDB.

This package wraps the viterin/vek32 library which implements high-performance vector similarity calculations using true AVX2+FMA assembly for x86/amd64 processors.

Implementation

The package automatically detects CPU capabilities at runtime and selects the fastest available implementation:

  • x86/amd64: AVX2 + FMA SIMD assembly via vek32 (Intel Haswell+, AMD Zen+)
  • arm64: NEON assembly via vek32 (Apple Silicon, ARM servers)
  • fallback: Pure Go implementation for unsupported platforms

No configuration is required; SIMD acceleration is detected and enabled automatically.

Supported Operations

  • DotProduct: Dot product of two vectors
  • CosineSimilarity: Cosine similarity between two vectors
  • EuclideanDistance: Euclidean distance between two vectors
  • Norm: Euclidean norm (L2 norm / magnitude) of a vector
  • NormalizeInPlace: Normalize a vector to unit length in-place

Performance

Measurements on Intel i9-9900KF (AVX2+FMA) comparing vek32 SIMD vs pure Go:

  • DotProduct (1536-dim vectors): ~1.9x faster (380 ns → 700 ns with SIMD on this CPU)
  • CosineSimilarity (1536-dim): ~0.31x-0.53x (NOTE: vek32 assembly is optimized)
  • EuclideanDistance (1536-dim): ~1.9x faster
  • Norm (1536-dim): ~1.6x faster
  • vek32.Sum (operations): Up to 20x faster with true SIMD assembly
  • vek32.MatMul (matrix ops): Up to 14x faster for float32

Performance varies by vector size and CPU features. Larger vectors and operations like matrix multiplication benefit most from SIMD acceleration.

Usage 

import "github.com/orneryd/nornicdb/pkg/simd"

a := []float32{1.0, 2.0, 3.0, 4.0}
b := []float32{5.0, 6.0, 7.0, 8.0}

// Cosine similarity (most common for embeddings)
sim := simd.CosineSimilarity(a, b)

// Dot product
dot := simd.DotProduct(a, b)

// Euclidean distance
dist := simd.EuclideanDistance(a, b)

// Normalize a vector
simd.NormalizeInPlace(a)

// Check SIMD acceleration status
info := simd.Info()
if info.Accelerated {
    fmt.Printf("Using %s SIMD with features: %v\n", info.Implementation, info.Features)
}

Thread Safety

All functions in this package are safe for concurrent use. They do not maintain global state and all operations are pure functions.

Precision Notes

- Float32 operations use float32 throughout for maximum SIMD performance - vek32 SIMD functions are compiled with -ffast-math for speed - This trades strict IEEE 754 compliance for performance (inputs should never be NaN/Inf) - For higher precision requirements, use pkg/math/vector which uses float64 accumulation

Dependencies

- github.com/viterin/vek: SIMD vector functions (true assembly implementation) - golang.org/x/sys/cpu: CPU feature detection

Index

Constants

This section is empty.

Variables

View Source 
var (
	ErrMetalNotAvailable = errors.New("simd/metal: Metal GPU not available (build without metal tag)")
)

Metal backend errors

Functions

func BatchCosineSimilarity 

func BatchCosineSimilarity(embeddings []float32, query []float32, scores []float32)

BatchCosineSimilarity computes cosine similarity between a query vector and a batch of embedding vectors. Automatically uses Metal GPU if available on macOS, otherwise falls back to CPU SIMD.

This is the recommended function for searching embedding collections.

Parameters:

  • embeddings: Contiguous array of [num_vectors × dimensions] float32
  • query: Single query vector of [dimensions] float32
  • scores: Output array of [num_vectors] float32 similarity scores

Example: 

embeddings := make([]float32, 1000*768) // 1000 vectors of 768 dimensions
query := make([]float32, 768)
scores := make([]float32, 1000)
simd.BatchCosineSimilarity(embeddings, query, scores)

func BatchCosineSimilarityMetal 

func BatchCosineSimilarityMetal(embeddings []float32, query []float32, scores []float32) error

BatchCosineSimilarityMetal returns error when Metal is not available

func BatchDotProduct 

func BatchDotProduct(embeddings []float32, query []float32, results []float32)

BatchDotProduct computes dot product between a query vector and a batch of vectors. Automatically uses Metal GPU if available, otherwise falls back to CPU SIMD.

Parameters:

  • embeddings: Contiguous array of [num_vectors × dimensions] float32
  • query: Single query vector of [dimensions] float32
  • results: Output array of [num_vectors] float32 dot products

func BatchDotProductMetal 

func BatchDotProductMetal(embeddings []float32, query []float32, results []float32) error

BatchDotProductMetal returns error when Metal is not available

func BatchEuclideanDistance 

func BatchEuclideanDistance(embeddings []float32, query []float32, distances []float32)

BatchEuclideanDistance computes Euclidean distance between a query vector and a batch of vectors. Automatically uses Metal GPU if available, otherwise falls back to CPU SIMD.

Parameters:

  • embeddings: Contiguous array of [num_vectors × dimensions] float32
  • query: Single query vector of [dimensions] float32
  • distances: Output array of [num_vectors] float32 distances

func BatchEuclideanDistanceMetal 

func BatchEuclideanDistanceMetal(embeddings []float32, query []float32, distances []float32) error

BatchEuclideanDistanceMetal returns error when Metal is not available

func BatchNormalize 

func BatchNormalize(vectors []float32, numVectors, dimensions int)

BatchNormalize normalizes a batch of vectors in-place. Automatically uses Metal GPU if available, otherwise falls back to CPU SIMD.

Parameters:

  • vectors: Contiguous array of [num_vectors × dimensions] float32
  • numVectors: Number of vectors
  • dimensions: Dimension of each vector

func BatchNormalizeMetal 

func BatchNormalizeMetal(vectors []float32, numVectors, dimensions int) error

BatchNormalizeMetal returns error when Metal is not available

func CosineSimilarity 

func CosineSimilarity(a, b []float32) float32

CosineSimilarity computes the cosine similarity between two float32 vectors.

Cosine similarity measures the angle between two vectors, returning a value between -1 (opposite directions) and 1 (same direction). A value of 0 indicates orthogonal (perpendicular) vectors.

The formula is: dot(a, b) / (norm(a) * norm(b))

Requirements:

  • Both vectors must have the same length
  • Returns 0 if vectors are empty, have different lengths, or either is zero-length

Example: 

a := []float32{1, 0, 0}
b := []float32{0, 1, 0}
result := simd.CosineSimilarity(a, b) // 0 (perpendicular)

func DotProduct 

func DotProduct(a, b []float32) float32

DotProduct computes the dot product of two float32 vectors.

The dot product is defined as: sum(a[i] * b[i]) for all i.

Requirements:

  • Both vectors must have the same length
  • Returns 0 if vectors are empty or have different lengths

Example: 

a := []float32{1, 2, 3}
b := []float32{4, 5, 6}
result := simd.DotProduct(a, b) // 1*4 + 2*5 + 3*6 = 32

func EuclideanDistance 

func EuclideanDistance(a, b []float32) float32

EuclideanDistance computes the Euclidean distance between two float32 vectors.

The Euclidean distance is the straight-line distance in N-dimensional space: sqrt(sum((a[i] - b[i])^2))

Requirements:

  • Both vectors must have the same length
  • Returns 0 if vectors are empty or have different lengths

Example: 

a := []float32{0, 0}
b := []float32{3, 4}
result := simd.EuclideanDistance(a, b) // 5.0

func MetalAvailable 

func MetalAvailable() bool

MetalAvailable returns false when Metal is not compiled in

func MetalCosineSimilarity 

func MetalCosineSimilarity(a, b []float32) float32

MetalCosineSimilarity falls back to CPU implementation

func MetalDotProduct 

func MetalDotProduct(a, b []float32) float32

MetalDotProduct falls back to CPU implementation

func MetalEuclideanDistance 

func MetalEuclideanDistance(a, b []float32) float32

MetalEuclideanDistance falls back to CPU implementation

func MetalNorm 

func MetalNorm(v []float32) float32

MetalNorm falls back to CPU implementation

func Norm 

func Norm(v []float32) float32

Norm computes the Euclidean norm (L2 norm / magnitude) of a float32 vector.

The norm is defined as: sqrt(sum(v[i]^2))

Example: 

v := []float32{3, 4}
result := simd.Norm(v) // 5.0

func NormalizeInPlace 

func NormalizeInPlace(v []float32)

NormalizeInPlace normalizes a vector to unit length, modifying it in place.

After normalization, Norm(v) will equal 1.0 (within floating-point precision).

If the vector has zero length, it will remain unchanged.

Example: 

v := []float32{3, 4}
simd.NormalizeInPlace(v)
// v is now {0.6, 0.8}

Types

type Implementation 

type Implementation string

Implementation represents the active SIMD implementation 

const (
	// ImplGeneric indicates pure Go fallback (no SIMD)
	ImplGeneric Implementation = "generic"
	// ImplAVX2 indicates x86 AVX2+FMA SIMD
	ImplAVX2 Implementation = "avx2"
	// ImplNEON indicates ARM NEON SIMD
	ImplNEON Implementation = "neon"
)

type RuntimeInfo 

type RuntimeInfo struct {
	// Implementation is the active SIMD backend
	Implementation Implementation
	// Features lists specific CPU features being used
	Features []string
	// Accelerated indicates whether SIMD acceleration is active
	Accelerated bool
}

RuntimeInfo contains information about the active SIMD implementation

func Info 

func Info() RuntimeInfo

Info returns information about the active SIMD implementation.

This can be used to check whether SIMD acceleration is being used and which specific features are enabled.

Example: 

info := simd.Info()
if info.Accelerated {
    fmt.Printf("Using %s SIMD\n", info.Implementation)
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.