goneural

goneural

A neural network and tensor library in Go. Designed for clarity, flexibility, and performance. Includes GPT-2 and Qwen2.5 transformer implementations with optional CUDA acceleration, pretrained weight loading, interactive chat, and a web UI.

Features

Core library

• Float32 tensor engine with CPU and optional CUDA backends
• Reverse-mode automatic differentiation (autograd)
• Stateless forward pass (safe for concurrent inference)
• Batched training support (forward + backward)
• Flexible network composition (dense layers, embeddings, transformer blocks, custom architectures)
• Activation functions: Sigmoid, ReLU, GeLU, SiLU
• Loss functions: MSE, Cross-Entropy
• In-place operations and memory optimizations

GPU (CUDA)

• Optional CUDA matmul via cuBLAS (FP16 GEMM); CPU is used when the library is missing or --no-gpu is set
• Runtime dlopen of libgoneural_cuda.so (no hard link to the CUDA toolkit at Go build time)
• Persistent GPU-resident weights, batched GEMM, and scratch buffers for inference
• Falls back to multi-threaded CPU automatically if GPU init fails

Transformer models

• GPT-2 — layer norm, learned positional embeddings, GELU MLP; loads HuggingFace safetensors weights
• Qwen2.5 Instruct — RMSNorm, RoPE, grouped-query attention (GQA), SwiGLU MLP
  • Sizes: 0.5B, 1.5B, 3B (default), 7B (sharded checkpoints supported)
• Shared llm utilities: safetensors loading, KV cache, top-k sampling with temperature, model download
• Autoregressive text generation with KV-cache prefill

Applications

• Interactive REPL for demos and model chat
• rest — HTTP/WebSocket server (:8080) for browser-based terminal access
• graph — optional compute-graph capture (--graph) served from the rest UI

Install

go get gitlab.com/cupla/goneural

CUDA (optional)

Build the shared library once (needs nvcc / CUDA toolkit):

cd cuda && make
# or without a local toolkit:
cd cuda && make docker

libgoneural_cuda.so is found next to the binary, under ./cuda/, or via NEURAL_CUDA_LIB.

Usage

Run without arguments for an interactive prompt, or pass a command directly. See main.go for full working examples.

go get ./...
go run .

# Qwen chat (GPU when available; default model 3b):
go run . qwen
go run . qwen 0.5b
go run . --no-gpu qwen 1.5b

# Web terminal + optional graph capture:
go run . rest
go run . --graph qwen 3b

goneural - Neural networks in Golang - (c) Cupla Software Oy

Options:
  --help           print usage and exit
  --no-gpu         force CPU matmul (GPU is default when available)
  --no-gpu-scratch fresh GEMM operand alloc per call (no scratch reuse)
  --graph          record compute graph and serve from rest
  --checks         enable runtime checks
  --no-kv-cache    qwen: full re-forward each token
  --profile        qwen: profile timings

Commands:
  sigmoid                      character-level text demo (dense + sigmoid)
  relu                         character-level text demo (dense + ReLU)
  embeddings                   character-level text demo (embeddings + dense)
  gpt2                         GPT-2 text completion
  test [kv [0.5b|1.5b|3b|7b]]  Qwen2.5 self-tests
  qwen [0.5b|1.5b|3b|7b]       Qwen2.5 chat (default: 3b)
  rest                         rest terminal

Model weights are downloaded automatically to data/models/ on first use.

Packages

Recommended reading

• "Build a Large Language Model (From Scratch)" - Sebastian Raschka
• "Attention is all you need" [https://arxiv.org/pdf/1706.03762]
• "The Crystallization of Transformer Architectures (2017-2025)" [https://jytan.net/blog/2025/transformer-architectures/]