README
ΒΆ
GoRAG is a production-ready Retrieval-Augmented Generation (RAG) framework built for high-scale AI engineering. Unlike complex "black-box" frameworks, GoRAG provides a transparent, pipeline-based architecture that combines Go's native concurrency with advanced RAG patterns.
From GraphRAG with automated triple extraction to Agentic RAG with self-correction, GoRAG is designed to move your AI applications from "prototype" to "production" with zero friction.
β¨ Why GoRAG?
- π Performance First: Built-in concurrent workers and streaming parsers with
O(1)memory efficiency. Perfect for indexing TB-scale knowledge bases. - ποΈ Pipeline-Based Architecture: Powered by
gochat/pkg/pipeline. Every retrieval step is explicit, traceable, and pluggable. No more "hidden magic" or deep inheritance hell. - π§ Smart Intent Routing: Automatically dispatches queries to the most suitable retrieval strategy (Vector, Graph, or Global) based on user intent.
- πΈοΈ Advanced GraphRAG: Native support for Neo4j, SQLite (Zero-CGO), and BoltDB. Includes automated LLM-driven knowledge graph construction.
- π Built-in Observability: Comprehensive distributed tracing across all core retrievers and steps. See exactly where your time and tokens go.
- π Enterprise-Grade Evaluation: Built-in benchmarking protocol for Faithfulness, Answer Relevance, and Context Precision (RAGAS-style).
π§° The RAG "Expert" Ecosystem
GoRAG doesn't just give you tools; it gives you pre-optimized strategies as first-class citizens:
| Strategy | When to use | Key Features |
|---|---|---|
| Native RAG | Standard semantic search | Vector-only, fast, low cost |
| Graph RAG | Complex relationship reasoning | Entities, Triples, Multi-hop reasoning |
| Self-RAG | High accuracy requirements | Self-reflection, Hallucination detection |
| CRAG | Handling ambiguous queries | Quality evaluation, fallback to Web Search |
| Fusion RAG | Multi-faceted queries | Query rewriting, RRF fusion |
| Smart Router | Dynamic workloads | Intent-based automatic dispatching |
π Quick Start
Installation
go get github.com/DotNetAge/gorag
1. The Smart Router: Intent-Based Retrieval
Automatically choose between Vector Search for domain facts and Graph Search for relationship reasoning:
package main
import (
"context"
"github.com/DotNetAge/gorag/pkg/retriever/agentic"
"github.com/DotNetAge/gorag/pkg/retriever/graph"
"github.com/DotNetAge/gorag/pkg/retriever/native"
)
func main() {
// 1. Setup retrievers
vectorRet := native.NewRetriever(vectorStore, embedder, llm)
graphRet := graph.NewRetriever(vectorStore, graphStore, embedder, llm)
// 2. Create a Smart Router
router := agentic.NewSmartRouter(
classifier,
map[core.IntentType]core.Retriever{
core.IntentRelational: graphRet, // Use Graph for relationship queries
core.IntentDomain: vectorRet, // Use Vector for specific facts
},
vectorRet, // Default fallback
logger,
)
// 3. Just ask! The router handles the "how"
results, _ := router.Retrieve(ctx, []string{"How are Project X and Person Y related?"}, 5)
fmt.Println(results[0].Answer)
}
2. Automated Knowledge Graph Indexing
Turn unstructured text into a queryable knowledge graph with one click:
// Initialize the triple-based indexing step
triplesStep := indexing.NewTriplesStep(llm, graphStore)
// Process your documents - GoRAG extracts (S, P, O) automatically
err := indexer.IndexDirectory(ctx, "./docs", true)
3. Production Observability & Benchmarking
Track every step and measure quality using built-in tools:
// Run a benchmark against your dataset
report, _ := evaluation.RunBenchmark(ctx, retriever, judge, testCases, 5)
fmt.Println(report.Summary())
// Output: Avg Faithfulness: 0.92, Avg Relevance: 0.88, Avg Precision: 0.85
β‘ Technical Integrity & Standards
- Go 1.24+: Leveraging the latest language features.
- Zero-CGO SQLite: Using
modernc.org/sqlitefor painless cross-compilation. - Clean Architecture: Strict separation of interfaces (
pkg/core) and implementations. - Modular Steps: Reuse
hyde,rerank,fuse, orprunesteps in any custom pipeline.
π€ Contributing
We aim to build the most robust AI infrastructure for the Go ecosystem. Whether it's a new VectorStore driver or an improved Parser, your PRs are welcome!
- Check our Contributing Guidelines.
π License
GoRAG is licensed under the MIT License.
Documentation
ΒΆ
Overview ΒΆ
Package gorag provides a Retrieval-Augmented Generation (RAG) framework for Go
GoRAG is a comprehensive framework for building RAG applications that combine large language models (LLMs) with vector databases for efficient information retrieval.
Key features include: - Circuit breaker pattern for service resilience - Graceful degradation for unreliable services - Lazy loading for efficient memory usage - Observability with metrics, logging, and tracing - Plugin system for extensibility - Connection pooling for efficient resource management - Support for multiple vector stores (Memory, Milvus, Pinecone, Qdrant, Weaviate) - Support for multiple embedding providers (Cohere, Ollama, OpenAI, Voyage) - Support for multiple LLM clients (Anthropic, Azure OpenAI, Ollama, OpenAI) - Support for multiple document parsers (CSV, JSON, Markdown, PDF, etc.)
To get started, see the examples in the cmd/gorag directory or refer to the documentation in the docs directory.
Source Files
Directories
ΒΆ
| Path | Synopsis |
|---|---|
|
pkg
|
|
|
core
Package entity defines the core entities for the goRAG framework.
|
Package entity defines the core entities for the goRAG framework. |
|
retrieval/answer
Package answer provides answer generation utilities for RAG systems.
|
Package answer provides answer generation utilities for RAG systems. |
|
retrieval/enhancement
Package enhancement provides query and document enhancement utilities for RAG systems.
|
Package enhancement provides query and document enhancement utilities for RAG systems. |
|
retrieval/graph
Package graph provides graph-related utilities for RAG systems.
|
Package graph provides graph-related utilities for RAG systems. |
|
steps/crag
Package crag provides evaluation steps for RAG retrieval quality assessment.
|
Package crag provides evaluation steps for RAG retrieval quality assessment. |
|
steps/decompose
Package decompose provides query decomposition steps for RAG retrieval pipelines.
|
Package decompose provides query decomposition steps for RAG retrieval pipelines. |
|
steps/filter
Package filter provides query preprocessing steps for RAG pipelines.
|
Package filter provides query preprocessing steps for RAG pipelines. |
|
steps/fuse
Package fuse provides result fusion steps for RAG retrieval pipelines.
|
Package fuse provides result fusion steps for RAG retrieval pipelines. |
|
steps/generate
Package generate provides answer generation steps for RAG pipelines.
|
Package generate provides answer generation steps for RAG pipelines. |
|
steps/image
Package image provides image retrieval steps for multimodal RAG pipelines.
|
Package image provides image retrieval steps for multimodal RAG pipelines. |
|
steps/indexing
Package indexing provides document indexing pipeline steps for RAG data preparation.
|
Package indexing provides document indexing pipeline steps for RAG data preparation. |
|
steps/rerank
Package rerank provides reranking steps for RAG retrieval pipelines.
|
Package rerank provides reranking steps for RAG retrieval pipelines. |
|
steps/rewrite
Package rewrite provides query rewriting steps for RAG retrieval pipelines.
|
Package rewrite provides query rewriting steps for RAG retrieval pipelines. |
|
steps/sparse
Package sparse provides sparse retrieval steps using BM25 algorithm.
|
Package sparse provides sparse retrieval steps using BM25 algorithm. |
|
steps/stepback
Package stepback provides query abstraction steps for RAG pipelines.
|
Package stepback provides query abstraction steps for RAG pipelines. |