gorag

package module
v2.0.12 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 17, 2026 License: MIT Imports: 25 Imported by: 0

README

🦖 GoRAG

The Expert-Grade, High-Performance Modular RAG Framework for Go

Go Report Card Go Reference Go Version Documentation

English | 中文文档


GoRAG is a production-ready Retrieval-Augmented Generation (RAG) framework built for high-scale AI engineering. It features a three-layer architecture that serves developers at all skill levels.

🏗️ Three-Layer Architecture

graph TB
    subgraph Pattern层["Pattern Layer (Application)"]
        P1["NativeRAG<br/>Vector Retrieval"]
        P2["GraphRAG<br/>Knowledge Graph"]
    end
    
    subgraph Pipeline层["Pipeline Layer (Assembly)"]
        I["Indexer"]
        R["Retriever"]
        D["Repository"]
    end
    
    subgraph Step层["Step Layer (Function)"]
        S1["Independent Modules<br/>Rewrite/Chunk/Embed..."]
    end
    
    Pattern层 --> Pipeline层
    Pipeline层 --> Step层
Layer Who Uses Responsibility
Pattern Application Developers Choose RAG mode, configure Options
Pipeline Advanced Developers Assemble Indexer/Retriever/Repository
Step Framework Developers Extend independent modules

✨ Key Features

  • 🚀 Performance First: Concurrent workers and streaming parsers with O(1) memory efficiency
  • 🏗️ Pipeline-Based Architecture: Every step is explicit, traceable, and pluggable
  • 🧠 Three-Phase Enhancement: Query enhancement → Retrieval → Result enhancement
  • 🕸️ Advanced GraphRAG: Native support for Neo4j, SQLite, and BoltDB
  • 🔭 Built-in Observability: Comprehensive distributed tracing
  • 📦 Zero Dependencies: Pure Go implementation with auto-download models

🚀 Quick Start

NativeRAG (Vector Retrieval)

Best for document QA and semantic search:

import "github.com/DotNetAge/gorag/pkg/pattern"

// Create a NativeRAG with auto-configuration
rag, _ := pattern.NativeRAG("my-app",
    pattern.WithBGE("bge-small-zh-v1.5"),
)

// Index documents
rag.IndexDirectory(ctx, "./docs", true)

// Retrieve
results, _ := rag.Retrieve(ctx, []string{"What is GoRAG?"}, 5)

GraphRAG (Knowledge Graph)

Best for complex relationship reasoning:

rag, _ := pattern.GraphRAG("knowledge-graph",
    pattern.WithBGE("bge-small-zh-v1.5"),
    pattern.WithNeoGraph("neo4j://localhost:7687", "neo4j", "password", "neo4j"),
)

// Add nodes and edges
rag.AddNode(ctx, &core.Node{ID: "person-1", Type: "Person", ...})
rag.AddEdge(ctx, &core.Edge{Source: "person-1", Target: "company-1", ...})

// Query neighbors
neighbors, edges, _ := rag.GetNeighbors(ctx, "person-1", 1, 10)

📚 Documentation

Getting Started

Advanced Topics

Step Layer


🔭 Built-in Observability

idx, _ := indexer.DefaultAdvancedIndexer(
    indexer.WithZapLogger("./logs/rag.log", 100, 30, 7, true),
    indexer.WithPrometheusMetrics(":8080"),
    indexer.WithOpenTelemetryTracer(ctx, "jaeger:4317", "RAG"),
)

⚡ Technical Standards

  • Go 1.24+: Latest language features
  • Zero-CGO SQLite: Painless cross-compilation
  • Clean Architecture: Strict separation of interfaces and implementations
  • Modular Steps: Reuse steps in any custom pipeline

🤝 Contributing

We aim to build the most robust AI infrastructure for the Go ecosystem.

📄 License

GoRAG is licensed under the MIT License.

Documentation

Index

Constants

View Source
const (
	GORAG_MODEL_PATH = "GORAG_MODEL_PATH"
	GORAG_BASE_URL   = "GORAG_BASE_URL"
	GORAG_API_KEY    = "GORAG_API_KEY"
	GORAG_AUTH_TOKEN = "GORAG_AUTH_TOKEN"
	GORAG_MODEL      = "GORAG_MODEL"
)

Variables

This section is empty.

Functions

func CheckModel

func CheckModel(modelId, modelFile string) (string, error)

CheckModel 检查模型文件是否存在,如果不存在则从 HuggingFace 下载 modelId: HuggingFace 模型 ID,如 "Xenova/chinese-clip-vit-base-patch16" modelFile: 模型文件路径,如 "onnx/model.onnx" 返回模型文件的完整路径

func New

func New(dataDir string, opts ...RAGOption) (core.Indexer, error)

New 创建新的 RAG 索引实例。

按代际自动选择索引器:

  • 有 LLM 模型配置(WithLLMModel)→ GraphIndexer(第三代,最强)
  • 无 LLM 模型配置 → HybridIndexer(第二代,semantic + fulltext)

如果数据目录不存在则创建,生成配置文件和子目录结构。

func Open

func Open(dataDir string) (core.Indexer, error)

Open 打开已存在的 RAG 索引实例 从数据目录读取配置文件并恢复索引器

Types

type Config

type Config struct {
	Name               string `yaml:"name"`                 // RAG 名称
	Type               string `yaml:"type"`                 // 索引器类型:hybrid, semantic, graph
	EmbeddingModelFile string `yaml:"embedding_model_file"` // 向量化模型(onnx 文件路径)

	// LLM 模型配置(用于 GraphIndexer)
	APIKey         string `yaml:"api_key,omitempty"`
	BaseURL        string `yaml:"base_url,omitempty"`
	Model          string `yaml:"model,omitempty"`
	Language       string `yaml:"language,omitempty"`
	MaxTokens      int    `yaml:"max_tokens,omitempty"`
	ThinkingBudget int    `yaml:"thinking_budget,omitempty"`
}

func (*Config) ToLLMConfig

func (c *Config) ToLLMConfig() *indexer.ModelConfig

ToLLMConfig 将 Config 中的 LLM 字段转换为 indexer.ModelConfig。 如果 model 为空,返回 nil。

type HybridIndexer

type HybridIndexer struct {
	// contains filtered or unexported fields
}

HybridIndexer 混合索引器(第二代 RAG) 组合语义索引器(semantic)和全文索引器(fulltext),实现查询结果融合与重排。 是 GraphIndexer(第三代)的降级方案,适用于无 LLM 配置的场景。

func NewHybridIndexer

func NewHybridIndexer(
	logger logging.Logger,
	vectorStore core.VectorStore,
	docStore core.FullTextStore,
	embedder core.Embedder,
	opts ...HybridOption) (*HybridIndexer, error)

NewHybridIndexer 创建混合索引器(语义 + 全文)

func (*HybridIndexer) Add

func (h *HybridIndexer) Add(ctx context.Context, content string) ([]*core.Chunk, error)

Add 将内容添加到所有子索引器。

func (*HybridIndexer) AddFile

func (h *HybridIndexer) AddFile(ctx context.Context, filePath string) ([]*core.Chunk, error)

AddFile 将文件添加到所有子索引器。

func (*HybridIndexer) AddIndexer

func (h *HybridIndexer) AddIndexer(indexer core.Indexer, weight float32)

AddIndexer 向混合索引器添加索引器

func (*HybridIndexer) Clear added in v2.0.11

func (h *HybridIndexer) Clear(ctx context.Context) error

Clear 清除所有子索引器中的数据

func (*HybridIndexer) Close

func (h *HybridIndexer) Close(ctx context.Context) error

Close 关闭所有索引器持有的资源

func (*HybridIndexer) Count

func (h *HybridIndexer) Count(ctx context.Context) (int, error)

Count returns the total number of indexed chunks across all sub-indexers.

func (*HybridIndexer) GetChunks

func (h *HybridIndexer) GetChunks(ctx context.Context, docId string) ([]*core.Chunk, error)

GetChunks returns all chunks belonging to the specified document.

func (*HybridIndexer) GetIndexer

func (h *HybridIndexer) GetIndexer(name string) (core.Indexer, bool)

GetIndexer 获取索引器

func (*HybridIndexer) GetWeights

func (h *HybridIndexer) GetWeights() map[string]float32

func (*HybridIndexer) List

func (h *HybridIndexer) List(ctx context.Context, offset, limit int) ([]core.Hit, error)

List returns paginated hits from the primary chunk-storing indexer.

func (*HybridIndexer) ListIndexers

func (h *HybridIndexer) ListIndexers() []string

ListIndexers 列出所有索引器名称(按字母排序,保证确定性输出)

func (*HybridIndexer) Name

func (h *HybridIndexer) Name() string

Name 返回索引器名称

func (*HybridIndexer) NewQuery

func (h *HybridIndexer) NewQuery(terms string) core.Query

func (*HybridIndexer) Remove

func (h *HybridIndexer) Remove(ctx context.Context, chunkID string) error

Remove 从所有索引器移除

func (*HybridIndexer) RemoveIndexer

func (h *HybridIndexer) RemoveIndexer(name string)

RemoveIndexer 移除索引器

func (*HybridIndexer) Search

func (h *HybridIndexer) Search(ctx context.Context, query core.Query) ([]core.Hit, error)

Search 从所有索引器搜索并融合结果

func (*HybridIndexer) StoreChunk

func (h *HybridIndexer) StoreChunk(ctx context.Context, chunk *core.Chunk) error

StoreChunk stores a pre-built chunk in each sub-indexer.

func (*HybridIndexer) Type

func (h *HybridIndexer) Type() string

Type 返回索引器类型

type HybridOption

type HybridOption func(*HybridIndexer)

HybridOption HybridIndexer 的可选配置

type IndexingService

type IndexingService struct {
	// contains filtered or unexported fields
}

IndexingService RAG 索引服务 支持批量索引和文件监控两种模式

func NewRAGService

func NewRAGService(dataDir string, opts ...ServiceOption) (*IndexingService, error)

NewRAGService 创建 RAG 索引服务 dataDir: RAG 数据目录(必须) opts: 可选配置项

func (*IndexingService) Index

func (s *IndexingService) Index() error

Index 执行批量索引 对监控目录下的全部文件进行全量索引

func (*IndexingService) Reindex

func (s *IndexingService) Reindex(ctx context.Context, file string) error

Reindex 重新索引指定文件(公开 API) 删除旧索引数据后重新分块并索引,适用于文件内容变更后的更新场景

func (*IndexingService) Stop

func (s *IndexingService) Stop() error

Stop 停止服务

func (*IndexingService) Watch

func (s *IndexingService) Watch() error

Watch 启动文件监控服务 当文件发生变更时自动进行索引 首次启动时会执行全量索引

type RAGOption

type RAGOption func(*Config)

func WithEmbeddingModelFile

func WithEmbeddingModelFile(modelFile string) RAGOption

func WithIndexType

func WithIndexType(indexType string) RAGOption

func WithLLMModel

func WithLLMModel(modelCfg indexer.ModelConfig) RAGOption

WithLLMModel 设置 LLM 模型配置(用于 GraphIndexer)。 配置内容会持久化到 config.yml 中,Open 时自动恢复。

func WithName

func WithName(name string) RAGOption

type ServiceOption

type ServiceOption func(*IndexingService)

ServiceOption 服务配置选项

func WithLogger

func WithLogger(logger logging.Logger) ServiceOption

WithLogger 设置日志记录器

func WithWatchs

func WithWatchs(dirs ...string) ServiceOption

WithWatchs 设置监控目录

func WithWorkerCount

func WithWorkerCount(count int) ServiceOption

WithWorkerCount 设置 worker pool 大小

Directories

Path Synopsis
Package core provides fundamental types and interfaces for the goRAG framework.
Package core provides fundamental types and interfaces for the goRAG framework.
Package logging provides structured logging capabilities for the goRAG framework.
Package logging provides structured logging capabilities for the goRAG framework.
Package minirag 轻量移动端 RAG 索引器 可通过 gomobile bind 编译为 iOS/Android 原生 framework。
Package minirag 轻量移动端 RAG 索引器 可通过 gomobile bind 编译为 iOS/Android 原生 framework。
Package result provides result fusion and re-ranking capabilities for multi-source search results.
Package result provides result fusion and re-ranking capabilities for multi-source search results.
store

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL