extractor

package

v1.1.13 Latest Latest Go to latest Published: May 26, 2026 License: MIT Imports: 8 Imported by: 0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/DotNetAge/gorag

Links

Open Source Insights

Documentation ¶

Index ¶

Constants
func BuildFromExtraction(ext *Extraction, chunk *core.Chunk) ([]core.Node, []core.Edge)
func ContentHash(content string) string
func ExtractBatch(ctx context.Context, client gochatcore.Client, chunks []*core.Chunk, ...) ([]core.Node, []core.Edge, error)
func ExtractBatchWithCache(ctx context.Context, client gochatcore.Client, chunks []*core.Chunk, ...) ([]core.Node, []core.Edge, error)
func ExtractNodesAndEdges(ctx context.Context, client gochatcore.Client, chunk *core.Chunk) ([]core.Node, []core.Edge, error)
func SetCachedExtraction(store core.CacheStore, hashKey string, ext *Extraction) error
func SummarizeCommunity(ctx context.Context, client gochatcore.Client, community *core.Community, ...) (string, []string, error)
func SummarizeMultipleCommunities(ctx context.Context, client gochatcore.Client, communities []*core.Community, ...) ([]*core.Community, error)
type ExtractedEntity
type ExtractedRelation
type Extraction
- func GetCachedExtraction(store core.CacheStore, hashKey string) (*Extraction, error)

Constants ¶

View Source

const DefaultBatchSize = 5

DefaultBatchSize 默认批处理大小，每次 LLM 调用处理的 chunk 数量

Variables ¶

This section is empty.

Functions ¶

func BuildFromExtraction ¶

func BuildFromExtraction(ext *Extraction, chunk *core.Chunk) ([]core.Node, []core.Edge)

BuildFromExtraction 将缓存的原始提取结果绑定到当前 chunk 的 ID/DocID

func ContentHash ¶

func ContentHash(content string) string

ContentHash FNV-1a 哈希，将内容映射为 16 字符的十六进制 key

func ExtractBatch ¶

func ExtractBatch(ctx context.Context, client gochatcore.Client, chunks []*core.Chunk, batchSize ...int) ([]core.Node, []core.Edge, error)

ExtractBatch 批量从多个 Chunk 中提取 Node 和 Edge 将多个 chunk 合并到一次 LLM 调用中，显著降低 LLM 调用次数 chunks 会被按 batchSize 分组，每组调用一次 LLM 如果提供了 cache，会优先从缓存中获取已提取的结果，仅对缓存未命中的 chunk 调用 LLM

func ExtractBatchWithCache ¶

func ExtractBatchWithCache(ctx context.Context, client gochatcore.Client, chunks []*core.Chunk, cache core.CacheStore, batchSize ...int) ([]core.Node, []core.Edge, error)

ExtractBatchWithCache 带缓存的批量提取 cache 可以为 nil，此时行为与 ExtractBatch 相同

func ExtractNodesAndEdges ¶

func ExtractNodesAndEdges(ctx context.Context, client gochatcore.Client, chunk *core.Chunk) ([]core.Node, []core.Edge, error)

ExtractNodesAndEdges 从 Chunk 中提取 Node 和 Edge 使用 LLM 进行实体和关系抽取，直接返回 GraphRAG 原语结构

func SetCachedExtraction ¶

func SetCachedExtraction(store core.CacheStore, hashKey string, ext *Extraction) error

SetCachedExtraction 将实体提取结果写入 CacheStore

func SummarizeCommunity ¶

func SummarizeCommunity(ctx context.Context, client gochatcore.Client, community *core.Community, nodes []*core.Node, edges []*core.Edge) (string, []string, error)

SummarizeCommunity 让 LLM 为一个社区生成摘要和关键词输入社区的节点和边信息，输出该社区的自然语言摘要

func SummarizeMultipleCommunities ¶

func SummarizeMultipleCommunities(ctx context.Context, client gochatcore.Client, communities []*core.Community, nodes []*core.Node, edges []*core.Edge) ([]*core.Community, error)

SummarizeMultipleCommunities 批量为多个社区生成摘要

Types ¶

type ExtractedEntity ¶

type ExtractedEntity struct {
	Name       string `json:"name"`
	EntityType string `json:"entity_type"`
}

ExtractedEntity 提取的实体

type ExtractedRelation ¶

type ExtractedRelation struct {
	Subject   string `json:"subject"`
	Predicate string `json:"predicate"`
	Object    string `json:"object"`
}

ExtractedRelation 提取的关系

type Extraction ¶

type Extraction struct {
	Entities  []ExtractedEntity   `json:"entities"`
	Relations []ExtractedRelation `json:"relations"`
}

Extraction 原始提取结果，与特定 chunk/doc ID 无关缓存的是"这段文字包含哪些实体和关系"

func GetCachedExtraction ¶

func GetCachedExtraction(store core.CacheStore, hashKey string) (*Extraction, error)

GetCachedExtraction 从 CacheStore 中查询实体提取缓存返回 nil 表示缓存未命中

Source Files ¶

View all Source files

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL