metrics

package
v0.5.10 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 13, 2026 License: Apache-2.0 Imports: 9 Imported by: 0

Documentation

Overview

Package metrics 提供高级评估指标

Package metrics 提供 AI 系统的评估指标

Package metrics 提供 AI 系统的评估指标

Package metrics 提供 NLP 评估指标

Package metrics 提供 AI 系统的评估指标

Package metrics 提供 AI 系统的评估指标

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type AnswerQualityEvaluator

type AnswerQualityEvaluator struct {
	// contains filtered or unexported fields
}

AnswerQualityEvaluator 评估回答的整体质量

func NewAnswerQualityEvaluator

func NewAnswerQualityEvaluator(llm evaluate.LLMJudge, opts ...AnswerQualityOption) *AnswerQualityEvaluator

NewAnswerQualityEvaluator 创建回答质量评估器

func (*AnswerQualityEvaluator) Description

func (e *AnswerQualityEvaluator) Description() string

Description 返回评估器描述

func (*AnswerQualityEvaluator) Evaluate

Evaluate 执行质量评估

func (*AnswerQualityEvaluator) Name

func (e *AnswerQualityEvaluator) Name() string

Name 返回评估器名称

func (*AnswerQualityEvaluator) RequiresLLM

func (e *AnswerQualityEvaluator) RequiresLLM() bool

RequiresLLM 返回是否需要 LLM

type AnswerQualityOption

type AnswerQualityOption func(*AnswerQualityEvaluator)

AnswerQualityOption 选项

func WithAnswerQualityCriteria

func WithAnswerQualityCriteria(criteria []string) AnswerQualityOption

WithAnswerQualityCriteria 设置评估标准

func WithAnswerQualityThreshold

func WithAnswerQualityThreshold(threshold float64) AnswerQualityOption

WithAnswerQualityThreshold 设置阈值

type BLEUEvaluator

type BLEUEvaluator struct {
	// contains filtered or unexported fields
}

BLEUEvaluator BLEU (Bilingual Evaluation Understudy) 评估器

主要用于机器翻译质量评估,通过计算 n-gram 精确度来衡量 生成文本与参考文本的相似程度。

func NewBLEUEvaluator

func NewBLEUEvaluator(opts ...BLEUOption) *BLEUEvaluator

NewBLEUEvaluator 创建 BLEU 评估器

func (*BLEUEvaluator) Description

func (e *BLEUEvaluator) Description() string

Description 返回评估器描述

func (*BLEUEvaluator) Evaluate

Evaluate 执行 BLEU 评估

func (*BLEUEvaluator) Name

func (e *BLEUEvaluator) Name() string

Name 返回评估器名称

func (*BLEUEvaluator) RequiresLLM

func (e *BLEUEvaluator) RequiresLLM() bool

RequiresLLM 返回是否需要 LLM

type BLEUOption

type BLEUOption func(*BLEUEvaluator)

BLEUOption BLEU 评估器选项

func WithBLEUMaxN

func WithBLEUMaxN(maxN int) BLEUOption

WithBLEUMaxN 设置最大 n-gram 长度

func WithBLEUThreshold

func WithBLEUThreshold(threshold float64) BLEUOption

WithBLEUThreshold 设置通过阈值

type CoherenceEvaluator

type CoherenceEvaluator struct {
	// contains filtered or unexported fields
}

CoherenceEvaluator 连贯性评估器

评估文本的逻辑连贯性和流畅度。

func NewCoherenceEvaluator

func NewCoherenceEvaluator(llm evaluate.LLMJudge, opts ...CoherenceOption) *CoherenceEvaluator

NewCoherenceEvaluator 创建连贯性评估器

func (*CoherenceEvaluator) Description

func (e *CoherenceEvaluator) Description() string

Description 返回评估器描述

func (*CoherenceEvaluator) Evaluate

Evaluate 执行连贯性评估

func (*CoherenceEvaluator) Name

func (e *CoherenceEvaluator) Name() string

Name 返回评估器名称

func (*CoherenceEvaluator) RequiresLLM

func (e *CoherenceEvaluator) RequiresLLM() bool

RequiresLLM 返回是否需要 LLM

type CoherenceOption

type CoherenceOption func(*CoherenceEvaluator)

CoherenceOption 连贯性评估器选项

func WithCoherenceThreshold

func WithCoherenceThreshold(threshold float64) CoherenceOption

WithCoherenceThreshold 设置通过阈值

type CompletenessEvaluator

type CompletenessEvaluator struct {
	// contains filtered or unexported fields
}

CompletenessEvaluator 完整性评估器

评估回答是否完整地回答了问题的所有方面。

func NewCompletenessEvaluator

func NewCompletenessEvaluator(llm evaluate.LLMJudge, opts ...CompletenessOption) *CompletenessEvaluator

NewCompletenessEvaluator 创建完整性评估器

func (*CompletenessEvaluator) Description

func (e *CompletenessEvaluator) Description() string

Description 返回评估器描述

func (*CompletenessEvaluator) Evaluate

Evaluate 执行完整性评估

func (*CompletenessEvaluator) Name

func (e *CompletenessEvaluator) Name() string

Name 返回评估器名称

func (*CompletenessEvaluator) RequiresLLM

func (e *CompletenessEvaluator) RequiresLLM() bool

RequiresLLM 返回是否需要 LLM

type CompletenessOption

type CompletenessOption func(*CompletenessEvaluator)

CompletenessOption 完整性评估器选项

func WithCompletenessThreshold

func WithCompletenessThreshold(threshold float64) CompletenessOption

WithCompletenessThreshold 设置通过阈值

type CompositePerformanceEvaluator

type CompositePerformanceEvaluator struct {
	// contains filtered or unexported fields
}

CompositePerformanceEvaluator 综合性能评估器 结合延迟、成本、吞吐量等指标

func NewCompositePerformanceEvaluator

func NewCompositePerformanceEvaluator(opts ...CompositePerformanceOption) *CompositePerformanceEvaluator

NewCompositePerformanceEvaluator 创建综合性能评估器

func (*CompositePerformanceEvaluator) Description

func (e *CompositePerformanceEvaluator) Description() string

Description 返回评估器描述

func (*CompositePerformanceEvaluator) Evaluate

Evaluate 执行综合性能评估

func (*CompositePerformanceEvaluator) Name

Name 返回评估器名称

func (*CompositePerformanceEvaluator) RequiresLLM

func (e *CompositePerformanceEvaluator) RequiresLLM() bool

RequiresLLM 返回是否需要 LLM

type CompositePerformanceOption

type CompositePerformanceOption func(*CompositePerformanceEvaluator)

CompositePerformanceOption 选项

func WithPerformanceWeights

func WithPerformanceWeights(latency, cost, throughput float64) CompositePerformanceOption

WithPerformanceWeights 设置权重

type ConsistencyEvaluator

type ConsistencyEvaluator struct {
	// contains filtered or unexported fields
}

ConsistencyEvaluator 一致性评估器

评估多个回答之间的一致性,用于检测模型输出的稳定性。

func NewConsistencyEvaluator

func NewConsistencyEvaluator(llm evaluate.LLMJudge, opts ...ConsistencyOption) *ConsistencyEvaluator

NewConsistencyEvaluator 创建一致性评估器

func (*ConsistencyEvaluator) Description

func (e *ConsistencyEvaluator) Description() string

Description 返回评估器描述

func (*ConsistencyEvaluator) Evaluate

Evaluate 执行一致性评估

func (*ConsistencyEvaluator) Name

func (e *ConsistencyEvaluator) Name() string

Name 返回评估器名称

func (*ConsistencyEvaluator) RequiresLLM

func (e *ConsistencyEvaluator) RequiresLLM() bool

RequiresLLM 返回是否需要 LLM

type ConsistencyOption

type ConsistencyOption func(*ConsistencyEvaluator)

ConsistencyOption 一致性评估器选项

func WithConsistencyThreshold

func WithConsistencyThreshold(threshold float64) ConsistencyOption

WithConsistencyThreshold 设置通过阈值

type ContextRelevanceEvaluator

type ContextRelevanceEvaluator struct {
	// contains filtered or unexported fields
}

ContextRelevanceEvaluator 评估上下文与回答的相关性

func NewContextRelevanceEvaluator

func NewContextRelevanceEvaluator(llm evaluate.LLMJudge, opts ...ContextRelevanceOption) *ContextRelevanceEvaluator

NewContextRelevanceEvaluator 创建上下文相关性评估器

func (*ContextRelevanceEvaluator) Description

func (e *ContextRelevanceEvaluator) Description() string

Description 返回评估器描述

func (*ContextRelevanceEvaluator) Evaluate

Evaluate 执行评估

func (*ContextRelevanceEvaluator) Name

Name 返回评估器名称

func (*ContextRelevanceEvaluator) RequiresLLM

func (e *ContextRelevanceEvaluator) RequiresLLM() bool

RequiresLLM 返回是否需要 LLM

type ContextRelevanceOption

type ContextRelevanceOption func(*ContextRelevanceEvaluator)

ContextRelevanceOption 选项

func WithContextRelevanceThreshold

func WithContextRelevanceThreshold(threshold float64) ContextRelevanceOption

WithContextRelevanceThreshold 设置阈值

type CorrectnessEvaluator

type CorrectnessEvaluator struct {
	// contains filtered or unexported fields
}

CorrectnessEvaluator 评估回答的正确性 通过与参考答案对比来评估

func NewCorrectnessEvaluator

func NewCorrectnessEvaluator(llm evaluate.LLMJudge, opts ...CorrectnessOption) *CorrectnessEvaluator

NewCorrectnessEvaluator 创建正确性评估器

func (*CorrectnessEvaluator) Description

func (e *CorrectnessEvaluator) Description() string

Description 返回评估器描述

func (*CorrectnessEvaluator) Evaluate

Evaluate 执行正确性评估

func (*CorrectnessEvaluator) Name

func (e *CorrectnessEvaluator) Name() string

Name 返回评估器名称

func (*CorrectnessEvaluator) RequiresLLM

func (e *CorrectnessEvaluator) RequiresLLM() bool

RequiresLLM 返回是否需要 LLM

type CorrectnessMode

type CorrectnessMode string

CorrectnessMode 正确性评估模式

const (
	// CorrectnessModeExact 精确匹配
	CorrectnessModeExact CorrectnessMode = "exact"
	// CorrectnessModeSemanticSimilarity 语义相似度
	CorrectnessModeSemanticSimilarity CorrectnessMode = "semantic"
	// CorrectnessModeFactual 事实正确性
	CorrectnessModeFactual CorrectnessMode = "factual"
)

type CorrectnessOption

type CorrectnessOption func(*CorrectnessEvaluator)

CorrectnessOption CorrectnessEvaluator 选项

func WithCorrectnessMode

func WithCorrectnessMode(mode CorrectnessMode) CorrectnessOption

WithCorrectnessMode 设置评估模式

func WithCorrectnessThreshold

func WithCorrectnessThreshold(threshold float64) CorrectnessOption

WithCorrectnessThreshold 设置通过阈值

type CostEvaluator

type CostEvaluator struct {
	// contains filtered or unexported fields
}

CostEvaluator 评估系统成本

func NewCostEvaluator

func NewCostEvaluator(opts ...CostOption) *CostEvaluator

NewCostEvaluator 创建成本评估器

func (*CostEvaluator) Description

func (e *CostEvaluator) Description() string

Description 返回评估器描述

func (*CostEvaluator) Evaluate

Evaluate 执行成本评估

func (*CostEvaluator) Name

func (e *CostEvaluator) Name() string

Name 返回评估器名称

func (*CostEvaluator) RequiresLLM

func (e *CostEvaluator) RequiresLLM() bool

RequiresLLM 返回是否需要 LLM

type CostOption

type CostOption func(*CostEvaluator)

CostOption CostEvaluator 选项

func WithMaxCost

func WithMaxCost(cost float64) CostOption

WithMaxCost 设置最大成本(美元)

func WithTargetCost

func WithTargetCost(cost float64) CostOption

WithTargetCost 设置目标成本(美元)

type DiversityEvaluator

type DiversityEvaluator struct {
	// contains filtered or unexported fields
}

DiversityEvaluator 多样性评估器

评估文本的词汇多样性和表达丰富度。

func NewDiversityEvaluator

func NewDiversityEvaluator(opts ...DiversityOption) *DiversityEvaluator

NewDiversityEvaluator 创建多样性评估器

func (*DiversityEvaluator) Description

func (e *DiversityEvaluator) Description() string

Description 返回评估器描述

func (*DiversityEvaluator) Evaluate

Evaluate 执行多样性评估

func (*DiversityEvaluator) Name

func (e *DiversityEvaluator) Name() string

Name 返回评估器名称

func (*DiversityEvaluator) RequiresLLM

func (e *DiversityEvaluator) RequiresLLM() bool

RequiresLLM 返回是否需要 LLM

type DiversityOption

type DiversityOption func(*DiversityEvaluator)

DiversityOption 多样性评估器选项

func WithDiversityThreshold

func WithDiversityThreshold(threshold float64) DiversityOption

WithDiversityThreshold 设置通过阈值

type F1Evaluator

type F1Evaluator struct {
	// contains filtered or unexported fields
}

F1Evaluator F1 分数评估器

用于分类任务的准确性评估,计算精确率和召回率的调和平均。

func NewF1Evaluator

func NewF1Evaluator(opts ...F1Option) *F1Evaluator

NewF1Evaluator 创建 F1 评估器

func (*F1Evaluator) Description

func (e *F1Evaluator) Description() string

Description 返回评估器描述

func (*F1Evaluator) Evaluate

func (e *F1Evaluator) Evaluate(ctx context.Context, input evaluate.EvalInput) (*evaluate.EvalResult, error)

Evaluate 执行 F1 评估

func (*F1Evaluator) Name

func (e *F1Evaluator) Name() string

Name 返回评估器名称

func (*F1Evaluator) RequiresLLM

func (e *F1Evaluator) RequiresLLM() bool

RequiresLLM 返回是否需要 LLM

type F1Option

type F1Option func(*F1Evaluator)

F1Option F1 评估器选项

func WithF1Threshold

func WithF1Threshold(threshold float64) F1Option

WithF1Threshold 设置通过阈值

type FaithfulnessEvaluator

type FaithfulnessEvaluator struct {
	// contains filtered or unexported fields
}

FaithfulnessEvaluator 评估回答对检索上下文的忠实度 检查回答是否完全基于提供的上下文,没有编造信息

func NewFaithfulnessEvaluator

func NewFaithfulnessEvaluator(llm evaluate.LLMJudge, opts ...FaithfulnessOption) *FaithfulnessEvaluator

NewFaithfulnessEvaluator 创建忠实度评估器

func (*FaithfulnessEvaluator) Description

func (e *FaithfulnessEvaluator) Description() string

Description 返回评估器描述

func (*FaithfulnessEvaluator) Evaluate

Evaluate 执行忠实度评估

func (*FaithfulnessEvaluator) Name

func (e *FaithfulnessEvaluator) Name() string

Name 返回评估器名称

func (*FaithfulnessEvaluator) RequiresLLM

func (e *FaithfulnessEvaluator) RequiresLLM() bool

RequiresLLM 返回是否需要 LLM

type FaithfulnessOption

type FaithfulnessOption func(*FaithfulnessEvaluator)

FaithfulnessOption FaithfulnessEvaluator 选项

func WithFaithfulnessStrict

func WithFaithfulnessStrict(strict bool) FaithfulnessOption

WithFaithfulnessStrict 设置严格模式

func WithFaithfulnessThreshold

func WithFaithfulnessThreshold(threshold float64) FaithfulnessOption

WithFaithfulnessThreshold 设置通过阈值

type HallucinationEvaluator

type HallucinationEvaluator struct {
	// contains filtered or unexported fields
}

HallucinationEvaluator 检测回答中的幻觉 与 Faithfulness 互补,专注于检测明显的编造

func NewHallucinationEvaluator

func NewHallucinationEvaluator(llm evaluate.LLMJudge, opts ...HallucinationOption) *HallucinationEvaluator

NewHallucinationEvaluator 创建幻觉检测评估器

func (*HallucinationEvaluator) Description

func (e *HallucinationEvaluator) Description() string

Description 返回评估器描述

func (*HallucinationEvaluator) Evaluate

Evaluate 执行幻觉检测

func (*HallucinationEvaluator) Name

func (e *HallucinationEvaluator) Name() string

Name 返回评估器名称

func (*HallucinationEvaluator) RequiresLLM

func (e *HallucinationEvaluator) RequiresLLM() bool

RequiresLLM 返回是否需要 LLM

type HallucinationOption

type HallucinationOption func(*HallucinationEvaluator)

HallucinationOption 选项

func WithHallucinationThreshold

func WithHallucinationThreshold(threshold float64) HallucinationOption

WithHallucinationThreshold 设置阈值

type LatencyEvaluator

type LatencyEvaluator struct {
	// contains filtered or unexported fields
}

LatencyEvaluator 评估系统延迟

func NewLatencyEvaluator

func NewLatencyEvaluator(opts ...LatencyOption) *LatencyEvaluator

NewLatencyEvaluator 创建延迟评估器

func (*LatencyEvaluator) Description

func (e *LatencyEvaluator) Description() string

Description 返回评估器描述

func (*LatencyEvaluator) Evaluate

Evaluate 执行延迟评估

func (*LatencyEvaluator) Name

func (e *LatencyEvaluator) Name() string

Name 返回评估器名称

func (*LatencyEvaluator) RequiresLLM

func (e *LatencyEvaluator) RequiresLLM() bool

RequiresLLM 返回是否需要 LLM

type LatencyOption

type LatencyOption func(*LatencyEvaluator)

LatencyOption LatencyEvaluator 选项

func WithMaxLatency

func WithMaxLatency(ms float64) LatencyOption

WithMaxLatency 设置最大延迟(毫秒)

func WithTargetLatency

func WithTargetLatency(ms float64) LatencyOption

WithTargetLatency 设置目标延迟(毫秒)

type PerplexityEvaluator

type PerplexityEvaluator struct {
	// contains filtered or unexported fields
}

PerplexityEvaluator 困惑度评估器

评估语言模型生成文本的质量,困惑度越低表示文本越符合预期。 注意:这个实现是简化版本,实际困惑度需要访问模型的对数概率。

func NewPerplexityEvaluator

func NewPerplexityEvaluator(opts ...PerplexityOption) *PerplexityEvaluator

NewPerplexityEvaluator 创建困惑度评估器

func (*PerplexityEvaluator) Description

func (e *PerplexityEvaluator) Description() string

Description 返回评估器描述

func (*PerplexityEvaluator) Evaluate

Evaluate 执行困惑度评估

注意:这是简化实现,使用统计方法估算困惑度。 真实困惑度需要模型的对数概率。

func (*PerplexityEvaluator) Name

func (e *PerplexityEvaluator) Name() string

Name 返回评估器名称

func (*PerplexityEvaluator) RequiresLLM

func (e *PerplexityEvaluator) RequiresLLM() bool

RequiresLLM 返回是否需要 LLM

type PerplexityOption

type PerplexityOption func(*PerplexityEvaluator)

PerplexityOption 困惑度评估器选项

func WithPerplexityThreshold

func WithPerplexityThreshold(threshold float64) PerplexityOption

WithPerplexityThreshold 设置通过阈值

type ROUGEEvaluator

type ROUGEEvaluator struct {
	// contains filtered or unexported fields
}

ROUGEEvaluator ROUGE (Recall-Oriented Understudy for Gisting Evaluation) 评估器

主要用于文本摘要质量评估,通过计算 n-gram 召回率来衡量 生成摘要与参考摘要的覆盖程度。

func NewROUGEEvaluator

func NewROUGEEvaluator(opts ...ROUGEOption) *ROUGEEvaluator

NewROUGEEvaluator 创建 ROUGE 评估器

func (*ROUGEEvaluator) Description

func (e *ROUGEEvaluator) Description() string

Description 返回评估器描述

func (*ROUGEEvaluator) Evaluate

Evaluate 执行 ROUGE 评估

func (*ROUGEEvaluator) Name

func (e *ROUGEEvaluator) Name() string

Name 返回评估器名称

func (*ROUGEEvaluator) RequiresLLM

func (e *ROUGEEvaluator) RequiresLLM() bool

RequiresLLM 返回是否需要 LLM

type ROUGEOption

type ROUGEOption func(*ROUGEEvaluator)

ROUGEOption ROUGE 评估器选项

func WithROUGEThreshold

func WithROUGEThreshold(threshold float64) ROUGEOption

WithROUGEThreshold 设置通过阈值

func WithROUGEVariant

func WithROUGEVariant(variant string) ROUGEOption

WithROUGEVariant 设置 ROUGE 变体

type RelevanceEvaluator

type RelevanceEvaluator struct {
	// contains filtered or unexported fields
}

RelevanceEvaluator 评估检索文档与查询的相关性

func NewRelevanceEvaluator

func NewRelevanceEvaluator(llm evaluate.LLMJudge, opts ...RelevanceOption) *RelevanceEvaluator

NewRelevanceEvaluator 创建相关性评估器

func (*RelevanceEvaluator) Description

func (e *RelevanceEvaluator) Description() string

Description 返回评估器描述

func (*RelevanceEvaluator) Evaluate

Evaluate 执行相关性评估

func (*RelevanceEvaluator) Name

func (e *RelevanceEvaluator) Name() string

Name 返回评估器名称

func (*RelevanceEvaluator) RequiresLLM

func (e *RelevanceEvaluator) RequiresLLM() bool

RequiresLLM 返回是否需要 LLM

type RelevanceOption

type RelevanceOption func(*RelevanceEvaluator)

RelevanceOption RelevanceEvaluator 选项

func WithRelevanceThreshold

func WithRelevanceThreshold(threshold float64) RelevanceOption

WithRelevanceThreshold 设置通过阈值

type ThroughputEvaluator

type ThroughputEvaluator struct {
	// contains filtered or unexported fields
}

ThroughputEvaluator 评估系统吞吐量

func NewThroughputEvaluator

func NewThroughputEvaluator(opts ...ThroughputOption) *ThroughputEvaluator

NewThroughputEvaluator 创建吞吐量评估器

func (*ThroughputEvaluator) Description

func (e *ThroughputEvaluator) Description() string

Description 返回评估器描述

func (*ThroughputEvaluator) Evaluate

Evaluate 执行吞吐量评估

func (*ThroughputEvaluator) Name

func (e *ThroughputEvaluator) Name() string

Name 返回评估器名称

func (*ThroughputEvaluator) RequiresLLM

func (e *ThroughputEvaluator) RequiresLLM() bool

RequiresLLM 返回是否需要 LLM

type ThroughputOption

type ThroughputOption func(*ThroughputEvaluator)

ThroughputOption 选项

func WithMinThroughput

func WithMinThroughput(tps float64) ThroughputOption

WithMinThroughput 设置最小吞吐量

func WithTargetThroughput

func WithTargetThroughput(tps float64) ThroughputOption

WithTargetThroughput 设置目标吞吐量

type ToxicityEvaluator

type ToxicityEvaluator struct {
	// contains filtered or unexported fields
}

ToxicityEvaluator 毒性评估器

评估文本是否包含有害、冒犯性或不当内容。

func NewToxicityEvaluator

func NewToxicityEvaluator(llm evaluate.LLMJudge, opts ...ToxicityOption) *ToxicityEvaluator

NewToxicityEvaluator 创建毒性评估器

func (*ToxicityEvaluator) Description

func (e *ToxicityEvaluator) Description() string

Description 返回评估器描述

func (*ToxicityEvaluator) Evaluate

Evaluate 执行毒性评估

func (*ToxicityEvaluator) Name

func (e *ToxicityEvaluator) Name() string

Name 返回评估器名称

func (*ToxicityEvaluator) RequiresLLM

func (e *ToxicityEvaluator) RequiresLLM() bool

RequiresLLM 返回是否需要 LLM

type ToxicityOption

type ToxicityOption func(*ToxicityEvaluator)

ToxicityOption 毒性评估器选项

func WithToxicityThreshold

func WithToxicityThreshold(threshold float64) ToxicityOption

WithToxicityThreshold 设置通过阈值(低于阈值则通过)

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL