compare

package
v0.0.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 27, 2026 License: MIT Imports: 7 Imported by: 0

Documentation

Overview

Package compare provides multi-provider benchmarking: run the same prompt against several AI providers simultaneously and compare results.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func JudgePrompt 

func JudgePrompt(originalPrompt string, entries []*Entry) string

JudgePrompt builds the evaluation prompt for an AI judge.

func ParseJudgeOutput 

func ParseJudgeOutput(output string, entries []*Entry)

ParseJudgeOutput parses the judge's structured output and annotates entries. Expected format per line: SCORE <N> <provider>: <feedback>

func Truncate 

func Truncate(s string, maxRunes int) string

Truncate shortens a string to maxRunes, appending "…" if cut.

Types

type Entry 

type Entry struct {
	ProviderName string
	Model        string
	Output       string
	Duration     time.Duration
	InputTokens  int
	OutputTokens int
	CostUSD      float64
	CostKnown    bool
	Err          error

	// JudgeScore is 0–10, set only when a judge provider is used.
	JudgeScore    int
	JudgeFeedback string
}

Entry holds the result (or error) from a single provider.

func Run 

func Run(ctx context.Context, prompt string, providers []provider.Provider, model string, timeout time.Duration) []*Entry

Run sends prompt to each provider in parallel and returns one Entry per provider. model is the model override (empty = use each provider's default).

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.