Documentation
¶
Overview ¶
Package benchmark provides published success rate data for attack techniques against known model families. Data is sourced from peer-reviewed research, vendor red-team reports, and standardized benchmarks (JailbreakBench, HarmBench).
NOTE: Benchmark evaluation criteria may differ from LLMrecon's success detection. Published WASR (Weighted Average Success Rate) uses specific rubrics that may not align with this tool's evaluateSuccess functions.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func GetExpectedSuccessRate ¶
GetExpectedSuccessRate returns the expected success rate for a model+technique combination, or -1 if no benchmark data exists.
Types ¶
type BenchmarkEntry ¶
type BenchmarkEntry struct {
// ModelFamily is the model family (e.g., "gpt-5", "claude-4.5", "deepseek-r1")
ModelFamily string `json:"model_family"`
// Technique is the attack technique ID
Technique string `json:"technique"`
// SuccessRate is the published ASR (Attack Success Rate) as a percentage (0-100)
SuccessRate float64 `json:"success_rate"`
// Source is the citation for the published result
Source string `json:"source"`
// Benchmark is the benchmark framework used (e.g., "JailbreakBench", "HarmBench")
Benchmark string `json:"benchmark,omitempty"`
// Notes contains any caveats about the measurement
Notes string `json:"notes,omitempty"`
}
BenchmarkEntry records a published success rate for a specific attack technique against a specific model family.
func DefaultBenchmarkDatabase ¶
func DefaultBenchmarkDatabase() []BenchmarkEntry
DefaultBenchmarkDatabase returns the built-in benchmark data from published research.
func GetBenchmarksForModel ¶
func GetBenchmarksForModel(modelFamily string) []BenchmarkEntry
GetBenchmarksForModel returns all benchmark entries for a given model family.
func GetBenchmarksForTechnique ¶
func GetBenchmarksForTechnique(technique string) []BenchmarkEntry
GetBenchmarksForTechnique returns all benchmark entries for a given technique.
Source Files