Documentation
¶
Overview ¶
Package config defines run configuration for bench-cli.
Index ¶
Constants ¶
const DefaultNamespace = "bench"
DefaultNamespace is the default Kubernetes namespace for benchmark scenarios.
const GracefulStopTimeout = 30 * time.Second
GracefulStopTimeout is the timeout for graceful shutdown of River workers.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Config ¶
type Config struct {
EnvironmentProvider string
Scenario string
ScenariosDir string
Adapter string
A2AAgentURL string
AgentCommand string
RunsDir string
KubeconfigPath string
Timeout time.Duration
ReuseCluster bool
ClusterName string
DryRun bool
BenchURL string
BenchAPIKey string
EvidenceDir string
Model string
Provider string
MemoryWindow int
SystemPromptFile string
ContractVersion string
SkillFile string // local skill prompt file; runner host must already have this path
SkillID string // stable skill identity for comparison/filtering
SkillVersion string // stable skill version for comparison/filtering
SkillSource string // source label such as local-file, local-temp, or registry name
SkillSHA256 string // optional expected sha256 for the skill file
MCPServer string // MCP server command
ToolServerID string // stable MCP server identity for comparison/filtering
ToolServerVersion string // stable MCP server version for comparison/filtering
ReportID string // stable public/private report campaign identifier
Parallel int // number of parallel workers (0 or 1 = sequential, >1 requires --database-url)
DatabaseURL string // PostgreSQL connection string for River job queue (env: BENCH_DATABASE_URL)
}
Config holds all settings for a benchmark run.
func Default ¶
func Default() Config
Default returns a Config with sensible offline-first defaults. Reads BENCH_API_URL and BENCH_API_KEY from environment for bench reporting.
func (*Config) ResolveA2AAgentURL ¶
ResolveA2AAgentURL returns the A2A agent URL from flag, env, or empty. Priority: flag > INFRA_BENCH_A2A_AGENT_URL > empty.
func (*Config) ResolveDatabaseURL ¶
ResolveDatabaseURL returns the database URL from flag, env, or empty. Priority: flag > BENCH_DATABASE_URL > empty.
func (*Config) ResolvePromptFile ¶
ResolvePromptFile returns the prompt file used for agent execution. Skill files are more specific than the legacy system-prompt file.
func (*Config) ResolveSkillFile ¶
ResolveSkillFile returns the local skill prompt path from flag, env, or empty. Priority: flag > BENCH_SKILL_FILE > empty.
func (*Config) ResolveSystemPromptFile ¶
ResolveSystemPromptFile returns the system prompt file path from flag, env, or empty. Priority: flag > INFRA_BENCH_SYSTEM_PROMPT > empty (use default).
type VersionInfo ¶
type VersionInfo struct {
// infra-bench
InfraBenchVersion string `json:"infra_bench_version"`
InfraBenchCommit string `json:"infra_bench_commit,omitempty"`
// prompt/contract
ContractVersion string `json:"contract_version,omitempty"`
SkillVersion string `json:"skill_version,omitempty"`
PromptVersion string `json:"prompt_version,omitempty"`
PromptFile string `json:"prompt_file,omitempty"`
SkillFile string `json:"skill_file,omitempty"`
SkillID string `json:"skill_id,omitempty"`
SkillSource string `json:"skill_source,omitempty"`
SkillSHA256 string `json:"skill_sha256,omitempty"`
}
VersionInfo holds all version metadata for reproducible benchmarks. Every run must record these so results can be compared across time, models, prompts, and skill releases.
func CollectVersions ¶
func CollectVersions(infraBenchVersion, infraBenchCommit string, cfg Config) VersionInfo
CollectVersions gathers version information from the prompt file and the infra-bench build.
func (VersionInfo) ToMetadata ¶
func (v VersionInfo) ToMetadata() map[string]string
ToMetadata converts VersionInfo to a flat string map for run metadata.
Source Files