agent-bench

Overview ¶

Command agent-bench drives a real tacklr agent through multi-turn scenarios aligned with industry agent/memory/tool evaluation shapes.

Environment (same as cmd/testserver):

OPENAI_BASE_URL, OPENAI_API_KEY, OPENAI_MODEL — required unless -dry-run
OPENAI_EMBEDDING_MODEL — dense model (default text-embedding-3-small); same base URL/key
EXA_API_KEY — optional; web_augmented cases skip when unset

Hybrid retrieval: seed Puts and agent saves embed via OpenAI-compatible /embeddings unless -lexical-only is set.

Example:

go run ./cmd/agent-bench -suite all -out /tmp/bench.json
go run ./cmd/agent-bench -dry-run -list