create-model-evaluation-run

Overview ¶

Command create-model-evaluation-run creates a new model evaluation run via the GradientAI API.

Required env vars:

• DIGITALOCEAN_TOKEN: a DigitalOcean API token.
• DIGITALOCEAN_EVAL_RUN_NAME: human-readable name for the run.
• DIGITALOCEAN_CANDIDATE_MODEL_UUID: UUID of the candidate model to evaluate.
• DIGITALOCEAN_JUDGE_MODEL_UUID: UUID of the judge model used to score responses.
• DIGITALOCEAN_EVAL_DATASET_UUID: UUID of the dataset to use for evaluation.
• DIGITALOCEAN_EVAL_METRIC_UUIDS: comma-separated list of metric UUIDs.