validation

func ValidateEvalBytes ¶

func ValidateEvalBytes(data []byte) []string

ValidateEvalBytes validates raw YAML bytes against the eval schema.

func ValidateEvalFile ¶

func ValidateEvalFile(evalPath string) (evalErrs []string, taskErrs map[string][]string, err error)

ValidateEvalFile validates an eval.yaml file at the given path against the JSON schema. Returns errors for the eval itself AND all referenced task files.

This assumes that the schema files are up-to-date with the implementation. Other YAML decoding errors may be reported if fields are removed from the implementation. Those will be caught by the strict YAML parsing in LoadTestCase and LoadBenchmarkSpec, but the schema validation has a much higher fidelity (better error location) than the validation in LoadTestCase and LoadBenchmarkSpec.

func ValidateTaskBytes ¶

func ValidateTaskBytes(data []byte) []string

ValidateTaskBytes validates raw YAML bytes against the task schema.