Documentation
¶
Index ¶
- func AvailableFocusCategories() []string
- func AvailableGraderTypes() []string
- func BuildPrompt(sk *skill.Skill, skillContent string) string
- func GraderSummaries() string
- func LoadGraderDocs(fsys fs.FS, types []string) string
- func ValidateFocus(focus string) error
- type FocusCategory
- type GeneratedFile
- type Options
- type Suggestion
- type WriteOptions
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func AvailableFocusCategories ¶ added in v0.38.0
func AvailableFocusCategories() []string
AvailableFocusCategories returns all supported --focus values.
func AvailableGraderTypes ¶
func AvailableGraderTypes() []string
AvailableGraderTypes returns supported grader kinds.
func BuildPrompt ¶
BuildPrompt builds a single-pass LLM prompt (no grader docs). Retained for backward compatibility and tests.
func GraderSummaries ¶
func GraderSummaries() string
GraderSummaries returns a formatted block of one-line grader descriptions suitable for the selection prompt (pass 1).
func LoadGraderDocs ¶
LoadGraderDocs loads the full documentation for the specified grader types from the embedded filesystem. Unknown types are silently skipped.
func ValidateFocus ¶ added in v0.38.0
ValidateFocus returns nil if focus is empty or a known category.
Types ¶
type FocusCategory ¶ added in v0.38.0
type FocusCategory string
FocusCategory steers the kinds of test cases the LLM should generate.
const ( FocusTriggers FocusCategory = "triggers" FocusNegativeTriggers FocusCategory = "negative-triggers" FocusEdgeFixtures FocusCategory = "edge-fixtures" FocusDoNotUseFor FocusCategory = "do-not-use-for" FocusParameters FocusCategory = "parameters" )
type GeneratedFile ¶
type GeneratedFile struct {
Path string `yaml:"path" json:"path"`
Content string `yaml:"content" json:"content"`
Confidence float64 `yaml:"confidence" json:"confidence"`
Rationale string `yaml:"rationale" json:"rationale"`
}
GeneratedFile is a single generated artifact. For tasks, Confidence and Rationale carry per-case metadata that is shown in dry-run output but is *not* written into the task YAML file (which must satisfy the strict task schema).
type Options ¶
type Options struct {
SkillPath string
TimeoutSec int
GraderDocs fs.FS // embedded grader documentation (optional)
// Count is how many test cases to propose. <= 0 means "use model default".
Count int
// Focus narrows generation toward a category. Empty means "balanced".
Focus string
}
Options configures suggestion generation.
type Suggestion ¶
type Suggestion struct {
EvalYAML string `yaml:"eval_yaml" json:"eval_yaml"`
Tasks []GeneratedFile `yaml:"tasks,omitempty" json:"tasks,omitempty"`
Fixtures []GeneratedFile `yaml:"fixtures,omitempty" json:"fixtures,omitempty"`
}
Suggestion is the structured output returned by the LLM.
func Generate ¶
func Generate(ctx context.Context, engine execution.AgentEngine, opts Options) (*Suggestion, error)
Generate runs the suggestion flow end-to-end. When opts.GraderDocs is set, uses a two-pass approach:
Pass 1: ask the LLM which grader types to use (lightweight) Pass 2: provide detailed docs for those graders and generate eval YAML
When opts.GraderDocs is nil, falls back to a single-pass prompt.
func ParseResponse ¶
func ParseResponse(raw string) (*Suggestion, error)
ParseResponse parses model YAML output into a Suggestion. Empty output is reported distinctly from malformed suggestion YAML.
func (*Suggestion) WriteToDir ¶
func (s *Suggestion) WriteToDir(outputDir string, opts WriteOptions) ([]string, error)
WriteToDir writes suggested files to outputDir and returns written paths. Existing files are preserved unless opts.Force is true. Generated task YAML files are validated against the task schema before being written.
type WriteOptions ¶ added in v0.38.0
type WriteOptions struct {
// Force overwrites existing files and duplicate task ids when true.
Force bool
// EvalFile is the eval filename to write/preserve in outputDir.
EvalFile string
// TaskGlob is the configured task glob, relative to outputDir.
TaskGlob string
// TaskFileSuffix is the configured suffix for generated task files.
TaskFileSuffix string
}
WriteOptions controls how a Suggestion is applied to disk.
Source Files