Documentation
¶
Overview ¶
Package runners provides interfaces and implementations for executing MindTrial tasks and collecting their results.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type AsyncResultSet ¶
type AsyncResultSet interface {
// GetResults returns the task results for each provider.
// The call will block until the run is finished.
GetResults() Results
// ProgressEvents returns a channel that emits run progress as a value between 0 and 1.
// The channel will be closed when the run is finished.
ProgressEvents() <-chan float32
// MessageEvents returns a channel that emits run log messages.
// The channel will be closed when the run is finished.
MessageEvents() <-chan string
// Cancel stops the ongoing run execution.
Cancel()
}
AsyncResultSet extends the basic ResultSet interface to provide asynchronous operation capabilities. It offers channels for monitoring progress and receiving messages during execution, as well as the ability to cancel the ongoing run.
type ResultKind ¶
type ResultKind int
ResultKind represents the task execution result status.
const ( Success ResultKind = iota Failure Error NotSupported )
Success indicates that task finished successfully with correct result. Failure indicates that task finished successfully but with incorrect result. Error indicates that task failed to produce a result. NotSupported indicates that task could not finish because the provider does not support the required features.
type ResultSet ¶
type ResultSet interface {
// GetResults returns the task results for each provider.
GetResults() Results
}
ResultSet represents the outcome of executing a set of tasks.
type Results ¶
Results stores task results for each provider.
func (Results) ProviderResultsByRunAndKind ¶
func (r Results) ProviderResultsByRunAndKind(provider string) map[string]map[ResultKind][]RunResult
ProviderResultsByRunAndKind organizes results by run configuration and result kind.
type RunResult ¶
type RunResult struct {
// Kind indicates the result status.
Kind ResultKind
// Task is the name of the executed task.
Task string
// Provider is the name of the AI provider that executed the task.
Provider string
// Run is the name of the provider's run configuration used.
Run string
// Got is the actual answer received from the AI model.
Got string
// Want are the accepted valid answer(s) for the task.
Want utils.StringSet
// Details contains additional information about the task result.
Details string
// Duration represents the time taken to generate the response.
Duration time.Duration
}
RunResult represents the outcome of executing a single task.
type Runner ¶
type Runner interface {
// Run executes all given tasks against all run configurations and returns when done.
Run(ctx context.Context, tasks []config.Task) (ResultSet, error)
// Start executes all given tasks against all run configurations asynchronously.
// It returns immediately and the execution continues in the background,
// offering progress updates and messages through the returned result set.
Start(ctx context.Context, tasks []config.Task) (AsyncResultSet, error)
// Close releases resources when the runner is no longer needed.
Close(ctx context.Context)
}
Runner executes tasks on configured AI providers.
func NewDefaultRunner ¶
func NewDefaultRunner(ctx context.Context, cfg []config.ProviderConfig, globalValidationRules config.ValidationRules, logger zerolog.Logger) (Runner, error)
NewDefaultRunner creates a new Runner that executes tasks on all configured providers in parallel. The individual runs on a single provider are executed sequentially. It returns an error if any provider initialization fails.
Source Files