Documentation
¶
Overview ¶
Package env builds the in-memory environment a scenario runs against: a memory repository (seeded with the scenario's prior cases), in-memory history and trace repositories, the system-under-test wired via usecase.New, a recording fake Slack service, and per-tool clients that are either simulated (ToolSimulator) or live (recorded). It exposes the AgentUseCase entrypoints the driver drives, plus the fake Slack, the tool-call recorder, and the trace repository for diagnostic dumps.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Env ¶
type Env struct {
AgentUC *usecase.AgentUseCase
JobRunner *job.JobRunner
Repo interfaces.Repository
Registry *model.WorkspaceRegistry
Entry *model.WorkspaceEntry
Slack *fakeSlack
Recorder *toolsim.Recorder
Trace *agentarchive.MemoryTraceRepository
SeededCases []*model.Case
MonitorChannel string
Language string
// contains filtered or unexported fields
}
Env is a prepared single-scenario environment.
func (*Env) AwaitJobRun ¶ added in v0.3.0
AwaitJobRun blocks until the newest run recorded for this Job has left the RUNNING stage, and returns that run's log.
A driver needs it for the same reason AwaitTurn exists: a Job now runs as a durable Process, so JobRunner.Run returns once the run is recorded and spawned. Everything the driver inspects — the outcome, the per-call timeline, the case and actions the run produced — is written afterwards, by the run's completion handler.
func (*Env) AwaitTurn ¶ added in v0.3.0
func (e *Env) AwaitTurn(ctx context.Context, channelID, threadTS string, before model.SessionEndReason) error
AwaitTurn blocks until the turn started on this thread has finished — the agent either recorded how it ended on the Session, or committed the case.
A driver needs it because a turn is a durable Process: the entry point returns once the run is recorded, and everything the driver then inspects (the pending question, the created case, the thread replies) is written afterwards by the run's completion handler.
func (*Env) Stop ¶ added in v0.3.0
func (e *Env) Stop()
Stop shuts the agent worker down. Callers must call it when the scenario ends; leaving it running leaks a goroutine per scenario.
It is deliberately not named Close: the project reserves that name for io.Closer, which must go through safe.Close, and this ends a worker rather than releasing a handle.
type Options ¶
type Options struct {
// LLM is the system-under-test agent's LLM client (required).
LLM gollem.LLMClient
// Models is which model each run generates through and what it may spend
// (required). A scenario Job naming a model reaches it the same way a
// deployed one does, so a scenario exercises the real resolution rather than
// a harness-only default.
Models agentkernel.ModelPolicy
// Completer drives the simulated tools (required when any tool is sim).
Completer evaltype.Completer
// Live* are real tool clients, used for tools marked live=true. Nil unless
// the corresponding live tool is requested.
LiveSlackSearch slacktool.SearchService
LiveNotion notiontool.Client
GitHub *githubtool.Client
// WebFetch holds the live-only webfetch HTTP settings; the eval LLM is
// injected as the screening client when the tool is built.
WebFetch *webfetch.ClientConfig
// JiraTools carries the already-expanded Jira read tools (see
// pkg/agent/tool/jira). Live-only, like GitHub.
JiraTools []gollem.Tool
// SlackToolLimits bounds what a Slack read tool's result may inject into
// the model context, exactly as it does in serve. It applies to the
// simulated Slack tools too: a scenario whose simulated result would be
// truncated in production must be truncated here, or the harness measures
// behaviour production never sees. The zero value disables both bounds.
SlackToolLimits slacktool.Limits
}
Options carries the externally-provided dependencies needed to build an env.