env

package
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 7, 2026 License: Apache-2.0 Imports: 34 Imported by: 0

Documentation

Overview

Package env builds the in-memory environment a scenario runs against: a memory repository (seeded with the scenario's prior cases), in-memory history and trace repositories, the system-under-test wired via usecase.New, a recording fake Slack service, and per-tool clients that are either simulated (ToolSimulator) or live (recorded). It exposes the AgentUseCase entrypoints the driver drives, plus the fake Slack, the tool-call recorder, and the trace repository for diagnostic dumps.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Env

type Env struct {
	AgentUC        *usecase.AgentUseCase
	JobRunner      *job.JobRunner
	Repo           interfaces.Repository
	Registry       *model.WorkspaceRegistry
	Entry          *model.WorkspaceEntry
	Slack          *fakeSlack
	Recorder       *toolsim.Recorder
	Trace          *agentarchive.MemoryTraceRepository
	SeededCases    []*model.Case
	MonitorChannel string
	Language       string
	// contains filtered or unexported fields
}

Env is a prepared single-scenario environment.

func Build

func Build(ctx context.Context, sc *scenario.Scenario, opts Options) (*Env, error)

Build assembles the environment for one scenario.

func (*Env) AwaitJobRun added in v0.3.0

func (e *Env) AwaitJobRun(ctx context.Context, key model.JobRunKey) (*model.JobRunLog, error)

AwaitJobRun blocks until the newest run recorded for this Job has left the RUNNING stage, and returns that run's log.

A driver needs it for the same reason AwaitTurn exists: a Job now runs as a durable Process, so JobRunner.Run returns once the run is recorded and spawned. Everything the driver inspects — the outcome, the per-call timeline, the case and actions the run produced — is written afterwards, by the run's completion handler.

func (*Env) AwaitTurn added in v0.3.0

func (e *Env) AwaitTurn(ctx context.Context, channelID, threadTS string, before model.SessionEndReason) error

AwaitTurn blocks until the turn started on this thread has finished — the agent either recorded how it ended on the Session, or committed the case.

A driver needs it because a turn is a durable Process: the entry point returns once the run is recorded, and everything the driver then inspects (the pending question, the created case, the thread replies) is written afterwards by the run's completion handler.

func (*Env) Stop added in v0.3.0

func (e *Env) Stop()

Stop shuts the agent worker down. Callers must call it when the scenario ends; leaving it running leaks a goroutine per scenario.

It is deliberately not named Close: the project reserves that name for io.Closer, which must go through safe.Close, and this ends a worker rather than releasing a handle.

type Options

type Options struct {
	// LLM is the system-under-test agent's LLM client (required).
	LLM gollem.LLMClient
	// Models is which model each run generates through and what it may spend
	// (required). A scenario Job naming a model reaches it the same way a
	// deployed one does, so a scenario exercises the real resolution rather than
	// a harness-only default.
	Models agentkernel.ModelPolicy
	// Completer drives the simulated tools (required when any tool is sim).
	Completer evaltype.Completer
	// Live* are real tool clients, used for tools marked live=true. Nil unless
	// the corresponding live tool is requested.
	LiveSlackSearch slacktool.SearchService
	LiveNotion      notiontool.Client
	GitHub          *githubtool.Client
	// WebFetch holds the live-only webfetch HTTP settings; the eval LLM is
	// injected as the screening client when the tool is built.
	WebFetch *webfetch.ClientConfig
	// JiraTools carries the already-expanded Jira read tools (see
	// pkg/agent/tool/jira). Live-only, like GitHub.
	JiraTools []gollem.Tool
	// SlackToolLimits bounds what a Slack read tool's result may inject into
	// the model context, exactly as it does in serve. It applies to the
	// simulated Slack tools too: a scenario whose simulated result would be
	// truncated in production must be truncated here, or the harness measures
	// behaviour production never sees. The zero value disables both bounds.
	SlackToolLimits slacktool.Limits
}

Options carries the externally-provided dependencies needed to build an env.

type Post

type Post struct {
	Kind      string // "thread_reply" | "thread_message" | "message"
	ChannelID string
	ThreadTS  string
	Text      string
	TS        string
}

Post is one recorded outbound message from the agent.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL