Documentation
¶
Overview ¶
Package mockllm serves scripted model replies over the three wire protocols BuildMax speaks, so an end-to-end suite can drive a real run without a provider, a key, or a paid call.
A scenario is a list of steps, replayed in order: one step answers one model call, whatever protocol asked. Nothing is inferred from the request, because a mock that answers what it thinks was asked stops being evidence — the run it produces is the mock's opinion rather than the agent's behaviour.
How long a reply takes is the exception, and it is set out of band rather than guessed: a step can script its own stall, and a suite driving a deployed mock arms one over the control route. A test that has to act on a run while the run is still going — cancel it, take its worker away — otherwise races the run to its own end, and duration is the one property of a scripted turn that says nothing about what the agent did.
See docs/design/end-to-end-testing.md §4.
Index ¶
Constants ¶
const ( ProtocolOpenAIChat = "openai_compatible" ProtocolOpenAIResponses = "openai" ProtocolAnthropic = "anthropic" )
Protocol names the wire protocol a request arrived on. The values match config.LLMProvider* so a suite can configure a model entry from one constant.
const ControlRequestsPath = "/control/requests"
ControlRequestsPath returns every request the mock has received so far, so a suite driving a deployed mock over HTTP can inspect a tool result a scripted final reply never echoes back. GET, unlike every other route here.
const ControlStallPath = "/control/stall"
ControlStallPath is the route that arms a stall on a mock a suite cannot rescript — one already deployed, answering a stack the suite only reaches over HTTP. It is matched by suffix so an ingress may serve it under a prefix.
const ControlToolCallPath = "/control/toolcall"
ControlToolCallPath arms a one-shot tool call the same way ControlStallPath arms a stall. See Handler.armedStep.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Handler ¶
type Handler struct {
// contains filtered or unexported fields
}
Handler replays a scenario over HTTP.
func (*Handler) ArmToolCall ¶
ArmToolCall queues times consecutive replies that answer with exactly one call to name, regardless of what Steps/Repeat scripts there. A dispatched task is rarely the very next call the runtime makes of the model — a routing or classification call ahead of it is common — so times covers however many of those come first; each is popped and discarded the same as the one the suite actually wanted.
func (*Handler) ClearArmedToolCalls ¶
func (h *Handler) ClearArmedToolCalls()
ClearArmedToolCalls drops every queued arm that was never consumed.
func (*Handler) Remaining ¶
Remaining reports how many steps were never consumed. A suite fails on a non-zero value: a run that stopped calling the model one turn early leaves the final output looking plausible, and this is the only signal that says otherwise.
type Request ¶
Request is one model call the agent made, kept so a suite can assert on the history it sent as well as on what came back.
type Scenario ¶
type Scenario struct {
Name string `json:"name,omitempty"`
Steps []Step `json:"steps"`
// Repeat replays the last step for every call past the end instead of
// failing. It exists for the deployment smoke, which asserts on outcomes
// rather than on how many turns reaching them took, and must stay opt-in:
// everywhere else, a call past the end of the script is the finding.
Repeat bool `json:"repeat,omitempty"`
}
Scenario is the ordered script for one run.
func LoadScenario ¶
LoadScenario reads a committed scenario file.
type Server ¶
type Server struct {
*Handler
// contains filtered or unexported fields
}
Server is a handler listening on a local port.
type Step ¶
type Step struct {
Text string `json:"text,omitempty"`
ToolCalls []ToolCall `json:"tool_calls,omitempty"`
Usage *Usage `json:"usage,omitempty"`
// Status and Error make the step a provider failure instead of a reply.
Status int `json:"status,omitempty"`
Error string `json:"error,omitempty"`
// DelayMS holds the reply back before writing it, so a run stays mid-turn
// long enough for a suite to act on it. A failure step delays too: a
// provider that fails slowly is what a timeout looks like from here.
DelayMS int `json:"delay_ms,omitempty"`
}
Step is one scripted model reply.
A step carries several tool calls because the runtime schedules a turn's calls concurrently: a format with one call per turn could not express the case that scheduling has to leave unchanged. See docs/design/parallel-tool-execution.md.
type ToolCall ¶
type ToolCall struct {
// ID is what the tool result will reference. Left empty, the handler names
// it after its position so a scenario does not have to invent ids.
ID string `json:"id,omitempty"`
Name string `json:"name"`
Args map[string]any `json:"args,omitempty"`
}
ToolCall is one call the scripted assistant turn makes.