mockllm

package
v0.2.0-alpha.13 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 18, 2026 License: Apache-2.0 Imports: 9 Imported by: 0

Documentation

Overview

Package mockllm serves scripted model replies over the three wire protocols BuildMax speaks, so an end-to-end suite can drive a real run without a provider, a key, or a paid call.

A scenario is a list of steps, replayed in order: one step answers one model call, whatever protocol asked. Nothing is inferred from the request, because a mock that answers what it thinks was asked stops being evidence — the run it produces is the mock's opinion rather than the agent's behaviour.

How long a reply takes is the exception, and it is set out of band rather than guessed: a step can script its own stall, and a suite driving a deployed mock arms one over the control route. A test that has to act on a run while the run is still going — cancel it, take its worker away — otherwise races the run to its own end, and duration is the one property of a scripted turn that says nothing about what the agent did.

See docs/design/end-to-end-testing.md §4.

Index

Constants

View Source
const (
	ProtocolOpenAIChat      = "openai_compatible"
	ProtocolOpenAIResponses = "openai"
	ProtocolAnthropic       = "anthropic"
)

Protocol names the wire protocol a request arrived on. The values match config.LLMProvider* so a suite can configure a model entry from one constant.

View Source
const ControlRequestsPath = "/control/requests"

ControlRequestsPath returns every request the mock has received so far, so a suite driving a deployed mock over HTTP can inspect a tool result a scripted final reply never echoes back. GET, unlike every other route here.

View Source
const ControlStallPath = "/control/stall"

ControlStallPath is the route that arms a stall on a mock a suite cannot rescript — one already deployed, answering a stack the suite only reaches over HTTP. It is matched by suffix so an ingress may serve it under a prefix.

View Source
const ControlToolCallPath = "/control/toolcall"

ControlToolCallPath arms a one-shot tool call the same way ControlStallPath arms a stall. See Handler.armedStep.

Variables

This section is empty.

Functions

This section is empty.

Types

type Handler

type Handler struct {
	// contains filtered or unexported fields
}

Handler replays a scenario over HTTP.

func NewHandler

func NewHandler(s Scenario) *Handler

NewHandler returns a handler that replays s.

func (*Handler) ArmToolCall

func (h *Handler) ArmToolCall(name string, args map[string]any, times int)

ArmToolCall queues times consecutive replies that answer with exactly one call to name, regardless of what Steps/Repeat scripts there. A dispatched task is rarely the very next call the runtime makes of the model — a routing or classification call ahead of it is common — so times covers however many of those come first; each is popped and discarded the same as the one the suite actually wanted.

func (*Handler) ClearArmedToolCalls

func (h *Handler) ClearArmedToolCalls()

ClearArmedToolCalls drops every queued arm that was never consumed.

func (*Handler) Remaining

func (h *Handler) Remaining() int

Remaining reports how many steps were never consumed. A suite fails on a non-zero value: a run that stopped calling the model one turn early leaves the final output looking plausible, and this is the only signal that says otherwise.

func (*Handler) Requests

func (h *Handler) Requests() []Request

Requests returns the calls made so far, in order.

func (*Handler) ServeHTTP

func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request)

func (*Handler) Stall

func (h *Handler) Stall(d time.Duration)

Stall makes every later reply wait d before it is written. Zero clears it. It applies to replies the mock has not started yet, never to one in flight.

type Request

type Request struct {
	Protocol string
	Stream   bool
	Body     []byte
}

Request is one model call the agent made, kept so a suite can assert on the history it sent as well as on what came back.

type Scenario

type Scenario struct {
	Name  string `json:"name,omitempty"`
	Steps []Step `json:"steps"`
	// Repeat replays the last step for every call past the end instead of
	// failing. It exists for the deployment smoke, which asserts on outcomes
	// rather than on how many turns reaching them took, and must stay opt-in:
	// everywhere else, a call past the end of the script is the finding.
	Repeat bool `json:"repeat,omitempty"`
}

Scenario is the ordered script for one run.

func LoadScenario

func LoadScenario(path string) (Scenario, error)

LoadScenario reads a committed scenario file.

type Server

type Server struct {
	*Handler
	// contains filtered or unexported fields
}

Server is a handler listening on a local port.

func Start

func Start(scenario Scenario) (*Server, error)

Start serves scenario on a loopback port until Close.

func (*Server) BaseURL

func (s *Server) BaseURL(protocol string) string

BaseURL is the api_url a model entry needs for this protocol. The two families disagree about where the version segment lives: the OpenAI client appends its path to the configured base, and the Anthropic client appends "v1/messages" of its own.

func (*Server) Close

func (s *Server) Close()

Close stops serving.

func (*Server) URL

func (s *Server) URL() string

URL is the server's root.

type Step

type Step struct {
	Text      string     `json:"text,omitempty"`
	ToolCalls []ToolCall `json:"tool_calls,omitempty"`
	Usage     *Usage     `json:"usage,omitempty"`
	// Status and Error make the step a provider failure instead of a reply.
	Status int    `json:"status,omitempty"`
	Error  string `json:"error,omitempty"`
	// DelayMS holds the reply back before writing it, so a run stays mid-turn
	// long enough for a suite to act on it. A failure step delays too: a
	// provider that fails slowly is what a timeout looks like from here.
	DelayMS int `json:"delay_ms,omitempty"`
}

Step is one scripted model reply.

A step carries several tool calls because the runtime schedules a turn's calls concurrently: a format with one call per turn could not express the case that scheduling has to leave unchanged. See docs/design/parallel-tool-execution.md.

type ToolCall

type ToolCall struct {
	// ID is what the tool result will reference. Left empty, the handler names
	// it after its position so a scenario does not have to invent ids.
	ID   string         `json:"id,omitempty"`
	Name string         `json:"name"`
	Args map[string]any `json:"args,omitempty"`
}

ToolCall is one call the scripted assistant turn makes.

type Usage

type Usage struct {
	PromptTokens     int `json:"prompt_tokens"`
	CompletionTokens int `json:"completion_tokens"`
}

Usage is the token report for one step. Runs assert on it, so it is scripted rather than invented per request.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL