scriptedmodel

package
v0.7.56 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 12, 2026 License: MIT Imports: 12 Imported by: 0

Documentation

Overview

Package scriptedmodel is a model server for tests. A test starts one on a local port, gives it a script of replies, points a model client at it and then reads back every request the client made.

It answers as four wire APIs, chosen by the request's path: the Anthropic Messages API (a path ending in /messages), OpenAI-style Chat Completions (a path ending in /chat/completions), the OpenAI Responses API (a path ending in /responses) and the Gemini API (a path ending in :generateContent or :streamGenerateContent). Each request takes the next step of the script. A step is a reply (text, tool calls, reasoning, usage) or a failure a real server produces: an answer cut at the output limit, an HTTP error, a refusal of a conversation that is too long worded as a given server words it, a stream that stalls, a connection that drops.

The package uses the standard library only and makes no outside connection.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func PrefixOf

func PrefixOf(prev, next Request) error

PrefixOf reports whether next continues prev: the same system prompt, the same tools, and prev's messages unchanged at the start of next's. That is what lets a server reuse the prefix it has cached. Cache markers (cache_control) are left out of the comparison, because a client moves them to the end of each request. A nil error means next continues prev; otherwise the error names the first difference.

Types

type API

type API int

API names the wire API a request used.

const (
	// Unknown is a request to a path the server does not serve.
	Unknown API = iota
	// Anthropic is the Anthropic Messages API.
	Anthropic
	// ChatCompletions is OpenAI-style Chat Completions.
	ChatCompletions
	// Responses is the OpenAI Responses API.
	Responses
	// Gemini is the Gemini API's generateContent.
	Gemini
)

func (API) String

func (a API) String() string

type Call

type Call struct {
	ID        string
	Name      string
	Arguments string
}

Call is one tool call. Arguments is the JSON text of its arguments.

type Dialect

type Dialect int

Dialect names a server whose refusal of a conversation that is too long the script can play.

const (
	// OpenAI refuses with a 400 and the code context_length_exceeded.
	OpenAI Dialect = iota + 1
	// VLLM refuses with a 400 in OpenAI's words and no code.
	VLLM
	// LlamaCpp refuses with a 400 of type exceed_context_size_error that
	// states its window as n_ctx.
	LlamaCpp
	// LMStudio answers 200 and sends the refusal as an error event in the
	// stream.
	LMStudio
	// Ollama refuses with a 400 that states by how much, not the window.
	Ollama
	// AnthropicAPI refuses with a 400 invalid_request_error.
	AnthropicAPI
	// GeminiAPI refuses with a 400 INVALID_ARGUMENT that states both
	// counts.
	GeminiAPI
)

type Message

type Message struct {
	// Role is system, developer, user, assistant or tool.
	Role string
	// Text is the message's text blocks, joined.
	Text string
	// Reasoning is the reasoning an assistant message carries back.
	Reasoning string
	// Calls is the tool calls an assistant message made.
	Calls []Call
	// Results is the tool results the message carries: any number in an
	// Anthropic user message, one in a Chat Completions tool message.
	Results []Result
	// Raw is the message as sent.
	Raw json.RawMessage
}

Message is one message of a request's conversation.

type Request

type Request struct {
	API    API
	Method string
	Path   string
	Header http.Header
	// Body is the request body as sent.
	Body []byte

	Model     string
	Stream    bool
	MaxTokens int
	// IncludeUsage is Chat Completions' stream_options.include_usage.
	IncludeUsage bool
	// Store is the Responses API's store, false when the request set it
	// so; Include its include list.
	Store   *bool
	Include []string
	// System is the system prompt's text: the Anthropic system field, or
	// the text of Chat Completions' system and developer messages.
	System string
	// Messages is the conversation, in order. A Chat Completions system
	// message is in it too.
	Messages []Message
	// Tools is the names of the tools offered, in the order sent.
	Tools []string
	// contains filtered or unexported fields
}

Request is one request the server received.

type Result

type Result struct {
	CallID  string
	Text    string
	IsError bool
}

Result is one tool result.

type Server

type Server struct {
	// contains filtered or unexported fields
}

Server plays a script of replies and records the requests it receives.

func New

func New(t testing.TB, steps ...Step) *Server

New starts a server that plays steps in order, one a request, and closes it when the test ends. A request that arrives after the last step is answered with a 500 and fails the test when it ends.

func (*Server) Close

func (s *Server) Close()

Close ends every stalled stream and stops the server. It is safe to call more than once.

func (*Server) Remaining

func (s *Server) Remaining() int

Remaining is the number of steps no request has taken yet.

func (*Server) Requests

func (s *Server) Requests() []Request

Requests returns every request received so far, in order.

func (*Server) Script

func (s *Server) Script(steps ...Step)

Script appends steps to the script.

func (*Server) URL

func (s *Server) URL() string

URL is the server's base address, with no trailing slash. A client's Anthropic base URL is URL() and its Chat Completions base URL is URL()+"/v1".

type Step

type Step struct {
	// contains filtered or unexported fields
}

Step is one scripted answer to one request.

func Cut

func Cut(text string) Step

Cut is a reply of text that stopped at the output limit.

func CutToolCall

func CutToolCall(name, partialArguments string) Step

CutToolCall is a reply that stopped at the output limit in the middle of a tool call's arguments. partialArguments is what arrived of them, which is not valid JSON.

func Drop

func Drop(text string) Step

Drop starts a stream, sends text, and then closes the connection.

func Overflow

func Overflow(dialect Dialect, window int) Step

Overflow is the refusal of a conversation that is too long, as dialect words it, from a server whose window is window tokens. Each dialect answers in its own server's shape whatever API the request used.

func RateLimited

func RateLimited(retryAfter time.Duration) Step

RateLimited is a 429 that says when to try again.

func Raw

func Raw(code int, contentType, body string) Step

Raw is an HTTP answer with exactly this status, content type and body.

func Repeat

func Repeat(n int, step Step) []Step

Repeat is step n times.

func Stall

func Stall(text string) Step

Stall starts a stream, sends text, and then sends nothing more until the client goes away.

func Status

func Status(code int, message string) Step

Status is an HTTP error with message in the error body of the API the request used.

func Text

func Text(text string) Step

Text is a reply of text alone, which ends the model's turn.

func ToolCalls

func ToolCalls(calls ...Call) Step

ToolCalls is a reply that asks for the calls.

func (Step) WithHeader

func (s Step) WithHeader(name, value string) Step

WithHeader adds a response header to an HTTP answer.

func (Step) WithReasoning

func (s Step) WithReasoning(text string) Step

WithReasoning puts reasoning before a reply's text and calls.

func (Step) WithText

func (s Step) WithText(text string) Step

WithText puts text before a reply's tool calls.

func (Step) WithUsage

func (s Step) WithUsage(u Usage) Step

WithUsage sets the token count a reply reports.

type Usage

type Usage struct {
	// Input is the input the server read fresh, not from its cache.
	Input int
	// CacheRead is the input the server read from its cache.
	CacheRead int
	// CacheWrite is the input the server wrote to its cache. Chat
	// Completions has no field for it.
	CacheWrite int
	// Output is the output, reasoning included.
	Output int
	// Reasoning is the part of Output that was reasoning. The Anthropic
	// API has no field for it.
	Reasoning int
}

Usage is the token count a reply reports.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL