Documentation
¶
Overview ¶
Package scriptedmodel is a model server for tests. A test starts one on a local port, gives it a script of replies, points a model client at it and then reads back every request the client made.
It answers as four wire APIs, chosen by the request's path: the Anthropic Messages API (a path ending in /messages), OpenAI-style Chat Completions (a path ending in /chat/completions), the OpenAI Responses API (a path ending in /responses) and the Gemini API (a path ending in :generateContent or :streamGenerateContent). Each request takes the next step of the script. A step is a reply (text, tool calls, reasoning, usage) or a failure a real server produces: an answer cut at the output limit, an HTTP error, a refusal of a conversation that is too long worded as a given server words it, a stream that stalls, a connection that drops.
The package uses the standard library only and makes no outside connection.
Index ¶
- func PrefixOf(prev, next Request) error
- type API
- type Call
- type Dialect
- type Message
- type Request
- type Result
- type Server
- type Step
- func Cut(text string) Step
- func CutToolCall(name, partialArguments string) Step
- func Drop(text string) Step
- func Overflow(dialect Dialect, window int) Step
- func RateLimited(retryAfter time.Duration) Step
- func Raw(code int, contentType, body string) Step
- func Repeat(n int, step Step) []Step
- func Stall(text string) Step
- func Status(code int, message string) Step
- func Text(text string) Step
- func ToolCalls(calls ...Call) Step
- type Usage
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func PrefixOf ¶
PrefixOf reports whether next continues prev: the same system prompt, the same tools, and prev's messages unchanged at the start of next's. That is what lets a server reuse the prefix it has cached. Cache markers (cache_control) are left out of the comparison, because a client moves them to the end of each request. A nil error means next continues prev; otherwise the error names the first difference.
Types ¶
type API ¶
type API int
API names the wire API a request used.
const ( // Unknown is a request to a path the server does not serve. Unknown API = iota // Anthropic is the Anthropic Messages API. Anthropic // ChatCompletions is OpenAI-style Chat Completions. ChatCompletions // Responses is the OpenAI Responses API. Responses // Gemini is the Gemini API's generateContent. Gemini )
type Dialect ¶
type Dialect int
Dialect names a server whose refusal of a conversation that is too long the script can play.
const ( // OpenAI refuses with a 400 and the code context_length_exceeded. OpenAI Dialect = iota + 1 // VLLM refuses with a 400 in OpenAI's words and no code. VLLM // LlamaCpp refuses with a 400 of type exceed_context_size_error that // states its window as n_ctx. LlamaCpp // LMStudio answers 200 and sends the refusal as an error event in the // stream. LMStudio // Ollama refuses with a 400 that states by how much, not the window. Ollama // AnthropicAPI refuses with a 400 invalid_request_error. AnthropicAPI // GeminiAPI refuses with a 400 INVALID_ARGUMENT that states both // counts. GeminiAPI )
type Message ¶
type Message struct {
// Role is system, developer, user, assistant or tool.
Role string
// Text is the message's text blocks, joined.
Text string
// Reasoning is the reasoning an assistant message carries back.
Reasoning string
// Calls is the tool calls an assistant message made.
Calls []Call
// Results is the tool results the message carries: any number in an
// Anthropic user message, one in a Chat Completions tool message.
Results []Result
// Raw is the message as sent.
Raw json.RawMessage
}
Message is one message of a request's conversation.
type Request ¶
type Request struct {
API API
Method string
Path string
Header http.Header
// Body is the request body as sent.
Body []byte
Model string
Stream bool
MaxTokens int
// IncludeUsage is Chat Completions' stream_options.include_usage.
IncludeUsage bool
// Store is the Responses API's store, false when the request set it
// so; Include its include list.
Store *bool
Include []string
// System is the system prompt's text: the Anthropic system field, or
// the text of Chat Completions' system and developer messages.
System string
// Messages is the conversation, in order. A Chat Completions system
// message is in it too.
Messages []Message
// Tools is the names of the tools offered, in the order sent.
Tools []string
// contains filtered or unexported fields
}
Request is one request the server received.
type Server ¶
type Server struct {
// contains filtered or unexported fields
}
Server plays a script of replies and records the requests it receives.
func New ¶
New starts a server that plays steps in order, one a request, and closes it when the test ends. A request that arrives after the last step is answered with a 500 and fails the test when it ends.
func (*Server) Close ¶
func (s *Server) Close()
Close ends every stalled stream and stops the server. It is safe to call more than once.
type Step ¶
type Step struct {
// contains filtered or unexported fields
}
Step is one scripted answer to one request.
func CutToolCall ¶
CutToolCall is a reply that stopped at the output limit in the middle of a tool call's arguments. partialArguments is what arrived of them, which is not valid JSON.
func Overflow ¶
Overflow is the refusal of a conversation that is too long, as dialect words it, from a server whose window is window tokens. Each dialect answers in its own server's shape whatever API the request used.
func RateLimited ¶
RateLimited is a 429 that says when to try again.
func Stall ¶
Stall starts a stream, sends text, and then sends nothing more until the client goes away.
func (Step) WithHeader ¶
WithHeader adds a response header to an HTTP answer.
func (Step) WithReasoning ¶
WithReasoning puts reasoning before a reply's text and calls.
type Usage ¶
type Usage struct {
// Input is the input the server read fresh, not from its cache.
Input int
// CacheRead is the input the server read from its cache.
CacheRead int
// CacheWrite is the input the server wrote to its cache. Chat
// Completions has no field for it.
CacheWrite int
// Output is the output, reasoning included.
Output int
// Reasoning is the part of Output that was reasoning. The Anthropic
// API has no field for it.
Reasoning int
}
Usage is the token count a reply reports.