Documentation
¶
Overview ¶
Package inferencetest provides a scripted, in-memory inference.Client for running and testing agents without a provider, an API key or a network.
A Client answers each call with the next Step of its script:
client := inferencetest.New(
inferencetest.ToolCall("read_file", `{"path":"go.mod"}`),
inferencetest.Text("The module is example.com/demo."),
)
loop.WithInference(client, inferencetest.Model())
Both Invoke and Stream are implemented, and they agree: folding a Stream through core's content/streamaccumulator yields the same message Invoke returns. Stream emits the chunk shapes a real codec produces — text in several TextChunk deltas, a tool call as a ToolUseChunk carrying its ID and Name followed by InputJSON fragments, reasoning as indexed ThinkingChunk deltas — and completes with an authoritative stream.StreamResult carrying the finish reason, the model name and token usage.
Like a real client, every call first validates the request (model.Model.Validate, then inference.ValidateRequestFeatures) and fails without consuming a step when it is invalid, so a request no provider would accept does not pass silently here. Use Model for a descriptor that validates.
Every request that reaches the script is recorded, so a test can assert what the model saw (Requests), and a step can check the request it answers (Step.Expect). A Client is safe for concurrent use.
Example (ToolRoundTrip) ¶
A tool-call round trip driven by a minimal agent loop: stream a reply, assemble it, run any tool it calls, send the results back, and repeat until the model stops calling tools. The second step checks that the tool result reached the model.
package main
import (
"context"
"encoding/json"
"errors"
"fmt"
"io"
"github.com/looprig/core/content"
"github.com/looprig/core/content/streamaccumulator"
"github.com/looprig/inference"
"github.com/looprig/inference/inferencetest"
)
func userMessage(text string) *content.UserMessage {
return &content.UserMessage{Message: content.Message{
Role: content.RoleUser,
Blocks: []content.Block{&content.TextBlock{Text: text}},
}}
}
func main() {
client := inferencetest.New(
inferencetest.Text("Let me add those.").ToolCall("add", `{"a": 2, "b": 3}`),
inferencetest.Text("2 + 3 = 5.").Expect(func(req inference.Request) error {
last, ok := req.Messages[len(req.Messages)-1].(*content.ToolResultMessage)
if !ok || last.ToolUseID != "call_1_0" {
return errors.New("the model expected the add result")
}
return nil
}),
)
tools := map[string]func(json.RawMessage) string{
"add": func(input json.RawMessage) string {
var args struct{ A, B int }
if err := json.Unmarshal(input, &args); err != nil {
return err.Error()
}
return fmt.Sprint(args.A + args.B)
},
}
req := inference.Request{
Model: inferencetest.Model(),
Messages: content.AgenticMessages{userMessage("What is 2 + 3?")},
Tools: []inference.Tool{{Name: "add", Schema: json.RawMessage(`{"type":"object"}`)}},
}
for {
reply, calls := streamTurn(client, req)
req.Messages = append(req.Messages, reply)
fmt.Println("model:", reply.Blocks[0].(*content.TextBlock).Text)
if len(calls) == 0 {
break
}
for _, call := range calls {
result := tools[call.Name](call.Input)
fmt.Printf("tool %s(%s) = %s\n", call.Name, call.Input, result)
req.Messages = append(req.Messages, &content.ToolResultMessage{
Message: content.Message{Role: content.RoleTool, Blocks: []content.Block{
&content.ToolResultBlock{ToolUseID: call.ID, Content: []content.Block{&content.TextBlock{Text: result}}},
}},
ToolUseID: call.ID,
})
}
}
fmt.Println("script error:", client.Err())
}
// streamTurn streams one reply and assembles it the way an agent loop does.
func streamTurn(client inference.Client, req inference.Request) (*content.AIMessage, []content.ToolUseBlock) {
reader, err := client.Stream(context.Background(), req)
if err != nil {
panic(err)
}
defer reader.Close()
var text streamaccumulator.Text
var calls streamaccumulator.ToolUses
for {
chunk, err := reader.Next()
if errors.Is(err, io.EOF) {
break
}
if err != nil {
panic(err)
}
switch c := chunk.(type) {
case *content.TextChunk:
text.Add(c)
case *content.ToolUseChunk:
calls.Add(c)
}
}
msg := &content.AIMessage{Message: content.Message{Role: content.RoleAssistant}}
if block := text.Block(); block != nil {
msg.Blocks = append(msg.Blocks, block)
}
for _, call := range calls.Blocks() {
msg.Blocks = append(msg.Blocks, &call)
}
return msg, calls.Blocks()
}
Output: model: Let me add those. tool add({"a": 2, "b": 3}) = 5 model: 2 + 3 = 5. script error: <nil>
Index ¶
- Constants
- func LastUserText(req inference.Request) string
- func Model(opts ...model.ModelOption) model.Model
- type Client
- func (c *Client) Append(steps ...Step)
- func (c *Client) Err() error
- func (c *Client) Invoke(ctx context.Context, req inference.Request) (*inference.Response, error)
- func (c *Client) Remaining() int
- func (c *Client) Requests() []inference.Request
- func (c *Client) Stream(ctx context.Context, req inference.Request) (*stream.StreamReader[content.Chunk], error)
- type ExhaustedError
- type ExpectationError
- type Step
- func (s Step) ChunkDelay(d time.Duration) Step
- func (s Step) ChunkSize(n int) Step
- func (s Step) Expect(check func(inference.Request) error) Step
- func (s Step) Finish(reason stream.FinishReason) Step
- func (s Step) Repeat() Step
- func (s Step) StreamError(err error) Step
- func (s Step) Text(text string) Step
- func (s Step) Thinking(text string) Step
- func (s Step) ToolCall(name, argsJSON string) Step
- func (s Step) ToolCallWithID(id, name, argsJSON string) Step
- func (s Step) Usage(usage content.Usage) Step
Examples ¶
Constants ¶
const ( // ProviderName is the provider label Model reports. ProviderName model.ProviderName = "inferencetest" // APIFormat is the API-format label Model reports. No codec speaks it: a // Model is meant to be paired with a Client from this package. APIFormat model.APIFormat = "inferencetest" // ModelName is the model name Model reports. ModelName = "scripted" // ContextWindow is the context window Model advertises. ContextWindow content.TokenCount = 200_000 )
Variables ¶
This section is empty.
Functions ¶
func LastUserText ¶
LastUserText returns the concatenated text blocks of the request's last user message, or "" when the request has none. It is the text Echo replies with.
func Model ¶
func Model(opts ...model.ModelOption) model.Model
Model returns a descriptor for a scripted Client. It validates, and it declares the capabilities an agent runtime gates on — tools, image input and native structured output, with and without tools — so a composition built around it takes the same paths it would with a capable provider model. It advertises a ContextWindow-token window and no endpoint.
Options apply after those defaults, so model.WithThinking or model.WithContextLimits can adjust them.
Types ¶
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client is a scripted inference.Client. Each call that passes request validation consumes the next Step of the script, in call order, whether it arrives through Invoke or Stream. Construct one with New; the zero value is an empty script. A Client is safe for concurrent use.
func New ¶
New returns a Client that answers calls with steps, in order.
New panics if a Repeat step is followed by another step, which could never be reached.
func (*Client) Append ¶
Append adds steps to the end of the script, for example to extend it between turns of a test. It panics if the script would contain a step after a Repeat step.
func (*Client) Err ¶
Err reports the first script failure: an *ExhaustedError when a call found no step left, or an *ExpectationError when a step's Expect check refused its request. The failing call also returned it, but an agent loop typically reports a failed model call only as a failed turn; Err keeps the cause. It is nil when every call so far was answered as scripted, including calls answered with a scripted Fail.
func (*Client) Remaining ¶
Remaining reports how many steps have not been consumed. A Repeat step counts as one and is never consumed.
func (*Client) Requests ¶
Requests returns the requests that reached the script, in call order. Each is a copy whose message and tool slices the caller may keep and modify; the messages' blocks are copied too.
Example ¶
Requests records what the model saw, for assertions after the fact.
package main
import (
"context"
"fmt"
"strings"
"github.com/looprig/core/content"
"github.com/looprig/inference"
"github.com/looprig/inference/inferencetest"
)
func userMessage(text string) *content.UserMessage {
return &content.UserMessage{Message: content.Message{
Role: content.RoleUser,
Blocks: []content.Block{&content.TextBlock{Text: text}},
}}
}
func main() {
client := inferencetest.New(inferencetest.Text("ok"))
_, _ = client.Invoke(context.Background(), inference.Request{
Model: inferencetest.Model(),
System: "Be brief.",
Messages: content.AgenticMessages{userMessage("Summarize the plan.")},
})
for _, req := range client.Requests() {
fmt.Println(strings.Join([]string{req.System, inferencetest.LastUserText(req)}, " | "))
}
}
Output: Be brief. | Summarize the plan.
type ExhaustedError ¶
type ExhaustedError struct {
// Call is the 1-based number of the call that found the script empty.
Call int
}
ExhaustedError reports a call made after every step was consumed.
func (*ExhaustedError) Error ¶
func (e *ExhaustedError) Error() string
type ExpectationError ¶
type ExpectationError struct {
// Call is the 1-based number of the refused call.
Call int
// Err is the check's error.
Err error
}
ExpectationError reports a request refused by a step's Expect check.
func (*ExpectationError) Error ¶
func (e *ExpectationError) Error() string
func (*ExpectationError) Unwrap ¶
func (e *ExpectationError) Unwrap() error
type Step ¶
type Step struct {
// contains filtered or unexported fields
}
Step is one scripted model reply. Build one with Text, Thinking, ToolCall, ToolCallWithID, Fail, Echo or Func, then refine it with its methods. Every method returns a modified copy, so a Step value can be shared and reused.
A reply's content is laid out as reasoning, then text, then tool calls — the order an agent loop assembles a streamed message in — and within each kind in the order it was added. Successive Text calls concatenate into one text block; each Thinking call is its own reasoning block; each tool call is its own tool-use block.
The zero Step is a reply with no content, which an agent loop treats as an empty response.
func Echo ¶
func Echo() Step
Echo returns a step that replies with the text of the request's last user message (see LastUserText). Combine it with Repeat for a keyless demo agent that answers anything:
inferencetest.New(inferencetest.Echo().Repeat())
Example ¶
Echo with Repeat answers anything, forever: a keyless demo model.
package main
import (
"context"
"fmt"
"github.com/looprig/core/content"
"github.com/looprig/inference"
"github.com/looprig/inference/inferencetest"
)
func userMessage(text string) *content.UserMessage {
return &content.UserMessage{Message: content.Message{
Role: content.RoleUser,
Blocks: []content.Block{&content.TextBlock{Text: text}},
}}
}
func main() {
client := inferencetest.New(inferencetest.Echo().Repeat())
for _, question := range []string{"ping", "is anyone there?"} {
resp, err := client.Invoke(context.Background(), inference.Request{
Model: inferencetest.Model(),
Messages: content.AgenticMessages{userMessage(question)},
})
if err != nil {
panic(err)
}
fmt.Println(resp.Message.Blocks[0].(*content.TextBlock).Text)
}
}
Output: ping is anyone there?
func Fail ¶
Fail returns a step whose call fails with err before any output: Invoke returns (nil, err) and Stream returns (nil, err), the way a provider error such as failure.NewAPIError(429, ...) surfaces from a real client. Content added to a failing step is never delivered.
Example ¶
A provider error, such as a rate limit, surfaces from the call itself.
package main
import (
"context"
"errors"
"fmt"
"github.com/looprig/core/content"
"github.com/looprig/inference"
"github.com/looprig/inference/failure"
"github.com/looprig/inference/inferencetest"
)
func userMessage(text string) *content.UserMessage {
return &content.UserMessage{Message: content.Message{
Role: content.RoleUser,
Blocks: []content.Block{&content.TextBlock{Text: text}},
}}
}
func main() {
client := inferencetest.New(
inferencetest.Fail(failure.NewAPIError(429, "rate_limit_exceeded", "", 0)),
inferencetest.Text("Recovered."),
)
req := inference.Request{Model: inferencetest.Model(), Messages: content.AgenticMessages{userMessage("Hi")}}
_, err := client.Invoke(context.Background(), req)
var apiErr *failure.APIError
fmt.Println("rate limited:", errors.As(err, &apiErr) && apiErr.Status == 429)
resp, err := client.Invoke(context.Background(), req)
if err != nil {
panic(err)
}
fmt.Println("retry:", resp.Message.Blocks[0].(*content.TextBlock).Text)
}
Output: rate limited: true retry: Recovered.
func Func ¶
Func returns a step computed from the request it answers. fn's step is used in its place — its content, finish reason, usage, failure and delivery settings — except that its Repeat and Expect settings are ignored; set those on the step Func returns.
func Text ¶
Text returns a reply that says text. It finishes with stream.FinishReasonStop.
Example ¶
A text reply, streamed word by word.
package main
import (
"context"
"errors"
"fmt"
"io"
"github.com/looprig/core/content"
"github.com/looprig/inference"
"github.com/looprig/inference/inferencetest"
)
func userMessage(text string) *content.UserMessage {
return &content.UserMessage{Message: content.Message{
Role: content.RoleUser,
Blocks: []content.Block{&content.TextBlock{Text: text}},
}}
}
func main() {
client := inferencetest.New(inferencetest.Text("Hello from a scripted model."))
reader, err := client.Stream(context.Background(), inference.Request{
Model: inferencetest.Model(),
Messages: content.AgenticMessages{userMessage("Say hello.")},
})
if err != nil {
panic(err)
}
defer reader.Close()
for {
chunk, err := reader.Next()
if errors.Is(err, io.EOF) {
break
}
if err != nil {
panic(err)
}
fmt.Printf("%q\n", chunk.(*content.TextChunk).Text)
}
result, _ := reader.Result()
fmt.Println("finish:", result.FinishReason)
}
Output: "Hello " "from " "a " "scripted " "model." finish: stop
func Thinking ¶
Thinking returns a reply that carries one reasoning block. Chain Text or ToolCall to give the reasoning something to lead to.
func ToolCall ¶
ToolCall returns a reply that calls tool name with argsJSON as its arguments. The call gets a deterministic ID, "call_<call>_<n>", where call is the 1-based number of the Client call it answers and n its position among the reply's tool calls. An empty argsJSON is sent as "{}". The arguments are sent verbatim, not validated: a model can emit malformed JSON, and a test may want to see how an agent copes. The reply finishes with stream.FinishReasonToolUse.
func ToolCallWithID ¶
ToolCallWithID is ToolCall with a caller-chosen tool-use ID.
func (Step) ChunkDelay ¶
ChunkDelay makes Stream wait d before each delta, so a demo streams at a visible pace and a test can interrupt a stream part-way. Next stops waiting when the call's context is cancelled or the reader is closed.
func (Step) ChunkSize ¶
ChunkSize sets how Stream splits text, reasoning and tool arguments into deltas. By default each delta is one word with its trailing whitespace; a positive n makes each delta at most n runes, and a negative n sends each block as a single delta.
func (Step) Expect ¶
Expect adds a check on the request the step answers. When check returns an error the call fails with an *ExpectationError wrapping it, the step is consumed, and the Client's Err reports it — an agent loop usually turns a failed model call into a failed turn, so Err is how a test learns why. Several checks run in the order they were added.
func (Step) Finish ¶
func (s Step) Finish(reason stream.FinishReason) Step
Finish overrides the reply's finish reason. By default a reply with tool calls finishes with stream.FinishReasonToolUse and any other reply with stream.FinishReasonStop; use Finish to script, for example, a reply cut off at the output limit (stream.FinishReasonLength).
func (Step) Repeat ¶
Repeat makes the step answer every remaining call instead of one: it is never consumed, so it must be the last step of a script.
func (Step) StreamError ¶
StreamError makes the reply fail part-way: Stream delivers all of the reply's chunks and then Next returns err instead of io.EOF, the way a dropped connection truncates a real stream, so the reader reports no Result. Invoke, which has no partial result to return, returns (nil, err).
func (Step) Thinking ¶
Thinking adds a reasoning block. It carries no signature, as a reasoning block from a model without signed reasoning would.
func (Step) ToolCallWithID ¶
ToolCallWithID adds a tool call with a caller-chosen ID. An empty id is replaced with the deterministic one ToolCall assigns.
Source Files