conformance

type Behaviour ¶

type Behaviour struct {
	// Text is the assistant text the backend returns for a normal completion.
	// For a schema-bound call the stub sends it as the structured payload.
	Text string

	// Hang makes the backend block until the request context is cancelled,
	// without ever responding. It is how the suite proves a client honours its
	// own timeout rather than waiting on the backend forever.
	Hang bool

	// FailStatus, when non-zero, makes the backend fail the request with this
	// HTTP status, so the module's registered status extractor is exercised. A
	// provider with no HTTP transport ignores it and the case skips.
	FailStatus int

	// ToolCall, when non-empty, makes the backend request this tool on the
	// first step and return Text on the second — the minimum ReAct exchange.
	ToolCall string

	// ToolInput is the raw JSON argument object sent with ToolCall. Empty means
	// an empty object.
	ToolInput string

	// Refuse makes the backend refuse, in whichever shape this provider's API
	// marks a refusal. Only a stub whose Control implements [Refuser] honours it.
	Refuse bool
}

Behaviour tells the module's stub backend how to respond. The zero value is a backend that returns empty text successfully.

type Control ¶

type Control interface {
	// Requests returns every request the backend received, oldest first. An
	// empty slice means the client never called out.
	Requests(t *testing.T) []Request
}

Control stands in for the backend the client under test talks to. It is how the suite asks what actually arrived, rather than inferring it from the client's own return values — which is exactly what a diverging provider gets wrong.

type Holder ¶ added in v0.27.0

type Holder interface {
	Hold(t *testing.T) (entered <-chan struct{}, release func())
}

Holder is the optional half of Control for a stub that can hold a request in flight, which the concurrency cases need (spec 0026 D10).

After Hold, the next request the backend receives waits: entered is closed when it arrives, and release lets it answer. A Control without it skips those cases.

type PromptObserver ¶ added in v0.37.0

type PromptObserver interface {
	// Prompts returns the user-turn texts of every request, oldest request
	// first, each in the order its turns were sent.
	Prompts(t *testing.T) [][]string
}

PromptObserver is the optional half of Control for a stub that can read the text of each request's user turns (spec 0038 D8).

Turn counts catch history carried over and requests combined, but not a request carrying the wrong caller's prompt, which still makes one turn. A Control without it leaves the stateless cases checking counts alone.

type Refuser ¶ added in v0.31.0

type Refuser interface {
	CanRefuse()
}

Refuser is the optional half of Control for a stub that can answer with a refusal when Behaviour.Refuse is set (spec 0029 D3).

Unlike the observers, it declares what the stub can send rather than what it can see. A stub that ignored Refuse would answer normally, and the refusal case couldn't tell that from an adapter that missed the refusal, so a stub says it can, or the case skips.

type Request ¶

type Request struct {
	// Turns is how many conversation turns the request carried, the prompt
	// included. A Control that cannot see turns implements [TurnBlind] rather
	// than reporting zero.
	Turns int
}

Request is one normalised observation of what the backend received. The fields are provider-independent concepts even though the wire formats carrying them are not.

type SchemaObserver ¶ added in v0.25.0

type SchemaObserver interface {
	// SchemaOffered reports whether the most recent request carried a response
	// schema.
	SchemaOffered(t *testing.T) bool
}

SchemaObserver is the optional half of Control for a stub that can report whether a request carried a response schema in the provider's structured-output field.

Without it the suite cannot tell a provider that sends the schema without a tool from one that silently dropped the schema. A Control that cannot see it omits the method and the schema case skips.

type StreamObserver ¶

type StreamObserver interface {
	// StreamClosed reports whether the streamed response body was closed.
	StreamClosed(t *testing.T) bool
}

StreamObserver is the optional half of Control for a stub that can detect its streamed response body being closed.

Without it the stream-lifecycle case can prove the callback's error reaches the caller but not that the connection was released, which is the half that actually leaks. An httptest-based stub implements it by wrapping the response body; a stub that cannot omits the method and the assertion skips.

type Suite ¶

type Suite struct {
	// Provider names the client under test. It is used in failure messages and
	// to build the snapshots the persistence cases restore.
	Provider chat.Provider

	// NewClient builds a client for cfg whose backend behaves as b describes,
	// together with the [Control] over that same backend.
	//
	// Each case calls it for its own client, so each gets a clean
	// conversation; a case gated on an optional interface may first build a
	// probe client to check for it. A factory should register its own teardown
	// with t.Cleanup rather than expecting the suite to tear anything down.
	NewClient func(t *testing.T, cfg chat.Config, b Behaviour) (chat.ChatClient, Control)
}

Suite describes the provider under test and the factory that builds it.

type ToolObserver ¶

type ToolObserver interface {
	// ToolsOffered reports whether the most recent request advertised any tool.
	ToolsOffered(t *testing.T) bool
}

ToolObserver is the optional half of Control for a stub that can report whether a request advertised tool definitions.

A Control that cannot see this omits the method and the tools-in-Ask case skips. Implement it wherever the wire format makes it visible — every HTTP provider can, since tools are a field in the request body.

type TurnBlind ¶ added in v0.37.0

type TurnBlind interface {
	BlindToTurns()
}

TurnBlind is implemented by a Control that cannot see a request's turns, such as one standing in for a subprocess whose prompt is flattened into argv (spec 0038 D9). The cases that count turns skip for it.

It is an opt-out rather than a zero in Request.Turns, so a stub that mis-parses a request and reports zero fails instead of skipping.