Documentation
¶
Overview ¶
Package protocoltest provides a framework for end-to-end validation of the model gateway's protocol transformation layer.
Architecture ¶
server_validate.VirtualServer — a mock HTTP provider that speaks OpenAI, Anthropic, and Google response formats. Conceptually a "virtual model" for testing.
TestEnv — wires a real gateway Server (with transform pipeline) to a VirtualServer, configures routing rules, and provides SendAs() for round-trip testing.
Matrix — executes the full cross-product of sources × targets × scenarios × streaming modes.
Note: The existing internal/virtualmodel package is a production Gin server. This package (protocoltest) is the test-only framework. Future integration with virtualmodel is planned once both stabilize.
Usage ¶
env := protocoltest.NewTestEnv(t) defer env.Close() env.SetupRoute(protocol.TypeAnthropicV1, protocol.TypeOpenAIChat, protocoltest.TextScenario()) result := env.SendAs(t, protocol.TypeAnthropicV1, protocoltest.TextScenario(), false) assert.Equal(t, "assistant", result.Role)
Index ¶
- type Assertion
- func AssertContentContains(substring string) Assertion
- func AssertContentEquals(expected string) Assertion
- func AssertFinishReason(expected string) Assertion
- func AssertHTTPStatus(expected int) Assertion
- func AssertHasThinking() Assertion
- func AssertHasToolCalls(count int) Assertion
- func AssertModelContains(substring string) Assertion
- func AssertNoThinking() Assertion
- func AssertRoleEquals(expected string) Assertion
- func AssertStreamEventCount(min int) Assertion
- func AssertToolCallArgs(index int, key, value string) Assertion
- func AssertToolCallName(index int, name string) Assertion
- func AssertUsageNonZero() Assertion
- type Matrix
- type MockResponseBuilder
- type RoundTripResult
- type Scenario
- func AllScenarios() []Scenario
- func ErrorScenario() Scenario
- func MultiTurnScenario() Scenario
- func StreamingTextScenario() Scenario
- func StreamingToolUseScenario() Scenario
- func TextScenario() Scenario
- func ThinkingScenario() Scenario
- func ToolResultScenario() Scenario
- func ToolUseScenario() Scenario
- type TestEnv
- type TokenUsage
- type ToolCallResult
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Assertion ¶
type Assertion struct {
Name string
Check func(r *RoundTripResult) error
}
Assertion is a named check applied to a RoundTripResult.
func AssertContentContains ¶
AssertContentContains returns an Assertion that the response content contains substring.
func AssertContentEquals ¶
AssertContentEquals returns an Assertion that the response content equals expected.
func AssertFinishReason ¶
AssertFinishReason returns an Assertion that the finish_reason equals expected.
func AssertHTTPStatus ¶
AssertHTTPStatus returns an Assertion that the HTTP status code equals expected.
func AssertHasThinking ¶
func AssertHasThinking() Assertion
AssertHasThinking returns an Assertion that thinking content is non-empty.
func AssertHasToolCalls ¶
AssertHasToolCalls returns an Assertion that exactly count tool calls are present.
func AssertModelContains ¶
AssertModelContains returns an Assertion that the model name contains substring.
func AssertNoThinking ¶
func AssertNoThinking() Assertion
AssertNoThinking returns an Assertion that no thinking content is present.
func AssertRoleEquals ¶
AssertRoleEquals returns an Assertion that the response role equals expected.
func AssertStreamEventCount ¶
AssertStreamEventCount returns an Assertion that at least min SSE events were received.
func AssertToolCallArgs ¶
AssertToolCallArgs returns an Assertion that tool call at index has key=value in its JSON args.
func AssertToolCallName ¶
AssertToolCallName returns an Assertion that the tool call at index has the given name.
func AssertUsageNonZero ¶
func AssertUsageNonZero() Assertion
AssertUsageNonZero returns an Assertion that at least one token count > 0.
type Matrix ¶
type Matrix struct {
Sources []protocol.APIType
Targets []protocol.APIType
Scenarios []Scenario
Streaming []bool
}
Matrix defines the cross-product of source protocols, target protocols, scenarios, and streaming modes to validate.
func DefaultMatrix ¶
func DefaultMatrix() *Matrix
DefaultMatrix returns the full validation matrix covering all supported protocol combinations, all built-in scenarios, and both streaming modes.
func (*Matrix) OnlyScenarios ¶
OnlyScenarios returns a copy of the Matrix filtered to only the named scenarios.
type MockResponseBuilder ¶
type MockResponseBuilder = server_validate.MockResponseBuilder
MockResponseBuilder re-exports server_validate.MockResponseBuilder for convenience.
type RoundTripResult ¶
type RoundTripResult struct {
// Source and target context
SourceProtocol protocol.APIType
TargetProtocol protocol.APIType
ScenarioName string
IsStreaming bool
// HTTP layer
HTTPStatus int
RawBody []byte
StreamEvents []string // raw SSE event lines (streaming only)
// Extracted semantics (populated by the framework after parsing)
Content string
Role string
Model string
FinishReason string
ToolCalls []ToolCallResult
ThinkingContent string
Usage *TokenUsage
}
RoundTripResult captures the full result of a round-trip request through the gateway.
type Scenario ¶
type Scenario struct {
Name string
Description string
Tags []string
// MockResponses keyed by provider APIStyle ("openai", "anthropic", "google").
MockResponses map[server_validate.APIStyle]MockResponseBuilder
// Assertions run after every round-trip for this scenario.
Assertions []Assertion
}
Scenario is a named test scenario describing:
- What the mock provider should return (MockResponses per APIStyle)
- What assertions to run on the round-trip result
func AllScenarios ¶
func AllScenarios() []Scenario
AllScenarios returns the full set of built-in validation scenarios.
func ErrorScenario ¶
func ErrorScenario() Scenario
ErrorScenario tests that provider error responses are forwarded to the client.
func MultiTurnScenario ¶
func MultiTurnScenario() Scenario
MultiTurnScenario tests a conversation with system prompt + 2 turns of history.
func StreamingTextScenario ¶
func StreamingTextScenario() Scenario
StreamingTextScenario tests SSE streaming for a plain text response.
func StreamingToolUseScenario ¶
func StreamingToolUseScenario() Scenario
StreamingToolUseScenario tests SSE streaming for a tool call response.
func TextScenario ¶
func TextScenario() Scenario
TextScenario is the baseline: a single user message and a plain text reply.
func ThinkingScenario ¶
func ThinkingScenario() Scenario
ThinkingScenario tests that extended thinking blocks are present in Anthropic responses.
func ToolResultScenario ¶
func ToolResultScenario() Scenario
ToolResultScenario tests a multi-turn conversation including a tool result message.
func ToolUseScenario ¶
func ToolUseScenario() Scenario
ToolUseScenario exercises a single tool/function call.
type TestEnv ¶
type TestEnv struct {
// contains filtered or unexported fields
}
TestEnv wires a real gateway Server (with the full transform pipeline) to a VirtualServer. It manages config, routing rules, and provides SendAs() for full round-trip testing.
func NewTestEnv ¶
NewTestEnv creates a TestEnv with a fresh gateway config and a new VirtualServer. All resources are cleaned up via t.Cleanup.
func (*TestEnv) Close ¶
func (env *TestEnv) Close()
Close is a no-op; resources are cleaned up via t.Cleanup registered in NewTestEnv.
func (*TestEnv) SendAs ¶
func (env *TestEnv) SendAs(t *testing.T, source protocol.APIType, s Scenario, streaming bool) *RoundTripResult
SendAs sends a request to the gateway as the given source protocol, using the request model configured by SetupRoute, and returns the parsed result.
Streaming requests use the real httptest.Server (env.gatewayServer) because httptest.ResponseRecorder does not support Gin's streaming/SSE machinery. Non-streaming requests use the recorder for simplicity.
func (*TestEnv) SetupRoute ¶
SetupRoute configures a gateway rule that routes source protocol requests to the virtual server acting as a target protocol provider.
The virtual server is pre-registered with the scenario's mock responses.
func (*TestEnv) VirtualCallCount ¶
VirtualCallCount returns the number of requests received by the virtual server.
func (*TestEnv) VirtualURL ¶
VirtualURL returns the URL of the underlying virtual server.
type TokenUsage ¶
TokenUsage holds token counts extracted from a provider response.
type ToolCallResult ¶
ToolCallResult holds a single tool/function call extracted from a response.
Source Files