contract

Contract Replay Tests

Contract tests in this project are adapter replay tests.

They do not call external APIs in CI. Instead, they replay recorded provider payloads from testdata/ through the real provider adapters and verify the normalized core output.

What is validated

• Real adapter parsing paths (ChatCompletion, StreamChatCompletion, ListModels, Responses, StreamResponses)
• Streaming conversion behavior ([DONE], chunk/event mapping)
• Provider-specific conversion logic (for example Anthropic -> OpenAI-compatible / Responses output)

How replay works

1. A custom in-memory http.RoundTripper returns recorded fixtures for expected method/path routes.
2. Provider adapters are constructed with NewWithHTTPClient(...) and pointed at a local replay base URL.
3. Tests call adapter methods directly and assert normalized outputs.

No sockets are opened and no network access is required.

Fixture layout

testdata/
├── openai/
├── anthropic/
├── gemini/
├── groq/
├── kimicode/
└── xai/

Each folder contains recorded JSON and SSE payloads used by replay tests.

Gemini has two recorded surfaces: the OpenAI-compatible payloads (chat_*, models.json) and the native API payloads (native_*), recorded with the gemini-native recorder provider:

GEMINI_API_KEY=... go run ./cmd/recordapi -provider=gemini-native   -endpoint=generate_content -output=tests/contract/testdata/gemini/native_chat_completion.json
# endpoints: generate_content, generate_content_stream, image_generate_content,
#            batch_embed_contents, models

The inlineData.data payload in native_image_generation.json is truncated post-recording to a 1x1 PNG — the recorded envelope (field names, casing, usage metadata) is unchanged.

Running

The CI workflow runs this suite in the test-contract job (.github/workflows/test.yml).

# Run contract replay tests
go test -v -tags=contract -timeout=5m ./tests/contract/...

# Make target
make test-contract

Updating fixtures

Contract tests under tests/contract/**/*_test.go must validate full normalized output against committed golden files. When finish_reason == "tool_calls", golden output must include message.tool_calls[] entries with id, type, and function{name,arguments}.

Use the canonical recorder target to refresh provider payload fixtures:

make record-api

Then refresh normalized contract-output goldens from replay tests:

RECORD=1 go test -v -tags=contract -timeout=5m ./tests/contract/...

Re-run the suite without RECORD=1 before committing.

recordapi Endpoints

When recording fixtures manually with cmd/recordapi, available endpoint options are:

• chat: POST /v1/chat/completions
• chat_stream: POST /v1/chat/completions with "stream": true
• models: GET /v1/models
• responses: POST /v1/responses
• responses_stream: POST /v1/responses with "stream": true

Example request body for responses:

{
  "model": "gpt-4o-mini",
  "input": "Say 'Hello, World!' and nothing else."
}

Example request body for responses_stream:

{
  "model": "gpt-4o-mini",
  "input": "Say 'Hello, World!' and nothing else.",
  "stream": true
}

Notes:

• The -model flag in cmd/recordapi overrides "model" for these request bodies.
• responses capability is currently supported by openai and xai in recording mode.
• Running responses/responses_stream for unsupported providers returns a local capability error before any network call.