agentapidoctor

From download to an answer

Linux and macOS can install the exact v0.1.1 release without Go:

curl --proto '=https' --tlsv1.2 -fsSL \
  https://raw.githubusercontent.com/whyiug/agentapi-doctor/v0.1.1/install.sh | sh
$HOME/.local/bin/doctor demo

The pinned installer verifies the release archive against checksums.txt before extraction. If you prefer to inspect it first, download install.sh, then run sh install.sh. Windows users can download the verified ZIP from GitHub Releases; the Installation guide includes checksum steps for every platform.

The demo needs no API key. It starts a random loopback fixture, runs four lifecycle checks, stores local evidence, and stops the fixture automatically:

Result: CHECKS PASSED
Cases: 4 candidate / 4 applicable / 4 executed
Verdicts: PASS 4 | FAIL 0 | WARN 0 | INCONCLUSIVE 0 | SKIPPED 0 | ERRORED 0
Important conditions:
  [candidate_interpretations_pending_review] Candidate raw-wire interpretations; not certification.

Demo success validates this exact synthetic fixture and the installed CLI. It does not certify another endpoint, SDK, provider, or deployment.

Check an authorized endpoint

No project initialization or YAML is required:

export DOCTOR_TOKEN='replace-with-a-test-token'

doctor test \
  --base-url 'https://your-endpoint.example/v1' \
  --protocol openai-chat \
  --model 'your-model-id' \
  --auth-env DOCTOR_TOKEN

Use openai-responses or anthropic-messages for those API shapes. Omit --auth-env for an unauthenticated endpoint. The endpoint can be local, on a private network, or remote; it only needs to be yours or explicitly authorized for testing.

Each run sends at most four requests under one 60-second deadline. The token is read from the named environment variable, not a command argument. Requests stay on the configured origin, redirects are not followed, and evidence remains in the local .agentapi/ directory. Treat that directory as private local state and add .agentapi/ to the tested project's .gitignore.

What Doctor checks

Today, each Quick Check selects four executable raw HTTP checks for one of:

• OpenAI Chat Completions;
• OpenAI Responses;
• Anthropic Messages.

It can render the same result as terminal output, JSON, JUnit, SARIF, Markdown, or a self-contained offline HTML report. Named baselines and stable exit codes make the result usable in CI.

See a failure—not just a self-test

When the checked-in synthetic server omits the Responses terminal event, Doctor rejects the stream even though its media type looks correct:

Result: CHECKS FAILED
Cases: 4 candidate / 4 applicable / 4 executed
Verdicts: PASS 3 | FAIL 1 | WARN 0 | INCONCLUSIVE 0 | SKIPPED 0 | ERRORED 0
Important conditions:
  [candidate_interpretations_pending_review] Candidate raw-wire interpretations; not certification.
PASS  stream media type
PASS  required response envelope
FAIL  terminal event exactly once
PASS  terminal status

Download the offline failure report and open it locally, or reproduce it with the documented Synthetic Fixture. This example is a real, deterministic wire/lifecycle observation. It is not by itself a real SDK run or automatic root-cause attribution.

Why a real SDK changes the answer

A status-only smoke test can accept 200 OK and text/event-stream without ever asking whether the terminal object is usable. The source tree now includes one deliberately narrow counterexample using the real, pinned OpenAI Python SDK:

Reproduce it on Linux amd64 with CPython 3.12.12:

doctor reproduce openai-python-responses \
  --python .venv/bin/python \
  --fixture null-completed-output \
  --bundle ./openai-python-null-output.zip

This command uses a random loopback fixture and a synthetic token; it never contacts a provider or reads an API key. See the reproducible case study for the hash-locked install and exact evidence boundary. The case proves one frozen SDK behavior, not compatibility or incompatibility of any vendor endpoint.

Where it fits

Doctor is not a model-quality benchmark, provider ranking, relay checker, or vendor certification service. The current catalog also contains candidate metadata that is not executable coverage; see Known Limitations.

Evidence and privacy

• Credentials are resolved from an environment or protected file reference and are redacted before persistence.
• Exact endpoint, model, plan, profile, pack, and evidence digests stay bound to the run so two results can be meaningfully compared.
• Structured model content and tool arguments are not necessarily anonymous. Review evidence before sharing it.
• The provider still receives the bounded synthetic prompts and may retain them under its own policy.
• Structural checks request 64 output tokens. The Chat/Responses terminal-status check requests 512 so default-thinking models can reach a natural terminal more often. A provider may reject or ignore either field, so these are not enforced cost ceilings.

Only test systems you are explicitly authorized to assess.

Why this project exists

Real reports repeatedly show the same gap: direct requests work, but an SSE terminal, tool-call delta, strict Responses event, proxy, or client state machine breaks later. Examples include Open WebUI #21768, llama.cpp #20607, and Codex #24973.

The public Roadmap keeps the next work focused on reproducible endpoint checks, report reuse, and verified client cases. Registry, public matrix, and hosted-service candidates remain outside the supported v0.1 scope.

Documentation and community

Quick Start · Installation · CLI reference · Troubleshooting · Known limitations · Real SDK case · Roadmap · All docs

Contributions are welcome. Read CONTRIBUTING.md, propose a real compatibility failure or fixture, and use SECURITY.md for private vulnerability reports. Please do not open a public issue containing a credential or unredacted provider response.

Source and documentation use the Apache License 2.0 unless a file says otherwise. Vendored dependency notices are recorded in THIRD_PARTY_LICENSES.txt.