Documentation
¶
Overview ¶
This sample demonstrates the base compat_oai plugin pointed at a custom OpenAI-compatible provider, here OpenRouter: a streaming flow that generates a joke with a model that resolves dynamically by name and takes the OpenAI SDK's own request type as its config.
This is the shape for a provider with no plugin of its own. OpenRouter has one, in plugins/compat_oai/openrouter, and it is the better way to reach OpenRouter: the SDK request type has no home for the routing, fallback, and reasoning fields that are the reason to use the gateway, and the config schema rejects them. See samples/compat_oai/openrouter.
Run it:
export OPENROUTER_API_KEY=... go run .
Or with the Dev UI, to call the flow from a browser and read a trace of every run at http://localhost:4000/traces:
curl -sL cli.genkit.dev | bash # install the Genkit CLI, once genkit start -- go run .
Or over HTTP. Streaming needs ?stream=true:
curl -N -X POST 'http://localhost:8080/jokesFlow?stream=true' \
-H "Content-Type: application/json" \
-d '{"data": {"topic": "bananas"}}'
Source Files