Documentation
¶
Overview ¶
This sample demonstrates composing the Retry and Fallback middlewares into a resilient model pipeline.
ai.WithUse composes outer-to-inner, so this:
ai.WithUse(&middleware.Retry{...}, &middleware.Fallback{...})
becomes Retry { Fallback { model } } at call time. Fallback moves down its model list on a fallback-eligible status (UNAVAILABLE, NOT_FOUND, DEADLINE_EXCEEDED, INTERNAL, ...); if the whole cascade still fails with a retryable error, Retry backs off and runs it again.
The primary model here is deliberately a non-existent id, so Google AI returns NOT_FOUND, Fallback catches it, and the real model answers on every run. Pointing the primary at a valid model just means Fallback never fires.
Run it:
go run .
Or with the Dev UI, to watch both attempts in a trace of every run at http://localhost:4000/traces:
curl -sL cli.genkit.dev | bash # install the Genkit CLI, once genkit start -- go run .
Or over HTTP:
curl -N -X POST 'http://localhost:8080/resilientFlow?stream=true' \
-H "Content-Type: application/json" \
-d '{"data": {"topic": "quantum computing"}}'
Source Files