basic-media

Overview ¶

This sample demonstrates media: reading a picture, making one, and making a video from one.

• describeFlow sends a picture and gets words back.
• editFlow sends a picture and an instruction and gets a square one back.
• generateFlow sends only words and gets a picture back.
• animateFlow sends a picture and gets a video back, a while later.

The first three call an image-capable text model, where only the request and what the response may contain change between them. animateFlow calls a background model instead: video generation is too slow to answer in one request, so it starts an operation and polls until it finishes, streaming a chunk on every check.

A picture attaches either by reference to a file in the Files API, which pays off for anything large or reused, or inline as a data: URI. Both appear here.

Run it:

go run .

Or with the Dev UI, which renders the generated images and keeps a trace of every run at http://localhost:4000/traces:

curl -sL cli.genkit.dev | bash    # install the Genkit CLI, once
genkit start -- go run .

Or over HTTP. Generated images come back as base64 data: URIs, so pipe through jq to keep a terminal readable:

curl -X POST http://localhost:8080/describeFlow \
  -H "Content-Type: application/json" \
  -d '{"data": {"question": "What time of day is this and how can you tell?"}}'

curl -X POST http://localhost:8080/generateFlow \
  -H "Content-Type: application/json" \
  -d '{"data": {"description": "a red panda reading a book in a library"}}' | jq '.result.images | length'

Watch the video operation report in while it runs. The video comes back inline, so ask jq for the path it was also written to and open that instead:

curl -N -X POST 'http://localhost:8080/animateFlow?stream=true' \
  -H "Content-Type: application/json" \
  -d '{"data": {"motion": "a slow drone push toward the ridge as clouds drift past"}}' \
  | grep '"result"' | sed 's/^data: //' | jq -r '.result.path'