metering

Overview ¶

This sample meters model usage per tenant with an observer.

A WrapModel hook sees only the model calls of the Generate it is attached to. Here the answering model calls a tool that asks a second, cheaper model, and that call happens in a Generate of its own. An aix.Observer added to the context with tracing.WithInstrumentation sees both, so the meter bills each model at its own rate:

ctx = tracing.WithInstrumentation(ctx, aix.Observer{ModelDone: ...})

Run it:

go run .

Or with the Dev UI at http://localhost:4000:

curl -sL cli.genkit.dev | bash    # install the Genkit CLI, once
genkit start -- go run .

Or over HTTP, then ask for the tenant's running total:

curl -X POST http://localhost:8080/answerFlow \
  -H "Content-Type: application/json" \
  -d '{"data": {"tenant": "acme", "question": "Why is the sky blue?"}}'

curl -X POST http://localhost:8080/usageFlow \
  -H "Content-Type: application/json" \
  -d '{"data": "acme"}'