langfuse-model-sync

langfuse-model-sync

Syncs model pricing from LiteLLM's open-source pricing database into Langfuse so that cost tracking works automatically for every model the agent uses.

Why This Exists

Langfuse only calculates costDetails (input/output/total cost) for models that have a pricing definition configured in its model management. When the agent starts using a new model (e.g. gpt-5.4-2026-03-05), Langfuse receives the token usage but can't compute costs because it doesn't know the per-token price — resulting in empty costDetails and totalCost: 0 in traces.

Rather than manually adding model definitions through the Langfuse UI every time a new model appears, this script automates the process by:

1. Fetching pricing from LiteLLM's database (2,500+ models with up-to-date pricing)
2. Checking which models already have definitions in Langfuse
3. Creating or updating definitions for models that are missing or have stale pricing

Local Pricing Overrides

If a model is missing from the LiteLLM database (e.g. a brand new model like gpt-5.4-nano-2026-03-17), you can provide manual pricing information in scripts/langfuse-model-sync/overrides.json.

Example overrides.json:

{
  "gpt-5.4-nano-2026-03-17": {
    "input_cost_per_token": 0.0000002,
    "output_cost_per_token": 0.00000125,
    "litellm_provider": "openai"
  }
}

This file is automatically merged with the LiteLLM database at runtime, with overrides taking precedence.

Usage

Local (via Make)

# Preview what would change (dry run)
make langfuse/sync-dry MODELS=gpt-5.4-2026-03-05,claude-sonnet-4-6

# Apply changes
make langfuse/sync MODELS=gpt-5.4-2026-03-05,claude-sonnet-4-6

Local (direct)

source .env  # needs LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY, LANGFUSE_HOST

go run -mod=mod ./scripts/langfuse-model-sync \
  --models gpt-5.4-2026-03-05,claude-sonnet-4-6 \
  --dry-run

GitHub Actions

The workflow at .github/workflows/langfuse-model-sync.yml can be triggered:

• Manually: Actions → "Langfuse Model Pricing Sync" → "Run workflow" → enter model names
• Scheduled: Runs weekly (Monday 09:00 UTC) with a curated default model list

How It Works

flowchart TD
    A["Fetch pricing from LiteLLM"] --> B{"Found pricing?"}
    B -->|No| C["⚠️ not_found"]
    B -->|Yes| D{"Model exists in Langfuse?"}
    D -->|"Yes, same price"| E["⏭️ skipped"]
    D -->|"Yes, stale price"| F{"Langfuse-managed?"}
    D -->|No| G["Create in Langfuse → ✅ created"]
    F -->|Yes| H["⏭️ skipped — won't override"]
    F -->|No| I["Delete + re-create → 🔄 updated"]

Environment Variables