Skip to content

Cost Tracking

TraceVerde includes automatic cost tracking with pricing data for 1,700+ models across 40+ providers. Every LLM call is enriched with per-request cost breakdown.

How It Works

Cost tracking is enabled by default. For every LLM call, TraceVerde:

  1. Reads the model name from the span attributes
  2. Looks up pricing in the built-in llm_pricing.json database
  3. Calculates cost from token usage (prompt + completion)
  4. Adds cost attributes to the span

No configuration needed - just instrument and go.

Cost Attributes

Every LLM span gets these attributes:

Attribute Description Example
gen_ai.usage.cost.total Total cost in USD 0.003250
gen_ai.usage.cost.prompt Prompt token cost 0.001250
gen_ai.usage.cost.completion Completion token cost 0.002000
gen_ai.usage.cost.pricing_source Where the price came from table
gen_ai.request.model.deprecated Present only when the provider has announced the model's end true
gen_ai.request.model.deprecation_note Why, and what to migrate to Moonshot v1 platform sunset 2026-08-31...

Spotting models that are about to stop existing

A deprecated model is not a pricing problem — it bills normally, at a real rate, right up until the provider withdraws it. That is exactly what makes it easy to miss: nothing in the telemetry looks wrong until the day the calls start failing, and the only prior warning was a line in a vendor changelog.

Spans for such models carry gen_ai.request.model.deprecated, so the question "what are we running that has an end date?" becomes a query instead of an audit:

retiring = {s.attributes["gen_ai.request.model"]
            for s in spans if s.attributes.get("gen_ai.request.model.deprecated")}

The attributes are absent on active models, so this costs nothing in the normal case. pricing_source stays table for deprecated models — conflating the two would make a retiring model look unpriced when it is still costing money.

Deprecations are recorded in the deprecated map in genai_otel/llm_pricing.json, keyed by pricing key so it covers every category, including the ones whose values are bare numbers and cannot carry an inline flag.

Audio prices declare their unit

A bare number cannot say what it is per. Text-to-speech bills per character, transcription per second, and some audio models bill per token — so audio entries in genai_otel/llm_pricing.json state the unit in the key:

"eleven_multilingual_v2":  { "per_1k_chars":  0.10     },  // synthesis
"elevenlabs/scribe_v1":    { "per_second":    6.11e-05 },  // transcription
"gpt-4o-transcribe":       { "per_1k_tokens": 0.0025   }

This is not tidiness. Storing both as an undifferentiated float is what allowed 42 entries to sit at a per-minute rate against a per-second contract — a 60× error that nothing in the data contradicted. With the unit declared, billing a per-second model by character is detected and refused rather than silently producing a plausible number:

calc.calculate_cost("elevenlabs/scribe_v1", {"characters": 1000}, "audio")  # -> 0.0 + warning

A bare number is still accepted for backwards compatibility and for custom pricing, and keeps the old inferred-unit behaviour.

Knowing which prices have actually been checked

Most of the pricing table is inherited from an upstream aggregate. That is a reasonable starting point and a poor source of truth — auditing in August 2026 found rates stale by a full model generation, transposed between tiers, and off by 2.5x, none of which looked wrong in the file.

Entries verified against the vendor's own pricing page carry the date that happened, in the prices_checked map:

from genai_otel.cost_calculator import CostCalculator
calc = CostCalculator()

calc.price_checked("elevenlabs/scribe_v1", "audio")   # '2026-08-12'
calc.price_checked("gpt-4o-mini")                     # None - never verified here

None means "nobody has checked this against the vendor", not "suspect". It is the honest default for an inherited number, and it makes the audit a query instead of a manual sweep:

for key, checked in calc.stale_prices(older_than_days=180):
    print(key, checked or "never verified")

Pass include_unverified=False to see only entries that were checked once and have since aged out.

Telling "free" apart from "not measured"

cost.total = 0.0 means one of two very different things, so every span also carries gen_ai.usage.cost.pricing_source:

  • table — matched the shipped pricing file. Billable.
  • estimated — no entry for this model, so the price was inferred from the parameter count in its name (e.g. ...-7b) via the local-model size tier. Useful for relative comparison, not for a bill.
  • unpriced — no price could be determined at all.

Filter on this before aggregating. Summing cost.total across all spans without it under-reports spend, because every unpriced model contributes 0.0 and looks identical to a genuinely free call:

# Spend you can defend, plus an explicit count of what could not be priced.
billable = [s for s in spans if s.attributes.get("gen_ai.usage.cost.pricing_source") == "table"]
unpriced = [s for s in spans if s.attributes.get("gen_ai.usage.cost.pricing_source") == "unpriced"]
print(f"${sum(s.attributes['gen_ai.usage.cost.total'] for s in billable):.4f}"
      f" across {len(billable)} calls; {len(unpriced)} calls could not be priced")

If unpriced is large, the model is missing from genai_otel/llm_pricing.json — add it via custom_pricing_json or open an issue.

Supported Providers

The pricing file carries 1600+ chat models, covering roughly 83% of the priced catalogue on models.dev. Coverage is deliberately limited to first-party providers and the major clouds that resell under their own SKUs — gateway and router listings are excluded, because they re-price the same underlying model and the recorded cost would then depend on which aggregator was indexed first.

Provider Models Pricing Type
OpenAI GPT-6 Astra/Luna/Sol, GPT-6.1 Sol/Terra, GPT Image 2.6/2.5/2/1.5/1 Mini, GPT-Live 1, gpt-realtime-2.1, gpt-transcribe, GPT-5.5 series, GPT-4o, GPT-4 Turbo, o1/o3, embeddings (50+) Per token (prompt/completion)
Anthropic Claude Opus 5.5, Claude Sonnet 5.5, Claude Haiku 5.5, Claude Fable 5.1, Claude Opus 5, Claude Sonnet 5, Claude 3.5/3 series (20+) Per token + cache pricing
Google AI Gemini 4 Argon, Nano Banana 2.1, Gemini 3.8 Flash / Flash TTS / Live, Gemini 3.5 Transcribe, Gemini 3.7 Flash, Gemini 3.6 Flash, Gemini 3.5 Flash / Flash Lite, Gemini 3.1/2.5 Pro/Flash, PaLM 2 (35+) Per token
AWS Bedrock Titan, Claude, Llama, Mistral (25+) Per token
Azure OpenAI Same as OpenAI Per token
Cohere Command A/R/R+, North Mini Code, Embed v4/v3, rerankers (15+) Per token
Mistral AI Large 4/Medium/Small, Mixtral, Voxtral 3 transcription and TTS, embeddings (25+) Per token
Moonshot AI Kimi K3.1 Pro / Highspeed, Kimi K3, Kimi K2.7 Code, K2.6, K2.5, Kimi Latest (20+) Per token + cache pricing
Alibaba / Qwen Qwen3.8-Max Prime, Qwen3.8-Max, Qwen-Image 2.1 Pro, Qwen3.8 Omni Flash, Qwen3.8 Flash, Qwen3.8 27B, Qwen3.7 Flash/Max, Qwen3.6 series (40+) Per token
Thinking Machines Inkling, Inkling Small Per token
Xiaomi MiMo V2.6 Flash/Pro/UltraSpeed, MiMo V2.5 Pro/UltraSpeed, MiMo V2 Flash/Omni/Pro (9+) Per token + cache pricing
Zhipu / Z.AI GLM-5.3 Prime, GLM-5.3, GLM-5.3-FlashX, GLM-5.3-Flash, GLM-5.2, GLM-5.1, GLM-5 series Per token + cache pricing
Meituan LongCat-2.5, LongCat-2.0, LongCat Flash Chat Per token (cache free)
Sakana AI Sakana Namazu, Fugu Ultra Per token + cache pricing
Nvidia Nemotron 3.5 Super / Lightning, Nemotron 3 Ultra, Nemotron 4/Nano/Super (15+) Per token
Together AI DeepSeek-R1, Llama 3.x, Qwen (25+) Per token
Groq Llama 3.x, Mixtral, Gemma (20+) Per token
Ollama All local models Token tracking (free)
Vertex AI Gemini models Per token
Replicate All models Per second (hardware-based)
HuggingFace Local models Estimated (parameter-based)
Sarvam AI sarvam-m, Saarika, Bulbul (12+) Per token
Voyage AI voyage-4/3.5/3 series (15+) Per token
Jina AI jina-embeddings-v3, jina-clip-v2 (5+) Per token
Deepgram Nova-3/2, Aura-2/1, Whisper Large (20+) Per second/character
AssemblyAI Universal-3.6 Pro (streaming), Universal-3.5 Pro, Universal-Streaming, slam-1 (8+) Per second
ElevenLabs v4 / v4 Turbo, v3 / v3 Conversational, Flash v2.5, Multilingual v2, Scribe v2, Realtime v1 (15+) Per character; STT and Realtime per second
IBM Granite Chat, vision, embeddings (10+) Per token
DeepSeek V4 Pro, V4.1 Flash, V3, R1, VL (15+) Per token (off-peak rate; peak hours bill 2x)
Qwen/Alibaba Qwen 3.5, VL, embeddings (25+) Per token
xAI Grok text, image/video generation, and voice APIs Provider-reported request cost; TTS per character; STT per audio second when duration is returned
Upstage Solar Pro 4, Solar Pro 2, Solar Mini Per token
Meta Muse Wonder 1.0, Muse Spark Open, Muse Spark 1.x, Muse Glimmer Per token + cache pricing
MiniMax MiniMax M3.1 Flash, MiniMax M3 / Highspeed Per token
Inception Mercury 3, Mercury 2.5, Mercury Voice, Mercury Decide (decisions) Per token + cache pricing
Microsoft (Foundry) MAI-Image 2.6 / Flash, MAI-Voice 2.1 / Flash, MAI-Transcribe 2 / Streaming Per token, per character, per second
Black Forest Labs FLUX 3 Image / Klein, FLUX.2 pro/flex/max/klein Per image by resolution; per megapixel
Speechmatics Standard, Enhanced, Melia, Oak, Linden Per second
Cartesia Sonic 3.6, Sonic 2, Ink 2 Per character (credit plan rate); per second

Special Pricing

  • Reasoning tokens: OpenAI o1/o3 series have separate pricing for reasoning tokens
  • Cache pricing: Anthropic prompt caching costs (read/write rates)
  • Batch pricing: Some providers offer discounted batch pricing
  • Tiered and scheduled prices: Claude Haiku 5.5 (above 100K prompt tokens), GPT-6.x (above 272K), DeepSeek (peak hours) and introductory prices (Gemini 4 Argon to 2026-11-30, Gemini 3.8 TTS to 2026-12-31) are stored at the base or current rate; each entry's note states the other rates
  • Hardware pricing: Replicate charges per second of GPU/CPU time

How a Model Name Finds Its Price

A call's model name is matched to a key in the pricing file:

  1. Exact match, ignoring case.
  2. Otherwise, a key found inside the name, on token boundaries (-, ., /, :, _), if what follows the key still names the same model:
  3. kept: a dated or numbered snapshot (-20260928, -2024-08-06, -002), a Bedrock version (-v1:0), a Vertex tag (@default), -latest, -preview, -instruct, a quantisation tag (-fp8), and other words that do not name a new model;
  4. refused: a new version (mistral-large + -4, kimi-k3 + .1-pro), a 4-digit release tag (mistral-small + -2603), and, for chat, another modality (-tts, -image, -realtime, -embedding).
  5. Among acceptable keys, the one that is the whole rest of the name wins (bedrock/us.anthropic.claude-opus-5-5 takes the regional key), then the longest.

A name no key prices is reported as unpriced (pricing_source), or estimated from a parameter count in the name. It is never billed at another model's price. Use CostCalculator().resolve_model_key(model, call_type) to see which key prices a name.

Custom Model Pricing

For models not in the pricing database:

# Chat models
export GENAI_CUSTOM_PRICING_JSON='{"chat":{"my-model":{"promptPrice":0.001,"completionPrice":0.002}}}'

# Embeddings
export GENAI_CUSTOM_PRICING_JSON='{"embeddings":{"my-embed":0.00005}}'

# Multiple categories
export GENAI_CUSTOM_PRICING_JSON='{
  "chat": {
    "my-custom-chat": {"promptPrice": 0.001, "completionPrice": 0.002}
  },
  "embeddings": {
    "my-custom-embed": 0.00005
  }
}'

Custom prices merge with defaults. If you provide pricing for an existing model, the custom price overrides the default.

Pricing format:

  • Chat models: {"promptPrice": <$/1k tokens>, "completionPrice": <$/1k tokens>}
  • Embeddings: Single number for price per 1k tokens
  • Audio: Price per 1k characters (TTS) or per second (STT)

OpenInference Cost Enrichment

When using OpenInference instrumentors (LiteLLM, Smolagents, MCP, Claude Agent SDK, or Agno), cost tracking is automatically applied via the cost enrichment exporter. It reads OpenInference semantic conventions and adds cost attributes:

  • llm.model_name -> model lookup
  • llm.token_count.prompt / llm.token_count.completion -> cost calculation
  • openinference.span.kind -> call type (LLM, EMBEDDING, etc.)

Claude Agent SDK records model and token usage on its agent span when reported by the SDK; Agno records usage on its model spans. Strands Agents and Microsoft Agent Framework emit native GenAI spans handled by the regular cost path. A cost is only reported when usage and a model price are available; unknown models are not invented or estimated.

For frameworks that put aggregate token usage on an agent span and per-call usage on child model spans, TraceVerde prices the model-call spans and leaves the aggregate token counts visible without adding the same cost again on the agent span. Claude Agent SDK's agent span remains billable when it is the only span with usage for that run.

Disable Cost Tracking

export GENAI_ENABLE_COST_TRACKING=false

Or programmatically:

genai_otel.instrument(enable_cost_tracking=False)

Grafana Dashboard

Import the pre-built GenAI overview dashboard to visualize costs over time by provider and model.