Configuration¶

TraceVerde is configured via environment variables or programmatically. All settings have sensible defaults - most users only need to set OTEL_SERVICE_NAME and OTEL_EXPORTER_OTLP_ENDPOINT.

A complete sample.env template is included in the repository.

Core Settings¶

Variable	Default	Description
`OTEL_SERVICE_NAME`	`genai-app`	Service name for traces and metrics
`OTEL_EXPORTER_OTLP_ENDPOINT`	`http://localhost:4318`	OTLP endpoint URL. Leave empty for console output
`OTEL_EXPORTER_OTLP_HEADERS`		Headers in `key1=val1,key2=val2` format
`OTEL_EXPORTER_OTLP_CERTIFICATE`		CA certificate file used to verify an HTTPS OTLP endpoint
`OTEL_EXPORTER_OTLP_TIMEOUT`	`60`	OTLP exporter timeout in seconds
`OTEL_EXPORTER_OTLP_PROTOCOL`	`http/protobuf`	Protocol: `http/protobuf` (default) or `grpc`
`OTEL_SERVICE_INSTANCE_ID`		Instance identifier (container ID, pod name)
`OTEL_ENVIRONMENT`	`dev`	Deployment environment (dev, staging, production)
`GENAI_OTEL_LOG_LEVEL`	`INFO`	Logging level (DEBUG, INFO, WARNING, ERROR)
`GENAI_FAIL_ON_ERROR`	`false`	Raise exceptions on instrumentation errors

For a TLS-enabled platform deployment, set OTEL_EXPORTER_OTLP_ENDPOINT to the HTTPS OTLP base URL and set OTEL_EXPORTER_OTLP_CERTIFICATE to the CA certificate file that signed the endpoint. The exporter appends the signal path (v1/traces or v1/metrics) to the base URL.

Feature Toggles¶

Variable	Default	Description
`GENAI_ENABLE_COST_TRACKING`	`true`	Automatic cost calculation for LLM calls
`GENAI_ENABLE_GPU_METRICS`	`true`	GPU utilization, memory, temperature, power metrics
`GENAI_ENABLE_MCP_INSTRUMENTATION`	`true`	Database, cache, vector DB, queue instrumentation
`GENAI_ENABLE_HTTP_INSTRUMENTATION`	`false`	HTTP/API instrumentation (disabled to avoid OTLP conflicts)
`GENAI_ENABLE_CO2_TRACKING`	`false`	CO2 emissions tracking from GPU power consumption
`GENAI_ENABLE_CONTENT_CAPTURE`	`false`	Capture prompt/response content as span events

Instrumentor Selection¶

Variable	Default	Description
`GENAI_ENABLED_INSTRUMENTORS`	all defaults	Comma-separated list of instrumentors to enable

Default instrumentors: openai, openrouter, cometapi, anthropic, google.generativeai, boto3, azure.ai.openai, cohere, mistralai, together, groq, ollama, vllm, llamacpp, vertexai, replicate, anyscale, sambanova, sarvamai, elevenlabs, langchain, langgraph, llama_index, transformers, autogen, autogen_agentchat, google_adk, pydantic_ai, openai_agents, bedrock_agents, crewai, smolagents (3.10+), litellm (3.10+), typesafe (3.10+)

Opt-in instrumentors (not enabled by default; add them to GENAI_ENABLED_INSTRUMENTORS explicitly):

Name	Purpose
`litellm_latency`	Streaming latency (TTFT/TPOT) for litellm routes that bypass provider SDKs

litellm routes OpenAI, Azure and OpenAI-compatible traffic through the OpenAI SDK, which is already instrumented, so those calls are measured on the inner provider span with no extra configuration. Every other provider (Anthropic, Bedrock, Gemini, Cohere, HuggingFace, ...) is implemented with litellm's own HTTP client, which no provider instrumentor sees. Enabling litellm_latency wraps litellm's own entry points so those routes report TTFT/TPOT too:

export GENAI_ENABLED_INSTRUMENTORS="openai,anthropic,litellm,litellm_latency"

The span it creates is the parent of any inner provider span. When an inner span already measured the request, this one records no tokens, cost or latency of its own -- one request is never counted twice. It is opt-in because it participates in token and cost accounting for every litellm call.

Example - enable only specific instrumentors:

export GENAI_ENABLED_INSTRUMENTORS=openai,anthropic,crewai

Sampling¶

Variable	Default	Description
`GENAI_SAMPLING_RATE`	`1.0`	Trace sampling rate (0.0-1.0). 1.0 = trace everything

Use lower values in high-traffic production to reduce telemetry volume:

export GENAI_SAMPLING_RATE=0.1  # Sample 10% of traces

Cost Tracking¶

Variable	Default	Description
`GENAI_ENABLE_COST_TRACKING`	`true`	Enable/disable cost tracking
`GENAI_CUSTOM_PRICING_JSON`		Custom model pricing (see Cost Tracking guide)

export GENAI_CUSTOM_PRICING_JSON='{"chat":{"my-model":{"promptPrice":0.001,"completionPrice":0.002}}}'

GPU Metrics¶

Variable	Default	Description
`GENAI_ENABLE_GPU_METRICS`	`true`	Enable GPU metrics collection
`GENAI_GPU_COLLECTION_INTERVAL`	`5`	Collection interval in seconds
`GENAI_POWER_COST_PER_KWH`	`0.12`	Electricity cost in USD per kWh

Common electricity rates: US average ~$0.12/kWh, Europe ~$0.20/kWh, Industrial ~$0.07/kWh.

CO2 Emissions Tracking¶

TraceVerde can track CO2 emissions from GPU power consumption. Two modes are available:

Manual mode: Uses a fixed carbon intensity value you provide. Codecarbon mode: Uses codecarbon for automatic region-based carbon intensity lookup.

pip install genai-otel-instrument[co2]  # Install codecarbon
export GENAI_ENABLE_CO2_TRACKING=true

CO2 Configuration¶

Variable	Default	Description
`GENAI_ENABLE_CO2_TRACKING`	`false`	Enable CO2 emissions tracking
`GENAI_CARBON_INTENSITY`	`475.0`	Carbon intensity in gCO2e/kWh (manual fallback)
`GENAI_CO2_USE_MANUAL`	`false`	Force manual calculation even with codecarbon installed
`GENAI_CO2_COUNTRY_ISO_CODE`		3-letter ISO country code (e.g., `USA`, `GBR`, `DEU`, `IND`)
`GENAI_CO2_REGION`		Region/state within country (e.g., `california`, `texas`)
`GENAI_CO2_CLOUD_PROVIDER`		Cloud provider: `aws`, `gcp`, `azure`
`GENAI_CO2_CLOUD_REGION`		Cloud region (e.g., `us-east-1`, `europe-west1`)
`GENAI_CO2_OFFLINE_MODE`	`true`	Run codecarbon without external API calls
`GENAI_CO2_TRACKING_MODE`	`machine`	`machine` (all processes) or `process` (current only)
`GENAI_CODECARBON_LOG_LEVEL`	`error`	Codecarbon logging verbosity

Example: CO2 Tracking for US West Coast¶

export GENAI_ENABLE_CO2_TRACKING=true
export GENAI_CO2_COUNTRY_ISO_CODE=USA
export GENAI_CO2_REGION=california

Example: CO2 Tracking on AWS¶

export GENAI_ENABLE_CO2_TRACKING=true
export GENAI_CO2_CLOUD_PROVIDER=aws
export GENAI_CO2_CLOUD_REGION=us-east-1

Example: Manual Carbon Intensity¶

export GENAI_ENABLE_CO2_TRACKING=true
export GENAI_CO2_USE_MANUAL=true
export GENAI_CARBON_INTENSITY=56.0   # France (mostly nuclear)

Reference carbon intensity values (gCO2e/kWh): - France: ~56 (nuclear) - UK: ~233 - Germany: ~350 - US average: ~420 - China: ~555 - India: ~700

CO2 Metrics¶

When enabled, these metrics are recorded:

Metric	Unit	Description
`gen_ai.co2.emissions`	kgCO2e	Cumulative CO2 emissions
`gen_ai.power.consumption`	kWh	Cumulative power consumption
`gen_ai.power.cost`	USD	Cumulative electricity cost

Content Capture¶

Variable	Default	Description
`OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT`	unset	Cross-library content-capture switch. Takes precedence over `GENAI_ENABLE_CONTENT_CAPTURE`
`GENAI_ENABLE_CONTENT_CAPTURE`	`false`	Capture prompt/response text in spans
`GENAI_CONTENT_MAX_LENGTH`	`200`	Max characters to capture (0 = unlimited)
`GENAI_CAPTURE_EMBEDDING_VECTORS`	`false`	Capture the raw embedding vector on embeddings spans

OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT is the spelling shared with other OpenTelemetry GenAI instrumentations, so an application migrating from one of them keeps working without relearning this switch. It accepts (case-insensitively):

Value	Effect here
`NO_CONTENT`	No capture
`SPAN_ONLY`	Capture
`SPAN_AND_EVENT`	Capture
`EVENT_ONLY`	No capture

EVENT_ONLY maps to no capture because this library has a single boolean capture switch rather than separate span and event sinks -- writing content onto spans when the operator asked for events only would put it somewhere they did not ask for. An unrecognised value logs a warning and captures nothing, rather than falling through to GENAI_ENABLE_CONTENT_CAPTURE: a typo on a privacy switch should fail closed.

Privacy

Content capture records full prompts and responses. This may expose sensitive data. Ensure proper data handling and access controls before enabling in production.

Shutdown and Flushing¶

Variable	Default	Description
`GENAI_FLUSH_ON_SIGTERM`	`false`	Flush pending telemetry when the process receives SIGTERM
`GENAI_SIGTERM_FLUSH_TIMEOUT`	`5.0`	Seconds to wait for that flush

The OpenTelemetry SDK registers an atexit hook, so a clean exit and an uncaught exception both flush already. Signals are the gap: Python does not run atexit handlers when the process is terminated by one, and docker stop and Kubernetes pod eviction both send SIGTERM. Every rolling restart therefore drops whatever is still queued in the batch processor - up to 5 seconds or 512 spans by default. Nothing raises, so the loss is silent and dashboards simply read a little low.

Enable the handler in containerised deployments:

export GENAI_FLUSH_ON_SIGTERM=true

It is off by default because installing a signal handler takes over a slot the host application may want. When enabled it never replaces an existing handler - whatever was registered before is called after the flush - and if nothing was, the default disposition is restored and the signal re-raised, so the process still exits with the conventional status rather than appearing to ignore SIGTERM.

If your application already owns its shutdown path, drain telemetry directly instead and leave the handler off:

import genai_otel
genai_otel.flush_telemetry(timeout_seconds=5.0)

SIGKILL, out-of-memory kills and segfaults cannot be handled from inside the process. To bound the loss window there, shorten the batch delay with the standard OTEL_BSP_SCHEDULE_DELAY, trading export frequency for exposure.

Host and Instance Identity¶

Spans carry host, OS, process and instance attributes so that traffic can be attributed to a machine and to one instance among several running on it. All of them are OpenTelemetry registry names, and all are controlled by standard OTEL_* variables - see Resource Attributes for the full list.

Variable	Default	Description
`OTEL_EXPERIMENTAL_RESOURCE_DETECTORS`	`host,os,process`	SDK detectors to run. Set to `host,os` to drop process attributes, or `otel` for none
`OTEL_SERVICE_INSTANCE_ID`		Instance identifier. Recommended - set it to something your orchestrator already guarantees unique and stable, such as a pod name
`OTEL_RESOURCE_ATTRIBUTES`		Any additional resource attributes, `key=value` comma-separated. Always wins over what the library detects
`GENAI_SERVICE_INSTANCE_ID_MODE`	`random`	How `service.instance.id` is generated when not set explicitly. `random` or `derived`

Defaults differ under the hardened profiles

GENAI_PROFILE=strict|bfsi|bank defaults the detector list to host,os, leaving out process so that no command line reaches the backend. Host and instance identity are unaffected. Setting OTEL_EXPERIMENTAL_RESOURCE_DETECTORS explicitly overrides this.

Choosing how `service.instance.id` is generated¶

service.instance.id is what separates two instances of the same service on the same host. If OTEL_SERVICE_INSTANCE_ID (or OTEL_RESOURCE_ATTRIBUTES) supplies one, it is used unchanged. Otherwise:

random (default) - a version 4 UUID, stable for the life of the process. This is OpenTelemetry's primary recommendation and is the only option guaranteed to distinguish instances that run at the same time. It changes on every restart.
derived - a version 5 UUID over the host name, the service name and the normalised startup arguments, using the namespace the specification designates for derived instance IDs. The same configured instance keeps its identity across restarts, which matters for any consumer that builds per-instance baselines: a key that changes on restart never accumulates enough history for one instance.

Its limitation is the mirror image: instances that differ in no argument - several identical workers started on one host - collapse onto a single ID. Prefer OTEL_SERVICE_INSTANCE_ID where the deployment can supply a real one, and reach for derived only where it cannot.

Telling the three apart from a span¶

A consumer reading service.instance.id needs to know whether it can be trusted as a grouping key, and the value says so on its own - no need to watch a service across restarts before classifying it:

Value	Meaning	Safe as a grouping key?
Not a UUID	An operator assigned it	Yes - stable and unique
A version 5 UUID	`derived` mode	Yes, but it identifies an instance configuration, not a process: identically-started workers on one host share it
A version 4 UUID	Nobody configured anything	No - it changes on every restart. Group on `host.name` + `service.name` instead

The version 4 case deliberately does not distinguish this library's random mode from the service.instance.id that opentelemetry-sdk 1.44 and later generate on their own. There is no difference worth acting on: both mean the instance was never configured, and both change on every restart.

Note the direction the version 4 rule can be wrong in. An operator who sets OTEL_SERVICE_INSTANCE_ID to a version 4 UUID they generate and manage themselves has a perfectly stable ID that reads as unconfigured. The cost is lost per-instance granularity rather than a corrupted grouping, so it fails safe - but prefer a non-UUID instance ID (a pod name, a worker index) anyway, since it is also the only kind a human can read in an alert.

process.pid is not an instance identifier

It separates concurrent processes within one lifetime, but a restart changes it. Use it as evidence, never as the key an entity is tracked by.

Semantic Conventions¶

Variable	Default	Description
`OTEL_SEMCONV_STABILITY_OPT_IN`	`gen_ai/dup`	Semantic convention mode

Options:

gen_ai/dup - Dual emission (default): emits the current names and the superseded ones (gen_ai.usage.prompt_tokens, gen_ai.usage.completion_tokens), so existing dashboards keep working during migration
gen_ai - Current conventions only: gen_ai.usage.input_tokens, gen_ai.usage.output_tokens

OpenTelemetry semantic conventions v1.27.0 renamed gen_ai.usage.prompt_tokens to gen_ai.usage.input_tokens and gen_ai.usage.completion_tokens to gen_ai.usage.output_tokens (semantic-conventions#1200). Backends that consume the current conventions - Arize AX, for example, maps them onto llm.token_count.* - report zero tokens if only the superseded names are present. Dual emission is therefore the default: both spellings are sent, so such backends resolve tokens and cost while dashboards still querying the old names keep working. Set gen_ai once nothing depends on the superseded names. See the Arize AX guide for a worked example.

Ollama Server Metrics¶

Variable	Default	Description
`GENAI_ENABLE_OLLAMA_SERVER_METRICS`	`true`	Poll Ollama's `/api/ps` for VRAM usage
`OLLAMA_BASE_URL`	`http://localhost:11434`	Ollama server URL
`GENAI_OLLAMA_METRICS_INTERVAL`	`5.0`	Polling interval in seconds
`GENAI_OLLAMA_MAX_VRAM_GB`	auto-detected	Override GPU VRAM size in GB

Evaluation Features¶

All evaluation features are opt-in and disabled by default. See the Evaluation guide for detailed examples.

PII Detection¶

Variable	Default	Description
`GENAI_ENABLE_PII_DETECTION`	`false`	Enable PII detection
`GENAI_PII_MODE`	`detect`	Mode: `detect`, `redact`, or `block`
`GENAI_PII_THRESHOLD`	`0.5`	Detection confidence threshold (0.0-1.0)
`GENAI_PII_GDPR_MODE`	`false`	Add GDPR-specific entity types (IBAN, EU passports)
`GENAI_PII_HIPAA_MODE`	`false`	Add HIPAA-specific entity types (medical records)
`GENAI_PII_PCI_DSS_MODE`	`false`	Add PCI-DSS entity types (credit cards)

Toxicity Detection¶

Variable	Default	Description
`GENAI_ENABLE_TOXICITY_DETECTION`	`false`	Enable toxicity detection
`GENAI_TOXICITY_THRESHOLD`	`0.7`	Score threshold (0.0-1.0)
`GENAI_TOXICITY_USE_PERSPECTIVE_API`	`false`	Use Google Perspective API (cloud) instead of Detoxify (local)
`GENAI_TOXICITY_PERSPECTIVE_API_KEY`		Perspective API key (required if using Perspective)
`GENAI_TOXICITY_BLOCK_ON_DETECTION`	`false`	Block requests with toxic content

Bias Detection¶

Variable	Default	Description
`GENAI_ENABLE_BIAS_DETECTION`	`false`	Enable bias detection
`GENAI_BIAS_THRESHOLD`	`0.4`	Detection threshold (0.0-1.0)
`GENAI_BIAS_BLOCK_ON_DETECTION`	`false`	Block requests with detected bias

Prompt Injection Detection¶

Variable	Default	Description
`GENAI_ENABLE_PROMPT_INJECTION_DETECTION`	`false`	Enable prompt injection detection
`GENAI_PROMPT_INJECTION_THRESHOLD`	`0.5`	Detection threshold (0.0-1.0)
`GENAI_PROMPT_INJECTION_BLOCK_ON_DETECTION`	`false`	Block detected injection attempts

Restricted Topics¶

Variable	Default	Description
`GENAI_ENABLE_RESTRICTED_TOPICS`	`false`	Enable restricted topics detection
`GENAI_RESTRICTED_TOPICS_THRESHOLD`	`0.5`	Detection threshold (0.0-1.0)
`GENAI_RESTRICTED_TOPICS_BLOCK_ON_DETECTION`	`false`	Block restricted topic requests

Hallucination Detection¶

Variable	Default	Description
`GENAI_ENABLE_HALLUCINATION_DETECTION`	`false`	Enable hallucination detection
`GENAI_HALLUCINATION_THRESHOLD`	`0.6`	Detection threshold (0.0-1.0)

Security Hardening (BFSI / on-prem)¶

For regulated on-prem deployments, GENAI_PROFILE=strict (aliases bfsi, bank) locks down the posture in one switch: audit content capture stays on, but all third-party network egress is disabled and runtime model downloads are blocked.

Variable	Default	Description
`GENAI_PROFILE`		`strict` / `bfsi` / `bank` expand to the hardened posture below (empty = no profile)
`GENAI_ALLOW_EXTERNAL_EGRESS`	`true`	When `false`, components must not send content/telemetry to any third party (e.g. the toxicity Perspective API is disabled)
`GENAI_AIR_GAPPED`	`false`	When `true`, components must not fetch models/assets at runtime (Detoxify weights, spaCy models)

Under GENAI_PROFILE=strict the profile sets GENAI_ALLOW_EXTERNAL_EGRESS=false, GENAI_AIR_GAPPED=true, GENAI_CO2_OFFLINE_MODE=true and GENAI_TOXICITY_USE_PERSPECTIVE_API=false, and enables content capture for audit unless you pin GENAI_ENABLE_CONTENT_CAPTURE yourself.

Blocking evaluation (GENAI_PII_MODE=block, *_BLOCK_ON_DETECTION=true) now intercepts the request before the LLM call and raises genai_otel.exceptions.PolicyViolationError, instead of only annotating the span after the response has already returned.

Performance and Metric Verbosity¶

Full per-request detail is always written to span attributes (audit is unaffected). These flags gate only the additional aggregated-metric instruments, which dominate per-call overhead at high throughput.

Variable	Default	Description
`GENAI_METRICS_PROFILE`	`standard`	`standard` = request count, latency, token counter, total cost. `full` = also token histograms, granular cost counters, finish counters
`GENAI_RECORD_TOKEN_HISTOGRAMS`	`false`	Record per-request token distribution histograms
`GENAI_RECORD_GRANULAR_COST_METRICS`	`false`	Record prompt/completion/reasoning/cache cost counters (span attributes are always set regardless)
`GENAI_RECORD_FINISH_METRICS`	`false`	Record finish-reason / success / failure counters
`GENAI_ENABLE_CONCURRENCY_METRICS`	`true`	Track the `gen_ai.server.requests.running` gauge
`OTEL_METRICS_EXEMPLAR_FILTER`	`always_off`	Metric exemplar sampling; set `trace_based` to re-enable (adds per-measurement cost)

See benchmarks/ for per-call overhead numbers and the measurement tooling.

Telemetry (Opt-in)¶

Anonymous usage telemetry is disabled by default.

Variable	Default	Description
`TRACEVERDE_TELEMETRY`	`false`	Enable anonymous usage reporting
`TRACEVERDE_TELEMETRY_URL`		Telemetry endpoint URL

See genai_otel/telemetry.py for details on what is collected.

Programmatic Configuration¶

All environment variables can be set programmatically:

import genai_otel

genai_otel.instrument(
    # Core
    service_name="my-app",
    endpoint="http://localhost:4318",
    enabled_instrumentors=["openai", "anthropic", "crewai"],
    sampling_rate=0.5,
    fail_on_error=False,

    # Features
    enable_gpu_metrics=True,
    enable_cost_tracking=True,
    enable_mcp_instrumentation=True,
    enable_content_capture=False,

    # CO2 tracking
    enable_co2_tracking=True,
    co2_country_iso_code="USA",
    co2_region="california",

    # GPU
    gpu_collection_interval=10,
    power_cost_per_kwh=0.15,

    # Evaluation
    enable_pii_detection=True,
    pii_mode="redact",
    pii_threshold=0.5,
    pii_gdpr_mode=True,

    enable_toxicity_detection=True,
    toxicity_threshold=0.7,

    enable_bias_detection=True,
    bias_threshold=0.4,

    enable_prompt_injection_detection=True,
    prompt_injection_threshold=0.5,

    enable_restricted_topics=True,
    restricted_topics_threshold=0.5,

    enable_hallucination_detection=True,
    hallucination_threshold=0.6,
)

Multimodal Observability¶

Capture image/audio/document content parts on spans. See the Multimodal Observability guide for full details.

Env var	Default	Notes
`GENAI_OTEL_MEDIA_CAPTURE_MODE`	`off`	`off` \| `reference_only` \| `full`
`GENAI_OTEL_MEDIA_STORE`	`none`	`none` \| `filesystem` \| `s3` \| `minio` \| `http`
`GENAI_OTEL_MEDIA_STORE_ENDPOINT`	—	URL or local path
`GENAI_OTEL_MEDIA_STORE_BUCKET`	`genai-otel-media`
`GENAI_OTEL_MEDIA_STORE_PREFIX`	`traces/{date}/{trace_id}/`	template
`GENAI_OTEL_MEDIA_STORE_ACCESS_KEY`	—	for s3/minio
`GENAI_OTEL_MEDIA_STORE_SECRET_KEY`	—	for s3/minio
`GENAI_OTEL_MEDIA_MAX_BYTES`	`10485760` (10 MiB)	per-blob cap
`GENAI_OTEL_MEDIA_ALLOWED_MODALITIES`	`image,audio,video,document`	comma-separated
`GENAI_OTEL_MEDIA_REDACTOR`	—	dotted path to callable applied before upload

The default off keeps text-only behaviour byte-identical with pre-1.0.0.

Session and User Tracking¶

For programmatic session/user tracking, provide extractor callables:

genai_otel.instrument(
    session_id_extractor=lambda instance, args, kwargs: kwargs.get("metadata", {}).get("session_id"),
    user_id_extractor=lambda instance, args, kwargs: kwargs.get("metadata", {}).get("user_id"),
)

CrewAI, LangGraph, and LangChain have built-in automatic session ID propagation. See the Multi-Agent Frameworks guide for details.