otelcol-genai-sketches

genaisketch is an OpenTelemetry Collector connector and ready-to-run distribution. It turns GenAI spans into token metrics, keyed rankings of users and sessions, and summary files, while keeping user, session and prompt identities out of metric labels. Your existing trace backend keeps receiving the original spans.

Use it to track LLM token usage by user, session and model, and to see where usage data is missing, in Prometheus or any OTLP metrics backend.

Run it on your traffic

Choose the path matching your source; keep your existing trace backend.

• Standard GenAI spans: model-call spans using gen_ai.operation.name (or gen_ai.request.model when absent) and gen_ai.usage.input_tokens / gen_ai.usage.output_tokens; add genaisketch beside your existing trace exporter.
• LiteLLM: an existing proxy exporting model spans through its OpenTelemetry callback.
• agentgateway: model spans with token usage, a verified user identity and your application's session header.
• Agent Router: GenAI spans and trusted identity headers; captured on Envoy AI Gateway 1.1.0.
• Copilot CLI: OpenTelemetry model spans with token usage and gen_ai.conversation.id for sessions.
• Copilot Chat: OpenTelemetry model spans with token usage and copilot_chat.session_id, or resource session.id.
• OpenInference / OpenLLMetry: vendor-shaped model spans, normalized on the sketch branch by a separate contrib collector before reaching v0.3.2.

The guides start with your own data: check a trace capture, review your collector configuration, or compare LiteLLM spend exports directly.

Try It

Synthetic demo traffic.

You need Docker with Compose v2, Git, and a POSIX shell (macOS, Linux, or WSL). The demo is synthetic and needs no model account or API key.

Clone the repository and start the stack:

git clone https://github.com/llm-measurement/otelcol-genai-sketches.git
cd otelcol-genai-sketches
sh examples/demo.sh up

You'll get a sample app, this collector, a Prometheus metrics server, and Grafana. Open the dashboard printed by the command to see request rate, reported token rate, distinct activity, and missing usage. The first build takes several minutes; allow one minute after startup for the first measurement window.

Watch the walkthrough, or open the demo guide for top-k rankings, controlled investigations, ports, secret handling, dashboard reading order, and cleanup. The token-consumption playbook gives queries for investigating your own traffic, including unexpected "token maxing."

Find Your Guide

How It Works

applications -> Collector fan-out -> current trace backend
                                  -> genaisketch -> Prometheus metrics
                                                -> keyed top-k snapshot logs
                                                -> optional summary files -> fleetdiff

Keep Datadog, Langfuse, Alloy, or another OTLP destination as your trace backend. The connector counts model attempts separately from agent, tool, retrieval, and MCP spans. The connector counts missing token usage separately, while the pinned standard token metrics record 0 with gen_ai.token.modality set to unknown when a response or its usage is absent. It uses llm-sketchkit to hash configured identities, estimate distinct activity, and rank heavy contributors. Compatible summary files let fleetdiff compare windows across collectors without uploading raw traces. Fan-out preserves the original traces, so disable content capture or redact before fan-out when raw content must stay local.

Read Shadow Mode, Accounting, Metrics, Configuration, and the deployment FAQ for fan-out, accounting rules, signals, and input fields.

Install

For a persistent deployment, use the signed release image or Helm chart. Follow Deployment to verify signatures and attestations, pin immutable digests, configure the hashing secret, and start the collector.

The image supports Linux amd64 and arm64. For Kubernetes, the deployment guide includes the verified chart install and an existing-Secret configuration.

For a custom collector build, add the standalone module using the component README. Read Sizing for capacity planning and Upgrading for feature availability, restarts, and rollback.

Privacy And Limits

• Keyed hashes are pseudonymous. Values are linkable under the same secret; secret holders can test candidate values. Restrict access to snapshots and summary files. Key rotation starts a new comparison period.
• Slice labels are bounded and cleartext. Choose non-sensitive dimensions such as model, team, or route. Excess slice values share an overflow bucket; user/session hashes and top-k items stay out of metric labels.
• Estimates carry uncertainty. Distinct counts are estimates; frequent-item rankings include lower and upper bounds. Use compatible summary state to combine measurements, rather than adding distinct-count gauges.
• Use exact records for billing and quota enforcement. Reported usage and coverage support investigation; optional Bloom deduplication can undercount. Keep provider records and an exact ledger for financial or admission decisions.

See Security for private vulnerability reporting and Accounting for measurement semantics.

Project

Evaluate with representative traffic; see the changelog and release and support policy for pre-1.0 changes, signed artifacts, SBOMs, provenance, and upgrades.

The benchmark report records reproduction commands, workloads, machine details, and all recorded accuracy, throughput, and soak runs.

Contributing covers development and signed, signed-off commits. For questions or feedback, open an issue.

Licensed under the Apache License 2.0.