Ready to supercharge your AI agents?

Install any skill in seconds. Free to start, no credit card required.

Phoenix Tracing · github | DecimalAI

Phoenix Tracing

SkillScore91Maintained

phoenix-tracing

OpenInference semantic conventions and instrumentation for Phoenix AI observability. Use when implementing LLM tracing, creating custom spans, or deploying to production.

AI/MLby github

Links this skill to your agents and keeps it current with the author’s updates.

Automatic

AI Engineer pack

1 · One-time — install the CLI (docs ↗)

pip install decimalai

2 · Get the skill — writes .claude/skills/phoenix-tracing/SKILL.md

decimalai skills pull phoenix-tracing --out .claude/skills/

No CLI? Get the same file — curl · raw file

curl -fsSL --create-dirs -o .claude/skills/phoenix-tracing/SKILL.md https://app.decimal.ai/s/phoenix-tracing/SKILL.md

Raw file: /s/phoenix-tracing/SKILL.md ↗ · machine-readable summary at /s/phoenix-tracing.json ↗

In productionGlobal

how this skill behaves on real traffic · last 30 days

No production data for this skill yet — usage numbers appear here once agents start running it.

Anonymized aggregate across every org using this skill — proof from real production use, not a demo.

Does this skill help? · Benchmark: with vs. withoutsign in to run this on your agents

22 test cases from the publisher’s eval.yaml · each run twice on gemini-3.6-flash — with the skill injected, then without · last run 44d ago

Yes — +5% lift · at higher cost+65% tokens0% turns91% of whole cases pass with skill (20/22)

Test case	Without → With	Effect	Δ tokens	Δ turns
case-02	✗→✓	▲ Improved	147%	0%
case-03	✗→✓	▲ Improved	15%	0%
case-21	✓→✗	▼ Worse	36%	0%
case-08	✓→✓	＝ Same ✓	264%	0%
case-13	✓→✓	＝ Same ✓	229%	0%

Showing 5 of 22 cases

SkillSafety

✓ 3 of 3 passed

Code scan15 of 15 checks clean

0 critical · 0 warning · 0 info — scanner v4 — 8/7/2026

live_secretremote_code_executionreverse_shelldata_exfiltrationssrf_metadataagent_snoopinganti_refusaldestructive_commandsobfuscated_unicodeunicode_homoglyphopaque_blobsuspicious_urltool_overreachtrigger_abuseprompt_injection_phrasing

IntentSafe

LLM judge · does the skill do what it claims

ContentClean

LLM judge · policy & harmful-content review

Source

Imported

github / awesome-copilot

skills/phoenix-tracing/SKILL.md

View on GitHub ↗

License Apache-2.0Imported 8/7/2026Version v2Bundled 30 files

Skill contents

references/

SKILL.md

Phoenix Tracing

Comprehensive guide for instrumenting LLM applications with OpenInference tracing in Phoenix. Contains reference files covering setup, instrumentation, span types, and production deployment.

When to Apply

Reference these guidelines when:

Setting up Phoenix tracing (Python or TypeScript)
Creating custom spans for LLM operations
Adding attributes following OpenInference conventions
Deploying tracing to production
Querying and analyzing trace data

Reference Categories

| Priority | Category | Description | Prefix | | -------- | --------------- | ------------------------------ | -------------------------- | | 1 | Setup | Installation and configuration | setup-* | | 2 | Instrumentation | Auto and manual tracing | instrumentation-* | | 3 | Span Types | 9 span kinds with attributes | span-* | | 4 | Organization | Projects and sessions | projects-*, sessions-* | | 5 | Enrichment | Custom metadata | metadata-* | | 6 | Production | Batch processing, masking | production-* | | 7 | Feedback | Annotations and evaluation | annotations-* |

Quick Reference

1. Setup (START HERE)

setup-python - Install arize-phoenix-otel, configure endpoint
setup-typescript - Install @arizeai/phoenix-otel, configure endpoint

2. Instrumentation

instrumentation-auto-python - Auto-instrument OpenAI, LangChain, etc.
instrumentation-auto-typescript - Auto-instrument supported frameworks
instrumentation-manual-python - Custom spans with decorators
instrumentation-manual-typescript - Custom spans with wrappers

3. Span Types (with full attribute schemas)

span-llm - LLM API calls (model, tokens, messages, cost)
span-chain - Multi-step workflows and pipelines
span-retriever - Document retrieval (documents, scores)
span-tool - Function/API calls (name, parameters)
span-agent - Multi-step reasoning agents
span-embedding - Vector generation
span-reranker - Document re-ranking
span-guardrail - Safety checks
span-evaluator - LLM evaluation

4. Organization

projects-python / projects-typescript - Group traces by application
sessions-python / sessions-typescript - Track conversations

5. Enrichment

metadata-python / metadata-typescript - Custom attributes

6. Production (CRITICAL)

production-python / production-typescript - Batch processing, PII masking

7. Feedback

annotations-overview - Feedback concepts
annotations-python / annotations-typescript - Add feedback to spans

Reference Files

fundamentals-overview - Traces, spans, attributes basics
fundamentals-required-attributes - Required fields per span type
fundamentals-universal-attributes - Common attributes (user.id, session.id)
fundamentals-flattening - JSON flattening rules

Common Workflows

Quick Start: setup-{lang} → instrumentation-auto-{lang} → Check Phoenix
Custom Spans: setup-{lang} → instrumentation-manual-{lang} → span-{type}
Session Tracking: sessions-{lang} for conversation grouping patterns
Production: production-{lang} for batching, masking, and deployment

How to Use This Skill

Navigation Patterns:

bash
# By category prefix
references/setup-*              # Installation and configuration
references/instrumentation-*    # Auto and manual tracing
references/span-*               # Span type specifications
references/sessions-*           # Session tracking
references/production-*         # Production deployment
references/fundamentals-*       # Core concepts
references/attributes-*         # Attribute specifications

# By language
references/*-python.md          # Python implementations
references/*-typescript.md      # TypeScript implementations

Reading Order:

Start with setup-{lang} for your language
Choose instrumentation-auto-{lang} OR instrumentation-manual-{lang}
Reference span-{type} files as needed for specific operations
See fundamentals- files for attribute specifications

References

Phoenix Documentation:

Python API Documentation:

Python OTEL Package - arize-phoenix-otel API reference
Python Client Package - arize-phoenix-client API reference

TypeScript API Documentation:

TypeScript Packages - @arizeai/phoenix-otel, @arizeai/phoenix-client, and other TypeScript packages

Illustrative Want to see what phoenix-tracing would put in front of your model? Preview the rendered body and check its trigger phrases — no LLM call, no signup.

Latest benchmark · publisher runVerified

8/8/2026, 15:31 UTC · 22 cases

Lift

+5%

91% vs 86%

Agent turns

1.0 vs 1.0

Tokens

+65%

2,977.09 vs 1,807.14

fail → pass · 2pass → pass · 18fail → fail · 1pass → fail · 1

Per-case results (22)

Case	Status	Duration (ms)			Turns			Tokens			Tool calls
Case	Status	Without	With	Δ	Without	With	Δ	Without	With	Δ	Without	With	Δ
case-08 We want to track unique end-users across multiple LLM interaction traces. What standardized OpenInference attribute key should we set on spans for the user's ID?	pass→pass	2,828	3,276	+16%	1	1	0%	579	2,108	+264%	0	0	—
case-01 Our team is deploying a Phoenix-traced LLM agent to production in Python. We need to handle high trace volume efficiently and ensure sensitive user data like PII is scrubbed before exporting. Please outline a production setup guide covering batch export configuration, data masking strategies, and how to attach feedback annotations to spans.	fail→fail	20,772	15,651	-25%	1	1	0%	4,649	4,843	+4%	0	0	—
case-02 We want to instrument our Python LLM application to send OpenInference telemetry to a local Phoenix instance running at http://localhost:6006. What standard package should we install and initialize in our code?	fail→pass	4,777	5,184	+9%	1	1	0%	1,016	2,507	+147%	0	0	—
case-03 We are building a Node.js API with TypeScript and want to record OpenInference traces to a Phoenix collector endpoint. Which package should we install to initialize tracing?	fail→pass	8,976	3,248	-64%	1	1	0%	1,814	2,082	+15%	0	0	—
case-13 We pass input strings to an embedding API to produce dense vector representations. What OpenInference span kind should be assigned to this vector generation span?	pass→pass	3,956	5,000	+26%	1	1	0%	750	2,466	+229%	0	0	—
case-04 In an OpenInference-instrumented application, we have an orchestration function that sequences multiple LLM prompts and vector retriever calls. Which OpenInference span kind should represent this multi-step pipeline wrapper?	pass→pass	5,150	6,436	+25%	1	1	0%	971	2,607	+168%	0	0	—
case-05 When recording telemetry for our vector database query step in OpenInference, which span kind should we assign to record document retrieval and similarity scores?	pass→pass	8,474	6,455	-24%	1	1	0%	1,706	2,735	+60%	0	0	—
case-06 In an agentic pipeline instrumented with OpenInference, an LLM generates a JSON payload requesting an external calculator function execution. Which span kind represents the execution of this function?	pass→pass	3,099	2,890	-7%	1	1	0%	623	2,007	+222%	0	0	—
case-07 We pass user input through a toxicity checker before feeding it to our language model in an OpenInference trace. Which span kind accurately categorizes this safety moderation gate?	pass→pass	5,351	3,156	-41%	1	1	0%	1,092	2,106	+93%	0	0	—
case-09 Our chat application allows multi-turn conversations. What standardized OpenInference attribute key should we add to spans to group traces belonging to the same chat thread?	pass→pass	4,300	3,698	-14%	1	1	0%	878	2,281	+160%	0	0	—
case-10 We want to wrap a custom Python function to generate an OpenInference span without manually managing start and end trace timestamps. How should we instrument this function concisely?	pass→pass	6,125	6,680	+9%	1	1	0%	1,246	2,707	+117%	0	0	—
case-11 We are instrumenting an autonomous agent in OpenInference that runs ReAct loops with multiple inner thought-action cycles. What span kind should represent the top-level agent execution?	pass→pass	6,804	8,511	+25%	1	1	0%	1,312	3,221	+146%	0	0	—
case-12 After retrieving candidate documents from search, we run a cross-encoder model to re-order the results before feeding them into the prompt context. What OpenInference span kind should be assigned to this step?	pass→pass	5,283	4,283	-19%	1	1	0%	1,132	2,475	+119%	0	0	—
case-14 We run an offline batch LLM-as-a-judge process in Phoenix to score answer faithfulness across our traces. What OpenInference span kind should be assigned to spans recording these evaluation runs?	pass→pass	7,264	4,400	-39%	1	1	0%	1,595	2,260	+42%	0	0	—
case-15 When attaching complex nested metadata objects as span attributes in OpenInference format, how should nested dictionary keys be formatted?	pass→pass	7,571	7,972	+5%	1	1	0%	1,571	3,094	+97%	0	0	—
case-16 We run three separate microservices that send traces to the same Phoenix server. How should we logically separate these applications in telemetry so traces don't mix?	pass→pass	12,301	9,032	-27%	1	1	0%	2,283	3,420	+50%	0	0	—
case-17 Users hit 'thumbs down' on an LLM response in our UI. How do we programmatically record this human feedback score onto the specific trace span using the Phoenix Python SDK?	pass→pass	9,438	8,432	-11%	1	1	0%	2,062	3,431	+66%	0	0	—
case-18 To comply with strict privacy regulations, we need to redact credit card numbers before Phoenix trace telemetry is emitted. How should this redact step be implemented in tracing instrumentation?	pass→pass	15,956	11,148	-30%	1	1	0%	3,200	3,866	+21%	0	0	—
case-19 We are using OpenAI SDK and LangChain in Python. Rather than decorating every function manually, how can we capture all API calls and chain executions automatically with Phoenix?	pass→pass	9,768	7,295	-25%	1	1	0%	2,142	3,058	+43%	0	0	—
case-20 We need to evaluate output text quality on a dataset of summarization outputs using standard BLEU and ROUGE n-gram overlap metrics in Python. How should we write code to calculate these metrics?	pass→pass	14,940	11,546	-23%	1	1	0%	2,844	4,204	+48%	0	0	—
case-21 We are setting up Qdrant HNSW vector index tuning for RAG search optimization. How should we set parameters like m and ef_construct in Qdrant collection configuration?	pass→fail	15,436	13,907	-10%	1	1	0%	2,963	4,035	+36%	0	0	—
case-22 How do we write a Kubernetes Horizontal Pod Autoscaler manifest for Kubernetes HPA GPU autoscaling for vLLM inference server deployments?	pass→pass	15,517	12,556	-19%	1	1	0%	3,329	3,983	+20%	0	0	—

These cases were also screened by the skill’s intent / safety pass.See the Trust review ↗Reproduce this benchmark in Colab ↗

How this skill was measured

DecimalAI ran this skill against gemini-3.6-flash twice over the same eval suite — once with the skill loaded and once without — and compared the two runs case by case. 22 cases were attempted. The headline lift of +5 percentage points is the difference between those two pass rates over the 22 comparable cases. 1 case got worse with the skill loaded, and it is included in that figure.

Model measured: gemini-3.6-flash
Graded by: gemini-3.6-flash
Grading method: judged
Verification: verified
Pass rate without the skill: 86.4%
Pass rate with the skill: 90.9%
Cases attempted: 22
Cases counted toward lift: 22
Cases that regressed: 1
Skill version measured: v2
Run completed: 8/8/2026, 15:31 UTC

Measured example — same prompt, with vs. without the skill · from the benchmark run · case case-02

Prompt

We want to instrument our Python LLM application to send OpenInference telemetry to a local Phoenix instance running at http://localhost:6006. What standard package should we install and initialize in our code?

Without the skill loaded, the model failed this case. With it loaded, the same prompt on the same model passed. This is one improved case from the latest verified run; every case, including any that regressed, is in the table above.

Show both answers

SkillScore blends 4 signals1 of 4 complete

BenchmarkPass rate WITH skill on a verified run91

Live evalNeeds ≥10 evaluated traces in 30 days

AI ratingNeeds ≥10 rated traces

AdoptionNeeds ≥10 serves across ≥2 orgs in 30 days

Efficiency · separate axis (at equal correctness)+0% turns+78% tokens

Shown for transparency; not credited unless a verified run shows no correctness regression.

Backed by 1 of 4 signals — each added signal makes the score more robust.

Verified on gemini-3.6-flash · re-tested 44 days ago

Model	Method	Date	Lift
gemini-3.6-flash	verified	7/24/2026	+18%

Other measured skills in the registry, with their headline benchmark lift.

Ready to supercharge your AI agents?

Phoenix Tracing

SkillSafety

Source

Phoenix Tracing

When to Apply

Reference Categories

Quick Reference

1. Setup (START HERE)

2. Instrumentation

3. Span Types (with full attribute schemas)

4. Organization

5. Enrichment

6. Production (CRITICAL)

7. Feedback

Reference Files

Common Workflows

How to Use This Skill

References

How this skill was measured

Measured example — same prompt, with vs. without the skill · from the benchmark run · case case-02

Related skills