Implementation:Arize ai Phoenix Contextvars Async Demo

Overview

The contextvars_async_gen_demo.py file is an internal vignette that empirically investigates the behavior of Python's contextvars module when used within async generators, particularly across suspend/resume boundaries and exception propagation. The findings directly impact how OpenTelemetry span context is managed in Phoenix's evaluation and streaming pipelines.

The core discovery is that a ContextVar token created inside an async generator before a yield becomes invalid in that generator's finally block when the consumer raises an exception. The reset(token) call fails with ValueError("Token was created in a different Context"). This justifies the design decision to avoid start_as_current_span inside async generators such as chat_completion_create.

Code Reference

Attribute	Value
Source File	internal_docs/vignettes/otel-contextvars-async/contextvars_async_gen_demo.py
Lines	335
Domain	Internal_Documentation, OpenTelemetry
Language	Python (asyncio)
Test Framework	None (plain asyncio + assert to avoid test-framework confounders)

Scenarios Tested

The script tests six distinct scenarios and asserts expected outcomes:

#	Scenario	Token Reset Result
1	Generator sets token, consumer raises (no aclosing)	FAILS -- ValueError("different Context")
2	Generator sets token, consumer raises (with aclosing)	SUCCEEDS -- aclosing ensures cleanup in the correct context
3	Consumer sets token, generator raises	SUCCEEDS -- consumer's finally runs in its own context
4	Consumer sets token, generator raises (with asyncio.sleep)	SUCCEEDS -- additional context switches do not affect consumer
5	Normal exit (no exception)	SUCCEEDS -- no context mismatch occurs
6	Chained generators: outer uses aclosing(inner), consumer raises without aclosing(outer)	FAILS -- inner generator's finally runs in finalizer task (wrong context)

Key Functions

Demonstration Functions

• f() -- Basic async generator demonstrating token reset failure when consumer raises
• f_normal() -- Generator that yields twice and exits normally (no context issues)
• gen_that_yields_then_raises() -- Simulates chat_completion_create: yields then raises an API error
• consumer_with_context() -- Simulates the evaluator: sets context then iterates over a failing generator
• consumer_with_sleep() -- Same as above but with asyncio.sleep(0) to force extra context switches

Empirical Assertion Functions

• _gen_sets_token_consumer_raises() -- Records token reset failure (Scenario 1)
• _gen_sets_token_consumer_raises_record_both() -- Records success/failure for aclosing variant (Scenario 2)
• _consumer_sets_token_gen_raises() -- Verifies consumer token survives generator exception (Scenario 3)
• _outer_a_chain() / _inner_b_chain() -- Chained generator pattern (Scenario 6)
• run_empirical_tests() -- Orchestrates all six scenarios with assertions

Design Implications

The demonstrated behavior has the following implications for Phoenix:

• Do not use start_as_current_span (or any ContextVar.set/reset pattern) inside async generators that are consumed across suspend/resume boundaries
• Use aclosing when consuming async generators to ensure cleanup occurs in the correct context
• Consumer-side context management is safe: when the consumer (e.g., evaluator) sets context and iterates over a generator that raises, the consumer's token reset in finally still works correctly

Usage

uv run python internal_docs/vignettes/otel-contextvars-async/contextvars_async_gen_demo.py

Related Pages

• Arize_ai_Phoenix_Pyproject_Config - Main package depending on OpenTelemetry SDK
• Arize_ai_Phoenix_Polymorphic_User_Vignette - Another internal vignette (SQLAlchemy patterns)