Implementation:BerriAI Litellm Nova Sonic Realtime Example

Attribute	Value
Sources	cookbook/nova_sonic_realtime.py
Domains	Realtime API, AWS Bedrock, Nova Sonic, WebSocket, Audio, Cookbook
Last Updated	2026-02-15 16:00 GMT

Overview

nova_sonic_realtime is a cookbook example that demonstrates a speech-to-speech realtime client connecting to the LiteLLM proxy's WebSocket endpoint for Amazon Bedrock Nova Sonic audio conversation.

Description

This script implements a full-duplex audio conversation client using the RealtimeClient class, which connects to the LiteLLM proxy's /v1/realtime WebSocket endpoint. The client handles:

• WebSocket connection with Bearer token authentication to the LiteLLM proxy
• Session configuration via session.update messages, configuring voice, temperature, modalities (text + audio), input/output audio format (PCM16), and server-side VAD (Voice Activity Detection) for turn detection
• Audio capture from the microphone at 16kHz (Nova Sonic input requirement) using PyAudio, sending base64-encoded audio chunks via input_audio_buffer.append messages
• Audio playback at 24kHz (Nova Sonic output rate) from a bounded asyncio queue, processing response.audio.delta events
• Message handling for various event types: session.created, response.created, response.text.delta, response.audio.delta, response.done, and error
• Concurrent task management using asyncio.gather to run receive, capture, and playback tasks in parallel

The audio queue size is configurable via the LITELLM_ASYNCIO_QUEUE_MAXSIZE environment variable (default: 10,000).

Usage

Run this script as a standalone client when testing or demonstrating the LiteLLM proxy's realtime audio capabilities with Amazon Bedrock Nova Sonic. Requires a running LiteLLM proxy with Bedrock configured, AWS credentials, and the pyaudio and websockets packages installed.

Code Reference

Source Location

cookbook/nova_sonic_realtime.py

Signature

class RealtimeClient:
    def __init__(self, url: str, api_key: str)
    async def connect(self)
    async def send_session_update(self)
    async def receive_messages(self)
    async def send_audio_chunk(self, audio_bytes: bytes)
    async def commit_audio_buffer(self)
    async def capture_audio(self)
    async def play_audio(self)
    async def close(self)

async def main()

Import

# This is a standalone cookbook script, run directly:
# python cookbook/nova_sonic_realtime.py

I/O Contract

Inputs

Parameter	Type	Description
url	str	WebSocket URL for the LiteLLM proxy realtime endpoint (default: ws://localhost:4000/v1/realtime?model=bedrock-sonic).
api_key	str	LiteLLM API key for authentication.
Microphone audio	PCM16 at 16kHz	Live microphone input captured via PyAudio.

Outputs

Output	Type	Description
Text transcription	str	Printed to stdout from response.text.delta events.
Audio playback	PCM16 at 24kHz	Played through speakers from response.audio.delta events.

Configuration Constants

Constant	Value	Description
INPUT_SAMPLE_RATE	16000	Microphone capture rate (16kHz, required by Nova Sonic).
OUTPUT_SAMPLE_RATE	24000	Audio playback rate (24kHz, Nova Sonic output).
CHANNELS	1	Mono audio.
CHUNK_SIZE	1024	Audio buffer size per frame.
AUDIO_QUEUE_MAXSIZE	10000	Maximum queued audio chunks (configurable via env var).

Usage Examples

# Run from the command line:
# python cookbook/nova_sonic_realtime.py

# Or use the RealtimeClient programmatically:
import asyncio

async def demo():
    client = RealtimeClient(
        url="ws://localhost:4000/v1/realtime?model=bedrock-sonic",
        api_key="sk-12345",
    )
    await client.connect()
    await client.send_session_update()

    # Run receive, capture, and playback concurrently
    await asyncio.gather(
        client.receive_messages(),
        client.capture_audio(),
        client.play_audio(),
        return_exceptions=True,
    )

asyncio.run(demo())

Related Pages

• Implementation:BerriAI_Litellm_Veo_Video_Generation_Example - Another cookbook example using LiteLLM proxy pass-through
• Implementation:BerriAI_Litellm_Passthrough_Utils - Utilities for pass-through endpoint handling