> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.jambonz.org/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.jambonz.org/_mcp/server.

# OpenAI Real-Time API

> **Note**
>
> The jambonz application referenced in this article can be
> found [here](https://github.com/jambonz/openai-s2s-example).

> **Migrating from preview**
>
> OpenAI deprecated the Realtime preview API on 2026-05-12. The example below uses the GA-format payloads (`response.create.audio.output`, `session.update.audio.input`, `output_modalities`, `session.type: "realtime"`).
>
> Apps written for the preview API still work without changes — jambonz auto-converts the legacy flat format (`voice`, `input_audio_format`, `input_audio_transcription`, `turn_detection` directly under `session_update`) to the GA format on the wire, logs a one-time deprecation warning, and aliases GA event names back to the preview names so your existing `events` filters keep matching. Plan to update to the GA format before that compatibility layer is removed.
>
> See the [`llm` verb article](/verbs/verbs/llm) for the canonical GA-shape example and field reference.

This is an example jambonz application that connect to the OpenAI Realtime API and illustrates how to
build a Voice-AI application using jambonz and OpenAI.

## Authentication

You must have an OpenAI API key that has access to the Realtime API.
Specify it as an environment variable when starting the application.

```bash
OPENAI_API_KEY=sk-proj-XXXXXXX node app.js
```

## Configuring the assistant

All of the configuration (in fact, all of the relevant code) can be found
[in this source file](https://github.com/jambonz/openai-s2s-example/blob/main/lib/routes/openai-s2s.js). This is the file you will want to edit as you play
with this example.

You can see that application first answers the call, pauses one second, and the connects to the OpenAI Realtime API
using the jambonz `llm` verb.  We specify the vendor and model, and provide options specific to that
LLM (in this case `gpt-realtime`) in the `llmOptions` property.

In the case of the OpenAI Realtime API,
configuration is provided in the form of the
[response\_create](https://platform.openai.com/docs/api-reference/realtime-client-events/response/create)
and [session\_update](https://platform.openai.com/docs/api-reference/realtime-client-events/session/update) client
events that are sent to OpenAI.  These specify the instructions to the assistant as well as things like
vad and function calling options.

## Function calling

The example illustrates how to implement client-side functions and provide them to the assistant.
In this example, we implement a simple "get weather" function using the freely-available APIs from
[open-meteo.com](https://open-meteo.com/). The function is described in the session\_update client message,
and a `toolHook` property for the llm verb defines the hook that will be called in the application when
the LLM wants the application to call a function.  Finally, the `session.sendToolOutput()` method is called
to send the results of the function call back to the LLM.

## Interrupting the assistant

When the user begins speaking over the assistant (i.e. "barge in") jambonz sends a
[response.cancel](https://platform.openai.com/docs/api-reference/realtime-client-events/response/cancel) client
event to interrupt the assistant.  Any queued audio that has been received from the assistant is flushed.

## Events

There are [28 server events](https://platform.openai.com/docs/api-reference/realtime-server-events)
that OpenAI sends, and your application can specify which it wants to receive.
(The only exception is the [response.output\_audio.delta](https://platform.openai.com/docs/api-reference/realtime-server-events/response/audio/delta)
server event — formerly `response.audio.delta` in the preview API — which contains actual audio content that jambonz itself processes. mod\_openai\_s2s handles both names so legacy and GA models both work.)
You specify which events you want to receive in the `events` property of the `llm` verb,
and as you can see in the example you can use wildcards to include a whole class of server events
(e.g. "conversation.item.\*").

## actionHook properties

Like many jambonz verbs, the `llm` verb sends an actionHook with a final status when the verb completes.
The payload will include a `completion_reason` property indicating why the llm session completed.\
This property will be one of:

* normal conversation end
* connection failure
* disconnect from remote end
* server failure
* server error

In the case of an error an `error_code` object is returned.
We use this, for example, in this sample application to detect if the user's
OpenAI's rate limits have been exceeded so as to notify them why the session is ending.