Skip to navigation
Agent types

Realtime voice API

One vendor's model listens, thinks and speaks. jambonz carries the call, adds the transfer and hangup tools, and records every turn.

A Realtime voice API agent hands the whole voice loop to one vendor: speech recognition, turn-taking, barge-in, the model and the voice all run inside the vendor’s session. You choose the vendor, the model and the voice. It gives the lowest latency and the most natural turn-taking, with one vendor to configure. jambonz runs it with the llm verb.

You write the prompt and add tools and knowledge in Studio, as for a pipeline agent. The vendor, model and voice are in the agent’s Model & voice section.

Vendors and credentials

Each vendor needs its own credential in your jambonz account, added in the jambonz portal under LLM Services, on the Speech-to-speech tab. It is a credential of its own, separate from any language model or speech credential for the same vendor.

VendorStarting modelStarting voiceNotes
OpenAI Realtimegpt-realtimemarinAlso suggests gpt-realtime-mini.
OpenAI GPT-Livegpt-live-1marinTool calls, transfers and hangups run on a separate Model for delegated work (starts on gpt-6.1-terra).
Rime PrismnoneastraNo model choice. Four voices: astra, aurora, brando and wawona. English only.
Deepgram Voice Agentgpt-4.1-miniflux-kit-enDeepgram listens (Flux for English, Nova-3 otherwise), runs the LLM and speaks with a Flux or Aura voice, and bills all three to your Deepgram account.
Ultravoxultravox-v0.7OliviaUltravox has many voices; Studio suggests a few English ones.

You choose the vendor in the New agent dialog. If the account has no credential for it yet, you can still build the agent and add the credential later.

Model & voice

FieldWhat it is
VendorOne of the five vendors above. Switching vendor clears the credential and starts from that vendor’s model and voice.
CredentialThe vendor’s credential from your jambonz account. When you have more than one, the label picks which.
ModelThe vendor’s model. For Deepgram Voice Agent this is LLM (managed by Deepgram). Not shown for Rime Prism.
VoiceThe vendor’s voice.
LanguageThe language of the conversation (en-US to start).
Model for delegated workOpenAI GPT-Live only: the model its tool calls, transfers and hangups run on.

For every field you can pick a suggestion or type any model, voice or language code the vendor accepts.

Options

The Options fold takes extra vendor settings as JSON:

  • OpenAI Realtime and OpenAI GPT-Live: any key from the OpenAI section of the llm verb docs, for example turn_detection or temperature. Studio asks OpenAI Realtime to transcribe the caller, so the call history has the caller’s words; set input_audio_transcription here to change that.
  • Rime Prism: only turn_detection (interrupt_response, eagerness, prism_end_of_turn_timeout_ms) and truncation. Anything else is left out.
  • Deepgram Voice Agent: listen, think and speak are merged into Deepgram’s providers; mip_opt_out, tags and flags go in the settings.
  • Ultravox: any setting of an Ultravox call, such as temperature, vadSettings or inactivityMessages.

Do not put the prompt, tools or voice here; Studio adds them. For Rime Prism, Deepgram Voice Agent and Ultravox, Studio notes any option it leaves out.

Handled by the vendor

Speech recognition, turn detection, barge-in, the voice and noise handling happen inside the vendor’s session. The pipeline’s turn-taking and interruption settings do not appear in the Conversation section for these agents; tune them through the vendor’s options.

Greeting

An OpenAI Realtime agent always speaks first. If its greeting is set to wait for the caller, Studio notes that it will not wait. Rime Prism and OpenAI GPT-Live can wait for the caller.

Studio’s own lines

On a warm transfer, Studio speaks a brief to the person who answers (“Brief the person first”, in Transfer & hangup). The vendor’s voice is inside its session, so Studio speaks the brief itself, with a text-to-speech credential from your account. The secure detour’s message is different: a Realtime voice API agent says that line itself, in its own voice.

When the agent has a warm transfer with a brief, Model & voice shows Voice for Studio’s own lines:

  • Speak them in the agent’s voice: shown when the vendor also sells the agent’s voice as text-to-speech. Turn it on and Studio speaks the brief in the same voice as the agent:

    VendorWhen it is offeredText-to-speech credential needed
    Deepgram Voice Agenta Flux voice (flux-…) or an Aura voice (aura-…)Deepgram Flux for a Flux voice, Deepgram for an Aura voice
    Rime Prismthe astra or wawona voiceRimeLabs

    Studio offers only credentials of that vendor, and uses the agent’s voice. If the chosen credential is from another vendor, Studio notes that it needs one from the agent’s vendor.

  • Otherwise, choose a text-to-speech credential and a voice for Studio’s lines. OpenAI Realtime, OpenAI GPT-Live and Ultravox voices are not sold as text-to-speech, so for those the brief is spoken in a voice you choose here.

If no voice is chosen, Studio notes that it has no voice to speak the brief with. The transfer’s Brief for the person field in Transfer & hangup shows which voice will speak it, with a link back here.

Next