Voice AI

Ultravox Speech to Speech

Using jambonz to connect custom telephony to the Ultravox open-weight Speech Language Model

The jambonz application referenced in this article can be found here.

This is an example Jambonz application that connects to the Ultravox Realtime API and illustrates how to build a Voice-AI application using Jambonz and Ultravox.

When building agents with Ultravox, you can extend your agents’ capabilities by connecting them to external services and systems via tools—functions that agents can invoke to perform specific actions or retrieve information.
These tools can be implemented as either client or server tools; we’re covering both in this sample application. Read more about client vs server tools in the Ultravox docs.

This example application covers four scenarios:

  • weather agent using Ultravox clientTools
  • weather agent using Ultravox serverTools
  • call transfer agent using Ultravox clientTools
  • call transfer agent using Ultravox serverTools.

The weather agent utilizes a weather REST API to enable Ultravox to answer callers’ questions about the weather for specified locations. The call transfer agent uses Jambonz to redirect the inbound call to a destination number of your choice.

Prerequisites

Running instructions

Set Environment Variables

Duplicate the examples.env file as .env and fill in your credentials as shown below.

$ULTRAVOX_API_KEY=SoMe.ExAmpLeKeY
>PORT=3000
>
># Required for Call Transfer
>HUMAN_AGENT_NUMBER=+12125551212
>HUMAN_AGENT_TRUNK=MyCarrier
>HUMAN_AGENT_CALLERID=+14155551000
>
># Required for Server Tools
>HTTP_BASE_URL=https://example.ngrok.io
>JAMBONZ_ACCOUNT_SID=72c5c38f-test-test-test-aaa6be55e1b5
>JAMBONZ_API_KEY=1cf2f4f4-stub-stub-stub-5bb4cb597c2a
>JAMBONZ_BASE_URL=https://api.jambonz.cloud
Environment VariableValue
ULTRAVOX_API_KEYYou can generate a new Ultravox API key under Settings in your Ultravox account
PORTThe port your Express server is listening at, you can use 3000 by default.
HUMAN_AGENT_NUMBERThe destination phone number you’d like your call to be transferred to, in international format. For example, +12125551212.
HUMAN_AGENT_TRUNKThe name of your carrier of choice.
HUMAN_AGENT_CALLERIDThe caller ID that shows up when transferring the call to the destination number. It will vary based on what type of caller IDs your carrier of choice allows, but using your virtual number is typically a good call. Use international format, e.g. +14155551000.
HTTP_BASE_URLThe URL you’re serving this app up at. If running locally, you could use a tunneling service like ngrok. In your terminal, run ngrok http 3000 to get your base URL. E.g. https://your-example-domain.ngrok.io
JAMBONZ_ACCOUNT_SIDFind this in your jambonz.cloud account under the Account tab.
JAMBONZ_API_KEYGenerate a new API key in your jambonz.cloud account under the Account tab.
JAMBONZ_BASE_URLThe base URL where you’re running your Jambonz server. This will be https://api.jambonz.cloud if using jambonz.cloud.

Jambonz Setup

  1. Create a carrier entity in the Jambonz portal.

  2. Add your speech provider of choice in the Jambonz portal.

  3. Create a new Jambonz application in your portal under the Applications tab. Give it a name, then set both Calling webhook and Call status webhook values to the same URL, based on which scenario you’d like to run:

    • weather agent using Ultravox clientTools:
      ws://your-example-domain.ngrok.io/weather-agent-client-tool
    • weather agent using Ultravox serverTools:
      ws://your-example-domain.ngrok.io/weather-agent-server-tool
    • call transfer agent using Ultravox clientTools:
      ws://your-example-domain.ngrok.io/call-transfer-agent
    • call transfer agent using Ultravox serverTools:
      ws://your-example-domain.ngrok.io/call-transfer-agent-server-tool

  4. Having created a carrier, you also need to provision the phone number that you will be receiving calls on from that carrier. At the bottom of the page select the Jambonz application you just created to link your new virtual number to that application.

Run Your App

Ensure all environment variables are properly configured before starting the application. Navigate to the root of this project in your terminal, then run npm install and npm start. Call your virtual number and test it out!

To switch between scenarios, update the Calling webhook and Call status webhook values to another URL, as explained in step 3 of the Jambonz Setup section.

A note on ActionHook

Jambonz integrates with Ultravox via the llm verb.
Like many Jambonz verbs, the llm verb sends an actionHook with a final status when the verb completes. The payload includes a completion_reason property indicating why the llm session ended. Possible values for completion_reason are:

  • Normal conversation end
  • Connection failure
  • Disconnect from remote end
  • Server failure
  • Server error

Resources