WebSocket calls

Overview

The WebSocket API lets your own code hold a real-time voice conversation with an agent. Your server opens a session, your client connects to the session's WebSocket, streams the user's audio in and plays the agent's audio back as it arrives. The agent listens, answers in its own voice and handles interruptions, and the conversation is kept like any other call on your account: it shows in your call history, follows the agent's recording and transcription settings, and is billed the same way.Use it when your code already has the audio: a server, a device, a contact-center platform or a phone system you run. In a browser or mobile app, use WebRTC calls, which handle the microphone, echo cancellation and the network for you.Messages are JSON text frames with Aethex's own event types, and the audio is 16 kHz PCM by default. You get what the user and the agent said as text events alongside the audio.

Start a call

Your server opens the session; your client carries the audio.

1. 01

   Open a session

   From your server, call POST /conversation/connect with your API key, the agent and transport: "websocket". You get back a session_id and a ws_url. Add dynamic_variables to personalize the call, and caller_phone with the user's number in E.164 format when the call has one.

   Open a WebSocket session

   bash

   curl -X POST https://api.aethexai.com/api/v1/conversation/connect \
     -H "X-API-Key: $AETHEX_API_KEY" \
     -H "Content-Type: application/json" \
     -d '{
       "agent_id": "8a1b2c3d-4e5f-6789-abcd-ef0123456789",
       "transport": "websocket",
       "audio_format": "pcm_16000"
     }'
   
   # 201 Created
   {
     "session_id": "3f2a7b1c-8d4e-4a5b-9c2d-1e0f3a4b5c6d",
     "ws_url": "wss://api.aethexai.com/ws/v1/conversation/3f2a7b1c-8d4e-4a5b-9c2d-1e0f3a4b5c6d"
   }

2. 02

   Connect

   Open a WebSocket to ws_url. It works like a signed URL: whoever holds it can join that one session, once, so hand it only to the client that connects. A second socket on the same session is refused. The first message you receive is session_started.

3. 03

   Talk

   Send the user's audio as audio events and play the agent's audio events as they arrive. When you receive interruption, drop any agent audio you have not played yet: the user has started talking over the agent. user_transcript and agent_response tell you what each side said.

4. 04

   Hang up

   To end the call from your client, close the socket. When the agent ends the call, you receive call_ended and the socket closes. To end a call from your server, call POST /calls/{call_id}/end or POST /conversation/{session_id}/end, and your client receives call_ended.

Who speaks first

By default the agent speaks first: it greets the user with its first message as soon as the socket connects, as on a WebRTC call. If your side placed the call, for example an outbound phone call your system dialed, connect the socket once the call is answered and set direction to outbound. The agent then lets the person who answered speak first: it greets them right after their hello, or after 3 seconds of silence if they wait for you to talk, as on an outbound phone call.caller_phone is saved as the call's from_number on an inbound call and its to_number on an outbound one, the same way a phone call records them.

Messages

Every message is a JSON text frame with a type. Ignore any type you don't recognize: new events may be added. Aethex ignores types it doesn't use, too.

{"type": "audio", "audio": "AAABAP7/AgD9/wMA/P8EAPv/BQD6/wYA..."}
{"type": "dtmf", "digit": "5"}
{"type": "contextual_update", "text": "The user opened the pricing page"}
{"type": "pong", "event_id": 1}

Audio format

Choose the format with audio_format when you open the session; both directions use it. pcm_16000, the default, is 16-bit little-endian PCM at 16 kHz, mono. ulaw_8000 is G.711 µ-law at 8 kHz, mono, for audio that comes from a phone line. Audio travels base64-encoded in the audio field of audio events. Send 20 ms chunks at real-time pace (640 bytes of PCM or 160 bytes of µ-law before encoding), and keep sending through silence: the agent uses it to tell when the user has finished speaking.

Twilio Media Streams compatibility

If your system already speaks the Twilio Media Streams format, open the session with protocol: "twilio". The socket then uses Twilio's messages instead: media and dtmf from your client, and media, clear and stop from Aethex, with µ-law audio at 8 kHz. This mode has no transcript or agent-text events, and audio_format can only be ulaw_8000.

During the call

- The call shows in GET /calls with provider set to websocket and the direction you chose. Its transcript and recording come from the usual endpoints, keyed by the session_id.
- GET /conversation/{session_id}/status reads created until your client connects, then in-progress, then completed, or failed if the call could not start.
- The agent's maximum call duration applies. At the limit the call ends and you receive call_ended.
- These calls cannot be transferred: the audio runs through your client, so there is no line for Aethex to hand over. The agent's transfer tool is turned off for them.

Close codes

If the socket closes before or during a call for any reason other than a normal hang-up, the close code says why.

Next steps

See every field of POST /conversation/connect and the WebSocket in the Conversation API reference, personalize each call with Personalization, and receive call events with Webhooks.