WebRTC calls

For any in-app, in-browser, or mobile experience, WebRTC is the right transport for live voice. Three reasons:

• Native browser API. RTCPeerConnection already handles mic capture, encoding, jitter buffering, packet loss recovery, and echo cancellation. You do not need a custom audio library.
• NAT traversal. ICE handles corporate firewalls, hotel networks, and home routers. Calls connect from places a raw WebSocket can't.
• Adaptive bitrate. Opus tracks network quality in real time and degrades gracefully. Users on bad WiFi still hold a conversation; they don't cut out.

Three actors. Setup goes through your server so your API key stays put; once the handshake completes, audio flows directly between the browser and Aethex.

1. Browser asks your server to start a call.
2. Your server calls POST/conversation/connect with your API key.
3. Aethex returns a session_id and ice_config.
4. Your server hands those to the browser.
5. Browser builds an SDP offer, posts it through your server to POST/conversation/{session_id}/offer; your server returns the SDP answer. The browser sets it and audio starts flowing.

After step 5 your server is out of the audio path entirely. It only re-enters the loop to return a /tool-result when the agent invokes a tool, or to /end the call.

By default, sessions are opened from your server, which also proxies SDP signaling — the browser sends its offer to your server, which forwards it to Aethex with your API key and returns the SDP answer. Once the handshake completes, audio flows directly between the browser and Aethex. The one exception is the ephemeral call-token flow (below): the browser signals Aethex directly with a short-lived token, and your server only mints the token.

If you'd rather not proxy every signaling request, mint a short-lived call token on your server and hand it to the browser. The browser then talks to Aethex directly — the token stands in for your API key on /connect, /offer, /ice, /status, and /end. Your only server-side code is a one-line mint route, and the ae_live_ key still never reaches the browser.

1. Your server calls POST /conversation/token with your API key and an agent_id, and returns the token to the browser.
2. The browser calls /connect, /offer, /ice, /status, and /end itself, sending Authorization: Bearer <token> — no per-request round-trip through your server.
3. The token is scoped to that one agent and lives just long enough for the call — connect, trickle ICE, poll /status, and hang up with /end (its TTL defaults to the agent's max call duration plus a short margin). Short-lived and single-agent, so a leak is contained; mint a fresh one per call.

The minimum working setup: your server mints a session and proxies SDP signaling, your browser builds the offer and plays the answer audio.

import os, requests
from flask import Flask, jsonify, request as req

app = Flask(__name__)

@app.post("/api/start-call")
def start_call():
    """Browser POSTs here; we mint a session for an agent."""
    resp = requests.post(
        "https://api.aethexai.com/api/v1/conversation/connect",
        headers={"X-API-Key": os.environ["AETHEX_API_KEY"]},
        json={"agent_id": "8a1b2c3d-4e5f-6789-abcd-ef0123456789"},
    )
    return jsonify(resp.json()), resp.status_code  # session_id + ice_config

@app.post("/api/send-offer")
def send_offer():
    """Browser POSTs its SDP offer here; we forward it to Aethex with the API key."""
    data = req.json
    resp = requests.post(
        f"https://api.aethexai.com/api/v1/conversation/{data['session_id']}/offer",
        headers={"X-API-Key": os.environ["AETHEX_API_KEY"]},
        json={"sdp": data["sdp"], "type": data["type"]},
    )
    return jsonify(resp.json()), resp.status_code  # sdp (answer) + type + pc_id

The browser side connects with the session ID and the ICE config the server returned:

async function startCall() {
  // 1. Get session details from your server
  const session = await fetch("/api/start-call", { method: "POST" }).then(r => r.json())

  // 2. Build the peer connection with Aethex's ICE config
  const pc = new RTCPeerConnection(session.ice_config)
  const appChannel = pc.createDataChannel("aethex-app", { ordered: true })
  let pingTimer: ReturnType<typeof setInterval> | null = null
  const stopPing = () => {
    if (pingTimer) {
      clearInterval(pingTimer)
      pingTimer = null
    }
  }
  appChannel.onopen = () => {
    const sendPing = () => {
      if (appChannel.readyState === "open") appChannel.send(`ping:${Date.now()}`)
    }
    sendPing()
    pingTimer = setInterval(sendPing, 1000)
  }
  appChannel.onclose = stopPing
  appChannel.onerror = stopPing
  pc.ontrack = (e) => (document.querySelector("audio")!.srcObject = e.streams[0])

  // 3. Add the user's mic
  const mic = await navigator.mediaDevices.getUserMedia({ audio: true })
  mic.getTracks().forEach((t) => pc.addTrack(t, mic))

  // 4. Build the offer and wait for ICE gathering to complete before sending.
  //    All candidates land in pc.localDescription.sdp so Aethex can open the
  //    media path immediately — no trickle handshake needed for this demo.
  const offer = await pc.createOffer()
  await pc.setLocalDescription(offer)
  await new Promise(resolve => {
    if (pc.iceGatheringState === "complete") return resolve()
    pc.onicegatheringstatechange = () => {
      if (pc.iceGatheringState === "complete") resolve()
    }
  })

  // 5. Exchange SDP through your server proxy (it calls Aethex with your API key)
  const answer = await fetch(
    "/api/send-offer",
    {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({ session_id: session.session_id, sdp: pc.localDescription.sdp, type: pc.localDescription.type }),
    },
  ).then(r => r.json())
  await pc.setRemoteDescription(answer)

  // Audio is flowing. The agent speaks its first message; the user can interrupt.
}

Once audio is up, two endpoints let your server step back into the loop:

• POST/conversation/{session_id}/tool-result — return a client-side tool result back to the agent so it can continue. See Tools for the request/response contract.
• POST/conversation/{session_id}/end — hang up. Drains in-flight audio and runs post-call hooks before closing.

Transcript and recording (when enabled on the agent) settle within seconds. Fetch them with GET/conversations/{id}/transcript and GET/recordings/{id}/audio. The session_id returned by POST/conversation/connect doubles as the conversation ID and the call ID, so a single UUID correlates everything. See Recording & transcription for what gets captured.

Every WebRTC session counts against your account's active-call capacity, the same as a phone call. If you're at the limit, POST/conversation/connect returns 429 with a Retry-After header; honor it and back off. WebRTC calls bill per minute of agent-active time, at the same rate as phone calls; see Billing.

For the full request and response shape on every endpoint, see Conversation (WebRTC) in the API reference.