Realtime API reference

This endpoint is a long-lived WebSocket connection. The gateway relays the Bailian Qwen-Omni-Realtime protocol verbatim in both directions with no protocol translation; the full field definitions and the tool-calling (Function Calling / MCP) events follow Bailian's official client events and server events. This page covers the handshake, auth, event list, and audio conventions needed to integrate.

Request

Only a logical model whose public_protocols includes dashscope_realtime can use this surface.

Handshake errors

Failures before the WebSocket upgrade are returned as HTTP status codes with a body of:

{"error":{"type":"invalid_request_error","code":"unauthorized","message":"unauthorized: api_key_not_found"}}

After a successful upgrade (HTTP 101), later problems are delivered as an in-band error event, then the connection closes:

{"type":"error","error":{"type":"upstream_error","code":"provider_unavailable","message":"upstream connection failed"}}

Session configuration session.update

After connecting, the server first sends session.created; the client overrides with session.update, and a valid update is acknowledged with session.updated. Common fields:

turn_detection: null disables the server VAD so the client commits and triggers responses manually.

Client events

Server events

Tool-calling events (response.function_call_arguments.delta / .done) and the MCP events (mcp_*) are documented in Bailian's server-events reference.

Example event order for one voice turn:

session.created
session.updated
input_audio_buffer.speech_started
input_audio_buffer.speech_stopped
input_audio_buffer.committed
conversation.item.input_audio_transcription.completed
response.created
response.output_item.added
response.audio_transcript.delta …
response.audio.delta …
response.done

Audio formats

Send input audio in full 100 ms frames (3200 bytes per frame) for best reliability. input_audio_buffer.append only buffers; whether it is committed is decided by VAD or by a client input_audio_buffer.commit.

VAD and interruption

• With a non-null turn_detection, the server auto-detects speech boundaries and, after silence_duration_ms of silence (default 800 ms), auto-commits and triggers a response.
• When the user speaks again: the server emits input_audio_buffer.speech_started → the client stops playback and sends response.cancel → response.done (cancelled) arrives → a new turn begins.
• turn_detection: null is manual mode: after recording, the client sends input_audio_buffer.commit (wait for input_audio_buffer.committed) then response.create.

Usage and billing

Each turn's response.done.response.usage carries per-modality tokens:

{
  "type": "response.done",
  "response": {
    "status": "completed",
    "usage": {
      "total_tokens": 843,
      "input_tokens": 693,
      "output_tokens": 150,
      "input_tokens_details": { "text_tokens": 693 },
      "output_tokens_details": { "text_tokens": 28, "audio_tokens": 122 }
    }
  }
}

The gateway settles the accumulated session usage at close: a session-level estimate is held at connect (currently about ¥1.2; insufficient balance fails the handshake with 402), then refund/top-up applies based on actual usage; a session with no usage releases the hold. See the guide.

Limits

Related

• Realtime (WebSocket) models guide
• Authentication
• Errors and rate limits