Realtime API: connect a web or Flutter app to an agent over WebSocket

A phone call is one way to reach an agent. Realtime is the other: your user talks to the same agent from a browser tab, a mobile app or a kiosk, over a WebSocket. The engines, prompts, knowledge bases and appointment booking are the ones your phone calls use; only the line is different. Two steps, and the split is the security model:

Your server calls POST /v1/realtime/sessions with your API key and gets a client_secret.
Your client opens wss://api.voice.wixzel.com/v1/realtime?client_secret=… and streams audio.

The secret works once, for one minute, for one agent. Your API key never leaves your server.

1. Create a session (server)

curl -X POST https://api.voice.wixzel.com/v1/realtime/sessions \
  -H "Authorization: Bearer $WIXZEL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "agent_id": "6a96a3ead6e886d42462dd3e",
    "allowed_origins": ["https://app.example.com"],
    "metadata": { "user_id": "u_812" }
  }'

const session = await wixzel.realtime.createSession({
  agent_id: '6a96a3ead6e886d42462dd3e',
  allowed_origins: ['https://app.example.com'],
  metadata: { user_id: 'u_812' },
});
// send { url: session.url, client_secret: session.client_secret } to your client

final session = await client.realtime.createSession(CreateRealtimeSession(
  agentId: '6a96a3ead6e886d42462dd3e',
  metadata: {'user_id': 'u_812'},
));

Response

{
  "id": "rt-5b0c7c0e-9d1f-4c52-8a9e-2f0d3a4b6c7d",
  "object": "realtime_session",
  "client_secret": { "value": "rt_secret_…", "expires_at": "2026-09-12T10:01:00.000Z" },
  "url": "wss://api.voice.wixzel.com/v1/realtime",
  "agent_id": "6a96a3ead6e886d42462dd3e",
  "engine": "gemini_live",
  "max_duration_seconds": 600,
  "audio_format": "mulaw_8000"
}

Field
`agent_id`	Required. One of your agents.
`lead_id`	Optional. Attributes the session to a lead: `{{name}}` merge fields resolve from it and appointment booking uses its phone. Without one, merge fields resolve to nothing.
`metadata`	Optional. Your own ids, echoed on the resulting call and its `callCompleted` webhook. Never sent to the model.
`max_duration_seconds`	Optional. 10–3600, default 600. The server ends the session at this point.
`allowed_origins`	Optional, recommended for browsers. The socket is refused unless the page’s `Origin` matches, so a secret copied out of your page cannot be used from another site.

engine in the response is the agent’s voice engine: classic, sarvam, gemini_live, deepgram_agent, openai_live, openai_realtime or grok_voice. All seven work over realtime. Needs the calls:write scope. Nothing is charged here: a user who opens your page and then denies the microphone costs you nothing.

2. Connect (client)

In a browser

The TypeScript SDK ships a browser client that handles the microphone, encoding, playback and barge-in:

import { RealtimeSession } from 'wixzel-voice/realtime';

// `session` is what your server returned above
const call = await RealtimeSession.connect(session, {
  onTranscript: ({ role, text }) => render(role, text),
  onAgentSpeaking: (speaking) => setSpeaking(speaking),
  onError: ({ code, message }) => showError(code, message),
  onEnd: (reason) => showEnded(reason),
});

// later
call.setMuted(true);
await call.end();

It needs a secure page (https://, or localhost) and asks for the microphone when connect is called. To ask before you mint a session, call requestMicrophone() first and pass the stream to session.start().

In Flutter or Dart

The Dart SDK has the protocol client. Microphone and speaker access are your app’s: use any recording plugin that gives you 16-bit PCM at 8 kHz, and any PCM player:

import 'package:wixzel_voice/wixzel_voice.dart';

final conn = RealtimeConnection.forSession(session);
conn.events.listen((e) {
  switch (e) {
    case RealtimeAudio(:final mulaw): player.add(mulawToPcm16(mulaw));
    case RealtimeAudioClear(): player.clear(); // the user interrupted
    case RealtimeTranscript(:final role, :final text): print('$role: $text');
    case RealtimeError(:final code, :final message): print('$code: $message');
    case RealtimeEnded(:final reason): print('ended: $reason');
    default:
  }
});
mic.listen((Int16List pcm) => conn.sendAudio(pcm16ToMulaw(pcm)));
// later
await conn.end();

Turn on your platform’s echo cancellation. Without it the agent hears itself through the speaker and interrupts itself.

With the agent’s mascot

Every agent has a mascot, an animated face, and it can lip-sync to a session: it thinks while the session connects, listens while the user talks, and moves its mouth in time with the agent’s audio. In a browser, mountMascot from wixzel-voice/mascot draws it and face.connect(session) opens the session with the mascot wired in. In Flutter, AgentMascotView from wixzel_voice_flutter draws it and MascotRealtimeLink follows a RealtimeConnection. See Draw it in your own app and SDKs: Mascot widgets.

Anything else

The protocol is plain JSON over a WebSocket, below.

The protocol

Every frame is a JSON text message with a type. Audio is base64 G.711 µ-law, 8 kHz, mono in both directions, exactly what a phone line carries. Send it in 20 ms frames (160 bytes). From the server

`type`	Fields	Meaning
`session.started`	`session_id`, `call_id`, `agent_id`, `engine`, `max_duration_seconds`, `audio_format`	Admitted; the agent is connecting and will greet the user.
`audio`	`audio`	Agent speech. Play it in order.
`audio.clear`		The user interrupted. Drop everything you have queued.
`transcript`	`role` (`user` or `assistant`), `text`	A line of the conversation.
`session.warning`	`code`, `seconds_remaining`	`low_balance`: the account is about to run out of credit.
`error`	`code`, `message`	Something went wrong. Before `session.started`, it means the session was refused.
`session.ended`	`reason`, `duration_seconds`	Over. The socket closes next.
`tool.call`	`call_id`, `name`, `arguments`, `wait_for_result`	The agent wants your app to run a client tool.

From the client

`type`	Fields	Meaning
`audio`	`audio`	Microphone audio.
`session.end`		Hang up.
`tool.result`	`call_id`, `output` or `error`	The answer to a `tool.call`. The browser client sends it for you from its `tools` option.

While the agent is saying its opening line, keep sending audio but send silence (0xFF bytes): the agent then cannot be interrupted by room noise before the user has heard anything. The browser client does this for you.

Why a session ends

`reason`
`client`	You sent `session.end`.
`agent`	The agent ended the conversation (the user said goodbye).
`max_duration`	It reached `max_duration_seconds`.
`idle`	Nobody spoke for two minutes.
`insufficient_credits`	The account ran out of credit mid-session.
`error`	The voice engine failed.
`disconnected`	The socket dropped.

Refusals

A browser cannot read the HTTP status of a refused WebSocket, so refusals arrive as an error frame followed by a close with one of these codes:

Close	`code`	Fix
4401	`invalid_client_secret`, `invalid_api_key`	The secret is missing, expired or already used: mint a new one. Or the key that minted it was revoked.
4402	`insufficient_credits`	Top up.
4403	`origin_not_allowed`	The page’s origin is not in `allowed_origins`.
4404	`agent_not_found`	The agent was deleted after minting.
4409	`concurrency_limit`	The account is at its concurrent-call limit.
4429	`spend_limit`	The API key reached its spend limit.
4503	`engine_unavailable`, `server_busy`, …	Try again shortly.

Billing and records

A realtime session is a call to everything but the phone network:

It is billed at the same per-minute price as a phone call on the same engine, per second, from credit. See pricing.
It counts toward your concurrent-call limit (5 by default) and the API key’s spend limit.
It appears in GET /v1/calls with "channel": "web" (filter with ?channel=web), inbound, with no from or to, and with its transcript.
It ends with a callCompleted webhook carrying "provider": "web" and your metadata.

Human transfer is not available: there is no second line to hand the user to. Web sessions are not recorded.

Getting started

Guides

Carriers

Concepts

API reference

Realtime API: connect a web or Flutter app to an agent over WebSocket

1. Create a session (server)

2. Connect (client)

In a browser

In Flutter or Dart

With the agent’s mascot

Anything else

The protocol

Why a session ends

Refusals

Billing and records