Grok Voice API Quickstart: TTS, Voice Cloning, and Realtime Speech-to-Speech in Three Code Blocks

TL;DR: xAI just shipped Grok Voice Think Fast 2.0, topping the Tau Voice benchmark with 0.70s first-audio latency at $0.08/min. This guide walks the entire xAI Voice API in three code blocks — TTS, voice cloning, and a WebSocket realtime speech-to-speech assistant — all runnable today.

Why It Deserves Attention

On July 30, xAI formally launched Grok Voice Think Fast 2.0, one of the fastest speech-to-speech models available:

First on the Tau Voice benchmark (56.5%), ahead of OpenAI, Google, and Alibaba Qwen.
0.70s first-audio latency — the only model in the top five under one second.
$0.08/min pricing, roughly half of OpenAI GPT-Realtime-2.1.
WebSocket bidirectional streaming, compatible with the OpenAI Realtime API protocol.
26 flagship voices plus custom voice cloning, across 25+ languages.

This is not a watch-the-keynote release — the xAI Voice API is fully open, and everything below runs the moment you have an API key.

The Three Capability Tiers

TTS (text-to-speech) — one curl line turns text into audio, with 26 voices plus speed and pitch control.
Custom Voice (cloning) — upload 120 seconds of reference audio to create your own voice.
Speech-to-Speech (realtime conversation) — a bidirectional WebSocket stream with VAD turn detection, tool calling, and session resumption.

One runnable code sample per tier, starting now.

Step 1: TTS — Audio From a Single curl

The simplest call, and a five-second check that your API key works:

# Generate speech and save straight to an MP3 file
curl -X POST https://api.x.ai/v1/tts \
  -H "Authorization: Bearer $XAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "text": "Hello, and welcome to Grok Voice text to speech. This is a test.",
    "voice_id": "eve",
    "language": "en"
  }' \
  --output hello.mp3

The Python version is just as short:

import os, requests

resp = requests.post(
    "https://api.x.ai/v1/tts",
    headers={
        "Authorization": f"Bearer {os.environ['XAI_API_KEY']}",
        "Content-Type": "application/json",
    },
    json={
        "text": "Hello, and welcome to Grok Voice text to speech.",
        "voice_id": "eve",
        "language": "en",
    },
)
resp.raise_for_status()
with open("hello.mp3", "wb") as f:
    f.write(resp.content)
print(f"Saved {len(resp.content):,} bytes -> hello.mp3")

Optional parameters at a glance:

speed: 0.7–1.5, speech rate multiplier.
output_format: sample rate (8kHz–44.1kHz) and bitrate; supports MP3/PCM/μ-law.
with_timestamps: character-level timestamps for subtitles or lip sync.
text_normalization: converts numbers and symbols into spoken forms automatically.
optimize_streaming_latency: 0–2, priority knob for first-frame streaming latency.

Step 2: Custom Voice — Build Your Own Voice

No fine-tuning required — upload one reference clip (WAV/MP3, up to 120 seconds):

# Create a custom voice
curl -X POST https://api.x.ai/v1/custom-voices \
  -H "Authorization: Bearer $XAI_API_KEY" \
  -F "name=My Voice" \
  -F "language=en" \
  -F "[email protected];type=audio/wav"
# Returns: {"voice_id": "nlbqfwie", "name": "My Voice", "language": "en", ...}

Then use the returned voice_id in any TTS call:

curl -X POST https://api.x.ai/v1/tts \
  -H "Authorization: Bearer $XAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "text": "This is my custom voice. Not bad, right?",
    "voice_id": "nlbqfwie",
    "language": "en"
  }' \
  --output custom_voice_demo.mp3

Tip: use clean vocals with low background noise and natural pacing for the reference clip. A voice_id is reusable but does not support incremental updates — if you don’t like the result, create a new voice from a better recording.

Step 3: Speech-to-Speech — a Realtime Voice Assistant

The centerpiece: a bidirectional voice stream over WebSocket with automatic VAD turn detection, tool calling (search/MCP), and session resumption. Complete runnable code:

import asyncio, json, os, websockets

async def voice_agent():
    """Connect to Grok Voice over WebSocket for realtime conversation"""
    url = "wss://api.x.ai/v1/realtime?model=grok-voice-latest"
    headers = {"Authorization": f"Bearer {os.environ['XAI_API_KEY']}"}

    async with websockets.connect(url, additional_headers=headers) as ws:
        # 1. Configure the session: voice, system instructions, VAD
        await ws.send(json.dumps({
            "type": "session.update",
            "session": {
                "voice": "eve",
                "instructions": "You are a warm, helpful voice assistant. Keep answers short.",
                "turn_detection": {
                    "type": "server_vad",      # automatic speech detection
                    "threshold": 0.85,          # activation threshold
                    "silence_duration_ms": 800, # silence that counts as "done"
                },
                "tools": [{                     # mount a search capability
                    "type": "web_search",
                    "name": "web_search"
                }],
            }
        }))
        print("[session ready] voice: eve | VAD: on")

        # 2. Send a text message to start the interaction
        await ws.send(json.dumps({
            "type": "conversation.item.create",
            "item": {
                "type": "message",
                "role": "user",
                "content": [{
                    "type": "input_text",
                    "text": "What's the big AI news today?"
                }]
            }
        }))
        await ws.send(json.dumps({"type": "response.create"}))

        # 3. Listen to the server's event stream
        async for raw in ws:
            event = json.loads(raw)
            t = event["type"]

            if t == "response.audio.delta":       # audio chunk received
                pass  # audio_data = event["delta"] -> play it
            elif t == "response.text.delta":      # synced text (for captions)
                print(event["delta"], end="", flush=True)
            elif t == "response.done":            # turn finished
                print("\n[response complete]")
            elif t == "error":
                print(f"[error] {event}")
                break

asyncio.run(voice_agent())

Key parameters explained:

turn_detection.type: "server_vad" — the server decides when the user finished speaking; no manual push-to-talk.
silence_duration_ms — how much silence counts as “done”; higher values stop mid-sentence pauses from cutting the user off.
prefix_padding_ms — audio recorded before the VAD trigger (default 333ms) so sentence openings don’t get clipped.
idle_timeout_ms — if set, the assistant proactively follows up when the user goes quiet for a long time.
resumption.enabled: true — session resumption restores context automatically after a reconnect.

Practical Advice

Learn the flow with TTS first: curl, then a Python wrapper, then parameter semantics — five minutes, done.
Treat voice cloning as a demo, not a product: 120 seconds of reference audio is enough for prototypes but short of production grade. Record several takes and compare.
VAD parameters are where realtime quality lives: speaking pace differs by language — try silence_duration_ms around 800–1000ms for slower-paced conversational styles and 600–800ms for faster ones.
Use Ephemeral Tokens in the browser: never ship your API key to the frontend. Generate short-lived tokens server-side (the xai-client-secret. prefix) and hand those to the WebSocket.
Start from the xAI Cookbook: the GitHub repo carries four complete sample apps — iOS, Web (WebSocket/WebRTC), and Telephony (Twilio) — ready to fork.

Resources

xAI Speech-to-Speech API docs
xAI TTS API docs
xAI API Quickstart
xAI Cookbook (GitHub) — iOS/Web/Telephony samples
Grok Voice live demos
Get an API key