The strongest real-time voice model now captures laughter and switches languages mid-sentence. Per Zhidx coverage from August 29: overnight, OpenAI released GPT-RealTime, a speech-to-speech model built for developers, alongside API updates including remote MCP server support, image input, and SIP (Session Initiation Protocol) phone calling.
OpenAI calls it its most advanced speech model ever, with improvements in following complex instructions, calling tools precisely, and producing more natural, expressive speech. It reads repeated letters and numbers naturally, switches languages seamlessly, and picks up non-verbal signals like laughter. Two new voices — Cedar and Marin — ship exclusive to the Realtime API.
Pricing: GPT-RealTime costs $32 per million audio input tokens (cached input: $0.40) and $64 per million audio output tokens — a 20 percent cut versus gpt-4o-realtime-preview, available to all developers from day one. OpenAI also added fine-grained conversation-context control: developers can set smart token limits and truncate multiple turns at once, cutting long-session costs meaningfully. The original Realtime API beta shipped last October; thousands of developers have used it since. The X comment section, characteristically, split between excitement that voice apps just got fun and complaints that the voices still sound robotic and the old cast sounds only slightly more expressive.
Buying Homes, Tickets, and Doctor Appointments — Conversationally
OpenAI’s blog walks through five partner deployments. Zillow, the US real-estate platform, uses the model to talk users through filtering homes by lifestyle needs and analyzing purchase prices. As T-Mobile’s phone assistant, it handles rapid conversational switching — interrupt it mid-sentence with a new topic and it keeps up. StubHub uses it to guide users through ticket purchases and payment problems. On Oscar Health’s platform it phones to confirm open appointment slots, prep requirements, and locations. And insurance-tech firm Lemonade deploys it to help buyers navigate car insurance, gathering needs in conversation and executing purchases against stored user and payment information.
Capturing Laughter, Switching Languages, Adjusting Tone
The improvements target audio quality, instruction understanding, and instruction following. A voice agent needs tone, emotion, and rhythm to keep users talking, and GPT-RealTime produces more natural high-quality speech while honoring fine-grained directions — “speak quickly and professionally” or “speak compassionately with a French accent.” On understanding: it catches non-verbal cues like laughter, switches languages mid-sentence, and adjusts tone. OpenAI’s internal evaluations also show higher accuracy reading alphanumeric sequences like phone numbers across Spanish, Chinese, Japanese, and French. On the Big Bench Audio benchmark — which measures reasoning for audio-input language models — GPT-RealTime scores 82.8 percent, ahead of the December 2024 predecessor.
Building a speech-to-speech agent means giving the model behavioral instructions — how to speak, what to say in which situation, what to do and not do — and OpenAI focused on how precisely the model follows them, so even small instructions now carry more information. On the MultiChallenge audio benchmark for multi-turn instruction following, GPT-RealTime scores 30.5 percent, up sharply from 20.6. And tool calling — the requirement that a voice agent invoke the right function at the right moment with the right arguments — improved on all three axes: on ComplexFuncBench, GPT-RealTime scores 66.5 percent versus the December model’s 49.7. Asynchronous function calling also improved natively: long-running calls no longer break the conversation flow — the model keeps talking while waiting for results, with no developer code changes required.
Preserving the Nuance: Four Realtime API Additions
Unlike the traditional chain of speech-to-text then text-to-speech models, the Realtime API processes and generates audio through a single model — cutting latency and preserving the nuance in a voice, which is what makes responses feel natural. The new capabilities: remote MCP servers — pass a server URL in the session config and MCP support turns on, with the API handling tool calls automatically, no manual integration; point the session at a different MCP server and new tools are instantly available. Image input — add images, photos, and screenshots into a Realtime session alongside audio or text, so the conversation builds on what the user actually sees: “what do you see?” or “read the text in this screenshot.” The framing is images added to the conversation rather than a live video stream — the application decides which images to share and when, keeping control of what the model sees and when it responds. And SIP support with reusable prompts connects applications directly to the public phone network, PBX systems, office phones, and other SIP endpoints through the Realtime API.
The Crowded Field It Lands In
The voice race is accelerating everywhere. Earlier this month, Chinese lab MiniMax shipped Speech 2.5 covering 40-plus languages; Doubao’s app rolled out free real-time voice calls with persona mimicry and emotion sensing at the start of the year; and on the same day, Microsoft launched its first highly expressive natural speech model, MAI-Voice-1, producing different performances from the same prompt. The competitive math for developers: OpenAI’s pricing cut plus MCP, image, and SIP integration makes the Realtime API the fastest path to production voice agents — especially phone-based ones, where SIP support removes an entire integration layer. But the moat is thinning: the same week brought a cheaper Chinese rival at 40 languages and a Microsoft entrant with expression variety, and the X comments about robotic voices are a reminder that in voice, naturalness is judged by ears rather than leaderboards. The market that decided text-agent supremacy on benchmarks will decide voice-agent supremacy on whether callers hang up — and that race is only beginning. Source: Zhidx, August 29.
