SeedRealtime: what ByteDance's audio-visual model actually does

What SeedRealtime actually is

SeedRealtime is filed under GenMedia in Seed's model index, next to Seedance, Seedream and Seeduplex. The one-line version from ByteDance: a native audio-visual full-duplex LLM that can "jointly understand audio, visual, and temporal information," pick out which thing in the room you are talking about, and respond without waiting for a clean end-of-turn signal.

"Full-duplex" is worth unpacking, because it gets used loosely. A half-duplex voice model works like a walkie-talkie: you talk, it waits, you stop, it answers. A full-duplex model listens while it is speaking, so it can be cut off, notice you have gone quiet, or decide to stay silent because you were talking to someone else in the room. Most of the field reached audio full-duplex over the last two years, GPT Realtime included. SeedRealtime's claim is that the video stream is in the same duplex loop rather than bolted on as a side channel.

The launch post organises the capability into three parts, and each one is a different thing to ask of a model:

• Joint audio-visual understanding. Resolving a homophone using what is on screen, tracking a visual referent across turns, and binding a voice to a person so it knows who said what.
• Proactive interaction. Acting on a change in the scene rather than only on a question, and weaving tool calls into a reply mid-sentence.
• Conversational timing. Deciding when to join, pause or hold back, and filtering out side conversations and background noise instead of treating every sound as input.

Seven demo videos carry the announcement: identifying speakers around a dinner table, translating a Sichuan menu, reminding someone at the Hebei Museum that they have been standing too long, correcting an espresso pull, spotting a section of the ResNet paper, resisting false triggers in the noise of Daxing Airport, and coaching a child's pronunciation over background chatter. It is a good reel. It is also, as I will get to, the entire published evidence base.

How the architecture differs from a cascaded voice stack

Most voice assistants you have used are a relay race. Speech recognition turns your audio into text, a vision model captions your camera frame separately, a language model reads both transcripts, and a text-to-speech engine reads the answer back. A voice-activity detector sits outside the whole chain and decides when your turn ended.

That design has two costs, and ByteDance names both. Every seam adds delay, and every seam throws information away. Tone, hesitation, the fact that you were pointing at something while you spoke: none of that survives the trip through a transcript. SeedRealtime's pitch is that perception, understanding, decision-making and expression all happen inside one model over continuous streams, so nothing has to be flattened into text on the way through. Low latency comes from chunked audio-visual input, streaming generation, quantisation and inference optimisation rather than from any single trick.

The sharpest architectural detail is the one that gets the least attention: there is no external voice-activity detector. Turn-taking is a decision the model makes from the multimodal signal itself. That is why the museum demo works at all, since "this person has been standing for a while" is not a sentence anyone said out loud. It is also why the airport demo matters more than it looks, because a model that decides its own turns can decide wrongly, and false triggers in a noisy room are the failure mode.

A recap account on X put the contrast with OpenAI more precisely than most of the English coverage did:

"除了有视觉能力外，跟GPT-Live的委派推理方式不同的是 SeedRealtime将声音、画面、时序与表达纳入到同一个模型实时决策，听、看、理解、推理、说话全在一个模型里"

AIGCLINK, X

The gloss: beyond adding vision, the difference from GPT Live's delegated-reasoning approach is that SeedRealtime puts sound, image, timing and expression into one model making real-time decisions, so hearing, seeing, understanding, reasoning and speaking all live in the same place. That is a fair description of the design, and it comes from an account summarising the announcement rather than testing it.

The one number ByteDance published

I went looking for a benchmark table and found a sentence. The only quantitative claim in the entire launch is that end-to-end human evaluation showed conversational pacing issues reduced by half compared to cascaded models, alongside qualitative reductions in interruptions, latency and false triggers.

Take that claim seriously and it is still thin on every axis that would let you check it. No rater count. No sample size. No named evaluation set. No protocol. And the baseline is the generic category "cascaded models," not a named competitor you could go and measure yourself. I am not calling it wrong. I am saying it is a vendor claim, and it should be attributed as one every time it gets repeated.

Two other numerals float around the coverage and both are traps. The "2 to 3 seconds" in the espresso demo is the model's advice about espresso extraction, not a system latency figure. And the arXiv ID that has circulated as the SeedRealtime paper, 2508.02193, belongs to something else entirely: Seed Diffusion, a discrete-state diffusion code model with no connection to audio or video.

There is no SeedRealtime paper. An arXiv full-text search returns zero results, with working controls in the same session. The most telling signal is from inside the team: Seed researcher Panwang Pan lists SeedRealtime among his own selected works with Project, ModelCard and Blog links, and no Paper link, while every other entry on that page has one. Weights are closed, with no repo in either the ByteDance-Seed GitHub org or on Hugging Face.

That is a choice rather than a norm. Kimi K3 shipped with a full benchmark table. So did DeepSeek V4 Flash. A Chinese lab publishing scores you can check is unremarkable in 2026.

The benchmark nobody ran

Here is the finding that reframed the whole post for me. SeedRealtime does not appear anywhere on the Artificial Analysis speech-to-speech leaderboard. Neither does any ByteDance, Doubao, Seed or Volcengine model, across five charts and a 36-row summary table, pulled 18 August 2026. Alibaba Cloud and StepFun are both listed, and Qwen currently tops two of the three component benchmarks, so "Chinese lab, therefore absent" is not the explanation.

That matters because of what the two headline benchmarks measure.

Full Duplex Bench scores the mechanics of a live conversation: not interrupting during a pause, correctly taking the turn, handling a user cutting in, and carrying on through a "mm-hmm." It is the exact thing SeedRealtime's pacing claim is about. The ceiling is 98.4%, held by Qwen Audio 3.0 Realtime Plus, and the current GPT Realtime family sits in the 91 to 96% band. The top of that chart is compressed, which is the polite way of saying that among the leaders, conversational dynamics has stopped separating anyone. Gemini 3.1 Flash Live at 74.3% is the one frontier model it still distinguishes.

𝜏-Voice scores something else: the proportion of replica customer service scenarios resolved while acting as a support agent. A voice user simulator phones the model with a flight change, a disputed charge, a telecom fault. Each task has one valid database end-state, and grading compares the final state against ground truth. It measures whether the work got done, not whether the model sounded helpful. The ceiling there is 56.5%, held by Grok Voice Think Fast 2.0 High.

Sit with that gap. The best voice agent available fails roughly 43% of replica support tasks, while nearly every frontier model handles the rhythm of the conversation almost perfectly. The industry has solved how a voice AI sounds and has not solved whether it finishes the job. Full leaderboard, for the models that are measured:

Note where the fastest model lands. Deepslate Opal answers in 0.44 seconds and resolves 17.5% of tasks. Latency stopped being the bottleneck a while ago, which is the same conclusion I reached in my support-ticket model breakdown, from a different angle.

What people who actually used it say

Community signal on this launch is thin and lopsided, and that is itself informative. Reddit has nothing: r/LocalLLaMA was actively discussing full-duplex options in the same window and never mentioned SeedRealtime, which is what you would expect when there are no weights to download. Hacker News has one submission at 1 point and 0 comments. LinkedIn is roughly ten reposts of the same recap headline. X carries the real reaction.

The best-credentialed voice belongs to someone who works on voice models for a living, and it doubles as the access complaint:

"why aren't more people talking about this?? audio-visual native FDX model is impressive, especially its ability to proactively detect and interact with the user based on visual cues.

GPT Live 1 is pure audio-only and doesn't have this capability.

has anyone tried it on Doubao? i don't have a Chinese phone number but really want to try 🥹"

kei kumata, X

The account that broke the news in English said the same thing in fewer words: "I wish it would be testable globally 😬".

The one substantive hands-on account I found is in Chinese, and what it praises is not what the headline sells:

"测试让豆包陪我一起看一个Youtube视频，画面出现个人，直接问这人是谁，竟然答对了！画面有个一闪而过的图表，问："刚才有个图表，你能给我解读下吗？"。它竟然知道我指的是哪个图，并完整解释。"

@vista8, X

He had Doubao watch a YouTube video with him, asked who a person on screen was and got it right, then asked about a chart that had flashed past and the model knew which chart he meant and explained it fully. That is referent resolution, not speed, and it is the most specific verified claim anyone has made about this model. A follow-up covers pointing a finger at a button on a physical amplifier and getting a sensible answer.

Two reservations surfaced, both from single accounts with low engagement, so read them as individual takes rather than a groundswell. Alex Volkov of ThursdAI on the free-in-Doubao framing: "Free or charge... holy surveilance" (typo his). And a sharper methodological point about the dinner-party demo:

"holding onto four names through overlapping crosstalk is a harder party trick than watching and listening at the same time, most "multimodal" demos never actually test the crosstalk part."

Jens Honack, X

That is the right instinct. The crosstalk case is the interesting one, and a demo reel is not adversarial testing.

Can you build on it? The three-model naming trap

This is where most write-ups go wrong, so it is worth being precise. ByteDance has three realtime models with confusingly adjacent names, and only two of them are for sale.

The plain version: the only thing you can buy is audio-only. The vision that makes SeedRealtime interesting is not on sale. I checked the Volcengine voice billing spec, which lists every purchasable voice SKU, the Ark model price list, and the Doubao product pricing table. SeedRealtime has no line item on any of them, prepaid or postpaid. Every reference to it shipping is app-side.

Volcengine's docs also note that the O and SC versions of the older model have stopped independent iteration, with capability converging into the 2.0 line. So if you are starting today, Seeduplex is the one to build against.

What Seeduplex actually costs

The purchasable realtime model bills per token, not per minute, and postpaid only. It is the one voice SKU with no prepaid resource pack anywhere in Volcengine's discount ladder, which means you pay list rate no matter your volume. Rates below, converted at ¥6.74 to the dollar (USD/CNY 6.7411, 17 August 2026); Volcengine bills in yuan only.

Volcengine publishes the audio-to-token conversion, which is what makes this comparable to a per-minute rate: input audio runs about 6.25 tokens per second, output audio about 25 tokens per second. Do the arithmetic and the shape of the bill becomes obvious.

Those derived rows are my arithmetic from ByteDance's own stated conversion, not a published per-minute price, and a real bill lands higher because every turn also carries text tokens and the accumulating context. Volcengine's own worked example prices a single 40-second exchange at ¥0.0974, about $0.014.

For context on whether that is expensive: OpenAI lists GPT Realtime audio at $32 per million input tokens and $64 per million output, while Google's Gemini 3.1 Flash Live sits at $3.00 input and $12.00 output per million, roughly $0.005 and $0.018 per minute. ByteDance lands between them on input and above Google on output, and my Gemini pricing breakdown has Google's full card. GPT Realtime Mini pricing covers OpenAI's cheaper tier.

The lever is right there in the table. Output audio costs fifteen times more per second than input audio, so the single biggest thing you control is how much the model talks. One caveat on the rate card itself: the table lists output text at ¥80 per million tokens, but the worked example on the same page computes it at ¥30, and only ¥30 reconciles with their stated total. Treat that row as unresolved until you have seen a real invoice.

The access details that decide whether this is real for you

Cost is the easy part. The friction is everywhere else, and it is worth listing plainly because none of it appears in the announcement.

• WebSocket only, at wss://openspeech.bytedance.com/api/v3/duplex/realtime/dialogue, with a single X-Api-Key header and model fixed to 1.2.6.1.
• OpenAI-Realtime-shaped, not OpenAI-compatible. The event names are borrowed (session.create, conversation.item.create, response.cancel) but it is Volcengine's endpoint with Volcengine's auth. Familiar to read, not a drop-in, which is a different situation from the one described in my Realtime API and WebRTC walkthrough.
• Ceilings you cannot buy your way past. 60 sessions per minute, 100,000 tokens per minute, and the realtime model is absent from the concurrency add-on table that covers text-to-speech and speech recognition. Raising limits goes through sales.
• Chinese-first, and the docs say so. One page states Chinese and English support; the constraints section on the same page says the end-to-end model only has good support for Chinese and does not guarantee other languages. Four premium Chinese voices, three US-English voices available on one older model version only.
• A mainland-China account in practice. Yuan-only pricing, a cn-beijing region pin, explicit service activation, and content moderation on by default.
• Context is 12K total, covering the system prompt plus conversation history.

For a support team outside China, that list is the answer. This is not a platform decision yet.

What this means if you are putting AI on a support queue

I build AI agents for helpdesks, so let me be direct about why none of the above changes my week.

The reason a support rollout fails is almost never that the AI's turn-taking felt stiff. It is that the AI answered confidently and wrongly, or that it answered a ticket it should have passed to a human, which is an escalation design problem. One customer, a Danish vehicle-telematics team on Zendesk running around 200 tickets a month and scaling toward 2,000, hit this precisely: their knowledge base said they supported all models, so the bot cheerfully confirmed support for car brands that were not in the database. Nothing about a smoother conversational rhythm fixes that. Better retrieval and grounding does.

The second reason is control, and it is the single most common objection I hear. A CX lead at a DTC supplements brand on Gorgias and Shopify, handling about 7,000 tickets a month against 30,000 orders, put it to me like this:

"The AI will never be able to answer 100% of the questions, but if it tries and just answers 'sorry I don't know this,' I cannot go and check all my 7,000 tickets to see if the AI actually made a good answer, then the point is a little bit gone. I need an AI who is only handling the tickets that it's confident to handle and all the other ones, leave them alone."

That is a 𝜏-Voice problem wearing everyday clothes. He is asking about the 43%. A model that scores 98.4% on conversational dynamics and 56.5% on task completion is, from his seat, a very polite colleague who gets four in ten cases wrong and never says so. Which is why the useful lever is confidence thresholds and a clean escalation path, not a faster model.

So a practical read on SeedRealtime, if you work in support:

• Do not wait on it. No API, no price, no benchmark, Chinese app only. It is not a procurement option.
• Watch the vision axis, not the duplex axis. Resolving "that chart just now" from what is on screen is a real new capability, and where it lands first is screen-sharing support and visual troubleshooting, not phone queues.
• Judge every voice model on task completion. Ask any vendor for its 𝜏-Voice score. If there isn't one, that is the answer. My voice customer support roundup works through the ones that are measured, and AI answering support calls covers the operational side.
• Start where the volume is. Most queues are still overwhelmingly text, and ticket triage alone returns more than a voice pilot does, for a fraction of the risk. Ticket deflection is the other one worth doing first.

For the wider field, the best AI helpdesk software roundup covers what is actually shipping this year.

If you are earlier than that and still working out what these agents do, AI customer service software is the category primer.

Try eesel

If you are evaluating any of this for a real support queue, the question that actually decides the outcome is not which model watches your camera. It is whether the thing will be right on your tickets, using your help centre and your past conversations, before a customer meets it.

That is the part eesel is built around. It plugs into the helpdesk you already run, learns from your existing macros and ticket history rather than needing a fresh knowledge base, and replays your real past tickets in simulation so you see the resolution rate and the exact replies before anything goes live. You set which ticket types it touches and where it hands off, so the "leave the rest alone" problem above is a setting rather than a hope. It is free to try, and you can be looking at simulated results on your own history the same afternoon.

Full-duplex voice will land in support eventually, and when it does the models that win will be the ones that can prove a task-completion number. Until then, the boring work of grounding an AI support agent in your own knowledge is where the resolution rate comes from.