Think of the last time you had a real conversation while driving. You merged, checked a mirror, glanced at the navigation, maybe decided to skip the next exit, and the conversation never stalled. Nobody calls this multitasking. It’s just what people do. Now think of every voice assistant you have used: you talk, it waits for you to stop, it thinks, it talks. Say something while it’s talking and it either ignores you or stops dead. It’s a walkie-talkie with better manners.
In June I wrote about digital humans and ended the architecture paragraph with “the voice should not be doing the calculus”: a fast, emotionally fluent model at the edge holds the conversation, and a frontier model behind it does the hard thinking when hard thinking is called for. Today Boson AI is releasing Higgs Realtime Duplex, which is that design, built and running.
A society of mind
Marvin Minsky’s Society of Mind argued that what we call a mind is a large number of small processes running at the same time, none of them very smart on its own. You don’t need to buy the whole theory to see that conversation works this way. One part of you listens, another plans the next sentence, another notices that the other person just frowned, and yet another is still wondering whether you left the stove on.
Duplex is organized along the same lines. A single front model listens, watches, and talks at the same time. When it needs more than it knows, it has three options, and none of them pause the conversation:
- Think. Start reasoning while you are still talking, and refine it as more of your sentence arrives.
- Call a tool. Search, a knowledge base, a booking system, whatever the deployment defines. Results can come back in pieces, and the model can start talking before they are complete.
- Delegate. Hand the hard problem, with the full context, to a much larger backend model, ideally a frontier one, and pick the answer up when it arrives.
Thinking Machines’ interaction models take a different route: all reasoning and tool use go to the background model. That keeps the front model simple, but it can put ten seconds or more between question and answer, and the cost is set by the big model. We would rather have the front model handle what it can, and ask for help only when it needs it.
Everything in, everything out
The model works in steps of 160 ms. In each step, text, audio and video come in, and text, voice and a rendered face go out. Silence is a chunk too, which is how the model tells a pause from the end of a turn, and when an “mm-hm” fits. It answers within about 200 ms of you finishing, stops when you interrupt, and ignores remarks that aren’t meant for it.
This is cheaper than it sounds. A minute of audio and video is about 2,600 tokens, so the last five minutes plus compacted notes on everything older fit into 13K to 25K tokens. That is a modest context for an LLM.
The engineering demo is my favorite. An engineer reads a book while the model compares two engine revisions, gets Claude Code to build a missing mass-calculation tool in the background, runs compliance checks while the tool builds, emails a designer, and reports the new mass once the tool is done. The engineer never puts the book down.
Configuring it is prompting
The front model is a 45B mixture-of-experts LLM that learned to hear and see by continued pretraining on about 2 trillion tokens of text, audio and video. It is not a speech system with an LLM bolted on. As a result, setting it up looks like setting up an LLM. You write a text prompt saying who the agent is, what it may and may not do, and which tools it has. You add a few seconds of reference audio for the voice and a single image for the face. No fine-tuning, no per-customer training run. And because it is still an LLM underneath, it follows instructions like one (94.1 on IFEval).
Size and cost
Our design target is that an hour of model time costs at most a tenth of an hour of a person’s time. That is why the front model is mid-sized and the heavy lifting goes to a backend model only when needed. At 45B, Duplex is about a sixth the size of Thinking Machines’ 276B model. It has the highest score on Big Bench Audio (98.9) of the models compared, and results elsewhere are mixed in both directions. My reading is that data and training matter more than parameter count. The Boson post has the full table.
What we can measure now
I ended the digital humans post by saying that we lack a science of what makes an interaction good. With a model that can backchannel, interrupt and keep quiet, these behaviors become things we can measure rather than argue about: when “mm-hm” helps and when it grates, how long to wait before jumping in, when being helpful turns into interrupting. Turn-based models never had to answer these questions. Duplex does, every 160 ms.