Why the AI frontier has been optimizing for the wrong bottleneck — and what designing for full-duplex presence actually requires
We've been building AI wrong.
Not technically wrong. Architecturally wrong. Wrong at the level of the relationship.
For the last decade, every AI product we've shipped has been built on a fundamental assumption: the human speaks, the machine listens, the machine responds, repeat. We called this progress. We called it conversational AI. We even called it intelligent.
What we actually built was a very fast filing cabinet with good manners.
I've spent over a decade pioneering Cognitive Experience Design — the practice of using AI to invent products, services, and systems that enhance human ability rather than replace it. The single most important design principle I've held since my days building Watson at IBM and Alexa at Amazon is this: a human will always have the final say.
But here's what I didn't fully anticipate when I coined the term in 2014: the next design frontier isn't about who has the final say. It's about whether the AI is present for the whole conversation.
I laid out this thesis at the Good Growth Global Forum in 2024 — that the most consequential shift coming in AI wasn't capability, it was presence.[1] We are now living inside that prediction. And a new body of evidence, including a troubling Harvard Business Review finding on what AI overuse is doing to our brains, makes the design imperative more urgent than I've ever felt it.[2]
The Three Eras That Led Here
To understand where we are, you have to understand how we got here. Between 2022 and 2026, three paradigm shifts defined how AI systems were built — each relocating rigor to a higher abstraction layer.[3]
Prompt Engineering (2022–2024) operated on the belief that the quality of instruction text determined success. Andrej Karpathy's famous declaration — "The hottest new programming language is English" — captured the ethos. The failure mode: even perfect prompts couldn't help an agent that couldn't see the right information.[3]
Context Engineering (2025) was the realization, crystallized by Shopify's Tobi Lütke and Karpathy, that what fills the context window matters more than how the prompt is phrased. The failure mode: even perfect context failed when the system consuming it was poorly designed.[3]
Harness Engineering (2026) gave us what Mitchell Hashimoto formalized as "Agent = Model + Harness" — where the harness is everything in the agent minus the model itself.[3] Martin Fowler and Birgitta Böckeler distilled the governing principle: every time an agent makes a mistake, change the system so that mistake structurally cannot recur. This era produced genuine value — KV-cache optimization, multi-agent orchestration, structured feedback loops, production-grade error recovery.
But it exposed a fundamental ceiling.
By stitching together voice-activity-detection modules, speech recognition systems, large language models, and speech synthesis layers into cascaded serial pipelines, harnesses introduced compounding latency and information loss at every seam.[4] Each external add-on sacrificed some dimension of human communication — tone, pause, expression, overlap — in order to maintain technical tractability.
We were building intelligence. But we were building it without presence.
The Turn Boundary Was Never the Interface. It Was the Limitation.
Think about what we've been asking people to do.
You have a half-formed idea. You're mid-thought, searching for a word. You trail off. You redirect. In every AI system built in the harness era, that pause is interpreted as your turn ending. The model answers the incomplete version of what you meant.
We designed AI to respond to your tokens. Not to you.
The result? People became secretaries to their own intelligence tools. You attend a meeting, make a decision, have a breakthrough — then stop everything to re-narrate the full context to a model that wasn't in the room. You're not getting help thinking. You're transcribing your life into a prompt box.
This isn't a latency problem. This is an isolation problem. The AI was never with you. It was always processing a record of a moment that had already passed.
The Bandwidth Bottleneck
The root failure has a name: single-threaded real-world perception.[4]
Until the user finishes speaking, a harness-dependent model is perceptually frozen. While generating a response, it receives no new input. It cannot hear hesitation. It cannot see the frown that signals confusion. It cannot detect the pause that means "I'm searching for the right word" versus the pause that means "I'm done."
The consequence is that human-AI collaboration has been structurally capped at the bandwidth of an email exchange — when human-human collaboration naturally operates at the bandwidth of a face-to-face conversation: parallel, continuous, two-way, and information-rich every millisecond.[4]
The implications for brain health are real and measurable. A March 2026 Harvard Business Review study found that heavy AI users are experiencing what researchers are calling "brain fry" — a state of cognitive fatigue and atrophied independent thinking caused not by overwork, but by the wrong kind of AI interaction.[2] When users are forced to constantly context-switch between their own live cognition and the turn-based rhythm of a half-duplex model, the cognitive toll compounds. The MIT finding is consistent: cognitive activity scales down in direct proportion to generative AI use when users stop exercising their own reasoning.[2] This is what happens when a tool designed to reduce cognitive load instead restructures human cognition around its own limitations.
Mira Murati and the Bet That Changes Everything
In May 2026, Mira Murati — formerly CTO of OpenAI — released Thinking Machines Lab's first model, TML-Interaction-Small. The thesis behind it is the most important architectural argument in AI right now: the rest of the frontier has been optimizing for the wrong bottleneck.[6]
While every major AI lab has raced to make models smarter, larger, and more capable in isolation, Murati and her team bet that the real constraint was never model intelligence — it was the interaction model itself. As TML states directly in their technical report:
"For interactivity to scale with intelligence, it must be part of the model itself."[6]
This is not an incremental improvement. It is the abolition of the walkie-talkie — the end of the era where human-AI communication was structured like an email exchange, where the model waited in darkness while the user formulated a thought, and where humans were forced to adapt to AI interfaces rather than the reverse.[4][6]
TML-Interaction-Small is a 276-billion-parameter Mixture-of-Experts model that natively processes audio, video, and text in continuous 200-millisecond micro-turns, paired with an asynchronous background model that handles sustained reasoning, tool calls, and agentic work — with both sharing full context throughout. The performance gap is stark: on FD-bench V1.5 interaction quality benchmarks, the TML model scored 77.8 compared to GPT-realtime-2.0 at 46.8 and Gemini-3.1-flash-live at 54.3.[7] Turn-taking latency reached 0.40 seconds, approaching the human instinctive reaction limit.[7] Evaluators were forced to create entirely new benchmark dimensions — time-awareness and visual proactivity — because existing commercial models scored essentially zero on tasks requiring these capabilities.[6]
Murati's bet is that the frontier has been solving for capability when it should have been solving for collaboration. She is right.
Full-Duplex Presence: What It Actually Means
The telecommunications world gave us the right vocabulary. A simplex channel is one-way — a megaphone. A half-duplex channel is two-way but takes turns — a walkie-talkie: you press the button, speak, say "over," and wait. A full-duplex channel allows both parties to talk and listen simultaneously — a phone call.[8]
Applied to AI: half-duplex is what we've built for five years. You finish speaking, AI processes, AI responds, you wait, repeat. The AI is perceptually off while you speak, and you are informationally off while it generates. Every exchange has a turn boundary.
Full-duplex presence is what comes next. The AI continuously perceives you — your voice, your screen, your hesitations, your tone — even while it is generating a response. You can interrupt. It can interject. There is no "over."
The word presence is the critical addition. Full-duplex alone describes a communication channel. Presence describes the experiential quality that emerges from it — the sense that the other party is in the same moment as you, not processing a record of a moment that already passed.[8] That is what makes a phone call feel qualitatively different from exchanging voicemails.
The Architecture of the New Paradigm
The architectural innovation making this possible is what TML calls dual-track parallelism.[6]
Two tracks run in parallel. A Foreground Interaction Model is always online — processing continuous audio, video, and text in 200-millisecond micro-turns, managing turn-taking, and capable of proactive interjections. A Background Reasoning Model runs asynchronously — handling deep reasoning, tool calls, web search, and agentic workflows. The foreground model never sleeps. When a task exceeds its instantaneous capacity, it packages the full conversation context and dispatches it to the background asynchronously. Results stream back woven into the live conversation at a moment appropriate to what the user is currently doing — not as an abrupt context switch.[6]
This split delivers something previously impossible: the planning, tool-use, and agentic workflows of reasoning models at the response latency of non-thinking ones.[6]
Four architectural innovations make this work:
Time-Aligned Micro-Turns. Rather than compressing all input and output into a single ordered token sequence, continuous audio and video streams are divided into 200-millisecond chunks. Within each chunk, the model simultaneously receives input and generates output — eliminating artificial turn boundaries and enabling natural understanding of breathing pauses, overlapping speech, and conversational handovers.[4]
Encoder-Free Early Fusion. Traditional multimodal systems bolt together independent speech recognition, visual encoding, and language models. TML eliminates these external encoders. Audio is converted to dMel signals; video frames are processed through lightweight patch embeddings; all modalities are co-trained from scratch in a unified transformer. The model doesn't "translate" between modalities — it thinks in all of them simultaneously.[4][6]
Streaming Sessions Infrastructure. 200-millisecond chunks require frequent small prefills that existing inference libraries weren't optimized to handle. TML developed Streaming Sessions — technology that retains sequences persistently in GPU memory, avoiding frequent memory reallocations, now contributed to the open-source SGLang framework.[6]
Shared Context Architecture. When the interaction model delegates to the background, it sends the full conversation — not a standalone query, but complete context. This bidirectional sharing transforms two separate models into a single coherent intelligence rather than a relay race.[6]
Real-World Implications: Cognitive Load Redistribution
The most important thing this architecture does isn't make AI faster. It redistributes cognitive load.
The "brain fry" phenomenon identified in the HBR study is a design failure, not a human failing.[2] When users are forced to translate their live, embodied, half-formed thinking into structured prompts — to perform the clarity they don't yet have — the cognitive tax compounds over every interaction. What was supposed to offload thinking instead adds to it. The burden of re-narrating context, of waiting, of correcting misunderstandings after the fact — these aren't friction. They are cognitive overhead that accumulates into exhaustion.
The post-harness architecture addresses this at the root. The model doesn't replace human judgment. It eliminates the overhead of holding context that should be handled by the system.
Clark and Brennan's 1991 communication theory frames what's actually required for effective collaboration: copresence (interacting with what others are interacting with), contemporality (receiving information as it is produced with instant feedback), and simultaneity (receiving and producing information at the same time).[9] Current turn-based AI satisfies none of these conditions. The micro-turn architecture satisfies all three — for the first time, between human and machine.
Consider the practical applications across domains where I'm actively building:
For design engineers — a profession where iteration speed, spatial reasoning, and contextual judgment are paramount — this is a qualitative shift. The model watches a design review, understands which feedback matters based on speaker tone and emphasis, dispatches background calls to specification databases and prior art, and synthesizes a revised brief while the meeting is still happening. Engineering constraints, material compatibility, simulation parameters arrive as a spoken conversational contribution, not a document to open later.
For health and clinical care, the gains are already measurable in harness-era approximations. Ambient AI scribes decreased documentation time by 16 minutes per clinical encounter across five academic medical centers.[10] Mass General Brigham observed a 21% reduction in burnout prevalence after 84 days of ambient documentation use.[10] These gains came from systems that were still turn-based. The compounding effect of native multimodal presence will be substantially larger.
For creative work in music, sound design, and therapeutic audio — a model that processes audio as a first-class modality, perceives sonic qualities as a composition develops, and asynchronously generates variations and evaluates psychoacoustic properties, delivers creative suggestions woven into the session itself. Not a file to evaluate separately.
For consumer devices, the ambient intelligence vision Mark Weiser described in 1991 — technology that "disappears," present everywhere, demanding attention nowhere — becomes architecturally achievable for the first time.[11] The model doesn't need to be summoned. It is already watching, listening, and thinking — surfacing what matters at the moment it matters, silent when it doesn't.
Historical Aha Moments: When Technology Pivoted Toward Magic
Every generation or so, a technology's fundamental interaction model is reimagined. The scaffold falls away and what remains feels, for the first time, like something natural. These are inflection points, not iterations.
The Gutenberg Press (c. 1440) changed the interaction model between humans and ideas: from laborious mediation to flow. The aha wasn't the mechanism — it was the first time a literate person could hold a book and feel the ideas inside were for them.[12]
The Telephone (1876) eliminated the temporal gap in human communication. The aha was presence: two people in different places experiencing the same conversational moment.[13] Tone, interruption, laughter, and silence all crossed the wire. The walkie-talkie preceded it. The telephone's full-duplex nature was the revolution. We are at the telephone moment for human-machine interaction.[4]
The Personal Computer (1984) didn't invent computing. It changed who could use one. The GUI made computation legible to human intuition without requiring users to understand how the machine worked.[14]
The Browser (1993) transformed the internet from a technical network accessible to specialists into a navigable landscape for anyone. The aha was orientation — the vastness of interconnected information feeling traversable, not overwhelming.[14]
The iPhone (2007) eliminated the mediating layer between intent and action. Multi-touch made the device feel like an extension of the hand rather than a tool operated by it.[15]
Ford's Moving Assembly Line (1913) didn't invent electricity or the automobile. Ford used electric motors' precise speed control to redesign the entire production system — cutting build time from 12 hours to 93 minutes.[15] The lesson: early adopters who treat a new medium like the old one miss its full value. The harness era treated new AI like the old model — patching turn-based systems to simulate real-time behavior. The post-harness era redesigns around the medium's native capabilities.
The defining aha of the post-harness era won't be a feature. It will be a feeling: working with an AI and forgetting you are — not because it is invisible, but because its presence is as natural as a skilled collaborator sitting beside you.
What This Means for Designers and Builders Right Now
When I was designing diagnosis tools for healthcare professionals at IBM Watson, the most critical design challenge wasn't accuracy. It was trust. We had to show clinicians how much data the system had sorted through. We surfaced confidence scores. We created mechanisms that let users see the inputs that led to the conclusions. Trust wasn't a feature — trust was the architecture.
Full-duplex agentic AI introduces a new trust design challenge: how do you design for a system that is always listening, always watching, and speaks when something is worth saying?
Neuroadaptive calibration over static personas. The best agentic experiences won't have a single interaction mode. They'll sense whether you're in flow or struggling, moving fast or thinking slow, and calibrate — the way a great teacher does. Designing for this requires moving from user personas to cognitive state modeling.
Transparent presence, not surveillance. The more an AI's behavior is shaped by real-time personal context, the more the design must surface how it's listening and why. The role of AI shouldn't be to find the needle in the haystack — it should be to show the user how much hay it can clear so they can better see the needle themselves.[16] Full-duplex presence amplifies this principle.
Emotional register as a first-class design signal. Half-duplex AI reads tokens. Full-duplex AI perceives that your voice is tight, that you sighed between sentences, that your intonation shifted — and responds to who is saying something, not just what was said. Voice biomarkers aren't a gimmick. They're the data layer that makes an AI experience genuinely human-obsessed.[17]
Agency architecture, not just feature architecture. The boundary between AI judgment and human judgment becomes fluid in the agentic era. Designing for it requires explicit agency scaffolding — mechanisms that keep humans in the loop not because we fear the AI, but because human agency is where the value lives. TML's own framing is explicit: the post-harness architecture keeps humans as genuine collaborators in an ongoing conversation, not supervisors of an autonomous system.[6]
Designing against brain fry, not just for convenience. The HBR finding is a design brief.[2] If our agentic systems are creating cognitive dependency rather than cognitive liberation, we have failed. The goal of presence is to restore the human's capacity to think — not to create a new form of outsourcing that atrophies it.
The Loneliest Expert Problem
Here's the human moment that full-duplex presence solves that we don't discuss enough.
You are the most senior person in the room. There is no one who can keep up with the specificity of your domain, the nuance of your tradeoffs, the particular way you approach problems. Half-duplex AI gets you partway there — but the turn boundary forces you into teaching mode. You explain; it responds; you evaluate; you prompt again. You're not collaborating. You're lecturing a student and grading their answers.
Every domain expert, every pioneer, every person operating at the edge of what's known lives this.
Full-duplex presence solves the isolation that comes from thinking at the frontier. The model is with you in the same moment, perceiving the same reality, available before you have to ask.
That's not a feature. That's a collaborator.
From UX to AX: The Design Shift We Can't Delay
At the Good Growth Global Forum in 2024, I argued that the most important reframe in our field was the shift from User Experience to Agentic Experience — UX to AX.[1]
User experience assumed a human sat down, opened an interface, completed a task, and left. The design challenge was reducing friction in that bounded session.
Agentic experience assumes the AI is already present — ambient, perceiving, capable of acting across systems and time on your behalf. The design challenge is entirely different: trust, transparency, calibration, and human agency at every layer.
The CXD principles I've spent a decade developing — Human Effects, Transparent Truth, Breaking Bias, Algorithmic Accord, Systematic Sanctuary — were written for this moment.[18] Not because I was prescient, but because designing for human cognition always leads you here. When you start from "what does a human brain actually need to thrive?" you end up building for presence, not just for performance.
Rich Sutton's "Bitter Lesson" — which TML cites in their technical report — holds that all complex external add-on systems relying on manual human design will eventually be outperformed by the underlying models through unified architecture.[6] The harness era was the final expression of the add-on era. What replaces it is not a better harness. It is a model that breathes in sync with humans.
Build for Cognition, Not Consumption
The interface paradigm built on buttons, menus, and chat boxes is ending. Not because AI got smarter. Because we are finally designing the relationship correctly.
The best agentic AI experience isn't the one that does the most. It's the one that makes you feel most fully yourself — more capable, more clear, more connected to the work that only you can do. The one that reduces brain fry rather than inducing it.[2]
That has always been the north star of Cognitive Experience Design: AI as the material that makes humans more human, not more like machines.
Every great technology pivot in history looked like magic until it looked like infrastructure. We are in the magic moment.
It's time to build.
References
- Peña-Bickley, J. (2021). Principles: CognitiveExperience.design. cognitiveexperience.design/principles
- Peña-Bickley, J. (2024). Good Growth Global Forum Keynote and Interview. joannapenabickley.com
- Gerber, M., & Bastian, H. (2026, March). When using AI leads to "brain fry." Harvard Business Review. hbr.org/2026/03/when-using-ai-leads-to-brain-fry
- Bits-Bytes-NN. (2026, April 5). From prompts to harnesses — four years of AI agentic patterns. bits-bytes-nn.github.io
- 36Kr English. (2026). Ex-OpenAI tech steerer aims to disrupt OpenAI's rules. 36Kr. eu.36kr.com
- MIT Research / Peña-Bickley CXD Framework. (2025). Cognitive activity and generative AI use: findings on reasoning atrophy. cognitiveexperience.design
- Thinking Machines Lab. (2026). Interaction models: A scalable approach to human-AI collaboration. thinkingmachines.ai/blog/interaction-models
- Analytics Drift. (2026). Mira Murati's Thinking Machines says every AI lab built interaction wrong. analyticsdrift.com
- AI Data Insider. (2026). Thinking Machines Lab unveils advancement in real-time human-AI collaboration. aidatainsider.com
- Clark, H. H., & Brennan, S. E. (1991). Grounding in communication. In Perspectives on socially shared cognition (pp. 127–149). American Psychological Association.
- American Hospital Association. (2026, April 14). 6 health systems enhancing care delivery with ambient AI scribes. AHA Center for Health Innovation Market Scan. aha.org
- Weiser, M. (1991). The computer for the 21st century. Scientific American, 265(3), 94–104.
- Brookings Institution. (2023). Gutenberg's message to the AI era. brookings.edu
- Webex Blog. (2023). Technology milestones: The story of five key innovations. blog.webex.com
- Grokking Tech Career. (2024). Key technological paradigm shifts throughout history. grokkingtechcareer.substack.com
- GrowthBook. (2024). Building in the AI era: Lessons from past technological revolutions. growthbook.io
- Peña-Bickley, J. (2015). About Cognitive Experience Design. cognitiveexperience.design
- Kintsugi Health. (2024). Leveraging voice biomarkers for real world solutions. kintsugihealth.com