Engineered Anxiety: Your Apple Watch Knows You're Stressed. Now Your NPCs Do Too.

Emotion isn't accidental. It's architected.

Every death, every victory in a game is carefully calculated to trigger specific psychological responses. But most digital interfaces remain emotionally illiterate. They track clicks and time on page, but what if you could detect pupil dilation, heart rate, or facial expression?

This isn't manipulation. It's craft. The same way a composer builds tension through chord progressions, designers now build emotion through systems, feedback loops, and carefully calibrated states.

We are moving from designing for emotion to engineering with it. The interface is no longer a static screen but a dynamic, responsive ecosystem that can sense, interpret, and adapt to our internal states.

engineered anxiety

 
 

Detroit: Become Human is the perfect example of emotional engineering through narrative choice. The game doesn't just tell you a story about androids gaining sentience. It makes you feel the weight of that struggle by forcing you to make life-or-death decisions in real time, often with imperfect information .

The game's core mechanic is engineered anxiety. When you're negotiating with a hostage-taking android as Connor, you have a ticking countdown. Every choice has a time limit. Miss a clue? Fail to ask the right question? Your options narrow. The girl dies

Characters can die. Storylines can close off. You live with your mistakes. Quantic Dream spent two years writing 3,000 pages of script, mocapped 250 actors, and built a new game engine.

Now imagine negotiating that hostage situation with a NPC that has access to LLM and live input from a Smart Watch.


 

The New Engineering Stack

Here's what's happening right now in labs and studios.

 
 
 

1 Reading the Room

EmoEcho proved you can build a game around facial expressions. Two players, one screen one player's face triggers events, the other must mimic those expressions to play. 24 participants tested it. The verdict: emotion-based input creates stronger social presence than keyboard controls. Players weren't just playing; they were building an emotional feedback loop with each other.

EmoEcho paper (CHI 2025): https://dl.acm.org/doi/pdf/10.1145/3748605

EmoMotion combines VR exercise with physiological tracking pupil dilation, heart rate, skin conductance, facial expression to estimate arousal and valence in real time. Users cycle through virtual environments while observing live feedback based on their physical state.

Affectiva's Emotion SDK recently added two new metrics: Yawn and Pain. Pain detects physical reactions to uncomfortable or intense content. Yawn spots attention drops with low latency.

Affectiva Pain and Yawn metrics: https://imotions.com/blog/learning/product-news/pain-and-yawn/ Affectiva SDK has a web playground where you can test the facial coding tech live (though it runs on an older version). Link: https://imotions.com/products/affectiva-facial-coding-sdk/


2 Making Sense of the Data

Raw signals are useless without interpretation.

Marmara University built a framework that combines eye-tracking, facial expression recognition using CNNs trained on FER-2013 and AffectNet, and process mining to create a scalable system for real-time player emotion analysis. The framework works with Unity, Unreal Engine, and Godot.

In practice? Variables like speed, difficulty, and audio adjust based on the player's detected state. A Flappy Bird-style prototype using this approach saw engagement jump significantly.

NieNie takes it further. An Apple Watch captures your heart rate and HRV. An LSTM-based model processes it against the WESAD dataset. Then an LLM generates narrative prompts based on your state. Keep a steady rhythm and it says, "You're in sync. Imagine squeezing this fruit." Stumble, and it offers encouragement. It's designed for young people dealing with emotional stress—biofeedback and adaptive language wrapped into one loop.

NieNie system (UbiComp 2025): https://ar5iv.labs.arxiv.org/html/2510.17534

3 Responding in Real Time

NPCs are getting smarter. University of Twente tested affective mirroring NPCs that recognise and respond to your emotions. Subtle responses strengthen attachment. Over-the-top ones break immersion.

The Oxyde SDK went open source a Rust toolkit for NPCs with "advanced emotional intelligence." Six emotional dimensions (happiness, anger, fear, trust, energy, curiosity) drive every response. NPCs pursue their own goals "earn 1000 gold," "uncover criminal networks" and the system juggles multiple LLMs (OpenAI, Claude, Grok) with intelligent provider selection.

Oxyde SDK (GitHub): https://github.com/Oxyde-Labs/Oxyde has a live web RPG demo you can run locally to see emotional NPCs in action. cd examples/rpg_demo && cargo run, then visit http://localhost:5000. You'll see NPCs with six emotional dimensions (happiness, anger, fear, trust, energy, curiosity) react in real time to your chat, pursue goals like "earn 1000 gold," and generate emergent stories.

MAPPING Emotions

Researchers analysed YouTube streamers playing Outlast tracking facial expressions, voice signals, and verbal narration against game events. It's a clever way to see how horror level design actually triggers real fear. Read the paper here: https://osf.io/5jtx3/

 
 
 

Designing for Player States

Different games demand different UX approaches because they aim to evoke different emotional states. The MDA framework Mechanics, Dynamics, Aesthetics captures this fundamental truth: aesthetics describe how users actually perceive and interact with the experience .

  • Flow State: Tetris Effect keeps you in flow with peripheral feedback that doesn't distract.

    Stress State: Left 4 Dead 2 simplifies UI under pressure stressed players process 25% of normal information.

    Discovery State: Journey removes all markers. You navigate by landmarks, not arrows.

    Learning State: Portal uses colour coding and contextual guidance that fades as you improve.

    Social State: Overwatch keeps key info visible without voice chat pressure.

 

State Transitions: The Hidden Challenge

The most sophisticated games anticipate transitions between states and help players adjust smoothly.

God of War (2018) excels at this. As you approach combat, Kratos' stance shifts, Atreus becomes more alert, and combat UI elements become prominent. When combat ends, the game provides natural decompression loot collection and exploration take priority, shifting interface focus accordingly . This anticipatory design reduces cognitive friction and maintains immersion.

 

What to Engineer For: Your Checklist

Before shipping any feature, ask:

  • What emotion should the player feel here? Joy? Tension? Relief? Pride?

  • Does the mechanic support that feeling or undermine it?

  • Is the effort proportional to the reward?

  • Am I respecting the player's autonomy?

  • Does this build competence or just demand time?

  • Am I creating genuine engagement or cheap addiction?

UX Engineering: The Practical Layer

It's not just about using tech to detect emotions, it's about engineering the response. The game industry has been doing this for years through systems, feedback loops, and calibrated states. Now that thinking is moving into mainstream UX.

Ayodeji Moses Odukoya puts it bluntly: "We've mastered the technical aspects and created a beautiful interface, but we've missed an important aspect: emotion." His solution? Emotional Tokens a new variable for design systems that treats emotions like colours or fonts.

The UI Engineering Stack

First, you need to read the room. Client-side AI models assess typing speed, scroll patterns, mouse movements. TensorFlow.js runs locally in the browser low latency, privacy preserved. Someone typing frantically is in a rush. Someone hovering slowly is confused.

Raw input becomes Emotional Tokens the design system can use:

🟢 mood-calm : softer colours, slower animations, generous whitespace

🔴 mood-urgent : brighter accents, faster transitions, simplified paths

🟡 mood-empathetic : warmer tones, conversational UI, larger fonts

The frontend monitors these tokens and instantly modifies components. React or Vue handle reactivity. Tailwind CSS enables mood-based style changes.

The UX Engineering Stack

  • Sentient UI is a Flutter framework that builds this into a real engineering toolkit:

  • Emotion Recognition Engine: On-device facial expression analysis using a quantised MobileNet-based TFLite model. Camera frames process in memory only never stored or transmitted.

  • Behavioural Analysis: Tracks tap frequency, gesture intensity, and scroll irregularities to detect frustration or cognitive overload.

  • Adaptive Theme System: AnimatedEmotionTheme enables smooth interpolation between emotional states, adjusting colour, typography, spacing, and motion.

  • Adaptive Widget Layer: Drop-in replacements for common Flutter widgets that automatically respond to emotional state changes.

  • Privacy-First Design: All computation is local. No telemetry. No cloud processing. No data persistence.

The Design Engineering Layer

  • Sentient UI also comes with a design philosophy baked in—a three-phase workflow to prevent "happy-path bias and arbitrary styling" :

  • Phase 0: Omni-Context & Journey. Brainstorms 5-20 real-world contexts and maps a three-act emotional journey.

  • Phase 1: Content Priority Analysis. Classifies every element: Critical 🔴, Important 🟡, Supportive 🔵, or On-demand ⚪.

  • Phase 2: Visual Execution. Translates priorities to code using absolute design tokens. Runs an eight-point Sanity Gate.

The Bottom Line

Emotional engineering isn't just about detection—it's about response architecture. As Odukoya puts it: "The future of digital experience is about how our interfaces make us feel, not just what they do."

The tools exist: React, TensorFlow.js, Sentient UI, design tokens. The question is whether we choose to engineer empathy into the stack.

Links:

So what?

All this tech stack is useless without a vision. Now we know how we detect and respond to emotion and the player states show why different users need different approaches, but what are we actually trying to create?

Detroit: Become Human isn't beloved because of its telemetry. It's beloved because it makes people feel something. Here are some of my favourite examples.

Forza Horizon 5

The intro to Forza Horizon 5 is a showcase of how emotionally charged narratives can set the tone for a great user experience. Yes its a car racing game but It’s not just about driving fast cars; it’s about adventure, exploration, and immersion. This approach offers valuable lessons for game and UX designers in crafting emotionally resonant experiences.

The Forza Horizon 5 intro drops players into a vibrant, high-energy setting: the Mexican landscape, presented with stunning visuals and diverse terrains. It doesn’t rely on heavy exposition but creates a sense of immediacy and wonder.

For designers, this demonstrates the power of environmental storytelling. Whether you’re designing a game or a digital experience, setting the mood and immersing the user from the outset builds an emotional connection. For instance, a travel booking app might use evocative visuals and language to make users feel the excitement of an upcoming journey.

 

The Forza Horizon 5 intro is a great example of emotionally driven design, focusing on adventure, freedom, and immersion. It uses stunning visuals, a dynamic soundtrack, and varied gameplay to create excitement and connection. For designers, it highlights the importance of:

  • Setting the emotional tone with visuals and storytelling.

  • Understanding user aspirations, like freedom or exploration.

  • Creating an identity for the user, making them feel central to the experience.

  • Using sensory design (sound, visuals, haptics) to enhance engagement.

 

Need for Speed Rivals

Understanding emotionally driven messaging and narrative building is critical for designers, especially in fields like game design and UX. These skills shape how audiences connect with your work, ensuring your design isn’t just functional but memorable. Need for Speed Rivals offers an excellent case study.

 

The opening monologue of Need for Speed Rivals is a masterclass in setting the tone. It speaks directly to the core of the racing spirit, embodying the thrill of the chase and the allure of rebellion. The narrative uses powerful, evocative language to draw players into a world where speed is both a weapon and a way of life. It's a call to action, inviting players to embrace the chaos of the open road.

 

Gears of War

 

The Gears of War trailer featuring Gary Jules’ Mad World is an iconic example of using emotion to connect with an audience. It contrasts gritty visuals with haunting music to create a powerful narrative, making it a masterclass in emotional storytelling for designers.

 

Conclusion

Every tap, swipe, click, or controller input should evoke something. Joy. Tension. Wonder. Relief. The products we remember aren't the ones that worked flawlessly—they're the ones that made us feel.

The best designers don't just build systems. They build feelings. They create spaces where people can experience joy, growth, and connection in ways real life often doesn't offer.

We're moving from functional tools to emotionally resonant worlds. The future of design is about how our interfaces make us feel, not just what they do.

The technology exists. The science is proven. The question is whether we choose to use it.

Game design isn't about colours or buttons. At its core, it's about engineering emotions.

 
Abdi Jama