Who’s holding the remote control? [6/9]
Exploring the dangers of blindly allowing personalised AI to shape our future
AI in Our Health and Wellbeing - Are We Still Well When We Don’t Choose It?
Health and wellbeing are sacred ground. Or they used to be.
Now, they’re metrics. Mood points. Compliance scores. Push notifications that ask, “Did you hydrate today?” and silently log your failure when you swipe them away.
In the world of data, wellness isn’t something you feel. It’s something you perform—in dashboards, on wristbands, through biometric apps you didn’t consent to but still somehow have. And sometimes things happen in life that are just going to be helped by throwing data and AI at them… Humans are more complicated than that.
This week, we meet Eleanor:
PART 5: HEALTH & WELLBEING
“Just Breathe, Eleanor…” – a Wellness Compliance Story
Eleanor used to go for early morning walks by the river. Sometimes she’d just sit. Not for steps. Not for stress scores. Just because she enjoyed it..
But lately, that didn’t count.
It started when her employer rolled out the WellScore+ platform, partnered with the national health system and three major fitness brands.
“We care about your wellbeing,” the CEO said over Zoom. “So now, your wearable will guide you to your best self. It's completely optional. Though… it does affect bonus eligibility.”
Suddenly, Eleanor’s calendar filled with coloured dots:
7:30 AM: BreatheSync™ breathwork
9:00 AM: Mood Check-In
1:00 PM: Positive Self-Talk Loop (Guided)
4:45 PM: JoyEvent Reminder (Post a grateful moment!)
Each moment tracked. Each breath counted.
One Thursday afternoon, Eleanor felt a bit out of sorts - nothing major; she had just been busy socially and at work. She opened the WellScore+ app.
“Low serotonin potential detected. Initiating MoodNudge™ sequence.”
A calm AI voice chimed in her earbuds:
“Let’s find your smile, Eleanor. How about a 5-minute micro-dance? Or a gratitude selfie?”
Eleanor closed her eyes. She didn’t want to smile. She wanted the app to just SHUT UP.
She muted the prompt.
That night, she got a message from HR.
“We noticed an anomaly in your wellness trajectory. We’re here if you need us! Please remember: sustained disengagement affects compliance scores.”
Two weeks later, Eleanor was feeling down. Her Mum had told her the day before that she had been diagnosed with an early stage cancer. Prognosis was good, but still a worrying time ahead and treatments to be navigated. As she went about her day she felt a tightness in her chest, and a mind full of worries. She was struggling to focus.
But the WellScore+ app kept butting in…
….. you need to do more steps
….. don’t drink alcohol on a Sunday night
It did not seem to know or recognise that sometimes things happen in life that are just sad; and no amount of data-driven advice is going to change that.
She took off the wearable device and put it in a drawer.
Later, a system review flagged her as non-compliant. Her therapy sessions were moved to asynchronous AI chat. Her bonus was cut.
All the app told her?
“Just Breathe, Eleanor….”
The Trouble with AI Wellness
When AI gets between us and our wellbeing, something subtle and dangerous happens:
We stop being well.
We start appearing well.
We optimise. We log. We gamify. And in the process, we lose the messy, human signals—fatigue, grief, intuition, vulnerability—that real healing requires.
Personalised AI treats health like a problem to solve.
Personal AI would treat it like a life to support.
A personal AI might have said:
“You don’t seem like yourself today. Want me to book time with someone who’ll actually listen?”
It wouldn’t have docked her bonus.
It wouldn’t have scheduled a “resilience refresh session” with upbeat music and branded smoothies.
It would have let her sit by the river.
Your Breath, Your Business
We shouldn’t have to perform health to prove our worth.
We shouldn’t confuse “data-driven” with “deeply caring.”
We shouldn’t let emotional nudges override emotional needs.
Sometimes the most powerful act of wellbeing is not to optimise… but to stop.
Let’s choose technology that supports silence, slowness, and humanity.
What You Can Do:
→ Notice when your health is being performed rather than lived.
→ Ask if your tech is measuring wellness—or managing behaviour.
→ When an app tells you to smile, ask: Who benefits from that smile?
We hope you’ve enjoyed this week’s post in our “Who’s holding the remote control?” series. Each story explores a different fork in the road between AI that works on you, and AI that works for you.
If you missed the earlier parts, here’s some links:
Part 1: Travel
- “The Lure of Convenience”Part 2: Home
- “Relax, Just Let Me Handle Things”- “Now Entering a Pre-Approved Natural Moment”
Part 4: Money
- “When AI handles your money, who profits?”Part 5: Learning
- “Smart Tools, Dull Minds”
Come back next week to explore what happens when personalised AI reshapes your life at work.
Thanks for reading — please add your comments below!