←Back to writings
Measuring the Success of UI Changes
In a recent workshop i was part of, we were exploring a question that comes up more often than you'd think: how do you measure the success of a UI change when the impact isn't immediately visible in the numbers?
It's a tension I find genuinely interesting. As UX and product designers, we're often asked to justify decisions in terms of conversion rates or throughput metrics that are clean, reportable, and fast. But some of the most meaningful design changes work on a different timescale, and through a different mechanism entirely.
It made me think of a case study I keep coming back to.
The Problem
Doug Dietz, a designer at GE Healthcare, created a technically advanced MRI and CT scanner. But when he visited a hospital to see it in action, he saw a young girl crying and needing sedation before entering the machine.
The scanner worked perfectly from an engineering perspective yet it failed emotionally and experientially for patients.
The “UI” Redesign
Dietz partnered with the University of Pittsburgh’s Center for Innovation and IDEO to redesign the entire experience, not the technology.
They introduced themed environments such as:
- Pirate Adventure for CT scans
- Jungle Adventure and Space Voyage for MRIs
Rooms were transformed with murals, lighting, and sound effects. The scanner tunnel became the entrance to a pirate ship or spaceship turning fear into curiosity and play.
Changing What “Success” Meant
Originally, success was measured in operational terms, how many patients could be scanned per day, or how efficiently the machines ran. The assumption was that improving throughput was the goal.
After the redesign, that definition shifted:
From: Reducing scan wait times
To: Reducing sedation rates and improving children’s emotional experience
And that change revealed the deeper impact of visual design change:
- Sedation rates fell from ~80% to under 10%.
- Children went from fearful to excited about “going on an adventure.”
- Satisfaction scores and hospital throughput both improved — not because the process was faster, but because children were calmer and more cooperative.
It became a textbook example of how empathy and storytelling can drive measurable outcomes.
The Emotional Metric
For Doug, the most meaningful moment wasn’t in the data or GE’s bottom line. It came when a mother told him how her six-year-old daughter had just completed her “pirate ship” MRI scan. The little girl tugged her mum’s sleeve and asked:
“Mommy, can we come back tomorrow?”
That simple question captured the true success metric a shift from fear to joy.
The Takeaway
When evaluating UI changes, we may need to rethink what we measure.
Instead of asking only “Did conversion improve?”, we can ask:
- Did people feel more confident navigating?
- Did they hesitate less?
- Did the experience reduce stress or uncertainty?
These emotional and behavioural signals are often leading indicators of later business outcomes such as acquisition or retention.
←Back to writings
Measuring the Success of UI Changes
In a recent workshop i was part of, we were exploring a question that comes up more often than you'd think: how do you measure the success of a UI change when the impact isn't immediately visible in the numbers?
It's a tension I find genuinely interesting. As UX and product designers, we're often asked to justify decisions in terms of conversion rates or throughput metrics that are clean, reportable, and fast. But some of the most meaningful design changes work on a different timescale, and through a different mechanism entirely.
It made me think of a case study I keep coming back to.
The Problem
Doug Dietz, a designer at GE Healthcare, created a technically advanced MRI and CT scanner. But when he visited a hospital to see it in action, he saw a young girl crying and needing sedation before entering the machine.
The scanner worked perfectly from an engineering perspective yet it failed emotionally and experientially for patients.
The “UI” Redesign
Dietz partnered with the University of Pittsburgh’s Center for Innovation and IDEO to redesign the entire experience, not the technology.
They introduced themed environments such as:
- Pirate Adventure for CT scans
- Jungle Adventure and Space Voyage for MRIs
Rooms were transformed with murals, lighting, and sound effects. The scanner tunnel became the entrance to a pirate ship or spaceship turning fear into curiosity and play.
Changing What “Success” Meant
Originally, success was measured in operational terms, how many patients could be scanned per day, or how efficiently the machines ran. The assumption was that improving throughput was the goal.
After the redesign, that definition shifted:
From: Reducing scan wait times
To: Reducing sedation rates and improving children’s emotional experience
And that change revealed the deeper impact of visual design change:
- Sedation rates fell from ~80% to under 10%.
- Children went from fearful to excited about “going on an adventure.”
- Satisfaction scores and hospital throughput both improved — not because the process was faster, but because children were calmer and more cooperative.
It became a textbook example of how empathy and storytelling can drive measurable outcomes.
The Emotional Metric
For Doug, the most meaningful moment wasn’t in the data or GE’s bottom line. It came when a mother told him how her six-year-old daughter had just completed her “pirate ship” MRI scan. The little girl tugged her mum’s sleeve and asked:
“Mommy, can we come back tomorrow?”
That simple question captured the true success metric a shift from fear to joy.
The Takeaway
When evaluating UI changes, we may need to rethink what we measure.
Instead of asking only “Did conversion improve?”, we can ask:
- Did people feel more confident navigating?
- Did they hesitate less?
- Did the experience reduce stress or uncertainty?
These emotional and behavioural signals are often leading indicators of later business outcomes such as acquisition or retention.