How a UX Team Discovered an Underserved Audience and Made Millions for Their Organization

A case study about how a team became strategic and everything changed.

I’m going to tell you the top-line takeaways of this case study right now:

A UX team pushed their comfortable tactical practices aside and embarked on a strategic UX research project to tackle a years-old business problem…

…and discovered a large, underserved audience that few in their organization knew existed… 

…for which they proposed new capabilities that could substantially improve the lives of the people in that audience…

…which frightened their organization’s senior management due to the effort and the risk…

…so they put together compelling quantitative and qualitative data that showed that the risks could be worth it…

…and that substantially reduced the stakeholders’ fears…

…who made implementing and delivering the new capabilities the organization’s top priority…

…and then they saw that the new capabilities exceeded everyone’s expectations…

…and added millions of dollars in new recurring revenue in the first months…

…which changed everything for the UX team.

There you go. Now, you don’t have to read this exceptionally long case study. Unless, of course, you want to know how this UX team pulled this off.

The business objective: Improving retention

The UX team is part of a successful entertainment streaming service provider that streams TV shows, movies, games, and sports to hundreds of millions of subscribers. (Unfortunately, I can’t reveal this service provider’s name for reasons that should become obvious.)

Each year, a tiny percentage of subscribers choose to cancel. However, 1% of hundreds of millions of subscribers is still in the millions. So, we’re talking about a considerable number of cancellations.

The service provider has had a top-priority goal of improving retention for many years. After all, keeping their subscribers paying at this scale is worth millions of dollars, even if it is just for a few more months.

The product team has tried many strategies to address this top-priority retention goal, with unimpressive results. No matter what the team attempted, the rate of cancellations barely decreased. With each retention effort, the product team was sure this time would finally move the needle. It didn’t, and they never knew why.

Something new to try: Deep hanging out

Something was missing. Despite working on this project for years, there was one thing the product team never did. They never asked their subscribers why they were cancelling.

A senior researcher on the UX team decided to tackle the retention issue, using something they’ve never tried before: strategic UX research techniques. In the past, the team had only conducted tactical research methods, like usability tests, multivariate tests, simple interviews, and surveys. 

However, these methods never revealed why so many people cancelled the service every month. And staring at reports from the service’s analytics tools didn’t deliver meaningful insights.

The researcher proposed to the UX team that they conduct “deep hanging out” with people who had recently cancelled. They hoped to get a deep understanding of a subscriber’s cancellation experience. What triggered the subscriber’s desire to cancel, how did they go about it, and what were the results?

The UX team was excited to join the researcher on this adventure. They didn’t ask permission. Instead, they just set out on the adventure to understand why all these people were cancelling.

They started by reaching out to people who had recently cancelled their subscription and asked if they could come by and hang out to learn about their experiences. To the team’s surprise, several past subscribers said yes. 

That’s when they met Marina, a single mom with two young children and a subscriber who recently cancelled the service.

The revelation: Marina’s experience

Before this research project, the Product team had tried many different ways to reduce the churn of cancellations, none of which worked. These various attempts had the same underlying speculation about why people were cancelling.

They believed most subscribers cancelled when they no longer found the service valuable. They thought there was something about the service that was off-putting, leading their subscribers to feel that the monthly fee wasn’t worth it. 

Not surprisingly, this belief didn’t come from research. It was just what the product managers assumed was true.

Because that was the unquestioned underlying belief, every effort the Product team made to increase retention was some form of convincing subscribers that the service was worth it. They tried implementing new approaches that explained “hidden” benefits of the service, but that didn’t work. They tried adding new content and capabilities to make the service more interactive and exciting — that didn’t work either. 

Nothing that the Product team tried had worked. And now, having met Marina, they could see why.

Marina loves the subscription service. She told the research team that she tremendously enjoyed the content. She loved binge-watching her favorite shows. 

She thought the service was worth it. That’s not why she cancelled.

She revealed to the team that she had cancelled because she was on a limited financial budget. She couldn’t afford to pay for the service every month. 

Marina told the team she had to save up to “treat herself and her family” (these were her words). When she had enough savings for her treat, she would resubscribe and watch her favorite shows.

However, Marina would only subscribe for a single month at a time. During that month, she and her family would binge-watch everything they could. 

Then, at the end of the month, she’d cancel and start saving up again, which could take a month or two. She’d later treat herself to another month when she saw that her favorite shows had a new season or an update.

The realization: They are not their subscribers

In all the years of focusing on improving retention, the Product team only concentrated on subscribers who no longer thought the service was valuable, even though they could easily afford it. It never occurred to the Product team to focus on subscribers who loved the service and thought it was great value but couldn’t afford it.

It never occurred to the Product team because Marina’s life experience differed significantly from their own. Without any research, they tried to imagine why they might cancel a service like this. 

Because the Product team’s management was all highly paid professionals, often with two substantial household incomes and a net worth in the millions, they assumed a subscription was easy to afford. They had never personally been in a situation where they couldn’t afford to spend $25 a month on something they valued.

Marina’s situation marked the Product team’s first encounter with how people with financial constraints use their service.

The deep dive: What it’s like on a restricted budget

Sitting in her home with the research team, Marina demonstrated how she cancelled her account and reinstated it later. It was eye-opening.

The team learned that when Marina needed to cancel, she had to click through a long gauntlet of annoying offers the service provider had added over the years as remnants of their past retention efforts. The team saw that she was asked multiple times if she genuinely wanted to cancel. The service pestered Marina with several different offers to entice her to stay. 

Marina complained about how horrible the offboarding flow made her feel. She wanted to stay — she really did. If she could afford it, she wouldn’t need the offers. Each offer made her feel ashamed of her financial situation. 

She also showed the team what it took to reinstate her account. When she wanted to sign up again, the service, which was designed for onboarding brand new customers, required her to jump through a lengthy setup process. She was forced to use repetitive click-through tutorials, as the design assumed Marina was unfamiliar with the service, even though she’d used it many times before. 

It took several minutes for Marina to set up her account how she liked. Frustratingly, the service didn’t remember any of her past history and preferences because it was a new account. She couldn’t see which episodes she had already watched or receive notifications about new content she might like. Because she had previously cancelled the service, it had essentially erased her history. 

Marina shared with the team another aspect of the service that constantly stressed her out — missing her account’s monthly renewal date. If she didn’t cancel before that date, she risked paying for a month she didn’t want. Having to pay attention to this stressed her out.

She said she had missed the renewal several times in the past. Each time, she had to call customer support for a refund. The support representatives were never courteous. They would resist her request to undo the automatic renewal, though they’d eventually give in and refund her. Their resistance made her more ashamed of her financial situation. 

Marina told the research team how she was reluctant to restart her subscription. She loved the service but didn’t want to deal with that fuss again. She said she didn’t like feeling inadequate because she had to cancel all the time.

Marina’s situation blew away the research team. Here was someone who didn’t fit their preconceived notions about why people were cancelling their subscriptions. 

She loved the service and found it valuable. Signing up for a month was a treat. However, she was cancelling because of her financial situation, not how she felt about the service.

The discovery: Marina was not unique

By complete chance, Marina was one of the first cancelled subscribers the research team talked with. They immediately wondered if her situation was unusual.

Naturally, they wondered if these financial constraints were rare among their users. Marina’s experience was a situation they’d never seen or even considered before. 

The team quickly discovered that Marina’s situation wasn’t unique. As they continued, they met many subscribers who stopped and restarted the service like Marina did, with similar frustrations and feelings of shame.

That’s when the UX team started working with the Data Science team. If the research team kept finding people on restricted budgets, how many other cancelled subscribers were also limited by their finances? 

With clever investigation, the Data Science team found persuasive evidence that most of the service’s cancelled subscribers behaved like they were on restricted budgets. In their system data, they could see millions of people cancelling, then a few months later, signing up again, often repeating this multiple times a year. 

The research team continued to reach out and talk to the cancelled subscribers, encountering many who had similar behaviors to others on restricted budgets. Sure enough, these subscribers shared that they had cancelled because they couldn’t afford to stay subscribed for over a month or two.

Like Marina, many spoke about the process as “treating themselves.” They also expressed the hassle of cancelling and reinstating their accounts. And they uniformly commented about how challenging it could be to deal with the customer support team. One UX research team member commented that it was almost like each of these subscribers was reading off the same script.

The Data Science team uncovered another interesting fact: the period between a subscriber cancelling and when they signed back up got longer with every iteration. The research team asked subscribers about this, and they uniformly heard it was the “hassle” of the process that made them reluctant to go through it again.

The frustration of cancelling and reinstating their accounts was training these subscribers that the service wasn’t worth the hassle.

The idea: Build a ‘Pause’ capability

If the UX team could reduce the hassle of the current cancelling and reinstating process, they could see these restricted budget subscribers treating themselves to one or two more months each year. Reducing the hassle could result in tens of millions in additional revenue.

One of the senior executives encouraged the UX team to explore how they could change the service to reduce this hassle. This project was the first time the UX team undertook a design improvement that wasn’t already in the Product team’s roadmap. 

Never before had significant new enhancements to the service come directly from UX research findings. In the past, the Product team originated every service enhancement, often based on nothing more than a hunch and usually without consulting with the UX team first. 

This time was different. This time, the UX team originated the idea based on data from their strategic research. (And, yes, the Product team was involved in the research from the beginning.)

After studying options, the UX team proposed a new capability: ‘pausing’ the service. 

Pausing would work similarly to cancelling, except the subscriber’s account would remain active, while any automatic renewals on that account would temporarily pause. Subscribers would not make any monthly payments while they remained paused.

With this new functionality, the next time a subscriber, such as Marina, visited the current ‘Cancel’ screen, a new Pause button and an explanation would appear. Marina could now pause her subscription at any time. The remaining time left in her current month would become a pro-rated credit, which the system would apply to the next time she unpaused her account.

When Marina wanted to return to treat herself, all she had to do was find a show to watch and hit ‘Play.’ The system would then let her know how many days remained from any previous credit and when the service would automatically renew. 

When she was done binge-watching, she could hit pause again and repeat the process. Marina would no longer go through the retention gauntlet when she left or experience the onboarding hassle when she returned. Ideally, Marina would not need to contact customer service about a refund.

Pausing would improve Marina’s life tremendously. Removing the friction and the shame-inducing interactions would make it easy for her to return when she wanted to treat herself again. This new capability, built for her situation, would increase overall retention revenue, even if it wasn’t from a contiguous subscription, making it a win-win for Marina and the service provider.

The first reaction: Stakeholder resistance

It was not smooth sailing for the new ‘Pause’ capability. Immediately, there was resistance from critical stakeholders.

After all, this new capability would touch substantial parts of the business. Development, Marketing, Content development, and even Legal would need to get involved. There were many moving parts to implementing this functionality.

Development would need to account for the subscriber’s outstanding credits and turn off automatic renewals until they return. Marketing would need to develop enticing invitations to return, based on what the subscriber had been watching. Content development would be under pressure to add new seasons of favorite shows. Legal would need to state the implications of a paused account in the terms of service.

Implementing this functionality was a big deal. Could everyone pull it off? What if they didn’t?

The stakeholders also worried there wouldn’t be enough subscribers to take advantage of this new functionality. They would need millions of subscribers to pay a few additional months each year to make this worth the investment.

And some stakeholders feared this new capability would attract wealthier subscribers who didn’t have a restricted budget. They could reduce their annual payments to save a little money, even though they had been fine up until now without this capability.

The response: Mitigating risk with UX metrics

The UX team turned to metrics to make the case and show the value of this new functionality.

The team collected key UX metrics about the current experience for restricted budget subscribers to demonstrate that this work was worth it. They settled on:

• The number of people cancelling subscriptions each year.
• The number of people who shared in a newly added cancellation-flow survey that budget limitations were why they cancelled. (The survey had been there, but they’d never thought to ask about the subscriber’s budget.)
• The number of people who had reinstated their subscription and then watched new content.
• The annual revenue from folks who had cancelled and reinstated.

Working with the Data Science team, the UX team collected the data for these metrics to give their qualitative findings a sense of scale. They already found the data that showed millions of subscribers had cancelled and reinstated their subscriptions at least once. Stakeholders were astonished by these new metrics.

Particularly surprising was how much existing recurring revenue was from subscribers with restricted budgets. People were putting up with the friction and getting value from the service. It was clear that reducing the friction would increase revenue.

These metrics, plus Marina’s story and the rest of the qualitative findings, made the case for the stakeholders. The company’s executives fell in love with Marina’s story and the idea of reducing the friction. They also fell in love with the idea that it could be worth millions in new revenue. These executives told everyone that this project was a prime example of the company becoming more innovative.

(About the fear of wealthier subscribers using the pause functionality: This is why they decided only to show it to people after they went to the cancellation page. Hardly anyone goes to that page if they aren’t considering cancelling, so that it wouldn’t be otherwise discoverable. And it’s why I can’t reveal the service provider’s name.)

Implementation and delivery: The project was a success

The executives committed to making the Pause project a top priority for the organization. It was now critical for the Development, Marketing, Content, and Legal teams. A senior vice president assembled a cross-team task force, putting the UX team in charge. (Another first for the UX team!)

As the project was underway, the task force identified and reported on further UX metrics to show the project’s progress:

• The number of subscribers taking advantage of ‘pausing.’
• The number of times paused subscribers returned and ‘unpaused’ to watch new content.
• The number of days between pausing and returning.
• The additional revenue in the first year from paused subscribers who returned.

Once deployed, the numbers started rolling in. Pauses steadily increased over the first few months, while cancellations decreased. Subscribers were returning and watching new shows, taking advantage of their outstanding credits.

Because it was so easy to pause, restricted budget subscribers paused more frequently than when they used to cancel their subscriptions. However, the number of days before returning decreased dramatically. These subscribers were treating themselves more often.

It was that last metric, additional revenue in the first twelve months from paused subscribers, that everyone watched the closest. The good news was that the additional revenue dramatically exceeded everyone’s initial expectations in its first year. 

The Pause project was a success. 

The executives and senior stakeholders saw the value in deeply understanding the user’s experience. They encouraged more UX research efforts to uncover new opportunities to anticipate subscribers’ needs. They pressured the Product team to integrate more UX research (and fewer hunches) into roadmap planning and feature prioritization.

Follow-up: Marina’s new experience

After the release, the research team returned to the field and met with many of their early research participants. Every participant said they loved using the Pause capability. They found it easy to use and didn’t make them feel ashamed.

Marina was particularly excited about the new functionality. She had picked up more shows that she now treated herself to. She loved not having to stress about the automatic renewal or dread dealing with customer support for a refund.

She told the researchers that their streaming service was her favorite now, with all her favorite shows.

Follow-up: The UX team is now working strategically

Discovering the real reasons why people were unsubscribing unlocked new paths to success. It opened the organization’s eyes to what great UX can contribute. 

While the team occasionally conducts tactical UX research, they now invest heavily in conducting far more strategic UX research. They want to become the world’s foremost experts on who their subscribers are, what their subscribers need, and the experiences they have every day.

They’ve also changed their approach to using metrics. They’ve stopped relying on tactical metrics, such as customer satisfaction (CSAT), Net Promoter Score (NPS), task success rates, or conversion rates. These common metrics seldom provided the UX team with worthwhile insights on improving their users’ experiences.

Instead, the team now opts for metrics that map closely to their subscribers’ experiences. While these metrics change with every project, they are much easier for stakeholders to understand. The team has found that strategic UX research with the right metrics can be highly inspirational.

Because the team armed themselves with strategic UX research and metrics, they successfully persuaded the executive team and senior stakeholders, critical allies needed to prioritize and champion this project. Those executives are now eager to see what else the UX team can accomplish.