Usability Testing Delivers More Value Before You’ve Changed Your Design

by Jared M. Spool

Teams faced with only a single opportunity to conduct usability testing often decide to schedule those tests after they’ve made changes to their design. They think the testing will help them work the kinks out of the changes, giving them a final chance to polish it up before the final release.

Unfortunately, this rarely works out well. True, the tasks in the testing do focus on the changes, as they should. And the team may find a problem or two that could use refinement.

However, the team is likely to miss bigger problems in the design because they’ve focused the research too narrowly. Or worse, they’ll uncover those bigger problems, but not have enough time and budget left to tackle them, having spent it on the design changes.

We recommend that teams skip the “validation test” to identify and clean up these small elements. We tell our clients to hold off spending that usability testing budget until after the design has shipped, then do research on how it’s being used by the users.

Doing the testing after it ships gives the team richer insights into what the next set of design changes should be. This does imply that the decisions they made for the first set of changes weren’t as informed as they could’ve been.

That’s why the best course of action is to conduct the research before deciding what changes you’ll make in the design. This research doesn’t have to be expensive. It can cost the same (or possibly less) than you were planning for the validation usability test.

Your research should be open ended. You should not assume the team understands how users are really using the design today. Instead, your research should explore what’s happening in the real world. Ideally, you’ll look at both new and existing users. (Inherent Value Tests are a great way to do this.)

By researching early, your team learns what to change. You’ll learn what to keep (because users love it). And you’ll learn how to structure the work.

It makes it easier to build the requirements, define the use cases, and even create QA test scripts, because you can drive all those things right off what you saw in the research. It will likely reduce your development costs because you’ll have data to make decisions, instead of driving everything off some strong-willed individual’s opinions of what users need.

Pushing your user research as early as possible in the schedule is the best way to get value from your efforts.

Published here on April 6, 2018.