Casting Your User Research

There’s an old adage that 90 percent of filmmaking is in the casting. Throughout the process of making a movie, doing the work up-front to get the right performers pays off and ultimately leads to a superior result.

We’ve found this adage also proves true when we’re conducting user research, because the quality of the results comes from selecting the right users at the project’s outset. No matter how well designers observe and analyze users, it’s all for nothing if they study the wrong people. How can designers ensure they cast the right users?

Let’s begin by thinking about the desired end result of user research. When we recruit users at Adaptive Path, the main goal is to select users who would likely use our client’s site or product and observe enough of them so that trends emerge from their collective behavior. Ideally, we try and create designs primarily based on these user trends.

Clearly, designers of some sites have it easier than others. For intranet sites, designers can observe in-house employees. For customer extranet sites, they can simply call up some of their existing customers.

But, let’s imagine we’re creating a design for VisitFrisco.com, a fictitious site supporting travelers to San Francisco. To be successful, the site needs to support a wide range of user groups: the young and the old, the vacationer and the business traveler, the active outdoors person, the sightseer, even the press. Because it’s impossible to study every potential user, where should we focus our energies?

The first step is to identify key audience types. This does not mean using traditional demographic segmentation. When we work with clients, we identify audiences through their behavior, not demographics. For example, an older vacationer has more in common with a younger vacationer than with an older businessperson.

We can learn a lot from how the marketing team segments customers. Market research can help us identify specific types of vacationers, such as jet-setters, once-a-year travelers, or package tourists. Those distinctions are probably more helpful than using factors such as age and gender.

Next, we need to observe enough users on VisitFrisco.com to ensure that we uncover behavioral trends. At Adaptive Path, we aim for 4 to 6 per audience type. If we observe fewer users, atypical users can easily skew the data. If we observe too many users, we’ll see the same behavioral patterns over and over again, with rapidly diminishing returns.

It would be ideal to study every distinct audience type. Unfortunately, most projects have time and money constraints making this impossible. As a result, when conducting user research, we have to choose those groups which provide the greatest return from both business and research perspectives.

For instance, even though travel agents only comprise a small proportion of all users interested in VisitFrisco.com, their ability to influence travel decisions could make studying them a business imperative. We may also want to focus more on once-a-year travelers, as these users are the hardest to support. If we successfully design for them, we will serve the more active travelers as well.

In making some hard choices, we’ve decided to study 4 representatives from each of these three audience types: once-a-year vacation travelers, business travelers, and travel agents. But we need to get even more specific. We’ll want to find users with a recent travel experience—either they’ve just finished a trip or are in the process of planning one. People who aren’t in a “travel mode” will be poor subjects, as their responses will be abstract and ideal, and our best observations will come from people with actual travel experience.

We’ll also want to find users who live far enough away from San Francisco, where travel will require serious planning. However, we won’t need to find people who are planning to or have recently visited San Francisco—VisitFrisco.com will gain from understanding why people chose other cities.

In order to find the most appropriate people, we write a screener, a formatted series of up to 20 questions for screening potential users. We order the questions from general to specific so that we can weed out as many people as early as possible in the screening process.

In addition to filtering based on the criteria mentioned above, in almost every user research study, we try and exclude people who work in market research and usability, people who have taken part in a similar study recently, and people who are not articulate (good talkers make good user research subjects).

Next, our recruiter will use the screener as a resource to go out and find the right people. You could do your own recruiting, but be warned—recruiting research participants is an arduous task, involving lots of phone calls, emails, leaving messages, and calling people back.

At Adaptive Path, we prefer to work with a professional market research recruiter with experience finding people for focus groups. These recruiters have methods for finding just the right people, and charge anywhere from $100-$150 a head. For our research, that could mean spending $1200-$1800, but that’s a small price considering the amount of time it would take us, particularly with all the other responsibilities we have in our jobs. In our experience working with recruiters, it takes one week to schedule users.

All this up-front preparation may seem like a lot of time and energy. However, it’s worth it. By creating a good *casting* process, we guarantee that we’ll get excellent data from our research, ensuring that we please the product’s business owners and the users.