Designing Products That Work the Way People Work: An Interview with Kate Gomoll

by Christine Perfetti

Kate Gomoll is a recognized expert in the area of Field Research and Usability Testing. UIE’s Christine Perfetti recently had the opportunity to talk with Kate about how she and her team at Gomoll Research & Design conduct field studies. Here is what Kate had to say about her experiences.

UIE: Many of our clients say they stay away from conducting field research projects because they are too expensive and time-consuming to implement. Why should designers invest the time and resources in field studies?

Kate Gomoll: Nothing replaces the power of direct observation. Designers can gain so much information from watching people in their actual work environment and capturing a user’s unarticulated needs. The work we conduct takes place in the participant’s environment—such as home, work, stores, banks, and hospitals. Our findings are based on realities, not preconceptions. As a result, design teams can get much closer to their customers. Users will “show and tell” them things they would incorrectly consider unimportant in a focus group, interview, or usability test.

Also, in our experience, what people say rarely matches what they do. People don’t remember the steps of a process, especially if it’s something they do all the time. They may tell you something is easy, but when you actually observe users doing that task in a field study, you can see all the problems and inefficiencies they didn’t recall or couldn’t articulate.

In your full-day seminar at UI9, you and Ellen Story will demonstrate how to plan and conduct a field study. Can you describe the major steps involved?

Before the field visit, we develop a participant screener to ensure that the people we are visiting are representative users. We always make sure that key stakeholders agree on the mix of participants we will visit so there is no debate about the appropriateness of the participants after the data analysis. We also create a detailed study plan with the session script and all of the materials we need for the study.

A typical site visit lasts 1.5-2 hours. We spend the first part of our visit asking users to prioritize their most important and frequent tasks. We then observe them completing their most high-priority tasks. As we watch the user, we capture the steps, constraints, and tools needed to accomplish their tasks and any social interactions that take place. We also note any process inefficiencies or opportunities for improvement.

Our sessions are a blend of interview and observation. We observe the user as much as possible, but we also gather data from contextual interviews. We have a basic script for the session, with interview questions and observation opportunities. But, we move around in the script, asking questions when they are relevant and weaving observation opportunities between questions. That’s the best way to make sure we get as much data as possible for our efforts.

Do you have any tips for conducting the contextual interviews with users?

My colleague, Ellen Story, and I have worked out a really good approach to “tag-team interviewing.” We have a study script that we work from, but we are quite flexible with it. Ellen will ask a question, and while the participant is answering her question, I will look through the script for the next logical question, based on the participant’s response. This helps the session feel more natural and informal. It also speeds things up, because we can do the same thing with our observation opportunities.

Having our questions worked out ahead of time and reviewed by stakeholders also helps us avoid asking leading questions. Of course, the ability to work with people and draw out their honest responses takes some practice and finesse. The more we do this kind of work, the better researchers we become.

How many customers do you recommend visiting in the field?

We recommend 5-8 participants within a segment. So, the total number depends on how the design team decides to create segments.

How do you determine how many user segments are necessary?

Before we determine the user segments and the number of participants, we typically ask the clients to assemble a group of key stakeholders. This group typically includes people from the development team, marketing, engineering, usability/user interface, and domain experts. We make sure that this group has input and buys into the definition of a segment, as well the criteria for recruiting.

There are three main questions we ask stakeholders:

How many different user groups are there in your target market?
How do you distinguish the user groups from each other?
How greatly do you think the usage patterns and preferences will differ among groups?

Before we conduct any of the field visits, we gain agreement from all of the key stakeholders about the sites and people we visit, so that when the data rolls in, nobody is telling us “you visited the wrong people.”

Many design teams have tried field research but are unsuccessful when the time comes to interpret the piles of data they’ve collected. How do you go about analyzing the data gathered from field observation?

At Gomoll Research & Design, we follow a 3-stage design process of Investigation, Analysis and Design. During the Investigation Phase , we create user profiles and personas and visit customer sites to understand the goals, attitudes, and workplace of the intended users. This is the stage where the field research process fits. When we’re developing our site visit materials, we think about what deliverables we plan to develop after the study. For example, a typical field study will yield scenarios, user profiles, storyboards / task maps, incremental reports, and a summary report.

During the Analysis Phase, we’re looking for patterns in the data. We focus our efforts on finding these key usage patterns and prioritizing design opportunities. We create deliverables that will highlight the patterns and opportunities in a compelling way. This phase includes Workflow Modeling, Task Analysis, and Usability Benchmarks.

Finally, during the Design Phase , we use an iterative process to develop user models and begin prototyping. We gather user feedback and conduct usability tests to help us evaluate and revise our designs.

During each phase, we’re careful to gather and incorporate user feedback so that we design products that work the way people work. That is the key.

(Editor’s Note: You can read more about Gomoll Research & Design’s Process on their web site.

Can design teams gather insights from field visits they can’t uncover with usability testing?

By making direct observations, design teams can discover product opportunities they may never have imagined if they had only conducted usability tests. We observe users doing real work in real time, which enables us to paint an accurate picture of the how users interact with a product or process.

For example, we worked with Sonic Solutions to design MyDVD 4, the leading DVD creation software, to better understand how people create DVDs today. To accomplish this, we conducted observational research in Sonic’s customers’ homes. By observing their customers firsthand, we uncovered common user goals, behaviors, task flows, and the critical problems users encounter when creating DVDs.

What led Sonic Solutions to want to conduct field research?

When we initially met with Sonic’s executives, they were already aware of the power of field research. Their biggest challenge was deciding what kind of users to study. The design team believed that the majority of their users were *early adopters* of DVD creation technology, willing to do whatever it takes to become technically proficient with relevant video and PC hardware and software. For the next generation MyDVD, they wanted to target a larger mass market. But they weren’t sure how well field research on early adopters would help with a design for mass market users.

How hard was it to find customers who would let you in their homes?

To find customers, we sent email messages to registered customers. As is typical, our acceptance rate was approximately 10%. Since the list of registered customers in our target locale was quite large, the recruiting was fairly easy. We are often surprised at how willing people are to invite us into their homes. In this case, most of our participants were anxious to show us their DVDs because they were so proud of them. They were also eager to positively impact the next release of MyDVD and to get a few usage tips.

Were there any findings that surprised the team?

One of the biggest surprises was that the design team’s assumptions about early adopters were not always accurate. We visited users representing a variety of profiles, from a professional videographer, to a teenager whose parents purchased a more powerful PC and related video equipment just to get a VHS tape of his grandfather archived for perpetuity. Most users had a keen interest in video or film media, but little of the supposed relevant technical expertise of an early adopter.

Whether the MyDVD application came with the PC, the camera, the DVD burner, or the conversion device, the users were confused by the multiple shrink-wrapped applications that came with their purchases. Initially, they didn’t understand the total task flow or which parts of which applications to use to accomplish each step in the task. The MyDVD application was just a piece in a very confusing puzzle, not the focus of the user’s attention.

Can you share some of the design improvements that came from the research?

Our research findings led to immediate product design improvements, such as the creation of a “MyDVD Tutorial” to help new users understand the technology, the product, and the DVD-creation process. Our analysis made it clear that all user profiles would benefit from this tutorial, and that the tutorial should cover many aspects of the DVD creation process, even those not directly supported by MyDVD.

Thanks, Kate.

Published here on September 17, 2004.