Streamlining the Design Process with Paper Prototyping: An Interview with Carolyn Snyder

You can’t think of paper prototyping without acknowledging Carolyn Snyder’s influence on the technique. Her many years of experience teaching and utilizing paper prototyping have made her a recognized authority figure in the field. In 2003, Carolyn wrote the definitive book, Paper Prototyping, which explores many of the method’s nuances and acts as a complete guide for design teams.

Carolyn has 23 years of experience in the software industry and 14 years experience in user-centered design, usability testing, and paper prototyping. Prior to her major publication, Carolyn spent 6 years working as a principal consultant for User Interface Engineering before founding Snyder Consulting in 1999. She continues to teach, write articles, and present at major conferences while remaining active in professional organizations. Carolyn received a BS in Computer Science from the University of Illinois and an MBA from the University of Chicago.

Recently, UIE’s Ashley McKee talked with Carolyn about the increased popularity of paper prototyping, what the technique is, who can use it, and how it’s beneficial to the design process.

UIE: More and more organizations are incorporating paper prototyping into their development process. Is there a reason for this technique’s increased popularity?

Carolyn Snyder: I think it’s simply that more and more people have tried paper prototyping and found it useful. Fifteen years ago, when I tried to explain to design teams that they could prototype a computer interface without using a computer, they looked at me like I had three heads. These days, many designers are aware of the technique even if they haven’t used it.

It may be partly because we live in the era of Google and Wikipedia, which makes it easy to conduct a bit of quick research. Also, as the usability profession matures, I believe efforts like the UPA’s Body of Knowledge project (www.usabilitybok.org) will be an increasingly useful resource for design teams wanting to learn more about accepted usability testing methods.

What kinds of interfaces can you prototype?

By definition, if it has a user interface, you can prototype it. Design teams can prototype software, web applications, web sites, and handheld devices.

However, paper prototypes are less suitable for visually complex or highly interactive interfaces such as video games. It’s also more challenging to prototype mobile devices or those that operate in a rich contextual environment. But for the vast majority of business and consumer applications, paper prototypes are quite useful.

What types of problems can paper prototype tests find? What won’t they find?

Paper prototypes are great for finding most usability problems including issues with concepts, terminology, navigation, workflow, and screen layout. Although usability testing isn’t the best method for researching user needs—you should do that before you design—if you missed a requirement or got one wrong, a paper prototype will reveal it.

Paper prototypes are less useful for rapid, subtle, or complex interactions because the human “Computer” can’t respond as quickly and unobtrusively as a real computer. Low-level interactions, such as scrolling, are also hard to mimic. And because color and solution are different on screen than on paper, graphic designers may not get all of their questions answered with a paper prototype.

In your opinion, what team members are best suited for paper prototyping and usability testing?

Anyone who wants to understand the product and its users better should be involved in the testing process, including the designers, developers, tech writers, trainers, marketers, and managers.

On most projects, a subset of team members consisting of a few designers and a technical writer create most of the prototype and complete the other prep. However, I encourage everyone to review the tasks and attend at least two usability tests if possible. Watching only one test is better than not watching any at all, but if the test turns out to be atypical, it can skew the observer’s view of what needs to be changed.

What skills does someone need to create and test a paper prototype?

Well, you don’t need artistic talent—I’m evidence of that. Athough if you’re creating the interface, it’s best if you know something about user interface design. The skills needed to test the prototype are the same as those needed to facilitate any other type of usability test, namely the ability to interact pleasantly but professionally with the users in a manner that doesn’t contaminate your findings.

What recommendations do you have for usability professionals who want to get their development team excited about usability testing and paper prototyping?

In my work, I’ve found that some of the development team members really need to see the technique in action to understand how paper prototyping works. Create a small-scale prototype and show the design team.

Often the developers are concerned about showing an unfinished design and having it torn apart. Reassure them that usability testing is a way to get their own questions answered, and that it’s equally important to learn what works, not just what’s broken. For those with an eye on the schedule, discuss how it’s less painful to fix problems before launch rather than after.

Many designers may be thinking, “I’m proficient with Visual Basic and other electronic prototyping tools and create working prototypes with them fairly quickly.” Why would they want to use paper prototyping?

The real question is what method will provide good data with the least amount of effort. If there are just one or two designers who can mock things up very quickly in their favorite tool, the case for a paper prototype may not be compelling. For a larger or multidisciplinary team, paper prototyping may be a more efficient way to get a lot of ideas on the table.

Also, consider what it takes to set up the test system, including realistic databases, reset procedures, and user accounts. Sometimes these things are trivial and other times they can take more time than creating the prototype itself. Development is often a process of two steps forward and one step back. If many parts of the system are in active development, your usability test might be derailed by last night’s buggy build. In that situation, a paper prototype gives you more control over your testing schedule.

Many design teams face the decision of whether to create a paper prototype or test the existing product. What do you suggest?

I first try to find out whether the team agrees about the existing product’s problems. If so, the team is probably ready to explore solutions, and paper is great for that. If there’s disagreement or uncertainty, sometimes it’s best to stick with testing the existing product and understand it better first. Otherwise, the risk with a paper prototype is that team members might go off in several directions that don’t prove fruitful.

Here’s one of my favorite tips. Whenever someone suggests a clever design idea, I ask, “What problem are you trying to solve?” If the person has a clear answer, they’re ready to prototype. If they don’t, getting them to back up a few steps and test the existing product first might be a better use of time.

Do you recommend that design teams incorporate paper prototyping at the beginning of every project? Are there certain projects where you recommend design teams skip directly to testing an electronic prototype?

I never make definitive blanket statements. (Well, except for that one.) As a consultant, I get involved in a project when the client calls me, which isn’t always as early as I’d like. In an ideal world, it’s great to explore different ideas with prototypes, though they needn’t necessarily be paper ones. However, if the working version is fairly far along and there aren’t compelling reasons to avoid testing the real thing, then we usually suggest skipping paper prototyping.

I also look at the resources (people, time, and tools) and try to determine what method will be most efficient for that particular team. Most importantly, my recommendations depend on what the team’s questions are. Although many questions can be answered with a paper prototype, some questions require a more finished electronic one.

When you finish a usability test, many teams participate in a debriefing meeting to go over the data collected. One method of data collection is the affinity diagram, in which a design team sorts the top individual observations into related groups. Is this method superior to other debriefing methods? Are there any cases in which this method proves inferior?

This question actually applies to any usability test, not just a paper prototype. Affinity diagrams have two main benefits: they help you find patterns in a large set of qualitative observations, and they are a consensus method. Affinity diagrams work well when there are several observers with different perspectives and you need to agree on the priorities.

That describes the typical situation I face as an outside usability consultant. Other teams have a meeting where they create a big spreadsheet, or they send all their observations to one person who filters out duplicates and creates a master list. Those methods are fine too, as long as they result in agreement about what needs fixing.

Can you give me an example of a project you worked on where you incorporated paper prototyping into the process? What were the challenges that paper prototype testing helped solve, and what did you learn?

A while back during my days at User Interface Engineering, I worked on a pre-launch paper prototype of Priceline.com. The concept was radical at the time—you tell this unknown site where you want to go, how much you’re willing to pay, and you give it a credit card number, all without knowing what flights are available. Naturally, the development team wanted to know whether people would understand and accept this rather backwards purchasing process.

The good news was that people did “get it,” though as expected, they had some questions about how it worked. We also learned what kinds of things would help them trust that the site (unknown at the time) was in fact legitimate. These issues weren’t difficult to solve, but it took a few iterations to get it right.

The bad news was that, in its initial design, Priceline.com took up to three days to respond to the user about whether they received a plane ticket or not. Even for leisure travelers, this was clearly unacceptable. If the design team launched the site with the 3-day window, Priceline.com was bound to flop, but fortunately the issue was discovered in time to do something about it.

Has your philosophy changed at all since you first began usability testing paper prototypes?

Yes, some. I used to think paper prototyping was good for everything and my main question was how to get the client to embrace it. I now have a better understanding of what paper prototypes will and will not find, and my main questions are about the scope of the project, the composition of the team, what they want to learn, and whatever constraints may exist. When you get a handle on those things, you’re in a better position to pick the best method.