Taking the Netflix Experience to a New Level: An Interview with Sean Kane

UIE’s Jared M. Spool had the chance to talk with expert user interface engineer, Sean Kane, to discuss the benefits of a truly usable web site, measuring web site changes, moving from a large corporation to a start-up company, rapid iterations during product launch, and hiring the right people.

Sean Kane is co-founder of GetListed, a new start-up company that is set to revolutionize the way people find and fill jobs in the employment marketplace. Before starting GetListed, Sean was the former Director of User Interface Engineering at Netflix, where he led the development of Netflix’s award-winning web user interface, which was rated #1 in customer satisfaction by independent researchers five consecutive times.

UIE: During your time working as Director of UI Engineering at Netflix, the service grew tremendously. How many users did Netflix have when you first started?

Sean Kane: I was at Netflix for five years. During my tenure, Netflix grew from 500,000 to about 7 million members. I saw the company move from its old little building that was falling apart to its new glamorous headquarters. There was a lot of change over those five years, a lot of it for the better. I learned a ton about the way to build really fabulous web sites.

UIE: Netflix is a business that runs entirely off its web site. If the site isn’t working, is the company completely incapacitated?

Sean Kane: To some extent. But there’s also an offline component to the Netflix experience, which is the shipping portion of the business.

With Netflix, users build a queue of movies they want to see. Then, in a separate system, the movies get shipped off to the user through postal mail. If the site is down because of a technical problem or update, the shipping portion still carries on. People still can receive and return their movies. They just can’t pick them out on the web site. Unlike Blockbuster where people can go into the store, Netflix’s website is the only interface users can interact with. Users have the option of calling customer service or using the website, and that’s it.

UIE: Did you find that the management team at Netflix really understood the value of the website and why it had to be usable?

Sean Kane: Absolutely. In my experience, Netflix’s management is as good as it gets. Reed Hastings, the founder of Netflix, is a very smart guy who focuses on data mining and letting data prove everything. As a result, we were driven to create a compelling user experience across the board, with everything from picking a movie on the website to getting it shipped within a day to your house.

We wanted to create an incredible experience where brand new people show up to the Netflix site and think, “Oh, this might work, this might not.” But then, when the movie appears on their doorstep the next day, it’s always an amazing thing.

We not only wanted to create that experience, but we also wanted to make sure that we deeply understood the “why” of everything that was happening. We monitored and measured any user interaction on the site to ensure that the experience was a better one than any of the alternatives we worked with in the past.

UIE: Did Netflix’s user experience team measure how design changes performed on the site?

Sean Kane: Definitely. Often times, someone would suggest a feature idea that involved a set of sub-features. There are often several different ways to implement the feature idea. When we faced situations with differing opinions on the best way to implement an idea, we took the approach of building each of those experiences out.

We would build four, five, or six different implementations of the same experience and try to determine which one would best achieve the anticipated business goals. We also tried to balance the business goals with our users’ goals. We had a great research lab where we’d recruit people and watch them use the Netflix site. We used the research to inform the subtleties of how we go about building experiences.

We focused on the quantitative data, such as looking at how people were interacting with site features, whether users were adding more movies to their queue, and if they got their first movie choice more often.

Typically, one team member would come up with an idea that everybody else would disagree with. At many companies, that might become a political battle where people angle to get their idea accepted. But since we built all of the different implementations of an idea, we could see which implementation was most effective. There’s no room to argue when the data makes it quite clear. It’s been a huge learning experience watching how people interact with web sites.

UIE: You recently left Netflix and moved over to GetListed, a brand new startup. What is it like to be in a startup after being in this organization where you had all these resources? Has it been a shock?

Sean Kane: To some extent. The part that I’m really enjoying is that I have to wear many different hats in my new role instead of focusing on just the UI engineering of the site. I’m doing everything now and it’s forced me to learn about many things I haven’t given much thought to in awhile, including writing out product specifications, talking to vendors, and getting people in the door to review technologies. It takes a lot more personal time and I have to manage my time a lot more carefully because we’re still really trying to crank that big wheel.

The biggest challenge is handling all of our ideas. We have two years of features we want to build. We have to prioritize the most essential things to get out the door. Once out the door, we need to iterate so that we’re constantly evolving the product.

When we launch what we’re doing, it’s not going to be polished. It’s going to have some quirky elements and some features that aren’t fully realized yet. It will be very important after we launch to iterate constantly and show that the site’s improving. We want to keep our members engaged and interested in what we’re up to. I think that when users find a gem of a new site and constantly see it improving, they feel like they’re a part of that experience.

UIE: Do you have a vision that goes out several years in terms of what your product is going to be like? Have you segmented your product down to the small piece of what’s going to be the initial launch?

Sean Kane: Yes. We want to prove that we’re on the right track as quickly as possible. This approach is similar to how Netflix thinks about things. We try to implement things as fast and cheaply as possible because we know our design decisions will sometimes fail. The more we can reduce the time and cost of those failures, the more we can attempt to get things right. If we spend six or eight months developing something and it doesn’t work, we’ll have wasted a lot of time and money, with nothing to show for it.

There are a lot of advantages to evolving an experience over time. If we conduct a bunch of rapid iterations on things, users aren’t caught off guard by the fact that the entire site’s different and they have to re learn it all. The experience evolves over time versus just doing something that’s not going to change at all.

From my perspective, it’s fine to release something that’s not 100%, because 100% is going to cost a lot of money and take a lot of time. If we launch something that does what it needs to do, within a few quick iterations it starts to do what it does really well. We’ll then start enhancing the product and making it much more robust for down the road. It allows us to get to the final result without spending as much money and we learn a lot along the way.

UIE: Is being in a smaller organization giving you any sort of flexibility that you didn’t have before?

Sean Kane: Yes, definitely. At a larger organization, it’s harder to get some personal, passionate stuff out because of the business priorities, unless I did it as skunkwork projects. But right now, I know what needs to be done. It’s up to me to set those priorities and figure out how everything’s going to happen.

UIE: As you’re choosing the people to be on your team, are you thinking about the functionality you’re building and trying to match people to the specific functionality? Or are you trying to build a more general team that is going to be capable of all sorts of things that you haven’t even imagined yet?

Sean Kane: I learned at Netflix to always hire great, super bright people, because they have this innate ability to try new things. And it’s not really that big of a challenge. They enjoy learning new things.

We want to build our product in Java using the Tapestry framework. We have some other things we may want to do with Adobe AIR or some other technology that’s out there, and so we have “must haves” for the technology components. If we decide to change direction in the future or try new technologies, if I recruit bright folks, they can change on that dime with us. Because one thing is certain: we are a startup so we need to adapt very quickly to any changes.

UIE: How do you find people who meet that criterion of just being really bright?

Sean Kane: When you post a job ad, there’s a lot of luck in play. Some of it is on instinct and some of it is when somebody actually starts proving themselves in their ability to deliver. You can get at some of that though references and through the interview and just getting a general sense about somebody. But at the end of the day, it’s all about how somebody actually performs.

I also try to keep in good contact with the people I have interacted with in the past, or I’ve worked with before because your network is your best resource. So it’s always important to keep in contact with really great people. You may not work with them within the first couple years that you’ve met them, but you never know when things are going to change.

I’ve had some success stories with people that I’ve talked to for many, many months, and sometimes a year or so. And finally, when the situation was right, because that warm tie was there, I was able to get a great person in the door.

UIE: As you’re building out this team, are you thinking of it in terms of specific roles? Or are you going for generalists who are just really bright and can basically do everything?

Sean Kane: We have an alpha prototype built right now that was created by a designer and a few external consultants. We have designs, our logo, and some technology to help us prove our model and to make sure that it’s going to work like we think it will.

We also have a secondary platform that we’re going to build to scale, which is our beta product. I have the luxury of leveraging the designs that have been put into the alpha prototype. At the moment, I’m not looking to fill designer roles. I’m working on the information architecture piece.

There are specific roles where I’m looking to place somebody that really knows the technology. However, there’s always room for a really great generalist that can get everything done and help everybody else in their various areas.

Down the road, we’ll start looking more seriously at hiring designers and product managers and start growing things. That’s when we’re going to start looking at the various components of the site and think more deeply about how we can implement the various features on our various pages.

UIE: At the Web App Summit in March, you’re going to talk about your experiences at Netflix and GetListed. Can you tell our readers what you’re hoping to share?

Sean Kane: I will be giving specific examples of the things I learned from Netflix that I’ve successfully applied at GetListed. Since we haven’t launched a product yet, we can’t do a lot of testing at GetListed yet. Instead, I’m looking at how we can achieve our end goals and get the site up and running before we have a lot of traffic.

One of the common questions I get when I give presentations is, “Sure, at Netflix, you guys have all this traffic, so it’s easy for you to do the quantitative testing and get statistically significant results back. I’m a startup. I don’t have very much traffic. How do I deal with that?”

I didn’t really have an answer to this question in the past. But at GetListed, I’m figuring out a way to answer that question. At the Web App Summit, I hope to have that question answered and share what I’ve learned with attendees.