A visit to Sun's usability testing labs
San Francisco (February 11, 1998) -- Jakob Nielsen, distinguished engineer at Sun's Science Office, invited SunWorld to tour Sun's usability testing labs and to discuss the importance of usability testing.Sun has done extensive usability testing not only on its own software and Web site, but also on competitors' software as well. Sun recently redesigned its Web site with a focus on increasing download speed, facilitating navigation, providing a unified visual appearance, and making search capabilities more prominent. Nielson says that a lot of work was particularly done on Sun's product documentation site, docs.sun.com, to make it much more search oriented.
He says that although a company as large as Sun has the resources to spend on usability testing, even small companies without a large budget ought to do it. "If you don't have a lab you can have two people sit with a user in a normal office." According to him, more than two observers in this situation creates chaos.
Nielson led the way into a dimly lit room that looks like a recording studio sound booth. A one way glass mirror spans the length of one wall. The mirror faces a room with a large table and a lone computer.
"We always take the users into this room first," says Nielsen. "That way they don't wonder what's behind the mirrors." The room also houses a row of a half dozen television monitors connected to video cameras in the observation room that are suspended from the ceiling at different angles.
According to Nielsen, a paper prototype is the best way to begin designing a site. He describes it as a big sheet of paper with a drawing of the Web page. He says, "You ask a user what he would click on first and what he would expect to see when he clicked on it."
He recommends that companies go through two or three rounds of testing for each product developed. He says it is best to test up to five users -- individually, not in a group -- in each round of testing. Some companies are known to spend thousands of hours testing hundreds of users. "It's impossible to create a perfect user interface," says Nielsen. He stresses that rather than testing many users, "more rounds of testing is what's really important." Fix the problems found in the first round of testing and then move on to the next round, and so on.
Nielsen says good usability testing involves three key elements:
- Test people who are representative of your users.
- Have users do representative tasks. For Web site testing this could be
something like, "find the press release on our latest quarterly earnings."
- Keep quiet -- if you are the usability test conductor, your comments to the person testing the product can skew the results significantly. Let the users do all the talking.
What do Web users want out of a Web site? Nielsen recommends that Web architects minimize the artistic element of a Web site. Less graphics create higher speed. "People's patience is going down," says Nielsen. "Web users are extremely aggressive when demanding speed."
Nielsen also cautions Web designers who limit their sites' links to information contained only within the site. "The goal in designing a Web site is not to trap the user." The goal is to maximize value and "make them want to come back."
--Stephanie Steenbergen, SunWorld
Resources
- Jakob Nielson's explanation of Sun's New Web Design http://www.sun.com/980113/sunonnet/
- Jakob Nielson's Usability Test of Advanced Web Concepts http://www.sun.com/980113/sunonnet/concepts.html
- Usability Test of WWW Designs http://www.sun.com/sun-on-net/uidesign/usabilitytest.html
- Interface Design for Sun's WWW Site http://www.sun.com/sun-on-net/uidesign/
- Sun's documentation site http://docs.sun.com