Software
usability testing
By
Jeff Freeman
Why is usability testing so important to the success of a company's software? Today much software is provided over the Internet. If a person does not like the interface or it proves to be overly difficult, the person will likely find another Web site with the same software.
There is much bad usability on the Web. However, people are becoming less tolerant. This will likely cause companies increased revenue loss in the next decade unless corrected.
Jakob Nielson, one of the nation's usability gurus, says "The next 10 years will see the Web become routinely integrated into everyday life, with mobile devices that actually work as advertised and sites where people can find what they are looking for. . . .We have only scratched the surface in terms of making the Web useful and intranets productive."
Nielsen says he measured traffic at 42 sites where usability had been emphasized at the design stage and found that they performed 135 percent better on average.
"There is no reason we should continue to suffer the clumsy user experiences that currently dominate the Web," he adds. "I am optimistic for the future. By 2014 the Web may reach the level of user empowerment defined by the Macintosh in 1984. Web usability is improving but at a snail's pace. However, as long as the snail is moving in the right direction, we should be happy, and that is exactly what's happening."
This article discusses how to prepare a software prototype and conduct one-on-one usability testing. The prototype is not created haphazardly. Rather, much preliminary research goes into creating a prototype that will be beneficial to the users.
Creating a Prototype Before Usability Testing
The following phases are conducted to create a good prototype that can be used for usability testing:
- surveys
- one-on-one interviews
- interpretation sessions
- consolidation
- work redesign
- revision of user interface design
- creation of storyboards
Surveys
Preliminary surveys can be conducted on the user population as a means of collecting demographic data and other information.
One-on-One Interviews
After completing the surveys, you can conduct one-on-one interviews with key users. During the interviews, your audience may be asked to perform certain scenarios using a software prototype. You then let the students evaluate their experience. In particular, note the features they use and those they do not.
Each member of a team of usability testers may interview different subjects. At the end of the one-on-one interviews, the entire team meets for interpretation sessions.
Interpretation Sessions
A few days after the conclusion of the one-on-one sessions, the entire team should get together to conduct an interpretation session. During the interpretation session, each team member describes one of his or her interviews to the entire team. During the sessions, the team tries to explore interviewee interactions in terms of work models.
The goal of the interpretation sessions is to obtain a better understanding of the audience's work sequence as well as the artifacts that are used to perform tasks.
Consolidation Phase
During the consolidation phase, the goal is to broaden the knowledge obtained from the interviews by expanding them to cover everyone who uses the software or system that you will be testing.
The consolidated work models are based on the same work models discussed in the interpretation sessions (flow, sequence, artifacts, physical, and cultural), except that the entire population is considered.
After the consolidation sessions, you can create an affinity diagram. An affinity diagram is a technique for grouping common issues from a nonstructured discussion, such as a brainstorming session (Binstock 1999). A hierarchy is created on the basis of individual points captured during the team's interpretation session (Beyer & Holtz 1998). The individual points are summarized in groups. The small groups are then summarized to create a larger group. Finally, the larger group issues identify areas of concern that should be studied later. These concerns form the basis for redesign.
Work Redesign (Visioning)
After the workflow of the entire user population becomes clarified and understood, you are ready to redesign the work process so that it is more efficient. Note that the product being evaluated is not redesigned at this point, only the work process. By redesigning the work process first, the design of the product becomes easier.
When you redesign the work process first, the team members meet to discuss various design ideas (visions) for improving the product. At this time, a high-level storyboard can also be created to capture each vision (sequence) used to accomplish the tasks.
Revision of User Interface Design
Only after completing all of the above phases do you actually revise the user interface design.
If the product that you are revising is currently in use, your team needs to make a major decision. You need to decide whether to refine the current methods or to redesign the entire interface.
Microsoft, like many companies, uses usability tests to identify "show stoppers" and fixes them right away. Ken Dye, leader of Microsoft's usability testing program, says, "All observations on possible improvement are prioritized on the basis of how serious the problem is and how often the user is likely to encounter it" [Binstock 1999].
In contrast to Microsoft's approach, Allan Cooper, inventor of Visual Basic and founder of Cooper Interaction Design, believes the "fix it" mentality of Microsoft is what is wrong with the current approach to usability testing. Cooper says, "You look at a product like Microsoft Office, and you see this very complicated interface that is not at all intuitive. You realize that Microsoft has relied on its usability testing solely to confirm its decisions" [Binstock 1999]. Cooper stresses that interfaces should be redesigned to make solid improvements rather than refined.
Creation of Detailed Storyboards
The last step before you create your prototype is to prepare detailed storyboards representing each part of the new design. The storyboards are used to create the prototype. Once the prototype is created, you are ready to begin planning your usability test.
One-on-One User Testing
Prototype interviews may be in a group or one-on-one. I prefer one-on-one user testing on a prototype because it avoids the possibility that users can influence others.
The goal of one-on-one testing is to interview a small number of people and achieve the same findings as if you had interviewed many more.
The following steps are recommended to ensure the most accurate results during one-on-one testing.
- Set the interviews up a few weeks in advance.
- Offer some type of incentive for customers to participate in the one-on-one interviews.
- Prepare a list of tasks that you want the user to perform. This will become your usability test. Your job is to evaluate how each user responds to the provided tasks.
- Prepare a list of evaluation questions that you want the users to answer when they finish performing the tasks.
- Use a scale from 1 to 5 with 5 being best.
- Do not provide hints or assist users. This includes using hidden body language.
- If users request assistance, tell them that they should do the best they can to figure it out on their own.
- Be sure to ask the same questions in the same order for each user.
- Take written notes during the interview session.
- You may also want to videotape the users to review results later. Be sure you can get a shot of the screen as well as the user.
Usability Test Report
After obtaining 8 to 12 usability test results, prepare the usability test report.
In short, the usability test report is designed to provide a summary of the findings and recommendations of the usability tests conducted.
The following guidelines are recommended when you write your usability test report.
- Provide the average ratings for each evaluation question that you asked.
- Come up with recommendations, based on the findings, for each major area tested.
- End the report with a summary of the findings.
- Refer to users by their numbers, not their names. For example, "User 1" and "User 2."
Also, appendices can be included that contain the usability test plan questionnaire as well as your user notes.
Summary
Preparing a prototype and conducting usability testing requires much preparation involving detailed tasks. However, the efforts pay off when you produce a site that accomplishes the needs of users in the most efficient way. Despite the complexities of the process, the purpose and the goal are simple. Know and understand your audience, and use good design techniques. These two principles go a long way to achieving an efficient and easy-to-use product.
References
Beyer, H. and K. Holtzblatt. 1998. Contextual Design. San Francisco: Morgan Kaufmann Publishers.
Binstock, A. 1999. "New Mantra: Usability." Information Week. http://www.informationweek.com. September.
