Usability testing exercise

For today's exercise, you will be preparing and carrying out a usability test. This will invovle:

• reading these instructions
• deciding on the test focus and test task
• preparing the test task instructions and data collection sheet
• carrying out the usability test with your test subjects
• writing up the results (completing any portions of the data collection sheet that you could not fill in during the test itself)

For each pair of students who prepared a low-fidelity prototype during Tuesday's class, one student will be designated "tester #1" and the other student will be designated "tester #2". The following table provides the (very tight) timeline we'll be following:

Instructions

Your first task is to decide on the test focus, that is, the most important usability objective(s) for the system being tested. For example, do you care most about ease of learning, speed of task completion, number of errors made, general user satisfaction with the result, or some other factor entirely? This choice may be partly subjective, but it should make sense given your expectations of the intended user population and their needs for using the system.

Ordinarily, you would then choose a user focus, but in this highly constrained exercise, you don't have much choice, since you only have access to two test subjects. Tough luck.

Next, you need to decide on the test task, that is, what is the function that you want your users to perform with the system being tested? Although at different stages of evolution of your own project, you may want to test different functionality, for this exercise, you should decide on two or three tasks that you believe are most critical to the successful use of the system.

Now that you have decided what the test will focus on, and what tasks the users will be performing, it's time to prepare the testing materials. In general, these materials might include:

• Observer briefing
• User introduction
• Informed consent
• Non Disclosure Agreement
• Pretest questionnaire
• User or training documentation (optional)
• Test tasks
• Data collection sheet
• Posttest questionnaire
• Test script

However, for today, we're going to concentrate on only two of these items:

• test tasks: you need to write out, in plain, easy-to-read language, what you want the test subject(s) to do. Importantly, these instructions should tell the subject what to do, not how to do it
• data collection sheet: prepare a spreadsheet with appropriate column headings that include, at least:
  • action: what the test subject did with the interface at a particular time step, e.g., clicked on "submit"
  • user comments: if the test subject said or expressed anything of relevance, for example, "Woops, I didn't mean to do that", or began making fists at the computer screen
  • observations: any observations of the testing team regarding issues encountered by the user or additional feedback that may be helpful to the design team

As you might appreciate, it is exceedingly difficult for one person to facilitate the test, observe the test subject, and take notes for entry into such a form. This is why usability testing is typically carried out be teams of at least two people. Again, today, you don't have that luxury. You get to run the test solo and try to remember what you observe during the test.

Since you can assume that your classmates all understand the point of this exercise, we're going to skip the user introduction. Your test subjects know why they're here.

Once you've completed the solo components, get back together with your partner and debrief. Discuss the results of your test sessions and identify any common elements (if any) that you observed.

For the final phase of this exercise, prepare a document summarizing your ideation (sketching), prototyping, and testing process, focussing on the following items, and post the document on the Usability Testing Exercise forum:

• images of your favourite one or two sketches that led to your prototype design, and don't forget to include images of the prototype itself (we want to see the history that led to the particular design you are testing)
• your test task instructions and data collection sheet (with observations filled in)
• summarize the results of your respective test sessions
• were there common issues that you both observed? Any particularly strong or weak points of your design that became apparent during testing?
• discuss how you would go about changing the interface based on the usability test session you observed

And last thing: please include your names (since only of you should post on behalf of the pair)!