Effectiveness of Prototype Fidelity for Usability Testing

Project developed with Sandra Gabriele for the course HCIN 5200 – Software and User Interface Development (Fall 2016), Carleton University

LH-4
Low-fidelity version of existing website and smartphone app

Our research Question was: Are low fidelity prototypes equally effective at revealing usability problems as high fidelity prototypes?

Abstract

This study focuses on the fidelity level of prototypes and the impact on usability testing and describes a study that evaluated how low- and high-fidelity prototypes affect the identification of usability problems. We used a travel website and a smartphone application, each with low- and high-fidelity versions. Fourteen participants completed a pre- and post-test questionnaires and performed a series of tasks with one paper prototype (low-fidelity) and one screen-based prototype (high-fidelity) for a total of seven participants per condition. While we found that participants identified more usability concerns with the high-fidelity prototypes, the results were not significant. This result might be related to the small sample size. In examining the data further, we found that the types of usability problems most identified in the high-fidelity prototype were related to design and technical concerns, while the low-fidelity prototype seemed to uncover more content related concerns.

Preparation

I was responsible for choosing systems that had usability problems that could be revealed during usability tests. After initial analysis, I chose a website and an iPhone app, both related to travel.

I tested those systems and designed some use cases that could be proposed as tasks to participants. We evaluated them together and based on that, we developed our study script and selected the screens to be reproduced in low-fidelity by Sandra, while I handled the organization of the study sessions.

 

LH-1
Existing website and iPhone app chosen, and their low-fidelity versions

Usability tests

During the usability tests, I acted as experimenter, while Sandra was responsible for the shooting video and taking notes.

LH-2
Participant interactions with high-fidelity prototypes (above) and with low-fidelity prototypes (below)

 

Results

The video and notes were used to list errors found by participants in each study condition, which we coded based on Nielsen’s Usability Heuristics.

Although we did not find significantly different results between the two prototype fidelities, we would suggest that future research might look towards determining how prototype fidelity can support the Human-Computer Interaction (HCI) community in finding usability problems related to design, content and technical concerns.