The One-Way Mirror: Usability Testing Revealed

Paper prototyping, paper prototype testing, usability testing

Do you want to know how products and designs become great? They are tested and trialled. One cost-effective way to do this, without rolling out a limited release or going live with an untested product, is usability testing.

Usability testing is when actual users come in and actually use a product – but it’s not as easy as it sounds.

The basic format for a usability test involves three roles: the user (called the participant), the facilitator and the analyst. The participant, who has been carefully screened with specific requirements, is asked to carry out a series of mini-tasks to accomplish a goal they might have. They are encouraged to think aloud though their actions, and the whole thing is recorded.

The analyst focuses more on what the participant does and less on what they say – but notes particularly interesting ideas and quotations. These real-time notations are tremendously valuable during the analysis across the 8, 10 or more session that might be run in a typical usability test.

The facilitator has the hardest job. They have to guide the participant, encourage them to think aloud, be reassuring and positive, but still minimize their impact on results. It requires someone to deftly walk the line between being useful to the participant as they tackle each task, while avoiding aiding the participant in the test. It’s an acquired skill – and a very particular mindset.

At Usability Matters, we use three main types of usability testing: formal, informal, and remote. All have their own advantages and drawbacks, so really it’s all about selecting the right tool for the job.

Formal usability testing

The details

Formal usability testing uses a research facility, complete with one-way mirrors for observation. The test is usually performed on a live site or a high-fidelity electronic prototype and often results in a formal usability report.

The Gain

  • Stakeholders, designers and developers get to actually witness their users interacting with their product, an experience that can be humbling but incredibly useful.

The Trade-off

  • Renting the facility and formalizing the report increases the cost, so if budgets are tight this might not be the test for you.

Best used?

Formal testing is best used for large organizations that are heavily invested in a project already or are reviewing a digital product as an input to strategy or to make a business case for redesign.

Informal usability testing

The details

Informal usability testing is when the participants, rather than going to a research facility with a special room, simply drop by the office or another informal location (we use our meeting room). There are only two people watching – the facilitator and the analyst. The prototype can be lower-fidelity – maybe on paper or just barely electronic. The report is often less formal and might be directed at the design and development team, more than management.

The Gain

  • Informal facility combines affordability with the benefits of in-person testing

The Trade-off

  • It’s not possible for stakeholders to witness the live sessions, sacrificing an enlightening experience.

Best used?

This style of test is best used if in-person testing is needed but budget restrictions don’t allow for formal usability testing. It’s often used to test designs in progress including concepts, navigation, and basic interactions.

Remote usability testing

The details

Remote usability testing follows the same approach as formal and informal but the testing is conducted using web conference software. The participant, facilitator, analyst, and observers can all join from the location of their choice – watching the participant perform the tasks while guided by the facilitator.

The Gain

  • Participants and observers can join from anywhere in the world
  • Everyone uses the computer and set-up they are familiar with, including those with special needs and assistive devices

The Trade-off

  • Despite our best efforts, sometimes the technology is a barrier due to software, equipment and connectivity glitches – but most of the sessions run very smoothly.

Best used?

This style is best used when geography or accessibility is an issue. It is also cost effective – about the same as informal usability testing but with the opportunity for observers to watch one or all sessions from the comfort of their favourite chair.