Usability testing: 3 common mistakes to avoid

Usability Testing Setup

Usability testing has seen a significant rise in popularity over the last several years. Companies across industries and of all sizes are embracing the value of putting their digital products in front of real users before these products launch. With this added emphasis on user research, we’ve seen an increase in teams looking to run their own usability tests to help build user research into their typical design process.

While it might seem as easy as showing an unfinished version of your product to a person you grab off the street, there is more that goes into a usability study. In order to ensure your study is producing the best results, effort must be made to ensure that the findings are useful and actionable to the team and ultimately help drive further design optimizations. This is not a simple task, and it is very easy to forget key steps during the planning or execution phases that might leave your results lacking.

After doing several years of testing myself, I’d like to share with you some things people commonly overlook to help you avoid falling into these traps.

1. Recruiting the proper participants will be harder then you expect

The number of participants required for a typical usability study will often range. Based on our experience, we tend to recommend somewhere between 8-12 participants. While it might seem like that is not very many people, it can actually take a lot of effort to get that number of people to commit to help you with your study. If your team does not have the budget to use a professional recruiter to obtain the right candidates, then you might have to allow yourself more time then you would have initially thought. Most people work during the day, which can make it tricky for people to attend a session in-person, even when you offer participant incentives (cash, gift cards etc.). And that’s assuming that your participant profile is relatively general. As you start seeking out more specialized participants (say a highly specific age range), you are shrinking your potential pool of candidates significantly with each criteria added.

If you plan on recruiting participants yourself, then it is highly recommended that you give yourself at least 2 weeks to find the right candidates and ensure that someone on your team has at least a few hours per week to handle all the administrative work associated with contacting and scheduling all the participants.

2. Take the time to test your setup

In order for your team to get the most out of each session, you need to be sure that your test setup is functioning properly. You will need to be doing screen recordings and often you will also want to do a screen share for remote observation. The technology for these different purposes is constantly evolving so you will definitely want to make a point of setting aside some time before your sessions to test all of this technology running in tandem.

We recommend doing a rehearsal session to get a feel for how everything will actually work with real participants. This could be done with anyone around the office. Even though it might feel a little odd, make sure you treat the rehearsal session as seriously as you would a real one and run through every single task. You are bound to find some issues with the prototype and you’ll also want to make sure that the technology will function properly for the full duration of the session.

During our time running test sessions, we have seen the odd case where everything seemed to work in short bursts but had issues during longer sessions. Technology can be finicky, but as long as you are properly prepared you should be able to overcome any challenges and still capture everything you require.

3. Work with your team to find the best way to communicate the results

Different teams have different uses for the results of a usability study. This is why we recommend having a talk with your team or client before you begin analysing your results about how they will use the results, and the best way to provide them.

If you are working closely with a team of developers that understand your process, you might not necessarily need to produce a thorough PowerPoint presentation. They might be best served with a straightforward chart that lists off all the findings and very succinctly identifies the problems found and the recommended solutions. If you are working with a larger team of stakeholders that needs more context, then a presentation will make more sense. For those cases, you might even want to produce a video highlight reel from the recordings that shows some of the more striking moments to come out of the sessions.

This all depends on your situation, but it is always a good idea to have an open conversation with the broader team to ensure that expectations are being met with regards to what comes out of a usability study.

Usability testing, when conducted properly, can help save your company huge amounts of time and money by catching major issues before a product goes into development. It might seem intimidating at first, but once you get enough practice and begin to refine your methodology, you’ll start to wonder how you ever did without it. Hopefully these tips help as you seek to grow the practice within your organization!

Why Digital Accessibility Matters for everyone. Button: Download infographic