Usability Testing: How and Why?

By July 13, 2018 No Comments

If you are reading this, you’ve probably heard about or utilized usability testing. Just in case you haven’t, a usability test is a technique used in user-centered interaction design to evaluate a product by testing it on users. Great usability is one of the key factors in keeping your users engaged and satisfied with your website or app. With 79% of users abandoning websites if the usability is poor, it is crucial that usability testing is continually undertaken and perceived as a core part of the development process.

As a designer I heavily rely on my experience, research, and intuition to solve problems. Explaining and defending your decisions to stakeholders on a daily basis without evidence will get you only so far. With usability testing you’ll be able to justify your solutions with solid evidence and help build a great user-centered product.

Recently I had the chance to create and conduct usability testing on one of our projects and I ran into this problem. I had no evidence to back up my claims. That’s where usability testing shined. Numbers are powerful. They offer a simple way to communicate usability findings to a general audience. After the first round of interviews and testing, the site’s success rate was 30%. Which was a bit disheartening considering how much time and effort we put into developing the product. After collecting the feedback and iterating the design we came back for round two of testing. This time the success rate was 65%! We improved the usability by 35% in just a month! How and what UX methods did we use? The Think-Aloud method with Expectation testing was used in combination for round one of testing. In round two of testing we began with a combo of Light A/B testing and Visual Affordance testing, where the first helped us determine what users liked or didn’t like while the latter let us know what users found interactive on the site. Based on comments and feedback from these tests, we ran a second combo of the Think-Aloud method with Expectation testing.

There are many ways to conduct usability testing and it’s up to you to decide which is best for you and your project. The most important thing is to be productive and add value from your tests, otherwise you are wasting valuable time and effort which could be used elsewhere. Personally, I’m beyond excited with what the next rounds will bring us.

How to run a usability test

You will need to draft a test plan. The test plan should include two important points: 1) Determine what you are going to test and create tasks; and 2) How you will measure the results.

1. Determine what you are going to test and create tasks

When creating your tasks pay special attention to wording and be sure not to lead your users on. Rather than simply ordering test users to “do Task X” with no explanation, it’s better to situate the request within a short scenario that sets the stage for the action and provides a bit of explanation and context for why the user is “doing Task X.”

  • User goal: Browse product offerings and purchase an item.
  • Poor task: Purchase a pair of orange Nike running shoes.
  • Better task: Buy a pair of shoes for less than $40.
2. How you will measure the results

To collect metrics, I recommend using a very simple usability measure: the user success rate (percentage of tasks that users complete correctly). It may be a coarse metric but, if users can’t accomplish their target task, all else is irrelevant. User success is the bottom line of usability. What was our criteria? Every successfully completed task equates to 1 point, partial success is 0.5, and a failed task renders 0 points.

Conduct the testing

While conducting the tests be sure to let the participant know that you are not testing them, you are testing your design. There is no passing or failing grade just observations. Before starting the interview let the participant know to think aloud. Make sure the participants explain what they see and what they are expecting while going through the product. In one sitting you can conduct multiple UX methods for testing, like I mentioned before. Just make sure to plan and prepare for it. If the participants stop talking at any moment, remind them to think aloud and continue with their scenario and tasks. The crucial thing for successful testing is not to lead your participants on. You mustn’t answer any question nor confirm or deny them. Instead use that opportunity to ask the participant what they are expecting and continue with the rest of the scenario. Also, make sure to control your body language so the participants feel at ease and can’t get any clues from you.

After the participants are finished with their tasks, ask them final questions before wrapping up the interview. Questions like “If you had a magic wand, what is something that you would change about this site/app?” will give you extra insight and surprisingly great ideas for future features.

Reviewing and iterating

Compile your results and create a short recap. Try to use your recap to provide as much information as possible about what you learned from the test and its results. What are the main points gained from the user tests? What are the action items to consider from the user tests? When you run your next user tests, what would you like to see fixed? Present them to your team and stakeholders and jump into iterating your design and fixing the presented issues.


UX testing isn’t a one-time thing. You should aim to conduct tests after every major feature release or every 2 months. Why that often? Well, after extensive testing and iterating you are armed with the knowledge that the new iteration will significantly outperform the old one. The data backs it up. Do not fear the UX testing process — it is an integral part of putting together a functional product on the market. While preliminary work and a lot of research will be required to determine what the best testing environment is going to be, the return on investment is going to be high, and it will justify the time and effort put into every single step.