Over a million developers have joined DZone.

Usability Testing: Moderated or Unmoderated? Part 1

DZone's Guide to

Usability Testing: Moderated or Unmoderated? Part 1

Learn the characteristics and advantages of moderated and unmoderated usability testing when testing for your app's performance.

· Performance Zone ·
Free Resource

Maintain Application Performance with real-time monitoring and instrumentation for any application. Learn More!

The best way to evaluate the user experience of a product is by testing it with users. Observing people trying to perform tasks in the interface gives you the best understanding of how well it works and what problems need to be fixed. However, usability testing can be moderated or unmoderated. What's the difference between these, which is better, and why should you care? In this two-part blog post, I'm going to answer those questions.

Traditionally, usability testing involved moderated sessions, in which a researcher met with a participant in person to observe them performing tasks and to ask them questions about the experience. With the advent of online meeting software, researchers could conduct remote, moderated sessions by having participants share their screens and communicate through a conference call or the meeting audio.

Moderated usability testing is very effective, but it can be very time-consuming. Over the past decade, a variety of unmoderated usability testing tools were created to make testing easier and to reach a larger number of participants. These unmoderated tools were similar to surveys, in that participants went to an online study which gave them tasks to perform, asked survey questions, and captured all the data. Researchers only needed to set up the study, send the link out to participants, and come back later to see the results already compiled and presented in useful visualizations. However, most of these unmoderated testing tools had problems and provided limited usefulness.

Let's take a look at the current state of moderated and unmoderated usability testing, to compare the advantages and disadvantages of each method, with the goal of determining how best to use them.

Moderated Usability Testing

The key to moderated usability testing is that the researcher is present with the participant to moderate the session. The researcher provides instructions, gives the participant tasks to perform, and asks questions.

Advantages of Moderated Testing

Vastly superior information
  • The researcher directly observes and listens to participants.
  • The researcher can ask questions to learn more about participant actions and comments.
  • Participants provide much more in-depth, and higher quality, information through conversation than they do through survey questions.
  • The researcher can observe the participants' facial expressions and body language.
More control over the sessions
  • The researcher ensures that the participant correctly understands what to do, can answer questions, can fix problems with the prototype, can help the participant when he or she get stuck, and can solve any technical issues that arise.
  • The researcher ensures that the participant takes the test seriously, stays on task, and thinks aloud.
People can observe the sessions

Disadvantages of Moderated Testing

The need for the researcher and participants to be in the same place at the same time (in person or remotely) also provides a few disadvantages.

Time consuming
  • Moderated sessions require the researcher to be present in each session, with additional time for setup and between sessions.
  • Typing up notes and organizing them is time-consuming.
More expensive

Moderated sessions may have these additional expenses:

  • Recruiting costs
  • Travel costs
  • Higher participant expenses
  • Renting a usability lab or focus group facility
The need to recruit and schedule participants
Limits on participants
Because of all these disadvantages, usability testing can seem like a major effort, which limits how many rounds of usability testing that are included in projects.

Unmoderated Usability Testing

The alternative to moderated usability testing is unmoderated usability testing. Unmoderated means the researcher is not present when participants complete the study. The researcher sets up the tasks and questions in an online testing tool and gets a link to the study. Participants click the link to the study and complete it online, on their own time. These tools capture data such as task success rates, task time, errors, survey answers, and audio and video of the task.

In part two of this blog post, we'll examine the advantages and disadvantages of unmoderated usability testing and how you can use both moderated and unmoderated usability testing together to get the best results.

Collect, analyze, and visualize performance data from mobile to mainframe with AutoPilot APM. Get a Demo!

ui testing ,usability testing ,performance ,testing

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}