3 Key Metrics to Identify the Usability of Your Product
The ideal product is considered usable only when a specific type of user can get the task completed.
Join the DZone community and get the full member experience.Join For Free
"Is your product usable enough to release now?" is the most difficult question to be answered when not backed up by usability metrics. Before understanding the key metrics of usability, though, we need to understand what usability is in the first place.
Per ISO standards (ISO 9241-11), usability is defined as:
“Extent to which a product can be used by specified users to achieve specified goals with effectiveness (task completion by users), efficiency (task in time), and satisfaction (responded by user in term of experience) in a specified context of use (users, tasks, equipment & environments).”
In simplest terms, the ideal product or service is considered usable only when a specific type of user can get the task completed without taking too long, without making many mistakes or errors, and while feeling very pleased in the context of use.
As per the standards, the key metrics to identify the usability of the product/service are:
These metrics will help in quantifying the progress in design between development sprints or releases. It also supports in assessing and setting up a competitive benchmark for achieving in the product.
How do we measure usability?
Usability metrics are basically measured by considering number of users' performance on a set of goal-directed test tasks performed in the product. Ideally, the preferred number of users to be tested is 20 or more to have a better confidence interval on the result. Conducting a quantitative usability test is expensive, but it can give lots of return on investment.
Effectiveness can be measured by the “success rate” (or completion rate). It is the percentage of participants who have correctly and completely reached each goal without any assistance from the test moderator.
Other metrics which can also assist in measuring effectiveness are:
- Disaster rate: percentage of users who thought they were successful but actually failed to complete the task
- Number of errors per unit of time
- Percentage of tasks completed successfully on the first attempt
- Number of requests for assistance accomplishing the task
But the “Success rate” is more accurate and useful than the others.
Efficiency can be measured by taking the average "time on task." In simpler terms, time taken by the user to reach the goal.
Other similar metrics for measuring efficiency are:
- Time-taken on the first attempt
- Time to perform task compared to expert
- Number of clicks taken to achieve the task
The average time taken should be calculated by using geometric mean rather than the arithmetic mean because a single large value can skew the results.
Satisfaction can be measured by obtaining a mean square using an established questionnaire such as QUIS, SUMI, PSSUQ, and SUS. This questionnaire needs to be answered at the end of a usability study.
Other similar metrics are:
- The ratio of + to - adjectives used to describe the website
- NPS (Net Promoter Score): percent of users who would recommend to a friend
- Customer rating of the quality of output
- Percent of users that rate the website easier to use than a competitor
Among the above questionnaires, SUS (System Usability Scale) is one of the most used standards for measuring the overall impression of the usability of the software. SUS consists of ten item attitude questions with five responses from strongly agree to strongly disagree. The SUS score is calculated in percentile, and if the product scores above 68, then it is considered a usable system.
Even though the three metrics are important, it is the responsibility of the team to decide which metric needs to be prioritized for testing and optimization based on the problem that is being solved.
Opinions expressed by DZone contributors are their own.