Accessibility Testing Using Playwright

Today, I’d like to talk about accessibility testing using Playwright.

Accessibility testing refers to evaluating web applications or websites for compliance with accessibility standards, ensuring that they can be used by people with disabilities. Existing guidelines and other standards related to web accessibility Here are the links to them:

• w3 Standard Guidelines
• w3c Guidelines

The Accessibility Tree is used by assistive technology to augment or replace the existing UI for applications, as well as translate operations performed by the user into something that the application understands.

Playwright provides built-in support for accessibility testing by exposing methods and properties that allow you to interact with and inspect the accessibility features of web elements. You can use Playwright’s API to retrieve information about an element’s accessibility properties, such as its role, name, state, and description. This allows you to programmatically verify if the web application or website meets the required accessibility criteria. Additionally, Playwright has support for generating accessibility reports. You can use the page.accessibility.snapshot() method to capture a snapshot of the accessibility tree of a page. The Playwright site already has detailed documentation on using accessibility for tests, so that I won’t duplicate everything here. Still, I’d like to focus on the bottlenecks and possible solutions to some problems that I had on one of my projects.

Basics

To test Accessibility using Playwright, you need to additionally install the @axe-core/playwright library, with the help of which scanning of the webpage will take place.

Library of axe-core

Axe is an accessibility testing engine for websites and other HTML-based user interfaces. It’s fast, secure, lightweight, and was built to seamlessly integrate with any existing test environment so you can automate accessibility testing alongside your regular functional testing.

npm i @axe-core/playwright 

As you can see, everything happens in three steps in the test:

1. Open the desired page.
2. Scanning using AxBuilder()
3. Comparison of obtained and expected results.

It is important to understand that the scanning is based on the principle of photography: if the page has modal windows or drop-down lists, they must first be opened to be tested. In addition, it is worth considering that the scan results contain not only violations, so for a correct comparison, you need to pass violations to expect(). If there is a violation, after the test, we will receive a report with detailed information about the violated rules and places where it happened, as well as information about possible solutions.

Configuration of the Analyzer

To get the optimal effect from testing, the analysis can be configured quite flexibly. This effect is achieved by using methods that allow you to set the level of recommendations or a set of rules and correct them by excluding irrelevant rules from the check.

It is also convenient to exclude a certain part of the page from the test and test only the part that is necessary. For myself, I chose the following configuration, wherein, include() depending on the specific test, the required selector or body will be passed by default. Playwright offers a number of sophisticated tools that let you run more complex a11y tests. For instance, the AxeBuilder class offers a wide range of axe setup configs. These settings can be specified by utilizing a Builder pattern.

Lighthouse for Accessibility Testing

Lighthouse is an open-source, automated tool from Google for improving the performance, quality, and correctness of your web apps. When auditing a page, Lighthouse runs a barrage of tests against the page and then generates a report on how well the page did. From here, you can use the failing tests as indicators of what you can do to improve your app.

To be able to use Lighthouse in Playwright tests, you can use the playwright-lighthouse package.

In my project, I have created a TS spec file for the purpose of running directly against Lighthouse.

Reports

Same approach as for normal test cases. Playwright supports the configuration of different reports. In the case of accessibility tests, we just need to pass some additional information to the test itself.

As I said before, for my current project, I have a Lighthouse TS spec file, which I run from the command set in package.json.

npm run clean && npx playwright test tests/lighthouse.spec.ts --project=chromium

Conclusion

Playwright, in combination with the @axe-core library, is a fairly flexible and convenient tool for automated accessibility testing, although it does not cover all of its needs. As always, it depends on what your project needs. Playwright provides methods to access the accessibility tree of a page, allowing you to programmatically analyze and validate the properties of web elements. This makes it easier to identify and address accessibility issues.

However, it is true that the effectiveness of using any automation tool depends on who is using it, so always use it with common sense.