How ARIA Snapshot Testing Solves Common Playwright Issues

The challenges of testing various elements on a webpage, such as headings and links, noting that traditional code-based tests can become complex and brittle. I am not a huge fan of using screenshots for testing, as they can break with minor layout changes.

Playwright’s ARIA Snapshots offer a powerful method to validate the accessibility tree of your web applications, ensuring that content is both accessible and correctly structured. By capturing the hierarchical representation of accessible elements, you can compare snapshots over time to detect unintended changes. 

This article delves into how to utilize ARIA Snapshots for content validation, explores the features provided by Playwright’s recorder, and provides practical examples to enhance your testing strategy.

Playwright offers a feature to streamline updating expectations in test cases with npx playwright test --update-snapshots, which creates a rebaseline — a git patch file — to apply necessary changes. This automates the adjustment of expectations when fundamental changes occur in the application.

Understanding ARIA Snapshots

An ARIA Snapshot in Playwright provides a YAML representation of a page’s accessibility tree. This snapshot details the roles, names, and attributes of accessible elements, reflecting the structure that assistive technologies perceive. By comparing these snapshots, you can verify that the page’s accessible structure remains consistent or meets defined expectations.

- banner:
  - heading /Playwright enables reliable/ [level=1]
  - link "Get started"
  - link "Star microsoft/playwright on GitHub"
- main:
  - img "Browsers (Chromium, Firefox, WebKit)"
  - heading "Any browser • Any platform • One API"

In this snapshot, each node represents an accessible element, with indentation indicating hierarchy. The format includes the role, accessible name, and any pertinent attributes.

Validating Page Content With ARIA Snapshots

To validate page content, Playwright provides the expect(locator).toMatchAriaSnapshot() assertion method. This method compares the current accessibility tree of a specified locator against a predefined ARIA snapshot template.

Example:

await expect(page.locator('body')).toMatchAriaSnapshot(`
  - heading "Welcome to Our Site" [level=1]
  - navigation:
    - link "Home"
    - link "About"
    - link "Contact"
`);

In this example, the test verifies that the body of the page contains a level 1 heading with the text “Welcome to Our Site” and a navigation section with links to “Home,” “About,” and “Contact.”

Aria Snapshot | Accesibility | Chrome Dev Tools

Partial Matching

ARIA Snapshots support partial matching, allowing you to validate specific parts of the accessibility tree without requiring an exact match. This is particularly useful for dynamic content.

Example:

await expect(page.locator('nav')).toMatchAriaSnapshot(`
  - link "Home"
  - link
  - link "Contact"
`);

Here, the test checks that the navigation contains links to “Home” and “Contact,” with the middle link’s accessible name being irrelevant to the test.

Updating Area Snapshot

When using the Playwright test runner (@playwright/test), you can automatically update snapshots by running tests with the --update-snapshots flag:

npx playwright test --update-snapshots

This command regenerates snapshots for assertions, including aria snapshots, replacing outdated ones. It’s useful when application structure changes require new snapshots as a baseline. 

Note that Playwright will wait for the maximum expected timeout specified in the test runner configuration to ensure the page is settled before taking the snapshot. It might be necessary to adjust the --timeout if the test hits the timeout while generating snapshots.

Utilizing Playwright’s Recorder Features

Playwright’s recorder is a tool that allows you to generate tests by recording your interactions with a web application. It captures clicks, typing, submissions, and navigation, generating code that can be used for testing.

Key Features

• Action recording – Captures user interactions to generate test scripts.
• Live code generation – Displays the generated code in real-time as actions are performed.
• Editing capabilities – Allows deletion and reordering of recorded actions.
• Cross-browser support – Compatible with multiple browsers for comprehensive testing.

Example Workflow

1. Start recording. Launch the recorder to begin capturing interactions.
2. Perform actions. Navigate through the application, performing actions such as clicking buttons or entering text.
3. Review generated code. Observe the real-time generation of code corresponding to your actions.
4. Edit as needed. Modify the recorded actions by deleting unnecessary steps or reordering them.
5. Export and use. Save the generated code for integration into your testing framework.

For a visual demonstration, you can watch the following video:

Practical Examples

Example 1: Validating a Login Form

Suppose you have a login form with username and password fields and a submit button.

ARIA Snapshot template:

- form:
  - textbox "Username"
  - textbox "Password"
  - button "Log In"

Test implementation:

await expect(page.locator('form#login')).toMatchAriaSnapshot(`
  - textbox "Username"
  - textbox "Password"
  - button "Log In"
`);

This test ensures that the login form contains the appropriate fields and buttons with the correct accessible names.

Example 2: Validating a Dynamic List

Consider a dynamic list where items can be added or removed.

ARIA Snapshot template:

- list:
  - listitem "Item 1"
  - listitem "Item 2"
  - listitem "Item 3"

Test implementation:

await expect(page.locator('ul#dynamic-list')).toMatchAriaSnapshot(`
  - listitem "Item 1"
  - listitem "Item 2"
  - listitem "Item 3"
`);

This test verifies that the list contains three items with the specified accessible names.

Conclusion

ARIA Snapshots in Playwright provide a robust mechanism for validating the accessible structure of your web applications. Using the recorder’s features, you can efficiently generate and manage tests, ensuring that your content remains accessible and correctly structured. Incorporating these tools into your testing workflow enhances the reliability and inclusivity of your applications.