Review — Automated Visual Testing With WebdriverIO
If you use WebdriverIO regularly, and you are unfamiliar with the basics of using Applitools for automated visual testing, you will appreciate this course.
Join the DZone community and get the full member experience.Join For Free
I took Nyran Moodie's course on Test Automation University: Automated Visual Testing with WebdriverIO.
If you use WebdriverIO regularly, and you are unfamiliar with the basics of using Applitools for automated visual testing, you will appreciate this course. Nyran focuses you on how to add visual test automation with Applitools to your WebdriverIO tests.
Nyran expects you to know WebdriverIO. If you want to learn WebdriverIO, check out Julia Pottinger's excellent course — UI Automation with WebdriverIO on Test Automation University.
Nyran breaks the course into eight main chapters. He gives this description in the course overview:
- 1 — I am going to start by giving you an introduction to visual testing.
- 2 — We are going to look at how we can get our environment set up to start using Applitools eyes.
- 3 — We are going to create and run our first visual test using Applitools
- 4 — I want to introduce you to the different match levels that Applitools has and the concept of viewport sizes
- 5 — I will be talking about checkpoints.
- 6 — I will be looking at how we can organize our visual tests using batches
- 7 — We will look at how we can analyze our test results using the Test Manager.
- 8 — We will be looking at the integration and collaborations that Applitools provides.
Each of these chapters provides a methodical approach to getting going with Applitools. I'll give a quick overview of each.
If you read this review or take the course, you know why visual testing. Your UI and end-to-end tests result in the rendered output. You can write all the functional tests that grab locators, enter data, effect action, and cause the appropriate output locators to have the appropriate values. But, until you look at the result, you cannot tell if the input and output conform to design and usability expectations.
Nyran did not explain the most frequently-experienced reason for visual testing — unintended consequences of code changes over time. Our experience shows us that most expected application changes get tested, but unintended changes cause problems.
Nyran does a nice job explaining how to get started. You need an Applitools API key, which you can get from the Applitools console. Nyran explains why you set up a local environment variable for your API key (so you do not need to include your API key in your test code directly). He also points to the GitHub repo he uses for all the examples in the course.
Chapter 3 involves the first coding examples for setting up Applitools. With a simple:
npm install @applitools/eyes.webdriverio
You get the the node instructions for installing the Applitools Eyes to your WebdriverIO setup. After this, you can install the Applitools Eyes service to your tests. He shows code examples of what test code looks like when calling Applitools:
Once he walks you through a test page example and fills in the tests, he gets code that looks like a full test.
Finally, he shows you the Applitools UI and how it highlights differences found during a test. To do this, he shows a test site with the ability to show different content, and he shows how Applitools highlights the differences.
Nyran makes it clear that Applitools can find and highlight all the visual changes on a comparison page (the checkpoint) versus the original capture (the baseline). And, he explains that Applitools lets you accept the checkpoint as the new baseline, or reject it.
Chapter 4 — Viewports and Match Levels
Next, Nyran writes about match levels. Applitools default comparison level, called "strict", compares visually noticeable changes between a checkpoint and baseline. Strict use Applitools Visual AI to break items on a page into elements that it then compares.
However, sometimes strict is too strict. In the "content" match level, Applitools checks text and structures but ignores color variations. This match level helps when you apply a global change to color and want to ensure that — color outstanding — no other changes have taken place. And, if they have, you want those changes highlighted quickly.
In the "layout" match level, Nyran shows, Applitools lets you validate pages that have dynamic content sharing a common structure. For example, you might have a retail shopping page that shows user-specific and hot items updating from test run to test run. Or, you have a news site that updates top stories regularly. The layout match level pays attention to the layout structure (relationship of sections and text sizes) without comparing the specific contents of any set of elements within that structure.
Nyran spends this chapter reviewing the ways Applitools lets you make visual captures. First, you can capture the visible page — the currently visible screen. Alternatively, you can capture the full page. Applitools runs through the app horizontally and vertically and stitches the images into a single screen baseline and checkpoint.
Next, you can capture individual web elements. Finally, you can capture frames on a page. Your flexibility in setting checkpoints gives you plenty of power to control the details of your inspection at different times in your testing life cycle.
Batches provide a way of organizing common tests inside the Applitools Test Manager. Nyran explains how to code tests into batches. He also shows how batched tests get grouped inside the Test Manager. When you use batches, your test results become easier to interpret.
Nyran implies two things about batches. First, grouping tests into batches make your testing much easier to understand. Second, If you want the benefit of batches, you need to code those batch definitions yourself.
The Test Manager is the Applitools service user interface. Your user login provides you an API key, and tests run under your API key collect in your view in Test Manager.
Nyran shows you the basics, as well as some cool tricks in the Test Manager. He shows you how to compare the differences between the checkpoint and the baseline. You can approve expected changes and update the baseline to the checkpoint. Or, you can reject changes and have them routed back to development as bugs.
Next, Nyran shows you how tests get grouped and how to use statistics. He also shows you how to override the existing match level on all or part of a test. Finally, he shows you Automated Test Maintenance.
Automated Test Maintenance gives you huge powers of scale for validating changes to your application. When you find a change in one checkpoint and approve it as a valid change, Applitools finds all other similar changes and gives you the power to approve those identical changes at the same time. For example — you change your menu bar at the top of your app, and the change affects 145 pages. Following your validation of a change on one page, Applitools asks you if you want to approve the other 144 pages with the identical change. That's powerful.
In the last chapter, Nyran shows how Applitools integrates with other tools in development — especially your CICD workflows.
Nyran shows the range of Applitools test framework and language SDKs that work on web and mobile test frameworks. Applitools lets you capture screenshots to use Visual AI if you use an unsupported framework. Applitools even lets you compare PDFs to ensure that your document generators behave correctly even as you update your application.
Next, you see how to link Applitools with Jira. You can link issues found in Applitools with incidents in Jira. Also, you can link GitHub and Applitools to tie your image comparisons to the source code GitHub pull requests. Finally, you see the standard approach to having Applitools link with your favorite CICD workflow manager.
Nyran doesn't cover:
- Setting up tests in WebdriverIO,
- Approaches for running and managing tests and test results
- Managing WebdriverIO in your CI/CD workflow
Published at DZone with permission of Michael Battat. See the original article here.
Opinions expressed by DZone contributors are their own.
How To Approach Java, Databases, and SQL [Video]
Part 3 of My OCP Journey: Practical Tips and Examples
What Is Envoy Proxy?
Transactional Outbox Patterns Step by Step With Spring and Kotlin