Join the DZone community and get the full member experience.Join For Free
- Selenium to automate behavior.
- Applitools to measure response.
Applitools is an automated visual regression testing framework. By capturing rendered pages, Applitools lets you test pages visually. Once you have valid baseline snapshots, you can compare checkpoints against your baselines.
Applitools integrates with testing frameworks, such as Cypress.io, Storybook, and Selenium. It provides SDKs for use in your projects to communicate and interact with Applitools.
Throughout the article, I will use some terminology repeatedly. You need to know these terms to get the most from this article.
“WebDriver is a remote control interface that enables introspection and control of user agents. It provides a platform and language-neutral wire protocol as a way for out-of-process programs to remotely instruct the behavior of web browsers.” — WebDriver W3C Recommendation June 5th, 2018.
WebDriver aims at writing a standard set of instructions to query for DOM elements and controlling the behavior of browsers in a cross-browser and cross-platform fashion.
Chrome, FireFox, IE, Edge, and many other browsers provide their own implementation for the WebDriver Standard. Hence, they put in place the same standard API and provide browser-specific functionality.
“Selenium is a browser automation library. Most often used for testing web-applications, Selenium may be used for any task that requires automating interaction with the browser.” — Selenium WebDriver NPM.
Selenium WebDriver is a layer on top of the WebDriver Standard that provides seamless programmatic access to any browser that supports a WebDriver implementation.
For instance, let’s say you have ChromeDriver installed on your machine in its correct path; a script written with Selenium can automatically detect the Chrome Driver and can send all the instructions to the ChromeDriver to control the Chrome browser. Similarly, if the FireFox Driver is installed on the machine, Selenium would automatically interact with that driver to control the FireFox browser. In summary, Selenium WebDriver hides the different browser-specific implementations and provides a standard API, resembling that of the WebDriver API, to allow developers and testers to write a single script and to run it on multiple browsers and different platforms.
Applitools Eyes SDK is a thin layer on top of Selenium WebDriver that Selenium's function to provide developers and testers the tools to incorporate visual and UI testing into their applications.
To write a visual E2E test using the Applitools Eyes SDK, the script starts by initializing a Selenium driver object that is passed to the eyes.open() method as a first parameter. The SDK wraps the driver with easy to use methods. This allows the developer to start a test, capture a snapshot, and finally close the test.
We'll cover Applitools Eyes SDK in the step by step guide below.
In this article, I will demonstrate the Applitools Eyes SDK by building a visual E2E test script to test Wikipedia. The script starts by navigating the browser to Wikipedia. It then locates and clicks on the English version of Wikipedia, searches the Wikipedia for the term “Software Testing,” and finally verifies the results. The source code for this article is available on this Github repo.
First, ensure that these are properly installed on your machine:
Before we start coding, let’s ensure that you’re ready with ChromeDriver and Applitools.
ChromeDriver and Chrome Browser
Install Google Chrome browser from https://www.google.com/chrome/.
Install a ChromeDriver that's appropriate for your operating system and your Chrome browser's version from:
For example, if your Chrome browser is v67-v69, download ChromeDriver 2.41.
Unzip ChromeDriver to any folder of your choice.
Add ChromeDriver’s path to your system's PATH so Selenium can find it.
On a Mac:
Open the ~/.bash_profile file (Create one if it's missing).
Add export PATH="<PATH_TO_YOUR_CHROME_DRIVER>:$PATH".
Save the file.
Run source ~/.bash_profile.
For example, if the ChromeDriver is in /Users/apps/chromedriver, then it would be: export PATH="/Users/apps/:$PATH
Navigate to Computer (Right click) | Properties | Advanced System Settings | Advanced (Tab) | Environment variables | System Variables.
Select the line with the Path key.
Click on Edit.
Add a semicolon to the end of the values, and add the path where you unzipped the ChromeDriver.
For example, if the ChromeDriver is in c:\selenium\chromedriver, then append ;c:\selenium to the end of the Path.
Finally, save it.
If everything went okay and you typed ChromeDriver in the command prompt, you should see something like this:
Configure Applitools API KEY
Before you can locally run Applitools Eyes SDK, make sure you get an Applitool API Key and store it on your machine as an environment variable.
To set the APPLITOOLS_API_KEY environment variable, you can use the export command on Mac or the set command on Windows as demonstrated here.
Clone and Install Selenium and Applitools SDK
The section below will guide you step by step in writing your first Applitools Selenium e2e test.
Clone the application repo:
git clone :bhaidar/wikipedia-testing-applitools-selenium.git
Install the Selenium WebDriver package as a dev dependency by issuing the following command:
npm install selenium-webdriver --save-dev
Install the Applitools Eyes SDK for Selenium as a dev dependency by issuing the following command:
npm install @applitools/eyes-selenium --save-dev
Prepare for E2E Test
Let’s start creating our first E2E test. Create a new file wikipedia.spec.js at the root of the solution folder and paste the following two lines at the very beginning.
The code snippet starts by importing a few objects from the Selenium WebDriver NPM package. The second line imports the main Eyes object from the Eyes SDK for Selenium.
The code snippet makes use of the Selenium WebDriver API to build a new Chrome browser object.
Initialize an instance of the Eyes object.
The code simply creates a new object instance of the Eyes class. The code snippet then sets the Applitools API Key by making use of the APPLITOOLS_API_KEY environment variable.
Finally, the code configures the Eyes object instance to show logs while running. This is something I enable intentionally while writing my E2E tests so that I get the chance to see all the logs generated by Eyes SDK that could help me trace any bug in my code.
Let’s start a new Eyes SDK test by calling the eyes.open() method. The code snippet above defines a try/catch block so that we guarantee a safe closure for any opened test.
To start using the Eyes SDK you call the eyes.open() method passing to it the WebDriver object defined above together with test configuration settings. Calling this method initiates communication with the Applitools backend servers to start processing your tests.
Remember, the Eyes SDK for Selenium is a thin wrapper on top of the Selenium WebDriver, hence the need to pass the driver object to the Eyes SDK.
The App Name, Test Name, and Viewport settings are the essential configuration settings that you need to pass to the eyes.open() method.
Towards the bottom of the try block, it is essential to safely close the test. The final block closes the browser instance opened by the Selenium WebDriver and makes sure the Eyes test is completely closed by aborting it in case it is still pending.
Now, with that out of the way, let’s write our test!
Create a Test Script
Navigate to the Wikipedia Website by adding this line of code into your E2E test script.
Take a visual checkpoint using the Eyes SDK by adding this line of code into your E2E test script. The eyes.check() method accepts a simple string that labels the snapshot was taken.
Locate the “English” hyperlink on the Wikipedia Home page and click it with this line of code. The browser navigates to the English version of Wikipedia.
Verify that the page the browser navigated to has a header with text of “Welcome to Wikipedia”.
Take a visual checkpoint using the Eyes SDK by adding this line of code into your E2E test script:
Locate the search box, type the text “Software Testing,” and hit Enter. This code demonstrates the three steps.
Wait until the Wikipedia Software Testing page loads and then take a new snapshot. Every page in Wikipedia has a title element with an ID of firstHeading.
Finally, we close the current test and quit the browser.
Verify Test Results
Run the E2E test script by issuing this command: node wikipedia.spec.js
With that done, we'll visit the Applitools Dashboard and verify the test run results. Navigate to https://www.applitools.com and Sign In to access your account.
The dashboard lists all test runs on the left side of the page. By clicking a test run, you can then check all the snapshots that were taken during the test run.
In our case, there are three snapshots representing the Wikipedia Home page, Wikipedia English page, and Wikipedia Software Testing page.
The test has a status of Passed. The snapshots are stored as baseline snapshots, and everything seems to be working fine.
Let’s now introduce a visual change in the test run by running our E2E test to demonstrate how visual regression testing works with Applitools.
Make Changes and Validate Differences
Let’s change the search term from “Software Testing” to “Software Testability” and run the e2e test again by issuing the following command:
Running the above command results in the following error:
That’s great news! The Applitools Eyes SDK was able to detect visual differences between the two test runs and that’s exactly what we want.
Let’s switch back to the Applitools Dashboard and check the new test run results there.
Notice how Applitools was able to detect some visual changes in the third snapshot taken. Let’s expand on the third snapshot to see the visual differences detected:
The visual differences are clear between the snapshots. One snapshot displays the Software Testing page, and the other displays the Software Testability page.
The power of Applitools lies in the visual regression testing capabilities that make it not only a visual testing framework, but an obvious choice to run all regression tests in your application. Applitools will ensure there are no intentional changes across the life cycle of the software under development.
Here’s the source code repo for the completed E2E test application.
My upcoming articles will explore Applitools further, unveil more features, and allow you to carry on with E2E and Visual UI Testing easily.
Opinions expressed by DZone contributors are their own.