How to Perform Visual Regression Testing Using Cypress

Software testing is a critical phase of the SDLC process. Early testing saves a lot of cost and effort — low-quality software can lower customer retention by impacting user engagement negatively. It is therefore essential to build software that is bug-free to increase sales and make the product that much more popular. 

There are different types of testing, such as smoke, sanity, integration, regression, unit testing, and API testing. Regression testing is one of the most important kinds of testing, especially when new features or updates are added to an existing application.

What Is Regression Testing?

Regression testing ensures that new code changes have not adversely affected existing features. Any issues which come into existence due to the latest code changes should be caught in this type of testing.

Role of Automation in Regression Testing

Unlike sanity and smoke testing, the scope of regression testing is vast, since QA has to test the entire application or subset of features. Considering that manual testing for regression involves a lot of time and effort, it is not cost-effective. This is the reason why most organizations are opting for automated regression testing. Regression testing in Agile teams often works in a risk-based approach, opting for test automation frameworks for a streamlined test process.

Types of Regression Testing

Regression testing can be classified into functional regression testing and visual regression testing. Fundamentally both work differently, as explained below.

Functional Regression Testing

Function regression testing is the type of testing where workflows or use cases will be tested using manual or automation tools. It involves entering some values by fetching selectors from browser DOM. This also includes validating text-based values by getting them either programmatically or manually. Most of modern testing in practice is based on this approach.

Visual Regression Testing

A visual regression test checks what the user will see after any code changes have been executed. This is done by comparing screenshots taken before and after the code changes. This is why visual regression tests are also sometimes called visual snapshot tests. Visual test highlights any visual changes that occur after the code update.

Visual tests generate, analyze, and compare browser snapshots to detect if any pixels have changed. These pixel differences are called visual diffs (sometimes called perceptual diffs, pdiffs, CSS diffs, or UI diffs). You can set the threshold values, which give you the flexibility to compare images with small differences or ignore small differences.

Visual Regression Testing Using Cypress

Cypress is a popular test automation framework that allows a lot of extensibility. Visual regression testing is one of the extensible features of Cypress. There are a lot of plugins available in Cypress that can be used to capture visual images and compare them. This tutorial explains Cypress visual testing using the Cypress Image Diff plugin.

Step-by-Step Tutorial of Cypress Visual Regression

To set up the Cypress visual regression environment, you need to install the following:

• Install NodeJS
• Install Visual Studio Code
• Install Cypress

Step 1: Install the Cypress Image Diff npm package.

Navigate to your project root folder (the directory where package.json is located). Enter the below command:
npm i -D cypress-image-diff-js

Step 2: Configure the image diff plugin.

Navigate to cypress/plugin/index.js and enter the commands written below:

// cypress/plugin/index.js
module.exports = (on, config) => {
const getCompareSnapshotsPlugin = require('cypress-image-diff-js/dist/plugin')
getCompareSnapshotsPlugin(on, config)
}

Step 3: Import and add the Cypress image command.

Navigate to cypress/support/commands.js and enter the code written below:

// cypress/support/commands.js
const compareSnapshotCommand = require('cypress-image-diff-js/dist/command')
compareSnapshotCommand()

Step 4: Configure Reporter.

Navigate to cypress/support/index.js and enter the code written below:

after(() => {
cy.task('generateReport')
})

Step 5: Write your first Cypress visual test.

Navigate to the cypress/integration folder add new file example visual-test.js. Enter the code snippet written below:

// visual-test.js
describe('Cypress Visual Testing', () => {
	it('Compare fullpage of Google page', () => {
		cy.visit("https://www.google.com/?hl=hi");
		cy.compareSnapshot('google-page');
	})
})

The above code navigates to the Google Home page and compares if it is visually fine.

Step 6: Run your first visual regression test with Cypress.

Run your Cypress visual test using the below command.

npx cypress run --spec "cypress/integration/visual-test.js"

Step 7: View the report.

One of the key features of this plugin is that it generates a good HTML report. Once you run the test, two folders will be created:

• cypress-visual-report: It contains, an HTML report, if the test passes there will not be any images shown if the test fails it will show baseline, comparison, and difference in that image.
• cypress-visual-screenshots: This folder contains three subfolders, namely baseline, comparison, and diff, where each folder contains the respective image files.

A passed test report looks like this:

A failed test report looks like this:

How to Perform Cypress Visual Testing of an Element

Cypress can be used to perform visual tests on a specific element, where it compares the before and after screenshots of the specified elements, as seen in the example below.

describe('Visuals', () => {
	it('should compare screenshot from a given element', () => {
		cy.visit('www.google.com')
		cy.get('#report-header').compareSnapshot('search-bar-element')
	})
})

This code snippet takes a snapshot of element #report-header and compares visually if it is same or not.

Adding Threshold to Visual Test

By default, the value of the threshold will be zero, which means it should match exactly with the base image. However, you can make it flexible by applying other threshold values.

Check out other useful functions of cypress-image-diff plugin.

Executing Cypress Visual Comparision Test Using BrowserStack

To run Cypress visual comparison tests on a real device cloud like BrowserStack, follow the steps below.

Step 1: Install BrowserStack Cypress plugin.
Step 2: Create browserstack.json file using browserstack-cypress init command.
Step 3: Copy and paste the below code.

{
"auth": {
"username": "<my_username>",
"access_key": "<my_access_key>"
},
"browsers": [
{
"browser": "chrome",
"os": "Windows 10",
"versions": [
"latest",
"latest - 1"
]
}
],
"run_settings": {
"cypress_config_file": "./cypress.json",
"cypress_version": "9",
"project_name": "My sandbox project",
"build_name": "Build no. 1",
"parallels": "2",
"npm_dependencies": {
"cypress-image-diff-js": "^1.18.0"
}
}
}

Note:

• You can find the username and access key by logging into the BrowserStack website.
• You can also change the browser settings and platform settings from browserstack.json file.

Step 4: Configure cypress.json file to include the .js files.

{
"testFiles":["*.js"]
}

Step 5: Execute your Browserstack test.

Use the below command to execute a Cypress visual test in BrowserStack:
browserstack-cypress run –sync

Executing Cypress Visual Regression Test With Percy

Percy is a visual testing tool that helps in visual testing of an application. Since Percy is now a part of BrowserStack, you can access Percy Dashboard with BrowserStack credentials. You can use Percy with Cypress to perform a visual test by following the steps written below:

Step 1: Install Percy using the following command:

npm install --save-dev @percy/cli @percy/cypress

Step 2: To import Percy to Cypress, navigate to cypress/support/index.js file and enter the following command:

import '@percy/cypress'

Step 3: Write your first Percy visual test script as shown below:

describe('Cypress Visual Testing', () => {
	it('Compare fullpage of Google page', () => {
		cy.visit("https://www.google.com/?hl=hi");
		cy.percySnapshot('Google');
	})
})

This example does a visual comparison of the Google.com page using Percy and Cypress.

Go to http://percy.io create a sample project. Once you create a sample project, Percy will generate an API key as seen below. Copy the Percy token from here.

Enter the Percy token using the following command for MacOS:

export PERCY_TOKEN=<your_token>

For Windows OS, enter the Percy token using the command written below:

set PERCY_TOKEN=<your_token>

For Powershell, use the below command to enter the Percy token:

$env:PERCY_TOKEN ==<your_token>

Once the Percy token is entered, use the below command to run your Percy Cypress tests:

npx percy exec -- cypress run

The Cypress visual test will start running, and it gives you following result in the command line: