{{announcement.body}}
{{announcement.title}}

How Cross-Browser Testing Is Evolving

DZone 's Guide to

How Cross-Browser Testing Is Evolving

Cross-browser testing is no longer a nice-to-have, but essential to the testing process.

· DevOps Zone ·
Free Resource

Cross browser

Continuous improvements in cross-browser testing have made it essential in development.

Today I'm going to write about how cross-browser testing is evolving. And I'll do that with Cypress. If you know Cypress, you know it only runs on Google Chrome.

Functional testing has been around long before the introduction of Visual and UI Testing. Functional tests, also thought of as end-to-end, sit at the top of the software testing pyramid as shown in Figure 1.

The pyramid applies to both manual and automated testing.

The pyramid applies to both manual and automated testing.

The introduction of visual testing has seen its steady rise to the top tier, side by side with functional testing.

Adding visual testing

Adding visual testing

There is a fine line between writing E2E tests and writing visual tests. But don't worry, we will cover this topic in more depth. However, the desired end result is the main difference between the two.

Testing What Your User Sees

Visually testing your application across multiple browsers and devices is a building block when referring to visual testing. Maybe the most important, and ultimate goal, is the appearance of your application across multiple browsers, devices and screen resolutions.

Visual testing can be easily integrated into your functional testing with little effort thanks to tools like Applitools. Although writing functional and visual testing may be similar, each has a purpose with a different goal in mind.

Having both functional and visual testing at the top of the software testing pyramid, the question has to be asked:

  • Should functional testing be concerned with cross-browser testing?

Or

  • Should visual testing be concerned with cross-browser testing?

There is no direct or specific answer. The topic has been debated with many varying opinions.

Here, I will give my analysis in the hope of shedding light on this topic. You'll find out ways to go about functional and visual testing in software development.

What is Functional Testing?

Functional testing is a type of black-box software testing in which software is tested against functional requirements/specifications with the goal of finding bugs or defects.

Functional testing consists of six steps:

  1. The identification of functions that the software is expected to perform
  2. The creation of input dataset based on the function's specifications
  3. The determination of output based on the function's specifications
  4. The execution of the test case
  5. The comparison of actual and expected output
  6. To check whether the application works as per the customer's need

Functional testing is tightly coupled with the software functional requirements. At the end of the day, this is what you are testing. You are making sure the client's brief has been fulfilled in the application.

Take for instance the following case:

  • A login page in your application
  • The user has to enter their details and hit submit to login
  • Successful login should redirect the user to the homepage of the application

You take the above functional requirement, and dissect it to find:

  • The input of this operation is the user details
  • The output of this operation is that the user is redirected to the homepage upon a successful login

Whether you are doing manual or automated testing, you follow the same steps in performing functional testing.

One of the most popular frameworks to date, for functional or E2E testing, is Cypress.io.

Below is a sample Cypress.io script that automates the functional/E2E test:

/// <reference types="Cypress" />

context('Startup', () => {
  beforeEach(() => {
    cy.visit('/login');
  });

  it('should fill login form and redirect to homepage', () => {
   
    // Fill the username
    cy.get('[data-cy="username"]')
      .type('bhaidar')
      .should('have.value', 'bhaidar');

    // Fill the password
    cy.get('[data-cy="password"]')
      .type('123$567')
      .should('have.value', '123$567');

    // Locate and submit the form
    cy.get('login-form').submit();
    
    // Verify the app redirected you to the homepage
    cy.location('pathname', { timeout: 10000 }).should('eq', '/');
    
    // Verify the page title is "Home"
    cy.title().should('eq', 'Home');
   
  });
  
});


The script is straightforward and easy to follow:

  1. Opens the login page of the application
  2. Fills in the username and password
  3. Locates and submits the form
  4. Waits for the page to load, and checks if the "homepage" text is found on the page

If the login is successful, the application passes the requirements.

Now that you have an idea of what Functional or E2E testing is, let's look at Visual testing.

What is Visual Testing?

Visual testing addresses the behavior of the Graphical User Interface of your application, making it aesthetically pleasing to the eye. That is, the positioning of elements on the page, the colors, and many visual appearance aspects. This would include: running the same visual test across multiple browsers, devices, screen resolutions, operating systems, etc.

Visual testing is regression testing that follows certain steps to validate that the screens, or pages, have not changed unexpectedly from one test run to another, or due to codebase changes introduced since the last test run.

With the E2E test we just looked at, adding visual tests is a no brainer. In general, Visual tests are built on top of E2E tests. Yes, you can take an existing functional test, integrate a visual testing library into the mix and you got visual testing up and running.

Example: Visual Testing with Cypress and Applitools

For instance, you can start writing visual tests by using Applitools SDKs that integrates seamlessly with Cypress.io E2E tests.

Let's take the same E2E test, and add a few steps to insert a visual test:

Visual test steps

Visual test steps


Taking the same Cypress.io test script, you can add Applitools SDK calls to take snapshots at different stages of running this script:

/// <reference types="Cypress" />

context('Startup', () => {
  beforeEach(() => {
    cy.visit('/login');
  });

  it('should fill login form and redirect to homepage', () => {
   
   // Start the test
    cy.eyesOpen({
      testName: 'Login Form'
    });
    
    // Take a snapshot
    cy.eyesCheckWindow('Login page');
    
    // Fill the username
    cy.get('[data-cy="username"]')
      .type('bhaidar')
      .should('have.value', 'bhaidar');

    // Fill the password
    cy.get('[data-cy="password"]')
      .type('123$567')
      .should('have.value', '123$567');

    // Take a snapshot
    cy.eyesCheckWindow('Login page with credentials');

    // Locate and submit the form
    cy.get('login-form').submit();
    
    // Verify the app redirected you to the homepage
    cy.location('pathname', { timeout: 10000 }).should('eq', '/');
    
    // Verify the page title is "Home"
    cy.title().should('eq', 'Home');
    
    // Take a snapshot
    cy.eyesCheckWindow('Home page');

  });
  
});

Do you notice how visual test scripts enhance E2E tests, so they complement each other?

That's not the whole story! Now the state of the DOM captured along with the visual snapshots get sent to the Applitools Ultrafast Grid in the cloud. Based on how you set up your test, the Ultrafast Grid would configure and start a new testing environment — different browsers, device emulators, screen resolutions. It would then run your DOM snapshots, generate images, and finally, sends them to the Applitools AI server.

The AI server will do all the comparisons for all the snapshots by comparing a baseline snapshot (if it exists) to the current image snapshot. Depending on the comparison results, you may keep the baseline snapshot (if it existed in the first place) or make this the baseline snapshot to compare to the next time a visual test is run or decide this latest snapshot will be the new baseline to compare against.

This is a general overview of how visual testing is done when using Applitools, and has the capacity to test many times over.

The takeaway is that visual tests make use of E2E tests. By employing visual testing tools like Applitools, the return on writing E2E tests is doubled. You are writing your E2E tests once. As a result, you are now performing functional and visual testing to make sure your application passes both the functional and visual appearance tests.

Cross-Browser Automation Testing

What is it and why do we need it?

Cross-browser automation testing is the act of running your automated tests across multiple, and different browsers, belonging to different platforms, and devices.

Generally, in web development, an essential part of our job is to ensure that the JavaScript and CSS we write are cross-browser. That means that the application runs smoothly across all platforms.

You can relate this concept to automation testing. When you run your automated test scripts across multiple browsers, you are safeguarding your application to work properly across the board.

The above is not 100% true when it comes to functional or E2E tests. However, this is always true when discussing cross-browser visual testing.

Let's delve further, and compare which method is more effective when testing our applications by looking at cross-browser functional, or cross-browser visual, testing.

Cross-Browser Functional Tests

You've learned that functional, or E2E, tests deal with validating the logic of the application, and making sure the function satisfies the specifications given to you by your client.

A functional test follows a step by step path in your application, allowing it the ability to validate its primary function.

An example of this might be in an e-commerce application validating a credit card number being entered by the user at the checkout. In the case of an SPA (Single Page App) your JavaScript code is the validator.

Another example, mentioned earlier, where a user enters their details on a Login form, and is redirected to the homepage, is the responsibility of the JavaScript code if it's in the form of a SPA.

Yet another example of an E2E test is to make sure the Logout button is visible on the top navigation bar once a user is logged in. This feature involves writing a function to check the LocalStorage and show the Logout button. Once again, this E2E is testing the login process that is controlled by writing JavaScript code if it is a SPA.

The point I am making is this: if you follow the strictest procedure when writing your JavaScript code, and use native JavaScript APIs that are certified cross-browser when selecting third-party libraries that are cross-browser compatible, you are making sure that your E2E tests (once they run and pass against one browser) will run against other browsers too. There are exceptions to the rule, but only a small percentage of your E2E tests.

These exceptions might be a result of using third party libraries, or native JavaScript APIs, that are known to have issues on some browsers. You might opt to run your E2E tests on another browser to establish how the application fails, and try to provide a solution.

Establishing a cross-browser lab to run your tests is expensive. This involves setting up multiple computers, each with different operating systems, multiple devices, phones and tablets. It's just not feasible. Add up costs, time, and the never-ending advances in technology, and you will agree, it's a nightmare.

When using Applitools Ultrafast Grid, you can forget cross-browser functional testing because eventually your E2E tests will be converted to Visual tests. Ultrafast Grid will do the job of running your E2E tests across multiple browsers. More on this shortly.

Cross-Browser Visual Testing

Cross-browser visual testing is a must. You are testing the visual appearance of your application rather than the function of it.

Visual testing is about testing the placement of elements on the page, the coloring of elements, the relation between elements when the screen size changes. For instance, do elements overlap each other? Do they obscure each other? How responsive is your page for different screen resolutions? How responsive is your page on smaller devices? Bigger devices?

All these questions cannot be answered by doing cross-browser visual testing. Despite the that fact you might be using cross-browser CSS frameworks like Bootstrap, the probability of using CSS selectors and styles that are incompatible is highly likely. There's a ton of CSS stuff that's available in one browser and not on another. You are more prone to error by using non-cross-browser CSS styles than non-cross-browser JavaScript code.

Therefore, it makes perfect sense to use Applitools Ultrafast Grid to solve this issue. You will see how inexpensive and reliable it truly is to perform cross-browser visual testing.

Why Applitools Ultrafast Grid?

If you are new to Ultrafast Grid, read my article titled about how I ran 100 UI tests in just 20 seconds.

In brief, Ultrafast Grid is composed of two components:

  • Client-side component, representing SDKS, and library, to be used when writing visual tests, to capture DOM snapshots, and send these snapshots to the server-side component living in the cloud.
  • Server-side component is usually hosted in the cloud, and receives DOM snapshots from its counter client-side component. Based on a set of configuration settings, Ultrafast Grid will establish a cross-browser testing environment, run all of your DOM snapshots there, generate image snapshots, and, finally, send those snapshots to the Applitools AI Engine that takes over the mission of doing image comparison and analysis.

When you employ Ultrafast Grid in your visual testing, you usually configure the settings of your test runs.

This configuration file instructs Ultrafast Grid on what browsers and devices are needed for your tests. Ultrafast Grid runs all your visual tests in all the browsers, operating systems, and screen resolutions you specify. You see how inexpensive it is to use Ultrafast Grid for cross-browser testing?

Not only this, with Ultrafast Grid, your visual tests run faster. You are no longer sending the server image snapshots you have captured on the client-side, but rather DOM snapshots containing the required resources from HTML, CSS and JavaScript that are enough to re-generate on the server (by the Ultrafast Grid) in order to generate the real image snapshots.

By utilizing the Ultrafast Grid, you get a bonus! As mentioned, when writing visual tests in Applitools, you are taking your functional, or E2E tests, adding to them a few calls to hook into the Applitools SDKs, and you are ready to start visual testing. So eventually, and indirectly, your E2E tests are being run on multiple browsers, and you get a chance to run your functional tests on multiple browsers inexpensively.

Conclusion

Cross-browser testing is not a "Yes" or "No" thing. Many factors play into whether or not you should do it, especially when you are dealing with functional, or E2E testing.

However, it becomes mandatory when you are doing visual testing.

Establishing an efficient, reliable, and accurate testing environment has been made very simple if you opt to use tools like Applitools Ultrafast Grid.

Also published on Medium.

Further Reading

Top Tips for Better Cross-Browser Testing 

Functional Testing vs. Non-Functional Testing

Topics:
cross browser testing ,cross browser testing tool ,software testing ,test automation ,test automation tools ,devops ,visual testing ,functional testing

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}