Responsive Web Design Testing Strategy Leveraging Selenium Automation

There’s been a clear transition toward developing responsive website design (RWD) over the past few years. This adoption is driven by factors like:

• Consistent user experience (UX) and display across platforms (mobile, desktop, tablet, etc.).
• Easy maintenance of the product – write once, deploy anywhere concept.
• Cost savings due to specific dev and test skill set and tooling used across the RWD SDLC.

With the excitement and benefits above, there needs to be a proper strategy in place around the SDLC, and especially the test automation angle.

Source.

To achieve a good user experience and better quality, your RWD test plan should focus on the following important points:

• Does the application look okay (visually)?
• Are there any bugs due to variation in screen size?
• Is the site optimized for mobile/tablet/laptop?
• How are the image size and quality (this impacts UX and performance/load time)?
• How do the font size and type look across platforms?
• Are the navigations straightforward (UX/Usability wise)?
• Is there selective content display, and does it work?
• Does the website perform consistently across browsers (IE/Safari/Edge/Chrome)?
• Does the RWD site meet the accessibility guidelines?

The above is only a subset of questions to ask when planning your testing activities on your RWD site.

Selenium Automation For Responsive Websites

As indicated above, there are a lot of considerations and tests to be accomplished on a RWD site, therefore manually testing across platforms with the UX/performance/visual end-goal in mind is simply not an option.

When it comes to automation, it’s no secret that Selenium is the de-facto test automation tool for standard and responsive websites. While Selenium is powerful, it does require complementary tools to meet the excessive quality goals listed above.

Let’s list a few important requirements that will complement Selenium, using either open-source tools or products.

Tools like Galen test framework are great add-ons to Selenium, especially when targeting RWD testing, so if you’re in the process of building your Selenium-based test suite, consider leveraging the benefits that framework can add – it fully supports Selenium-Grid, and provides cross-platform insights through its ability to test the location of objects relative to each other on the page using a special syntax and comprehensive rule set.

So, what are the top capabilities to cover as part of your RWD test plan? I would list the following:

1. Visual validation.
2. Environment-based testing.
3. Accessibility compliance.
4. Quality analysis.

Visual Validations

Selenium cannot meet a goal of visually testing an RWD across different platforms and provide insights into a visual glitch. This is where tools like Perfecto and Applitools come into play – complementing Selenium to add the capabilities listed above. The blog contributed by Amir Rozenberg from the product management team at Perfecto, describes a step-by-step instruction on how to use visual validation within Perfecto using Applitools Eyes to visually test the Geico.com RWD site.

Environment Condition Testing

Since RWD sites are aimed to be used by almost any target platform in the market including smartphones and tablets, it is important to realize that these platforms are highly sensitive to varying conditions. Including in your test plan things like incoming events, background apps, location services, and changing network conditions can be the difference between a successful and engaging RWD and a bad one.

The cube below represents a test coverage matrix that at its base focuses on the test cases and test data (e.g. TDD), then on the target platforms (mobile, desktop, etc.), and lastly on the personas that represent the environment characteristics as mentioned above (network, location, apps, etc.)

Responsive Web Design Testing Strategy- What to Cover?

In almost any customer engagement, the topic of accessibility compliance comes up. Most organizations are struggling with such testing and, in most cases, they are running such tests manually.

Considering the number of platforms teams need to cover as part of an RWD site testing, adding manual into the mix is a bad practice, and a velocity blocker.

For that purpose, there is a very useful tool called WAVE, that allows teams to build their accessibility test steps on top of their Selenium suite.

In the following site, you can find a working example of the WAVE tool on top of Selenium that you can re-use for your RWD site and quite easily gain value and kick-start your accessibility automation branch.

As you’ll see below, the end-result of running such code on an RWD site is a report that highlights the accessibility issues that the tool (WAVE) detected on your page(s).

The first visual is within the Perfecto lab in the cloud, and the second one (Geico.com) is through the simple website of WAVE.

Quality Analysis

When thinking of thousands of tests being executed across platforms, it is clearly a challenge at the end of each execution to evaluate each failure, compare the results across platforms and drill down/triage the failures toward a fast resolution. Selenium-based reports, whether being executed through TestNG or another open-source execution framework, are not as insightful when it comes to quality analysis at scale.

For that – based on an open-source project – Perfecto implemented its DigitalZoom™. This tag-based reporting framework, allows test automation practitioners to put structure into the test code, so that at the end of each cycle, teams can easily see a dashboard, drill down and carry out a fast RCA (root-cause-analysis).

Finally, and as mentioned above – testing the UX, rendering, and performance of your RWD site, is crucial to assure continuous engagement both in “quiet” times as well as “peak” events.

For that, enabling timer measurements is a great way to complement the Selenium automation solution as can be seen in the code snippet (below) also taken from the Github sample contributed by Amir (above).

In this post, we covered both the great value and benefits RWD design of choice can bring to organizations that implement such, together with some challenges teams might face by simply leveraging the plain Selenium framework.

Having a clear RWD test strategy that extends Selenium and adds capabilities like those referenced above can enhance the test automation coverage, release cycle velocity and overall RWD site engagement. To learn more, download our Guide to Responsive Web Testing.