Build a BDD Automation Framework With A Vision

Building a test automation framework with Selenium is more of an art than a technical activity. A lot needs to be considered when building such frameworks. Understanding client expectations for what needs to be delivered is a key aspect of thinking about automation. 

Automated testing approaches with technologies that can build applications much faster than a traditional full-stack brings automation testing time in a squeeze. Technologies like OPA and FirmStep’s Achieve Forms and Microsoft Dynamics CRM, which have known to speed up application building time significantly, have also impeded test automation. This leads to a small window for automation testing and is ruled out of scope. I have worked with all of the above technologies and have faced the same experience.

This led me to build an automation framework using Cucumber which can be understood by all the members in the project. We could sit down with the BA and the Scrum Master and translate requirements and acceptance criteria into an automated test.

A framework where we automate things to the point where we can build tests just as fast as the development team is faster by any comparison. A framework should ideally be configured to the point that a new project can be set up and automated in under two days all the way to Jenkins CI, with dashboard reporting and the ability to conduct security testing alongside Selenium testing. This should include independent reporting on each front into a reporting dashboard to give the project team a good view of the current state of the application.

Keep Things Modular

Think about being modular when writing your automated scripts. A modular development tool can let the developer build an application by dragging and dropping the core components. If there are many applications built on this technology, then you can take this approach to make things faster for future integrations.

An example with XPaths might illustrate this better. An input field for the first name should have the same structure in the DOM except for the label, which is "First Name." This way, we can build an XPath with the First Name as a parameter that completes the XPath at runtime. This also has a greater benefit as we do not have to store web elements and manage them, so similar fields with different labels can use the same functions beneath. 

Parameterizing Cucumber Files

Parameterize the Cucumber feature files into the step definitions. This approach enables the reuse of the feature file lines, giving you an easy-to-maintain file and rapid test building capability.

To summarize this vision, we have feature files that have parameters like First Name, Last Name, Email Address, and so on, which roots a single step definition and a single method that builds specific web elements at runtime.

Practice and Increase the Scope

Such an approach can be practised and honed to be used across most functionalities of the application, and when many applications are built with the same technology, then automation test building speeds up significantly.

Use Configuration Files

Configure the framework as much as possible through configuration files where less intervention in code is made.

Introduce a Headless Browser

Always provide a headless browser mode for reducing testing time. Chrome Driver provides a headless driver capability that is easy to implement. Configure the ChromeOptions class and add arguments according to your needs. The headless browser works over the HTTP protocol and does not open a browser window, which can save some time when you have a huge set of tests.

Keep a Data-Driven Approach in Mind

Use a data-driven approach through Cucumber's Scenario Outline to provide a data table. If you have a very large set of data, we do not want to pass this thought the feature files; rather, read this data from XLS or CSV using Apache POI and write the result back to an Excel or CSV. An after-test hook can run a separate function to analyze the result files.

Make the Framework OS-Independent

Make the framework OS-independent so can it can run on any platform with few or no changes. Team members can have different OSs and this creates setup activity every time the framework is pulled. Use the System class to fetch the OS and configure paths through a special class such as FrameworkPropertyConfigurator, which could set the paths based on the OS name and version, so next time you won't have to worry about what OS you are on.

Advanced Logging

I cannot stress the importance of a good logging mechanism. An advanced logger like Log4j comes with an array of appenders to write logs to files, consoles, databases, etc. You may keep log files in the framework as they are easy to get. Set the number of files to maintain in the log folder as you do not want logs from the start of time. When the framework gets pulled down, then a test log folder should be created automatically to store log files.

Logfiles should have a timestamp and the user name. Use a config file to drive this from your Logging factory class in the framework where the logger is configured.

Pay special attention to the log4j.xml, like a ton of configuration can happen here: log pattern, log user name, a number of log files to maintain, the maximum size of a log file, and more. Be sure to also provide a log switch in the config file from where the logging mechanism can be turned on or off and is pretty useful during test building.

Reporting

Reports are vital for the project teams to know the progress of the application. If you are using IntelliJ Idea, it has an in-built reporting function that needs to be called manually. For more automated reports, use the Maven Cucumber Reporting plugin and configure it to work with your runner class. You will need to configure the runner class in the plugin. Extent Report is another option with some new ExtentX reports by Vimal Selva and Anoosh Arora.

Screenshot

Provide a Screenshot feature on failure with a timestamp and page name. This helps in identifying where the failures have occurred, which is a helpful feature with the reporting and debugging.

Cross-Browser Testing

Cross-browser is a must these days and some providers like Browser Stack and SauceLabs are helpful. Always keep the test for cross-browser testing separate as they can be slower.

OWASP Zap With Selenium

For someone who needs to perform security testing while doing a Selenium test, OWASP Zap is a perfect option.
Get the OWASP Zap dependency/jar, install and start the Zap server, configure the @After  or  @AfterClass to help with the ClientApi  class from Zap proxy. Finally, set the browser to the manual proxy to listen to the server.

You can report published at the end of the test to the desired location highlighting any security issues. It will work its Spiders, Crawlers, XXS engines and many more to find vulnerabilities in our application. Zap as a standalone is just as good, but struggles when we need to pass through application login. Integrating with Selenium lets it get through login and exposes the world inside. Always seek permission before such activity. I recommend using a System Input for it to trigger (disable this input carefully).

A word of caution as this is a Security Scan which can highlight some issues but more can be discovered through a dedicated security and penetration test.

A Bunch of Utilities

Utility classes are a real game-changer to make your framework perform well. Some of the must utilities a typical framework must have are:

File Readers and Writers

This is to read and write a file like XLS, CSV, Doc, Word, and XLSM.

PDF Stripper

Some applications produce PDF reports, and to test these reports, we need to strip the PDF and asset. I recommend PDFBox, an easy API to use for PDFs, with a multitude of functions.

Reading and Writing to Database

A database connector for SQL Server or Oracle or any of your needs should be built as a component to connect and perform actions on the database. Always store the database URL and credentials in a separate file and keep it safe.

Reading an Email Inbox

Email verifications are a part of most applications, so they must be tested as well. Using an SMTP server or API like javax mail can help you automate this.

Summary

To summarize, all of this has to be tailor-made to your needs, but you can always keep building your utilities from time to time when you see a need on the horizon.