Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Improving Your Regression Testing Projects Through Automation

DZone's Guide to

Improving Your Regression Testing Projects Through Automation

Learn how to build a smart plan to automate regression testing to save time and improve software quality.

· DevOps Zone ·
Free Resource

Is the concept of adopting a continuous everything model a daunting task for your fast moving business? Read this whitepaper to break down and understand one of the key pillars of this model in Continuous Governance: The Guardrails for Continuous Everything.

What Is Regression Testing?

Regression testing, also known as repeated testing, is the process of ensuring all the old functionalities still correctly work with new changes. In other words, regression testing tests an already tested application to find defects resulting from changes. This is a common step in any software development process and is done by testing specialists. Testers do regression testing by re-executing tests against a modified application to evaluate whether the revised code breaks anything which was working earlier. The only reason regression testing might not work is that changing or adding new code to a program can easily introduce errors into code that is not intended to be changed. Regression testing is required in the following scenarios:

  • When the code is modified according to a change of the requirements
  • When new functionality is added
  • While fixing defects
  • While fixing performance issues

Why Should Regression Testing Be Automated?

Let’s consider an example and how to perform regression testing in that situation. At the beginning of a project, assuming we have six developers and two testers, the Agile model is set up every two weeks for a sprint.

In the first sprint, we start with the basic features (about ten functions). The testers begin designing testing scenarios (about 100 scenarios). The very first sprint receives a good rating from customers.

In the second sprint, the developers continue to create new features — let's say 10, and the testers also do things like in the first sprint, with 100 new scripts plus 100 old scenarios that need to be retested. Well, that's only 200 scripts, everything is still under control.

In the next sprint, the developers need to make eight new features and update the two old features due to new customer requirements. At this point, the two testers not only have to design test scripts for the eight new features but also have to test and update 200 old scenarios. The whole implementation is about 300 scenarios. Do you feel there is something wrong?

Over the next few sprints, the developers still meet the number of features and the changing requirements, but with two testers, the number of scripts to create and update is much higher. Tiredness begins to spread. The lack of time and the risk of missing a bug is higher and higher. Too many problems arise.

Therefore, when regression testing is automated, it enables testers to check a variety of changes and unusual cases in the production environment. Not all regressions are caused by new features or the consequences of routine bug fixes; database updates or new browser versions can also cause them. Regression can also be an issue with efficiency and speed. Automating those cases which are stable and repeatable allows manual testers to spend more time testing various environments and merging more complicated cases at a higher level.

What's more, regression analysis is the key to success. It needs to deal with intelligence rather than hard work.

  • Highest Return: Execute tests that contribute to high coverage of the requirements, then any others.
  • Quick, Lower Risk: Execute tests for the most critical requirements, then any others.
  • Practically Safe: Execute tests for all the critical requirements, then any others, especially since often ~20% of the test cases cover ~80% of the business value.

How to Improve Automated Regression Testing

Two automated regression testing approaches are most used: data-driven testing and keyword-driven testing. Data-driven testing is a framework where test input and output values are read from data files (data pools, ODBC sources, CSV files, Excel files, DAO objects, ADO objects, etc) and are loaded into variables in captured or manually coded scripts. In this approach, variables are used for both input values and output verification values. Navigation through the program, reading of the data files, and logging of test status and information are all coded in the test script. Let’s consider a simple example of a data-driven script for setting up Admin accounts in a system:

private void Auto_ID_400_test()
{
     Common.InitializeWebDocument();
     DataInit.Init();

     BasePage.NavigateToHomePage();
     HomePage.NavigateToSettings();
     SettingBasePage.NavigateToBase();

     Base.AdminSection.AddNewAdmin(Data.Read("NewAdmin"));
}
public static void AddNewAdmin(DataTable dataTable)
{
     Report.Log(ReportLevel.Info, "Add new Admin:");

     OpenAddnewAdminPopUp();

     FillNewAdminData(dataTable);

     ConfirmAdmin(Read(dataTable.GetValue"NewAdminName"));
}
NewAdminName NewAdminPassword NewAdminPasswordRepeat AdminRole AdminTenant ExpectedResult
base_admin admin123 admin123 Basic tenant administrator tenantA Created
dasklfkj@#@!# admin333 admin333 Basic tenant administrator tenantF Invalid
invalid_admin admin444 admin444 Basic tenant administrator tenantA Invalid
base_admin1 admin555 admin555 Basic tenant administrator tenantB Created
base_admin8 admin888 admin888 Basic tenant administrator tenantB Invalid
g&32LKLG admin777 admin777 Basic tenant administrator tenantB Invalid
base_admin2 admin456 admin456 Basic tenant administrator tenantC Created
base_admin9 admin773 admin773 Basic tenant administrator tenantE Created

Keyword-driven testing is a technique that separates much of the programming work from the actual test steps so that the test steps can be developed earlier and can often be maintained with only minor updates, even when the application or testing needs change significantly. The keyword-driven testing methodology divides test creation into two stages: Planning Stage, Implementation Stage.

Example

We define a test case for login: Navigate to the login page, type a user ID, type the password, verify login.

Then, automated testing engineers develop scripts for the keywords:

Actions Keyword
Navigate to the login page NavigateTo
Type a user ID TypeUserID
Type the password TypePass
Verify login VerifyLogin

This strategy does not require programming skills, so it much easier for non-technical people to contribute to the regression testing process. By way of explanation, it is more suitable for the projects catering to a large audience and requiring a broader focus than the data-driven testing.

Read more for a practical example of writing test cases for the Facebook login function at Test Automation Resources

Are you looking for greater insight into your software development value stream? Check out this whitepaper: DevOps Performance: The Importance of Measuring Throughput and Stability to see how CloudBees DevOptics can give you the visibility to improve your continuous delivery process.

Topics:
test automation ,regression testing ,devops ,performance ,software quality

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}