Automated or Manual Testing: Keeping the Balance Right!
Automated or Manual Testing: Keeping the Balance Right!
It's not so much a matter of whether you should use automated or manual testing, but of when you should use each.
Join the DZone community and get the full member experience.Join For Free
If you are a tester, then you have likely had a discussion about automated and manual testing. This is nothing new, and lots of people have different views around this. Whether you are on a big team and already established an automation framework or you are a small team new to automation, it is always necessary to keep this balance right in order to get maximum efficiency.
Automation testing has the benefits of increasing efficiency, getting faster regressions, and thus contributing to timely project deliveries. It also removes the execution of repetitive test cases or regression cases manually and saves a tester’s time.
But before considering automation, there are certain points which you should evaluate. You have probably heard the statement “You can not automate everything,” which is very true.
Manual testing is required in many cases. In fact, the biggest drawback of manual testing is its biggest advantage: that it requires human intervention! There are certain cases in which require human instinct and intuitiveness to test a system. To name a few, these are the following cases where manual testing plays a vital role:
- Usability Testing: This is testing an application from the perspective of how easy or difficult is it to understand. This is to test how interactive the application is to the users who are going to use it. These kinds of tests cannot be automated and must be performed manually.
- UI and UX Testing: UI and UX testing cannot be automated. and even if you try, it would be only to some extent. Automation scripts can be used to test the layout, CSS errors, and HTML structure but the whole user experience cannot be automated as it is very subjective.
- Exploratory Testing: Cem Kaner who coined the term in 1984, defines exploratory testing as, “A style of software testing that emphasizes the personal freedom and responsibility of the individual tester to continually optimize the quality of his/her work by treating test-related learning, test design, test execution, and test result interpretation as mutually supportive activities that run in parallel throughout the project.”
- Ad-hoc Testing: This is completely unplanned testing which relies on tester’s insight and approach. There is no script ready for this testing and has to be performed manually.
Pros of Automation Testing
There are certain cases where automation testing is beneficial and can actually reduce efforts and increase productivity. Let’s have a look"
- Regression Testing: Regression cases are mostly repetitive and we can automate them once and execute in a timely manner.
- Load Testing: Automation is very much useful in case of load testing. Load testing identifies the bottlenecks in the system under various workloads and checks how the system reacts when the load is gradually increased, which can be achieved by automation.
- Performance Testing: Performance testing is defined as a type of software testing used to ensure whether software applications perform well under their expected workload. Automation is very useful in this type of testing.
Apart from that, the test cases which are repetitive can be automated. Keeping in mind the above points, you can decide on the what, how and why of automation. To maintain a balance between manual and automation can be very tricky at times. I have seen many aggressive managers pushing to automate everything. But is this the best approach?
Before starting to automate, you need to answer these three questions.
1) What Needs to Be Automated?
Let’s first think of what exactly needs to be automated. What part of the requirement, feature, or application is a candidate looking to be automated? Often the application which is going to be automated is called an AUT (Application Under Test). It is quite possible that a part of a feature can be automated while the rest should be tested manually.
This requires a deep-dive into the feature, its test cases, and effort which will be required. Sometimes knowing how a developer is going to implement that feature plays a vital role in deciding if it can be automated and to what extent.
2) Why Should You Automate?
This is very important. Why do you need to automate? Is it because it reduces effort and increases efficiency or because it would benefit you in the long run? Or is it just conventional? During my tenure as a quality analyst, I found some managers who aggressively wanted everything to be automated without analyzing that it might increase effort and reap no fruit. You might end up asking a few questions to yourself:
- Is it a one-time requirement? We probably don’t need to automate then.
- Is the automation solution complex? If automating it leads to building a parallel application itself, there is no point of automation. But in a case where the solution is complex but the feature or AUT keeps on changing and development is planned for the long term, then you may find automation beneficial.
- Are there any time constraints? There might be a time constraint in delivery. At that time, the manager’s role is very crucial in deciding to invest in automation or go for manual.
- Does your team have the appropriate skill and resources? This is also an important factor. How many automation engineers are available in the testing team to leverage their bandwidth for AUT? For small teams, this is the deciding factor to go for automation.
3) How Should You Automate?
This is where we decide how we are going to automate the AUT. Do not confuse this with AUT or feature as this stands for both if you are going to automate entire application or a small feature. Sometimes for a small application, you need not have a full-fledged automation framework.
There are various tools available which don’t require coding and can solve the requirement. TruBot from CloudQA is one such tool which has many handy tools for different types of testing and is very user-friendly for small applications.
Apart from that, there are various requirements which can be met by simply writing a shell script.
A full-fledged automation framework is required when the AUT is big and there are continuous enhancements going on. At that time, a regression suite can be executed before each feature release and automation can significantly reduce the effort.
Automation framework development requires both coding skills and time, so before jumping into that, a tester should always analyze the ROI and then make a plan accordingly.
Let’s conclude that using either only manual or only automated testing is not the right approach. There should be a balance between both and I hope that these points will be helpful in finding the right balance.
Opinions expressed by DZone contributors are their own.