Question: Can Selenium be used in testing real-world scenarios, and would spending time programming Selenium to create different scenarios and permutations take longer than using the system to manual creating scenarios through exploration?
The short answer is yes, it can be used for real-world scenarios, and it probably takes longer than interacting manually.
The rest of this article is the long answer.
Can Selenium Be Used in Real-World Testing Scenarios?
Yes. I, and teams I work with, use Selenium WebDriver in the real world for production work and to implement user scenarios.
Does It Take Longer Than Interacting Manually?
Generally, yes. Exploratory testing of the system app and interacting with it will likely take less time than automating that interaction, at least in the short term. But how many permutations are you talking about?
Probably still faster interacting manually.
At some point, the physical interaction would probably take longer than the automated interaction. And where are we considering adding permutations? Data? Flow?
We Have a Bunch of “What Ifs” to Consider
- What if we coded the automated interaction badly and it fails because of synchronization issues that we don’t understand? We might not get value from it because we don’t trust it and we have to keep fiddling with the code to make it work.
- What if the system kept changing and we didn’t abstract the different layers of the system correctly? With every system change, we'd have to amend so much code that the amendment would take longer than interacting physically with the system.
- What if the permutation was just input data and expected output data? Then, the automated scenario might add value almost immediately because the same path is executed, but with massive data permutation. Automated tactics can excel in these circumstances.
- What if the permutation is subtle tweaks to the interaction (i.e., different orders of inputs, different input mechanisms, different form navigation methods, etc.)? When the execution path changes, then the automated interaction becomes harder to code for and synchronization becomes essential. This is not impossible, just a different approach to the work.
- What if the result checking was incredibly time-consuming for a human to perform? Poring through long reports, calculating lots of different totals for different permutations, pulling results out of the database and merging with information from a REST interface and comparing to a GUI representation...perhaps an automated comparison routine would take less time to create and execute.
It Can Be Done...
I’ve successfully automated:
- Lots of data combinations across single execution path.
- Permutations of input order (i.e., surname, first name, address; address, first name, surname).
- Different navigation paths through a GUI.
- Different submission approaches for a form (submit, keypress, mouse click, etc.).
I've done thes e things all while the system is changing regularly.
Sometimes, the automated approach has taken longer than the manual effort, but over time, had we continued to interact with the same scenarios manually, we would either have had to not cover those scenarios or perform those scenarios at the expense of covering other areas of the system. Automating it seemed like the right thing to do.
Sometimes, the effort of automating the system seemed to take far too long given the short-term nature of the product, and the automated code was dropped in favor of manually interacting with the system.
We have to make choices when we choose our test approaches. Hopefully, with experience, we can learn to choose the most effective balance between manual interaction or automating without pursuing the wrong path first, wasting time, and then changing our approach.