Scripting Testing vs. Record-and-Replay Testing
The primary difference between these methodologies is the depth of technical expertise required for test strategy, planning, scripting, and execution.
Join the DZone community and get the full member experience.Join For Free
So, you're planning to make a move toward automation testing. But you're continuously debating what kind to opt for. Should you make a move toward record-and-replay automation testing? Or would you rather stick to good old scripting? In this article, we will help you gain clarity among the differences between these two approaches.
Software planning, development, and testing are critical activities that you'll come across in any project, and each of these activities cannot be executed in a silo since every project will have some internal and external dependencies. When it comes to testing, you should follow the right mix of black box testing and white box testing. The primary difference between the methodologies is the depth of technical expertise required for test strategy, test planning, test scripting, and test execution. A tester with expertise in black box testing tests the product against normal and boundary use cases without having any internal knowledge about the source code. This approach is completely contrary to white box testing in which the tester writes test code in order to test functionalities of the product.
Record-and-replay testing is a popular testing mechanism to perform black box testing. It's ideal for products with a static user interface where fewer changes are pushed to the production server (at least from UI/UX point of view).
Insights Into Record-and-Replay Testing
As the name signifies, pre-defined test scenarios are recorded and can be played at the push of a button (i.e. 'record' to record the use case and 'replay' to play back the recorded use case). This approach sounds interesting since the tool lets you create a script that can execute those recorded actions automatically. There is a huge upside to this approach since the QA team can get started with the product testing the earliest and the QA team doesn't need any programming expertise. It's a perfect choice for organizations preferring to perform shift-left testing.
It is a known fact that product requirements can alter bug fixes over a period of time which may result in changes of back-end code, as well as changes in the user interface (front-end). In such a situation, record and playback tests would break since they were created with the assumption that there would be infrequent changes to the UI.
Record-and-replay can be the first step in learning about test automation since the tester does not require any scripting/coding expertise. It's a good learning tool for someone with a manual testing background who wants to get a foothold in the world of automation testing. However, the recorded scripts can be large in size as the number of use cases increase and this would impact the execution speed of these scripts. It's an ideal approach for a product in which UI changes are far-fetched since it would not have much impact on the recorded scripts.
- Record-and-replay testing is suited for apps/websites with tight testing schedules.
- It is also ideal for test teams with a background in manual testing who want to venture into automation testing.
- It can be used by people outside development/test teams.
Potential Problems With Record-and-Replay Testing
Every technique has a downside; here are the issues with record-and-replay testing:
Problem 1: Too Much Recording
When you navigate on any website, you tend to perform many actions, and some of these actions might be unintentional. The issue with record-and-replay tools is that they may record too much information and a lot of it can be noise, i.e. not required to be a part of the test script. For example, below is a snippet of a recording:
- Move the mouse to location 100, 200 (relative X, Y coordinates)
- Click the 'User ID' text box
- Type the username
- Click the 'Password' text box
- Type the password
- Wait five seconds
- Click the 'Submit' button
The scenario looks perfectly fine, but a delay of five seconds could be misleading. There is a possibility that tester might have entered the wrong password and was trying to delete it or there could be some other reason.
Problem 2: Not Enough Recording
On the contrary, another potential problem with record-and-replay tests is that sometimes, the recording can be short recording and not transcribe into a valid test script. In such a scenario, the recorded test script has to be re-recorded, which takes more time and may take a couple of re-tries.
There is also a possibility that you record the wrong thing and the time taken to do the recording would be exponential depending on the complexity of the use case. For example, while recording a use case, you commit a mistake at the last step. Since you cannot record in parts, you need to re-record the whole scenario.
Problem 3: Modification of Test Cases and Execution of Test Cases in a Loop
Consider that you have a test scenario where you have to test the authenticity of a user. This would involve permutations and combinations of test cases based on usernames and passwords in the respective fields on your web app/website. If you plan to perform the testing using record-and-replay testing, you have to perform the same task for every set of usernames and passwords. You cannot modify the recorded test script to enter the details about each user in the script.
Since the recorded test script might not be understandable (e.g. it could be HexCode that can only be decoded by the record-and-replay tool), you would have a new script for each scenario even though the change in the test scenario might be minimal.
These are some of the major downsides of record-and-replay testing, but it can still be an ideal starting point for someone interested in automation testing. Plus, it can be executed by anyone belonging to your organization, from your DevOps team to your Marketing one.
Insights Into Scripting
Creating test scripts using Selenium (an open-source tool for automating web apps) is considered as the most sought-after an approach for automation testing. Unlike record-and-replay testing, a tester assigned with writing test scripts using Selenium needs a fair amount of coding expertise with Python/PHP/Perl/Ruby/other programming languages.
Depending on the functionalities that need to be tested, the tester should also have an understanding of the different modules available in Selenium for the corresponding programming language. Script-based testing is always a viable option for any kind of project (whether it is in the development phase or maintenance phase). Since Selenium is a widely used test automation framework, it is likely that members in the development team would also have a good amount of knowledge about it. In such a scenario, they can work with testers to come up with effective test cases and test scripts that have long-term benefits.
Unlike record-and-replay tests, modification of scripts (that are written using Selenium) is possible; though the turn-around time depends on the coding expertise of the script developer/tester. It is also suited for performing stress testing and regression testing, which makes it a more scalable approach for automation testing.
- Scripting using Selenium is preferred by testers with a development mindset who possess knowledge about frameworks like Selenium.
- It is suited for creating test suites/test cases in a more robust manner, as well as for performing end-to-end testing.
- Test scripts using Selenium can be used to achieve the best code coverage metrics, i.e. the test code can be used to test even the boundary scenarios along with the normal test scenarios.
Based on the timelines of the project and the technical know-how of the test team, you should choose either record-and-replay testing or scripting using a framework like Selenium. Though each approach has its own share of pros and cons, an ideal test plan is the one where tests are executed in parallel. As a tester who is responsible for testing the features of a web app/website, you should keep in mind the cross-browser compatibility of your web app across different browsers, operating systems, and devices. Parallel testing can provide the maximum throughput as far as automation cross-browser testing is concerned. If you are new to automation, record-and-replay can save the day for you. Once the tests are saved using record-and-replay, you can execute those scripts in parallel (on different combinations of browsers, operating systems, and devices). However, if you're already proficient with programming languages, Selenium can offer powerful, customizable multithreading capability for running automation cross-browser test cases in parallel with maximum efficiency.
Published at DZone with permission of Harshit Paul, DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.