I deal with lots of users who use automation tools to test their systems every day. I interact with them for technical support as well as training and consulting services. I have seen certain mistakes that are repeated more often than others. Here is a list of some of the common mistakes in automation testing.
1. Simple "Record and Replay"
Many testers seem to think of automation testing as little more than record and replay. The fact is that an effective automation testing requires you to customize the generated script. The record feature should be viewed as a way to generate a skeletal script and should rarely be viewed as the final step in script creation. Script customization could involve data parameterization and added checkpoints for validation. It could also involve modularizing the script to allow several testers to work on it at one time.
2. No Validation
It amazes me how many scripts are created and used without any validation. If you are testing a system that has a login page, you would want to find out if that login succeeded. You could do this by validating the resulting page by using checkpoints. Checkpoints could be used to detect web objects, page parameters, or perhaps some text on the page. Checkpoints should be placed at as many points as possible.
3. Visible Validation Only
Validations mentioned earlier should not be restricted to validations that are visible (i.e., page text and the like). If you are using an order entry system to place an order, you might want to query the database to ensure that the order was in fact saved successfully into the database. Similarly, if a particular operation results in the creation of a file, you might want to validate the contents of that file.
4. Replacing Human Testing
Automation is a great way to augment your testing efforts. However, don’t expect automation to allow you to completely replace human testing. Automation works best when you know what to look for. In some cases, you may generate a web page that has incoherent fonts. Unless you are specifically looking for this error (which is rather unlikely), you are likely to only find this issue.
5. Inappropriate Test Cases
The test cases chosen for automation need to represent a significant portion of user activity. There are an astronomical number of paths that can be taken by the user. However, the trick is to condense all possible paths to a small sample of highly representative test cases. This is more of an art than a science.