There are few topics in quality assurance testing that cause as much confusion as smoke testing versus sanity testing. The two names would seem to describe very different practices—and they do! But people still get them mixed up since the distinction is kind of subtle.
Whether you are developing a mobile app, web service, or Internet of Things program, you will probably undertake smoke as well as sanity testing along the way, likely in that order. Smoke testing is a more generalized, high-level approach to testing, while sanity testing is more particular and focused on fine-grain details.
Let's take a look at each one in depth.
The first thing you may be wondering about is, why is it called "smoke testing?"
The basics of smoke testing are 'name' and 'function.'
The 'name' is unusual but it makes sense.
- In fact, the term originates with hardware testing.
- Test engineers who turn on a PC, server, or storage appliance, check for literal smoke coming from the components once the power is running. If no smoke is detected, the test is passed. If not, all other project-related work has to be put on hold until it passes.
- As we can see, the idea is to verify that the most basic functionality—the ability to be powered on without catching on fire in the case of hardware, and successful startup and connection to various necessary libraries, services, etc., for software—is operating properly before additional testing is undertaken.
Smoke testing usually takes place at the beginning of the software testing life cycle. It verifies the quality of a build—i.e., a collection of files that make up (or "build") a program—and checks to see if basic tasks can be properly executed. The idea is to ensure that the initial build is stable. If it can't pass a smoke test, then it has to be redone before the actual testing phase can be pursued. At some organizations, smoke testing is called 'build verification testing.'
"In smoke testing, the test cases chosen cover the most important functionality or component of the system," explained a guide from Guru99. "The objective is not to perform exhaustive testing, but to verify that the critical functionalities of the system [are] working fine. For example a typical smoke test would be – verify that the application launches successfully, check that the GUI is responsive, etc."
Performing Smoke Tests
A smoke test might be done manually or be automated. So you could create manual test cases or come up with a script that would automatically check to see if the software could be installed and launched without incident. An enterprise test management suite can be used to help with your smoke tests.
Sanity testing is sometimes called a sanity check. Like a literal sanity check, it is not meant to be exhaustive and instead supposed to verify that recent changes are not causing any major problems. The "sanity" in the name just refers to the idea of making sure that developers were rational when updating an application.
Basics of Sanity testing and Differences from Smoke Testing
It is common to categorize sanity testing as a subset of acceptance testing, which is a much more thorough process. Sanity testing:
- Is usually done near the end of a test cycle, to see if bugs have been fixed and if any minor changes to the code are being well tolerated.
- Is typically executed after receiving a new build, to see if the most recent fixes break any part of the application.
- Is often unscripted and may take a "narrow and deep" approach as opposed to the "wide and shallow" route of smoke testing.
- Helps determine that an app can do the basics. So a sanity check could be used to see if a calculator app can give a correct result for 2 + 2. If it can't, then there's no point yet in doing further tests on its ability to handle more advanced things like trigonometric functions.
- Can be done manually or with the help of automated tools.
As we can see, there is some overlap between smoke testing and sanity testing, especially when it comes to the fact that neither is really designed to be a thorough process. However, there are also obvious and important differences.
Developers and testers rely on smoke and sanity testing to move through application development and deployment with as few delays and technical errors as possible. Overall, we can look at smoke testing and sanity testing as similar processes at the opposite ends of a test cycle. Smoke testing ensures that the fundamentals of the software are sound so that more in-depth testing can be conducted, while sanity testing looks back to see if the changes made after additional development and testing broke anything.