Adding Exploratory Testing to Your Testing Strategy for Better Coverage

What is Exploratory Testing?

By definition, exploratory testing is a software testing style that emphasizes the personal freedom and responsibility of the individual tester, to continually optimize the quality of his/her work, by treating test-related learning, test design, test execution, and test result interpretation as mutually supportive activities, that run in parallel throughout the project.

Exploratory testing relies on the tester’s experience and thought process, where the tester is involved in minimum planning and maximum execution, and that is why it is not heavily-documented or scripted. But that is not to say that it cannot be documented on some level and become an integrated type of testing to provide excellent QA coverage.

Exploration of software is basically looking at a certain feature, functionality, change and the overall behavior from learning as well as from a critical standpoint. Exploratory Testing is a crucial aspect of software testing, which almost every tester performs knowingly or subconsciously. That is to say, all testing is exploratory to some extent. Even when performing a test based on a written script, a good tester will do so with some degree of exploration.

Exploratory testing is also known as ad hoc testing, and was known solely by this term until the early 1990’s, when a group now called the “Context-Driven School” started using the term exploratory. Cem Kaner was the person that coined the term Exploratory Testing in his book, Testing Computer Software, and described it as:

“Simultaneous test design, test execution and learning with an emphasis on learning.”

As an experienced tester myself, I cannot stress enough upon the benefits of exploratory testing. Working in an Agile team, we are facing the struggles of time-crunch and release-time pressure. Though we did have scripted tests and automated regressions, my best defects were found during exploration!

Session-Based Test Management

Some test managers or project managers are wary of exploratory tests because of the lack of ways to manage or gather evidence of work performed or achieved by testers. This is where session-based test management comes in. Session-based test management aims to combine accountability and exploratory testing in order to provide rapid defect discovery, creative on-the-fly test design, management control, and metrics reporting.

For a novice team, the best way to start with exploratory tests would be to set aside an uninterrupted time box for exploration with a certain objective in mind. Set the goal and have a block of 1-2 hours set aside for looking around that feature/component/functionality/behavior or use case of the application, with the focus of finding defects. During this time, keep a note of your observations and report them at the end. This is called a test session and the pre-defined goal you set for your session is called a test charter. A test manager may then keep track of the number of test sessions allotted to a tester, their time spent, and their observations of value, which may include defects, questions on design, or doubts about the usage of features.

For teams that are larger in size, more mature in their exploratory test skills, or for teams in need of more structure around their exploration, there are many tools that support exploratory testing by providing standardized ways of creating, assigning, and fulfilling these test charters.

Exploratory Testing Tools

Test management tools like PractiTest, that support exploratory and session-based testing practices can give your QA coverage that extra advantage, by enabling all various testing efforts managed under one roof.

Exploratory tests tools allow you to define charters for your exploratory test sessions, document your notes, defects and observations as annotations as you are running your tests, report bugs directly from your runs, and finally, to review your exploratory test sessions with colleagues in order to gather feedback. They also help to time your session and keep a track of the history and logs of past runs of your test sessions. These exploratory testing tools’ features will help you create as well as manage exploratory test sessions, and also help easily combine them with scripted as well as automated test runs.

Adding Exploratory Testing for Optimum QA Coverage

It is no secret that software development is shifting more and more towards Agile and DevOps practices. These practices are pushing organizations that applying them to provide faster releases, without lowering their quality standards. This process has had some important effects on software testing practices as well.

The clearest and most visible of those effects is the obscureness of boundaries between developers and testers. From separate, and even at times opposing teams, to one joint force on a single team, with the common goal of releasing a quality product.

The increasing popularity of test automation on Agile teams is also contributing to the blurriness between developer and tester roles. Testers are now expected to understand code better and in most cases to know how to write code, while developers are to have a role in testing their own code.

The demands from QA teams are constantly getting more intense, and it is a struggle to keep up with the ever-changing testing world. Finding new testing strategies and following “best practices” that can add extra value to the product’s quality is an unceasing effort.

Foreseeing this trend, PractiTest has continuously made the effort to develop its platform to accommodate the various types of testing used in today’s software development market, with the understanding that there is no one solution for software quality. A combination, however, of different testing types and approaches can help meeting those quality goals and add this extra value.

So what should we mix into our Agile testing strategy for maximum coverage?

Scripted Tests

Scripted testing is a step-by-step testing approach for testing a specific functionality. This approach confides on the testers’ skills and common sense. Even though it is considered to be a precise testing method, it does has some disadvantages when it comes to Agile development. The main shortcoming is the time factor. Executing scripted tests can take a lot of the testing teams’ valuable time. Additionally, while the scripted tests method does offer great coverage for planned requirements, it does not offer coverage for the unexpected and the for the unplanned.

Automated Testing

As said earlier, the automation testing method is continuously expanding its reach, and it is now almost mandatory to include automation in the testing process in order to keep up with market demands. The use for automated testing (scripting) has overwhelmingly increased over the past five years, and for good reason. The process of automating some of the formerly manually executed scripted tests has allowed QA teams to speed up the process and save some valuable time. Automated tests can also be scripted using one of the test automation tools out there, and the skill of writing such scripts has become much needed for QA professionals.

By automating the repetitive manual tests, testers can free up more time for conducting more creative exploratory and session based tests.

Manual Exploratory Testing and Session-Based Testing

Exploratory testing is more intuitive-based and requires testers to be professional and responsible to continually optimize their work. It allows testers to gain a deeper knowledge of the system, and to get the user’s perspective on its functionalities, combined with their inside business knowledge. By performing exploratory sessions, the test team can better identify weak spots and design new test cases for better coverage of those weaknesses. Additionally, testers are more likely to find new defects in those ET sessions than while running regression tests that are executed over and over again as they tend to lead to new directions.

By adding exploratory and session based testing to the testing process, the QA team can save a lot of valuable time. When performed by skilled and experienced testers, the focus of the session can be altered on the fly, directing the testing team to find vulnerabilities and bugs faster and to address them more efficiently.

How to Combine Exploratory Practices Into Your Testing Routine

The two main ways for a test lead to manage exploratory tests are delegation and participation.

When delegating, the test lead will usually dictate a test charter, leaving the execution for a member (or few members) of his team, while the team reports back the results of the session. Participation of the lead in the sessions means the test lead will have an active role in the conducted session. Participation will allow the test lead to shift the session’s strategy in real time and serve as a live example for what is expected from the testers while performing the sessions.

For smaller and more novice teams, the best way for blending exploratory testing in the mix will be dedicating a designated time slot for sessions, while keeping in mind a certain functionality, feature, component or behavior to focus on (session based tests).

More experienced and mature teams, and teams that are in need for a more consistent way for creating, assigning, and fulfilling test charters can use a session-based test management tool to ensure clearer results and better structure.

Conclusion

So, if you are looking to improve your testing practices, add exploratory tests to the mix of scripted and automated tests to push up your software quality! And if you are looking for better and easier management around exploration, exploratory test tools can help and support you further in your quality endeavors.