Mobile Testing in the Context of BDD, TDD, and ATDD

Last week, I presented in the Joe Colantonio AutomationGuild online conference. (Kudos to Joe for a great event!)

Among multiple interesting questions that I got post my session (such as, "What is the best test coverage for mobile projects?" and "How do I design effective non-functional and performance testing in mobile and RWD?"), I also got a question about the differences between BDD and ATDD.

My session was around an open-source test automation framework called Quantum that supports cucumber BDD. This obviously triggered the question.

ATDD: Acceptance Test Driven Development

Based on Wikipedia’s definition (referenced above), ATDD is a development methodology based on communication between the business customers, the developers, and the testers. ATDD encompasses many of the same practices as specification by example,behavior-driven development (BDD), example-driven development (EDD), and support-driven development also called story test-driven development (SDD).

All these processes aid developers and testers in understanding the customer’s needs prior to implementation and allow customers to be able to converse in their own domain language.

ATDD is closely related to test-driven development (TDD). It differs by the emphasis on developer-tester-business customer collaboration. ATDD encompasses acceptance testing, but highlights writing acceptance tests before developers begin coding.

BDD: Behavior Driven Development

Again, based on Wikipedia’s definition, BDD is a software development process that emerged from test-driven development (TDD). Behavior-driven development combines the general techniques and principles of TDD with ideas from domain-driven design and object-oriented analysis and design to provide software development and management teams with shared tools and a shared process to collaborate on software development.

Mobile Testing in the Context of BDD and ATDD

The way to look at the Agile like-practices of BDD, ATDD, and TDD is from the context of higher velocity and quality requirements.

Organizations are aiming to release faster to market with great quality and sufficient test coverage, and in the same time, of course, meet the business goals and customer satisfaction. To achieve these goals, teams ought to be strongly collaborative from the very beginning app development and design stages.

Once organizations have the customer product requirements, and they can start developing through user stories and acceptance criteria so that several goals can be met:

• High customer-vendor alignment. This equates to customer satisfaction.
• Faster time to market; the app is tested along the SDLC.
• Quality is in sync with customer needs and there are much less redundant tests.
• There are no communication gaps or barriers between Dev, Test, Marketing, Management.

Looking at the below example of a BDD-based test automation test code, it is very easy to understand the functionality and use cases under test, the desired test outcome.

As can be seen in the screenshot above, the script installs and launches on an available Samsung device. The TestApp.APK file performs a successful login and presses on a menu item. As a final step, it also performs a mobile visual validation to assure that the test also passes, and also, as an automaton anchor, the test code reached the expected screen.

It is important to mention that the test framework and tools that support TDD, ATDD, and BDD can be similar, and one can still develop and test from a BDD or ATTD standpoint by using a Cucumber test automation framework (i.e., Cucumber or Quantum).

If we compare the above functional use case (or, as stated in the Cucumber language, scenario) to a scenario that would fit an ATDD-based approach, we would most likely need to introduce the known three amigos approach: three perspectives of customer (what problem are we trying to solve?), development (how might we solve this problem?), and testing (what about…).

Since a real ATDD best practice will determine a Gherkin-like app scenario before the development even starts, the above BDD example will be a precondition test for the app development team to make sure that they develop against acceptance criteria (which is, in our example, a successful app install and login).

An additional example of an acceptance test that also involves a layer of login/register that I can reference would look like this:

I can understand the confusion between BDD and ATDD since, as mentioned above, they can look a lot similar.

The bottom line is that BDD, ATDD, and TDD are all methods to better sync the various counterparts involved in shipping a working product to the market faster, with higher quality, and with the right functionality that meets the customer requirements. Implementing it using the Gherkin method makes a lot of sense due to the easy alignment and common language these counterparts use during the SDLC workflow.