Automation Testing Pyramid Today
In this article, explore the familiar concept in software engineering, the automation testing pyramid.
Join the DZone community and get the full member experience.Join For Free
The testing pyramid is a familiar concept in software engineering. The model drew greater attention in Mike Cohn’s 2009 book Succeeding with Agile.
The bottom level of the pyramid consists of unit tests. Unit tests are written by developers and cover methods and functions behaviors, using test doubles to mock inputs, outputs and external dependencies.
Using clean code concepts, these tests are easy to create and maintain. That way all developers are able to run these tests locally and check code change impacts.
The middle level consists of service tests for APIs and general integrations, these tests cover the composition of units of code and their communication with the external world (Database, web services, etc).
The top of the pyramid consists of automated UI testing, final users flows throughout the entire system. Normally these tests are slow and hard to maintain.
In general, the core idea is:
- Separate tests with different granularity
- The more high-level you get the fewer tests you should have
The concept has always seemed a little simplistic, but the main idea is still alive and in a Continuous delivery world it is more important than ever because even with this widespread concept, is very common to see automated tests written in a wrong way.
- Tests written in the wrong layer (UI Tests, checking business logic) can result in testing that is rarely executed because it takes too long and they often fail due to problems that are not related to something important for the end user.
- Unit tests are written just for coverage and not to testing business logic.
- Integration tests that just check the results of APIs (input and output) and don't covers the composition between them.
Testing Pyramid Today
The discipline of software development and software architecting has matured and evolve, and software testing as well.
In moderns software architectures (Microservices, Micro frontends, SOA), tests have to manage multiple and independently deployable components, that are managed by different teams, and valuable features that need to be tested with greater attention, normally are the composition of these different components.
These kind of architectures are a game changer for software testing, bringing new challenges and also new benefits, So today a testing pyramid need to have:
The foundation of the test suite should still be unit tests, the unit tests guarantee that units of your code works as intended. and also can help teams to improve their software design, because often difficulty in writing a unit test can show that a module/class should be broken down into independent pieces.
Is also important to measure the quality of unit testing, in order to make sure that your unit tests are really testing important pieces of the code, so then comes the concept of mutation testing.
Mutation Testing tools basically runs your unit tests against a modified version of your application code. The expected behavior when code conditionals are changed is that unit tests fail. If a unit test does not fail, it may indicate an issue with the test suite.
Splitting a system in many small components, ofter means that these components have to communicate with each others via certain interfaces.
- Events / Messaging
The contract consists of expectations of input and output data structures, side effects, performance, and errors mapping.
Each consumer forms a different contract based on his requirements, so the sum of all consumer contract tests defines the overall service contract, this concept is called (Consumer Driven Contract)
Tools for writing contract tests:
The goal is to test the behavior of the integrated system. End-to-end tests can directly manipulate microservices through their public API using HTTP client. In this way, a feature is tested independent of the UI/channel that uses it, this can be great and very productive for companies that have an omnichannel platform based on microservices architecture, in that way these tests can run fast and validate the whole company features business logic in a single test suite.
Write and maintaining end-to-end testing can be difficult because of the number of components involved and their different integration methods, may also have to account for asynchrony in the system or backend process between services. So it is important to follow the concept of the testing pyramid and write as few end to end tests as possible, and to do that is important:
- Focus on user journeys
- Make tests data-independent
UI Tests guarantee that the user interface works correctly. These kinds of tests can run independently of the rest of the system and can be achieved with tools like Cypress, Selenium.
Exploratory tests need to be at the top of the pyramid.
It is extremely important to manually explore the system with a destructive approach, it can result in some issues scenarios that are not part of tests scripts.
Exploratory tests also allow teams to learn more about the whole system and improve their automated tests.
Regardless of the new layers we add to the pyramid, mike's concepts are still there and help us to find ways to organize our test suites and make them easy to run in our software development cycle.
Opinions expressed by DZone contributors are their own.