DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Please enter at least three characters to search
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Zones

Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

The software you build is only as secure as the code that powers it. Learn how malicious code creeps into your software supply chain.

Apache Cassandra combines the benefits of major NoSQL databases to support data management needs not covered by traditional RDBMS vendors.

Generative AI has transformed nearly every industry. How can you leverage GenAI to improve your productivity and efficiency?

Modernize your data layer. Learn how to design cloud-native database architectures to meet the evolving demands of AI and GenAI workloads.

Related

  • 7 Best Practices for Salesforce Testing
  • API Security Is Finally Gaining Attention That it Deserves
  • Android Cloud Apps with Azure
  • Enterprise Computing and the Public Cloud Dilemma

Trending

  • How to Use AWS Aurora Database for a Retail Point of Sale (POS) Transaction System
  • How Large Tech Companies Architect Resilient Systems for Millions of Users
  • Caching 101: Theory, Algorithms, Tools, and Best Practices
  • Data Lake vs. Warehouse vs. Lakehouse vs. Mart: Choosing the Right Architecture for Your Business
  1. DZone
  2. Testing, Deployment, and Maintenance
  3. Testing, Tools, and Frameworks
  4. Your Automated Testing Strategy: Pyramids, Triangles, and Beyond

Your Automated Testing Strategy: Pyramids, Triangles, and Beyond

Learn how to formulate your automated tests with the help of the Test Automation Pyramid.

By 
Lisa Crispin user avatar
Lisa Crispin
·
Oct. 21, 18 · Tutorial
Likes (2)
Comment
Save
Tweet
Share
15.9K Views

Join the DZone community and get the full member experience.

Join For Free

If your team struggles with automated testing, don't feel alone or ashamed. I meet lots of people whose teams do all their regression testing manually. Other teams have automation but are finding it expensive to maintain and aren't sure if the tests are providing a benefit.

Teams who lack automated regression tests are incapable of delivering value to their customers frequently. Changing the code takes longer and longer, because there's more and more manual regression testing needed to feel confident about releasing.

Where to Start

Get a diverse group of delivery team members together to talk about what's holding you back and whether automation testing might help with that roadblock. Think about the trade-offs between automating and doing testing manually. Both have a cost.

The Test Automation Pyramid

Over the years, my teams have found Mike Cohn's "Test Automation Pyramid" a useful model to help us formulate our automation strategy. Yes, it is not actually a pyramid, it's a triangle, but what's important is that it helps you visualize the steps to take as you tackle automation testing.

We also need to check functionality and other quality characteristics such as accessibility and security across larger pieces of the application with functional testing. For example, does an API endpoint return the correct values? These tests may take more time to create and, if they access databases and other services, provide slower feedback. If business stakeholders need to collaborate on acceptance tests to verify business rules, these middle layer tests can be written in plain language that everyone can understand.

Products that have a user interface (UI) generally need to be tested across all these layers. The feedback loop from UI level tests is longer, but they can find the most important regression failures. Today's technology makes it easier to automate robust tests at the UI level, but many factors can still make these tests "flaky" and more costly to maintain. That's why the model has the fewest tests at the top level.

The little cloud at the top of the pyramid represents testing that's done manually. Even with solid automation at all layers of the triangle, your product might require a huge cumulonimbus cloud that includes exploratory testing, accessibility testing, security testing, and many other critical testing activities. You might be able to use automation to speed up this manual testing.

The Test Automation Triangle, Simplified

There are many variations on the pyramid model, so take time to check those out and see if they work better for your team. Your context is unique, so experiment to find what helps you map out your first steps to automation testing success. I like the way that Seb Rose maps out the concept behind the pyramid. You want as many tests as possible that test smaller chunks of the application, but you still need to check the "big picture".

An Actual Pyramid!

In Chapter 15 of More Agile Testing, Sharon Robson explains a four-step process she uses to talk about automation testing. She starts with the basic pyramid, but adds on the tools that can be used to execute the tests, the types of tests or system attributes to be tested, and finally, the automated regression tests needed.

A DevOps Model

If none of these models sings to your team, check out the links in the reference section at the end of this post. And if none of those help you get started formulating your strategy, grab some markers and a whiteboard (physical or virtual) and start drawing together to visualize what levels of your application need automation testing, and where you want to start.

Starting Your First Experiment

Your whole delivery team needs to agree on a starting strategy for automation. Automation testing can be scary and difficult, so you need to be willing to stick to it during the painful parts.

You may decide on a strategy that involves more than one layer of the triangle. I worked on a team that committed to learning how to do test-driven development, knowing the process was going to take many months. We shared the burden of manual regression testing through the UI at first, and had no automated tests at all, just a cloud of manual regression tests:


We decided to automate UI smoke tests for the critical areas of the application. After eight months, we no longer had to do manual regression testing. By then, we also had a small, growing library of automated unit tests. We still did a lot of manual testing on new features and at the services layer of our application. We had an upside-down sort of triangle that is typical of teams trying to succeed with automation testing:


We still had a lot of holes to fill, but since we didn't spend a day or two every sprint doing manual regression testing, we had time to tackle other automation challenges. Eventually, our test automation triangle looked like the classic "pyramid" above.

Successful automation testing requires a big investment over time. Start small. A good way to get started with automating unit tests is to write them for all bug fixes - or even better, test-drive them. You may decide to start automating API tests by doing them for all new stories or features going forward. Or, if manual regression testing is sucking up all your time, take advantage of tools that let you quickly get up to speed on UI test automation and create some robust UI tests that cover the most critical areas, so you can devote more time to expanding your automation efforts to other levels. For sustainable automation testing that lets you release frequently and confidently, your team will need to invest time in learning good automation principles and patterns.

What if your first experiment fails? Maybe you automate some tests and then find they are unreliable or impossible to maintain. Awesome - you have learned a lot, and that will help you going forward! Don't be afraid to scrap ineffective tests or a tool that isn't working for you and start over. Don't fall victim to the "sunk cost fallacy." As Seb Rose has said, "if a test doesn't help you learn something about the system, then delete it!"

References and more reading:

  • Seb Rose, " http://claysnow.co.uk/architectural-alignment-and-test-induced-design-damage-fallacy/

  • Katrina Clokie, A Practical Guide to Testing in DevOps

  • Lisa Crispin and Janet Gregory, More Agile Testing: Learning Journeys for the Whole Team

  • Noah Sussman, “Abandoning the Pyramid of Testing in favor of a Band-Pass Filter model of risk management”

  • Alister Scott, “Testing Pyramids & Ice-Cream Cones”

unit test agile Regression testing Test automation application Database Manual testing Trade-off Cloud

Published at DZone with permission of Lisa Crispin, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Related

  • 7 Best Practices for Salesforce Testing
  • API Security Is Finally Gaining Attention That it Deserves
  • Android Cloud Apps with Azure
  • Enterprise Computing and the Public Cloud Dilemma

Partner Resources

×

Comments
Oops! Something Went Wrong

The likes didn't load as expected. Please refresh the page and try again.

ABOUT US

  • About DZone
  • Support and feedback
  • Community research
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends:

Likes
There are no likes...yet! 👀
Be the first to like this post!
It looks like you're not logged in.
Sign in to see who liked this post!