Testing Automation Antipatterns: When Good Practices Become Your Worst Enemy

Note: This article is a summary of a talk I gave at VLCTesting in 2023. Here's the recording (Spanish).

Test automation is a fundamental tool for gaining confidence in what we build in a fast and efficient way. However, we often encounter practices that, while seemingly beneficial in the short term, generate significant problems in the long term: antipatterns.

What Is an Antipattern?

First, let's establish what I consider an antipattern, as it's not simply an obvious bad practice. It's characterized by:

• Offering an immediate benefit that's intuitive
• Seeming like the right solution at the moment
• Leading to negative consequences over time

Understanding and identifying them is important to avoid test suite degradation, slowness, unexplained failures, and ultimately abandoning automation efforts.

Let's look at some examples I've encountered recurrently in teams and testers.

1. The Testing Pyramid as Dogma

The testing pyramid suggests a specific distribution: many unit tests at the base, some integration tests in the middle, and few end-to-end tests at the top. The problem arises when it's applied as a universal rule without considering the project's specific context.

Why is it an antipattern?

The testing pyramid becomes an antipattern due to these apparent short-term benefits:

• Intuitive and visual model: Easy to understand and explain to stakeholders
• Widespread popularity: "Cargo cult" - if Martin Fowler says it, it must be right
• Feeling of doing the right thing: Following a recognized model gives immediate confidence
• Simplifies decisions: No need to think about strategy, just follow the distribution

Why it happens: It's much easier to follow an established model than to analyze each project's specific context. Also, questioning the pyramid might seem like you're questioning a "universal truth" of testing.

Long-Term Problems

• Misalignment with business objectives: We should focus testing efforts on what makes sense for our application and what customers value. In some cases, this might be visual aspects, performance, stability, or the ability to integrate with other systems
• Low-value tests: Adding low-value unit tests just to "comply" with the pyramid's base
• Strategic rigidity: Applying the distribution without considering that frontend-heavy projects might benefit from other models
• Resource waste: Time invested in tests that don't add real business value

How to Solve It

• Identify your business core: Is it performance? Visual experience? Complex API logic?
• Risk analysis: Where are the biggest failure points in your specific application?
• Consider alternatives: "Trophy" model for SPAs, "diamond" for hybrid applications
• Remember the iceberg: The pyramid is the visible result, but it needs a solid foundation of team culture, knowledge, and strategy.

2. Local Test Execution Only

This antipattern occurs when automated tests can only be executed on one specific person's local machine, usually the tester, who must run them manually instead of having them integrated into a CI/CD system.

Why is it an antipattern?

Local-only execution offers these irresistible immediate benefits:

• Development speed: No dependency on complex CI/CD configurations
• Total control: You have absolute dominion over the execution environment
• No blockers: No need to wait for others to configure pipelines or environments
• Initial ease: It's the quickest way to start with automation
• No dependencies: No need to coordinate with DevOps or system administrators
• Immediate feedback: You see results instantly without CI queues

Why it happens: It's simpler to have total control and see immediate results. Also, setting up CI/CD can seem complex and "not urgent" when tests work locally.

Long-Term Problems

• Knowledge silos: Only one person can execute the tests
• Slow feedback for the team: Others don't receive immediate information about code status
• Critical dependency: If that person isn't available, tests don't run
• Impossible continuous integration: No real automation in the development pipeline
• Investment loss: When the person leaves, all automation is lost

How to Solve It

• CI/CD from day one: Every test must be able to run unattended
• Dockerization: Use containers to ensure consistency across environments
• Shared repositories: Test code must be versioned and accessible to the entire team
• Clear documentation: Procedures so anyone can execute the tests
• Investment in configuration: Dedicate initial time to properly configure environments

3. Cucumber: Misunderstood and Misused

Cucumber allows writing tests in natural language (Gherkin) that are then linked to technical code. It becomes an antipattern when adopted, expecting it to automatically improve collaboration with business or simplify testing, without a clear BDD strategy behind it.

Why is it an antipattern?

Cucumber offers very attractive benefits:

• "Wow effect": Translating natural language to executable code seems magical
• Promise of collaboration: "Now business can write tests"
• Ubiquitous language: Everyone will understand tests, technical, and non-technical
• Living documentation: Scenarios serve as executable specifications
• Methodological justification: "We're doing real BDD"
• Professional differentiation: Using Cucumber makes you seem more advanced than using only unit tests

Why it happens: The promise of democratizing testing is very attractive. Also, once you invest time learning Gherkin and creating step definitions, it's hard to admit it doesn't add value in your specific context.

Long-Term Problems

• Unnecessary complexity: You add an extra layer that isn't always justified
• Expensive maintenance: Imperative scenarios become fragile with changes
• False collaboration: Business people rarely maintain or write scenarios
• Implementation coupling: Steps become too specific about the "how"
• Loss of real value: Used as a post-development testing tool instead of facilitating prior conversations

How to Solve It

• Evaluate real context: Is there active collaboration between non-technical roles in criteria definition?
• Use it a priori: To generate conversations before development, not as a post-development testing tool
• Declarative scenarios: Focus on what (behavior), not how (implementation), following Gherkin syntax best practices
• Consider alternatives: If your entire team is technical, direct tests might be more efficient

4. Testing Through the Interface vs. Testing the Interface

This antipattern consists of using UI tools (like Selenium/Cypress) to test the entire application stack when what you intend is to validate only the specific functionality of the user interface.

It's the difference between using the UI as a vehicle to test the entire application versus testing specifically that the UI works correctly.

Why is it an antipattern?

Testing the entire stack through UI with an e2e tool offers very attractive immediate benefits:

• Total security feeling: "If it works in the browser, the entire system works"
• Manual testing replication: It's the most direct automation of what we'd do manually
• Less analysis required: No need to think about layers, dependencies, or architecture
• Less communication: No need to coordinate with other teams about what they test
• Universal understanding: Anyone can understand what the test does just by watching it execute
• One tool for everything: Selenium solves "all" testing needs

Why it happens: It's natural to want to replicate what we do manually. Also, thinking in layers and dividing testing responsibilities requires more mental effort and team coordination.

Long-Term Problems

• Slow and confusing feedback: When something fails, you don't know if it's the UI, API, database, or external service
• Expensive maintenance: Changes in any layer break end-to-end tests
• Extreme redundancy: You validate the same logic in multiple layers (e.g., email validation)
• Slow and unstable tests: More components = more failure points
• Limited scalability: Adding more end-to-end tests makes the suite progressively slower

How to Solve It

• Divide responsibilities: Each test type in the most efficient layer for what it validates
• Strategic mocking: Isolate the specific functionality you want to test
• Risk analysis: Identify which critical flows DO require complete end-to-end tests
• Team communication: Coordinate to avoid unnecessary validation duplication

5. The Danger of Retries in Flaky Tests

Flaky tests are tests that sometimes pass and sometimes fail without apparent changes in the code. The antipattern arises when we configure automatic retries to make these tests "go green" instead of investigating and solving the root cause of their instability.

Why is it an antipattern?

Configuring automatic retries for flaky tests offers irresistible immediate benefits:

• Quick and easy solution: A simple retry: 3 in your configuration and "problem solved"
• Instant green pipeline: No more broken builds due to "temporary issues"
• Fewer interruptions: The team isn't interrupted by false positives
• External problem attribution: "It's the tool's or environment's fault, not ours"
• Delivery pressure: You need it green "now" to not block the release
• Less investigation required: You don't have to analyze each individual failure

Why it happens: It's much easier to configure a retry than investigate the root cause. Also, when the test passes on the second attempt, it reinforces the belief that "it was just a temporary problem."

Long-Term Problems

• Total confidence loss: Nobody trusts tests that "sometimes fail"
• Hidden real problems: Race conditions, memory leaks, concurrency issues get masked
• "Watermelon" suite: Green on the outside in the dashboard, but red inside with real failures
• Progressive degradation: Underlying problems worsen until impossible to ignore
• Eventual abandonment: Teams end up disabling or completely ignoring tests

How to Solve It

• Mandatory investigation: Every failure must be analyzed before any retry
• Exhaustive root cause analysis: Logs, screenshots, manual reproduction, team consultation
• Deep technical knowledge: Understand how your testing tool works internally
• Quality culture: "It's green" isn't enough if there were retries without prior investigation

6. The Coverage-Oriented Testing Illusion

This antipattern arises when the main goal of writing tests becomes reaching a specific code coverage percentage (typically 80%), instead of focusing on testing critical system behaviors. The result is tests that increase metrics but don't add real value.

Why is it an antipattern?

The "80% coverage" goal offers very seductive immediate benefits:

• Easy metric to measure: A clear number you can report to management
• Objective justification: "We have 80% coverage, our code is good"
• Fulfilled contractual obligation: Many contracts explicitly require it
• Feeling of professionalism: "A good developer has high coverage"
• Gamification: It's satisfying to see the percentage rise like a video game
• Easy comparison: You can compare projects and teams with a simple metric

Why it happens: It's much easier to chase a number than to analyze whether tests actually add value. Also, high unit test numbers give a false sense of security, but with a clear conscience.

Long-Term Problems

• Tests without real value: Testing getters/setters just to increase the percentage
• Extreme coupling: One test class for every production class
• Abusive mocking: You end up testing your mocks, not the real code
• Refactoring obstacle: Changing a constructor breaks 50 tests that didn't even detect bugs
• False security: High coverage doesn't mean error-free code

How to Solve It

• Focus on behavior: Tests that verify real user flows, not lines of code. "Test behavior, not implementation"
• Coverage delta: More important that it doesn't decrease when you add new code
• Social tests: Tests that exercise multiple classes working together
• Mutation testing: Verify your tests actually detect errors using tools like PIT or Mutmut

After identifying the most common antipatterns in test automation, it's time to transform these learnings into practical actions to avoid falling into the same traps.

Conclusions: Strategy, Context, and Collaboration

To build robust test automation that adds value:

1. Define Your Strategy (Golden Circle)

• Why: What specific business problems do you want to solve?
• How: Will you focus on APIs, performance, and visual experience?
• What: Only at the end, choose specific tools

2. Context Determines Validity

• A practice can be an antipattern in one context and a solution in another
• Evaluate your specific situation: team, product, constraints

3. Collaboration and Consensus

• Decisions should be made by the complete team
• Avoid unilateral impositions

4. Invest in Fundamentals

• Successful automation requires culture, knowledge, and time
• Visible results depend on a solid, invisible foundation

5. Continuous Learning

• Antipatterns evolve with tools and practices
• Stay updated and willing to question your own practices

Final reflection: These antipatterns don't arise from incompetence, but from rational decisions with limited information. The key is maintaining a long-term perspective and being willing to change when context requires it.

Remember: In test automation, what works today can be tomorrow's antipattern. The true professional isn't one who never makes mistakes, but one who constantly questions their own practices.