DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones AWS Cloud
by AWS Developer Relations
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones
AWS Cloud
by AWS Developer Relations
The Latest "Software Integration: The Intersection of APIs, Microservices, and Cloud-Based Systems" Trend Report
Get the report
  1. DZone
  2. Testing, Deployment, and Maintenance
  3. Testing, Tools, and Frameworks
  4. Thoughts about "UI Test Automation Tools are Snake Oil"

Thoughts about "UI Test Automation Tools are Snake Oil"

Alex Ruiz user avatar by
Alex Ruiz
·
Jan. 19, 10 · Interview
Like (0)
Save
Tweet
Share
6.44K Views

Join the DZone community and get the full member experience.

Join For Free

I just finished reading "UI Test Automation Tools are Snake Oil" by Michael Feathers. Although I agree with many of the ideas in the article, I also think it contains some hasty generalizations and misplaced blame.

Mr. Feathers points out that “selling UI test automation tools is irresponsible” and these tools are sold “with a dream, a very seductive and dangerous one.” The scenario that he refers to, to my understanding, is the one where tool vendors sell their (expensive) tools promising:

  • “click and type,” quick test generation with a record/playback tool, no coding required
  • test the whole application, including all different use case and data permutations, using this tool

This can be easily interpreted as “with my tool, you will have a comprehensive test suite in no time, and you can hire cheap monkeys to do all the testing.” It is indeed a very seductive promise: spend some money and get what you want fast and with minimal effort, just like “weight loss, without diet and exercise.”

I completely agree with Mr. Feathers that selling something based on a fallacy is irresponsible. At the same time, I also think this is the point where blame is misplaced. Instead of blaming the tool, blame should be shared between tool vendors and the people buying the tool, as long as the business transaction is based on the “seductive dream.” IMHO, this is a people problem: on one hand we have vendors taking advantage of the customers’ lack of knowledge and the customers, that did not do their proper homework on time and by now have a big, untestable mess.

Another good point that Mr. Feathers makes is that UI testing tools are brittle. Selling a UI testing tool promising that the generated test suite will never break is also careless. The fact is, when testing UIs there are many external elements that can introduce false failures. IMHO, it is impossible to fix all of them, but at least, the tool vendor should warn their customers about these limitations and offer ways to overcome them (more on that later.)

I also agree with Mr. Feathers that record/playback doesn’t work. I wish he went into more details to see if we agree based on the same premises. In my opinion, the major weakness of existing record/playback tools is expensive maintenance of the generated tests. Recorded scripts are often long and written in proprietary languages lacking object-oriented features. Modularization/refactoring is hard or impossible, resulting in duplicate test code. The common end result is that changes in the application requires re-recording all tests scenarios, a labor-intensive and error-prone task.

Not everything is black and white. UI test automation tools still have a place. Not every shop has the resources or time to rewrite an application, just to make it more testable. I’ve seen applications with massive code bases, that were not written with testability in mind. Extending or refactoring is scary due to the lack of the safety net that automated tests provide. The only way to test them is through the UI. UI testing can provide the initial safety net, which later on can be enhanced (or replaced) with API-based unit tests.

In his post, Mr. Feathers asks “Developing them open-source? Well, let your conscience be your guide.” I don’t sell a UI test automation tool per se. What I provide is an open source, API-based UI testing library, called FEST (you can find some testimonials here.) I think I’ve been pretty responsible by pointing out that UI testing is fragile and offering ways to overcome this limitation. What I’m missing, and I have to thank Mr. Feathers for it, is a guide on how to test the UI layer in isolation (which I will be doing soon.)

Like I mentioned in a previous post, I’m currently working on my spare time on a playback/recording tool. My intention is to create a tool that generates clean Java code, as if it was hand-crafted, to overcome to what I think are the problems with this technique. We’ll see if I can accomplish this ambitious goal :)

Overall, I liked and enjoyed Mr. Feathers article. It has pretty good points and observations. I wish he backed up his ideas with more details and examples, to sound less emotional and more objective. The problem is in the human factor, not in the tool. Ideally, vendors (in general) should set the right expectations about their products, and customers should have enough knowledge to avoid being taken advantage of. At least, some of us are trying to do the right thing :)

Feedback is always welcome :)

From http://alexruiz.developerblogs.com/

Testing Test automation

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • Application Architecture Design Principles
  • Stop Using Spring Profiles Per Environment
  • Fargate vs. Lambda: The Battle of the Future
  • Introduction to Container Orchestration

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends: