Additional Considerations for Automated Testing
Additional Considerations for Automated Testing
Our 31 respondents, all industry executives, raised a number of thought-provoking topics and questions on the state of automated testing.
Join the DZone community and get the full member experience.Join For Free
The need for DevOps innovation has never been greater. Get the results from over 100 business value assessments in this whitepaper, Digital Darwinism: Driving Digital Transformation, to see the positive impact of DevOps first hand.
To gather insights for the current and future state of automated testing, we asked 31 executives from 27 companies, "What have I failed to ask you that you think we need to consider with regards to automated testing?" Here's what they told us we still need to consider:
- Speed, accuracy, and integration are key. You will not achieve your goals without all three.
- Testing is not a filter. Testing is measurement. The frequency of measurement is important. Once you think about testing that way you understand why the speed of testing is important. Measure the quality of the code. More data points result in greater accuracy for performance and security. Performance testing has to be done over the course of the lifecycle.
- I would say that there is a debate about the ownership and responsibility of writing different kinds of automated tests. In a more traditional world, you can see that Quality Engineers would write automation outside of the definition of the tasks or in more agile worlds the next sprint after code is release. The industry is moving towards a world where Quality Engineers are responsible for providing the tools and frameworks for developers to write tests and giving advice about what to test and how to test it, but they are not responsible anymore for writing the tests. Quality is shifted to the left to the point that automated tests are quality requirements for any development and they are done before shipping the code.
- I'm interested in knowing more about why some individuals are not adopting automated testing. I'd also like to learn more about when some people attempt to adopt automated testing what causes them to not move forward with it or not use it. And again, I think it's important to have a control point after deployment. Though automated testing is a great checkpoint, it can't be solely relied on to ensure everything is going to go as intended. Having that secondary control point post-deployment in your code–using a feature flag and feature management–allows you to move forward more quickly with confidence.
- How many people, developers, and testers, are in the specification? Is there a description of the business user problem they are trying to solve? Testing cannot answer that but it's critical for testing to know.
- We are going to see more regulation over the next 10 to 20 years. Are developers becoming responsible for security and regulatory aspects? Do they understand the importance of what they’re doing? Do developers understand the implications of this?
- The landscape of solutions is at a critical point in maturity with regards to quality within CI/CD. Vendors shift and change. Paradigms are changing. Within five years we will see a very different landscape of players impacting the landscape. We’re in the midst of the disruption. We will see new gorillas impacting the market positively. Test, automation, managing quality process.
- Maturation of mobile app development market requires testing on real devices and emulators/simulators. Latest survey 77% use both for mobile up from 34% in 2014.
- Inquire as to the ability for automated testing to enforce either company and or development policies. The automated enforcement of security policies via automated testing is a huge win for Development Teams and Security Teams in DevOps/Agile environments. In addition to security policy, I would expect other contributors to focus on enforcing coding guidelines, pull request expectations (ex: including a test with every commit), etc.
- Principally two things: 1) the terrible state of visual tools; and, 2) whether and when it’s appropriate to have tests that are not stored as source code.
- Key points: 1) be educated, follow best practices, test-driven environment, 2) make sure you write successful tests with coverage and you don’t have to go back and rewrite.
- Big area if looking into modern distributed systems it becomes interesting with testing, debugging, become intermingled. Appreciate what it means to operate in that development traditional software development becomes less relevant. Tracing and log tracking becomes important. Continuously testing and upgrading the airplane while it is flying. How to manage and update live systems.
- Automated testing offers a new career path for manual testers. Becoming empowered with AI with what’s happening with UI will make it easier for manual testers to make the transition.
- We need to be running longer integration tests from end-to-end.
- People are so focused on the trees they’re not focused on securing the forest.
- From a security perspective. Tailor test to organization versus something out of the box.
- Well, what about the challenges? Certainly, a very important aspect for those who’ve tried implementing automated testing in their organizations but may have failed due to certain challenges. So, what are those challenges? Some examples: not enough budget, not enough knowledge, lack of support and awareness from management.
- Gil Sever, CEO and James Lamberti, Chief Marketing Officer, Applitools
- Shailesh Rao, COO, and Kalpesh Doshi, Senior Product Manager, BrowserStack
- Aruna Ravichandran, V.P. DevOps Products and Solutions Marketing, CA Technologies
- Pete Chestna, Director of Developer Engagement, CA Veracode
- Julian Dunn, Director of Product Marketing, Chef
- Isa Vilacides, Quality Engineering Manager, CloudBees
- Anders Wallgren, CTO, Electric Cloud
- Kevin Fealey, Senior Manager Application Security, EY Cybersecurity
- Hameetha Ahamed, Quality Assurance Manager, and Amar Kanagaraj, CMO, FileCloud
- Charles Kendrick, CTO, Isomorphic Software
- Adam Zimman, VP Product, LaunchDarkly
- Jon Dahl, CEO and Co-founder, and Matt Ward, Senior Engineer, Mux
- Tom Joyce, CEO, Pensa
- Roi Carmel, Chief Marketing & Corporate Strategy Officer, Perfecto Mobile
- Amit Bareket, CEO and Co-founder, Perimeter 81
- Jeff Keyes, Director of Product Marketing, and Bob Davis, Chief Marketing Officer, Plutora
- Christoph Preschern, Managing Director, Ranorex
- Derek Choy, CIO, Rainforest QA
- Lubos Parobek, Vice President of Product, Sauce Labs
- Walter O'Brien, CEO and Founder, Scorpion Computer Services
- Dr. Scott Clark, CEO and Co-founder, SigOpt
- Prashant Mohan, Product Manager, SmartBear
- Sarah Lahav, CEO, SysAid Technologies
- Antony Edwards, CTO, Eggplant
- Wayne Ariola, CMO, Tricentis
- Eric Sheridan, Chief Scientist, WhiteHat Security
- Roman Shaposhnik, Co-founder V.P. Product and Strategy, Zededa
Opinions expressed by DZone contributors are their own.