DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Please enter at least three characters to search
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Zones

Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

Last call! Secure your stack and shape the future! Help dev teams across the globe navigate their software supply chain security challenges.

Modernize your data layer. Learn how to design cloud-native database architectures to meet the evolving demands of AI and GenAI workloads.

Releasing software shouldn't be stressful or risky. Learn how to leverage progressive delivery techniques to ensure safer deployments.

Avoid machine learning mistakes and boost model performance! Discover key ML patterns, anti-patterns, data strategies, and more.

Related

  • The Rising Risks and Opportunities in API Security
  • What Is API-First?
  • Five Steps To Building a Tier 1 Service That Is Resilient to Outages
  • DevOps: CI/CD Tools to Watch Out for in 2022

Trending

  • From Zero to Production: Best Practices for Scaling LLMs in the Enterprise
  • Performing and Managing Incremental Backups Using pg_basebackup in PostgreSQL 17
  • Intro to RAG: Foundations of Retrieval Augmented Generation, Part 1
  • Why High-Performance AI/ML Is Essential in Modern Cybersecurity
  1. DZone
  2. Testing, Deployment, and Maintenance
  3. Testing, Tools, and Frameworks
  4. What Are the Keys To Automated Testing?

What Are the Keys To Automated Testing?

Strategic automation of the entire CI/CD/DevOps process reduces the amount of human interaction with testing and data.

By 
Tom Smith user avatar
Tom Smith
DZone Core CORE ·
Jul. 03, 17 · Interview
Likes (7)
Comment
Save
Tweet
Share
11.5K Views

Join the DZone community and get the full member experience.

Join For Free

To gather insights on the state of automated testing today, we spoke with twenty executives who are familiar with automated testing and asked, "What are the keys to using automated testing to improve speed and quality?"

Answers were around two themes: strategic automation, and use of testing and data.

Strategic Automation

  • Don’t believe people when they say they cannot automate manual tests. Gary Gruver at HP automated testing for printers and fax machines.
  • Have a good process around testing. Link automation into the CI/CD/DevOps process. You cannot achieve speed and agility without it.
  • Nirvana happens without engineers coding in testing. Automate the entire CI/CD process. Develop unit tests and regression testing with Artifactory and other common workflow automation tools. 1) Developers integrate the proper unit tests. 2) Automatically run a nightly build test. 3) Regression testing.
  • There’s front end/UI testing and backend/API testing. The focus is now more on the back end with mobile and more API connections. The key for both the front end and the back end is to reduce the amount of manual testing. Run test overnight as part of the continuous integration process. Parallelize your tests since the shift to automation can take six to eight hours and become a point of friction. Start running tests in parallel, especially in DevOps.
  • Better integration into DevOps for continuous testing. Make the test team independent from the IT and operation team. This is critical for speed and quality.
  • Application and orchestration automation built in extensive APIs so it will scale. Multiple customers in the enterprise and the cloud. Get an in-view portal. Enterprise needs to see protection and throughput. Can make part of VM onboarding. Ongoing reporting and visibility.
  • Increase speed and quality at every step. Build the pipeline and set-up for automated tests to run in. Branch only when passing the test. Regression testing before release of the framework. Ensure confidence in the framework. Speed leverage browser renders to increase the number of VM’s = 50, 40,000 tests in multiple versions, 420,0000 test cases in 20 to 60 minutes. Concurrency multi-threaded tests. Must run in parallel since IE 10 and 11 runs slower than Chrome.
  • Both aspects have the same answer for the most part. Automation stops the need for repeating the same tests repeatedly. Instead, they are afforded the time and freedom to explore the more exotic test cases. In modern development, everyone is very aware of the common areas to test, along with developers writing more and more unit level tests. This means that we are less likely to find the usual bugs as before; instead, we need to be more creative with our testing - automation allows us to pursue that given better coverage and quality.
  • You first must have a stable test infrastructure in place, where you are catching regressions and able to notify developers appropriately.  At that point, you need clear policies for what is done when regressions are detected: who is assigned to fix them, how fast must they be resolved vs completing other tasks, what happens to ambiguous regressions (is the code wrong or is the test wrong), etc. We’ve seen a recurring type of dysfunction in several organizations: they’ve built an automated test system, but the noise from broken tests is drowning out the signal from the working tests, so everyone ignores the test system.  That’s worse than having no automated test infrastructure at all. You must actively maintain both the tests and the people processes around them, or you end up with this dysfunction.
  • Rapid development and execution are core components of delivering quality software on a schedule. Repetitive, automated coverage allows us to focus on the current product changes while the regression content – content that has either already been delivered into the field or has already been checked in previously - is adequately covered for any integration level defects. With most automation, less human interaction is desirable. This applies to the actual test, but also to things like setting up the test harness, triaging results, finding out historical details about the test and providing a general stability metric for the content that has already been shipped or delivered.
  • Architecture agility is key to support multiple vendors. It requires more agility and good quality automated testing – integration testing and endurance testing 24x7 on specific test scenarios to detect memory leaks. We start with manual tests until we get it automated. We must address the gap when developing a new feature. The automated testing team writes automation pieces. We are moving to Test-Driven Development (TDD) but we’re not all the way there yet. You need a platform that enables you to consistently leverage and use automated test development on a consistent basis and understand how to write automated test cases.
  • For web testing, use Selenium or Appium to test cross-browser and cross-platform. You can write tests without having to learn a test scripting language. Write tests that automate repeatable core functionality. Developers should learn how to write tests and QA people should learn a little bit of code. You need the teams to work cross-functionally to be able to scale. Run tests in parallel to scale. It cuts testing down to just minutes. Once test automation is up and running, start thinking about CI/CD (continuous integration and delivery).
  • If you just constrain your efforts to the pure automation of tests, you very likely will fail. That’s because the maintenance of the test cases will be a huge burden. To succeed with test automation, you will need to solve for the stability of infrastructure for the systems you’re acting on. Plus, you need to assure that the test data that needs to be present so that test cases can run smoothly is here in the desired state and quantity. Only if you solve for the core automation, and in addition, the test data management issue, and service virtualization to make sure that the systems are stable and available, will you succeed with automation.

Using Testing and Data

  • We track 140,000 events per second with the goal of optimizing the UX.  1) Start with analytics and see where people are getting stuck. 2) Optimize the UX. 3) Campaign messaging, conversations, A/B tests, layer, language, timing, content, and location. 4) Data and event automation with marketing systems of record.

  • 1) The main key that we’ve discovered is to maintain and sharpen our test scope. This starts with always determining the best level for tests – Should we be testing this in the UI, or is it more effective on a unit level? Next, we must ensure that our test strategy isn’t allowed to experience scope creep, especially with UI tests, which can be expensive to create and maintain. To do this, we focus on prioritizing our use cases and subsequent tests and seek to automate in priority order. This will allow us to produce sanity checks quickly that span critical end to end scenarios. Keeping our scope focused greatly impacts both speed and quality of our efforts. 2) Another key is to heavily leverage a data-driven approach. This helps our teams write tests that are deterministic and only fail when real application changes are made, cutting down on maintenance time for flaky tests. Solutions for this can depend on context, but mainly leverage tools like Chance.js to generate randomized data, and use Factory and Data Builder patterns to allow tests to set up and clean up data effectively.

  • Identifying and aligning to true business value is critical with respect to improving speed and quality. It is important to be able to measure and clearly demonstrate the value to the organization in terms of quicker speed to market with new products, fewer customer impacting incidents and an overall reduction in overhead and expense. This understanding helps with creating the business case for continued investment in test automation.

Developer Adoption

  • Developers need to take accountability for their work. Testing is not someone else’s job. Quality from a functional and non-functional point-of-view. Inspect quality in.

What do you think are the keys to automated testing from your perspective?

 Here’s who we talked to:

  • Murali Palanisamy, EVP and Chief Product Officer, AppViewX
  • Yann Guernion, Director of Product Marketing, Automic
  • Eric Montagne, Technology PM, Barclaycard
  • Greg Luciano, Director of Services and Amit Pal, QA Manager, Built.io
  • Donovan Greeff, Head of QA, Currencycloud
  • Shahin Pirooz, CTO, DataEndure
  • Luke Gordon, Senior Solutions Engineer and Daniel Slatton, QA Manager, Dialexa
  • Anders Wallgren, CTO, ElectricCloud
  • Charles Kendrick, CTO, Isomorphic
  • Bryan Walsh, Principal Engineer, NetApp
  • Derek Choy, V.P. of Engineering, Rainforest QA
  • Subu Baskaran, Senior Product Manager, Sencha
  • Ryan Lloyd, V.P. Products, Testing and Development and Greg Lord, Director of Product Marketing, SmartBear
  • Christopher Dean, CEO, Swrve
  • Wolfgang Platz, Founder and Chief Product Officer, Tricentis
  • Pete Chestna, Director of Developer Engagement, Veracode
  • Harry Smith, Technology Evangelist, Zerto
unit test Continuous Integration/Deployment Data (computing) dev Test data Regression testing Integration teams

Opinions expressed by DZone contributors are their own.

Related

  • The Rising Risks and Opportunities in API Security
  • What Is API-First?
  • Five Steps To Building a Tier 1 Service That Is Resilient to Outages
  • DevOps: CI/CD Tools to Watch Out for in 2022

Partner Resources

×

Comments
Oops! Something Went Wrong

The likes didn't load as expected. Please refresh the page and try again.

ABOUT US

  • About DZone
  • Support and feedback
  • Community research
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends:

Likes
There are no likes...yet! 👀
Be the first to like this post!
It looks like you're not logged in.
Sign in to see who liked this post!