Webinar Summary: The State of Test Automation in 2020
COVID-19’s impact on test automation, today’s biggest hurdles, and looking ahead
Join the DZone community and get the full member experience.Join For Free
In June, our friends at Testim surveyed end-to-end test automation. So, what were the results?
Watch this webinar recording where Abstracta COO, Federico Toledo, joined Testim CEO Oren Rubin to uncover what developers, QA, and managers are saying about test automation.
They discussed the results of the survey and compared it to other data points gathered from related industry sources and their own experiences.
- How has COVID-19 affected test automation?
- What are the top challenges and priorities for automation?
- What do we expect in the future?
Watch the Webinar
TL; DW (Too Long, Didn’t Watch)?
Who Took the Survey?
To start, here’s a breakdown of the survey respondents by role:
- 46% – Automation Engineers
- 18% – Developers / Dev managers
- 16% – QA Testers
- 13% – QA Managers
- 7% – Other
And by team size:
- 29% – 100 or more
- 78.1% – Less than 100
What’s maybe even more interesting, is how the respondents answered they organized their teams:
From our experience, we were not too surprised to see these results, since, in most of our own projects, we integrate our testers with our client’s development team, whether they’re using Scrum or kanban, etc. We see it’s highly valuable to have the testers participate in the team’s daily calls, retrospectives, planning, and even in product demos for the customer. This helps everyone in development to remain aligned and work towards a common goal.
What Skills Are Most Critical for Testers?
Coming in at number one for most important skills was test design and architecture, which makes it seem that the respondents emphasized the need for coding skills. Oren Rubin agreed that this was very important: being able to design and implement tests is critical.
Federico also wanted to emphasize the need for the human side of testing, looking beyond the technical answer choices that were provided:
What Are Your Biggest Challenges in Test Automation?
The fact that most people complained that test maintenance is one of the greatest challenges they face when it comes to testing automation didn’t surprise the Testim team (given that they built a tool to help solve for that!). The next biggest pain that the survey indicated was the long time it takes to write UI functional tests.
Trends Impacting Test Automation in 2020
The top trends that respondents indicated around test automation today were:
- CI/CD – 55.6%
- AI-assisted Testing – 46%
- Continuous Testing – 38.1%
- Shifting Left – 33.3%
- Agile Testing – 55.4%
- Testing in production – 25.4%
As Federico commented, these answers are consistent with our own experiences, as the shift-left testing approach becomes more popular, we see more engagement from different team members, taking part in automation, whether its unit testing or API testing. So these methodologies that are gaining more traction impact the way we automate and of course, this relates to CI/CD because we’re able to benefit more from automation since we’re using it in different stages before each release. If we’re continuously testing, then we should be shifting left and right at the same time, so these trends are fairly related to one another.
Automation Goals for the Future
Most respondents were looking to increase how many cross-browser tests they run while many were satisfied with their current suite.
Oren commented that some of those who do not aim to grow their cross-browser testing may be shooting for increasing their test coverage in a specific browser first. It may also be due to the high level of maintenance involved.
For teams that haven’t managed to already, it is clear that common goals include more automating of tests, integrating test automation with CI/CD, and increasing test coverage.
Federico commented that instead of making more tests, maybe it’s smarter to focus on making current tests more stable and more reliable.
What’s probably more interesting is where testers think the focus will shift in the next three years? From the results, it would seem that most expect to have already implemented CI/CD, with shifting left also losing importance as a priority. However, almost 80% answered AI-assisted testing will be a major trend looking ahead.
COVID-19’s Impact on Test Automation
And last but not least, we couldn’t fully understand today’s state of test automation without mentioning COVID-19. The survey touched on how test automation has been impacted by the pandemic. According to the results, two-thirds (67%), responded that it did not impact their team.
Conversely, as for their budget, over three-fourths responded that it did cause their budgets to be reduced.
Oren speculated that the respondents must have had two different budgets in mind while going through the questions: one for people and the other for tooling. Teams may have invested in more tooling to compensate for not being able to hire more development talent during COVID-19 as their goals most likely remained the same.
For us at Abstracta, one of the main impacts of COVID-19 was an acceleration in a cultural shift. Nowadays, we see that our clients and other companies are more ready to accept the possibility that teams can work seamlessly in a distributed manner. As a benefit of this, companies can source the brightest talent, wherever it may be. We have been forced to learn more communication tools during the pandemic, which can help us to work in a more fluid, and even, asynchronous way.
As for COVID-19’s impact on the team’s test automation initiatives, 45% answered that they had been slowed down, 40%, accelerated, and 15% reported no impact.
What do you think of these survey results on the current state of test automation? Does anything surprise you? Let us know in the comments!
P.S., Major kudos to Tristan, Testim’s Community Manager for live-tweeting the webinar!
Published at DZone with permission of Kalei White. See the original article here.
Opinions expressed by DZone contributors are their own.