DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Join us today at 1 PM EST: "3-Step Approach to Comprehensive Runtime Application Security"
Save your seat

Automated Test Changes

The most significant change to automated testing is its integration into the DevOps methodology and the CI/CD pipeline.

Tom Smith user avatar by
Tom Smith
CORE ·
Feb. 28, 19 · Interview
Like (1)
Save
Tweet
Share
14.57K Views

Join the DZone community and get the full member experience.

Join For Free

To understand the current and future state of automated testing, we spoke to 14 IT professionals intimately familiar with automated testing. We asked them, "What are the most significant changes to automated testing over the last year?"

Here's what the respondents told us:

DevOps/CI/CD

  • 1) As we have adopted an Agile methodology of development, we have to have an Agile testing strategy. Continuously improve automated tests and integrate them into the CI/CD pipeline. 2) Another change is fast feedback by running tests in the development cycle so we can find defects during development. We started providing cloud offerings with UX feedback. UI automation is now a core part of our automation test suite. 3) As we adopt microservices, we have also broken down our monolithic automated test repository into smaller reports managed by individual teams. This has helped us scale and release features quickly. We have also built a contribution model for automation for individual teams to test their own repos.
  • People are starting to realize the importance of developing custom solutions. Organizing to a more product-centric approach, break down silos, and have more shared responsibility. More than technology, it’s about culture and philosophy. We're also seeing a focus on DevOps with security integrated from the beginning. 
  • Automated testing has gained a more critical role in the software development process. Before, test automation was considered a nice-to-have and was typically developed in a silo without much visibility into the results. Now, automated tests have become the gatekeeper for teams who are integrating or deploying continuously. The status of automated tests determines whether customers will receive this feature right now or not.

Embedded

  • Automated tests are becoming embedded in the developer process. We are assimilating to the developers' technology – with the Jenkins plugin, automated security testing is done in preproduction. Full testing is taking place in the SDLC before production.

API-Centric

  • We see a shift from UI automation to API automation. Automation used to be automating a complete workflow; however, this does not help from a DevOps perspective where small modular design automated tests are needed. We’re still seeing Selenium for browser-level automation. Also, we’re seeing a movement toward scriptless automation tools. Tool vendors are expanding on scriptless automation. 
  • In the industry, with the evolution from monoliths to distributed systems it is more and more common to focus on testing the contracts between the APIs and put more efforts in post-deployment verifications in production. The reasoning behind this is reacting quickly to outages is more important in such architectures.
  • Some of web testing is getting less Selenium-centric. Also testing via REST API’s is getting more attention.

Other

  • Organizations are trying to do more with less and are experimenting more with artificial intelligence (AI) and machine learning (ML) to optimize testing and test strategies. They are also focusing more on how much business value they can deliver at speed, by connecting and measuring all of their end-to-end activities, including those related to automated testing, across their portfolios.
  • 1) A big thing in technology right now is people taking AI, algorithms, and neural nets and using them to make a meaningful difference. 2) There is far more focus on the end user. Companies are looking forward rather than backward. More people are thinking about testing software to see that it’s good. They are determining if it will help customers do what they want to do. The stars in the app store have become more important as has knowing whether or not users accomplishing what they want to do. There has been a broadening of what people are testing – performance and usability to improve UX and CX. LOB owners want to see test results, the impact on UX and CX, more shopping, more articles read. There is an opportunity to see real ROI from automated testing. Testing should be a profit center. Be able to determine that you increased the revenue of an app by X%. Go one step further to focus on the business outcome to validate what you are doing. Monitor website or app, conversions, all technical factors. Use AI/ML to see correlations. Quantify the effect of testing on outcomes and users.
  • In past years, we have seen the most significant changes in all aspects automation testing including strategy, tools, and skills. 1) Strategy: We focus on business test scenarios automation rather than test cases. Shift left strategy: In-Sprint automation to align with DevOps and sprint based development and Shift towards Service level/API automation rather than at UI only Layer. Progressive test automation with CI/CD pipelines. Service vitalization, test data, and test environment automation. Automation for business outcomes instead of focusing % test case automation. 2) Tools: Adaptation of freeware tools to reduce automation cost. Integration with build deployment, requirement management, and defect tracking tools for test lifecycle operations automation. 3) Skills: Re-Skilling of the testing workforce on emerging automation tools and frameworks.
  • A lot of startups are looking for more standard ways of doing things. They do not have the time or money to explore industrial solutions. Open source choices with more universal tests are gaining favor.
  • Functional UI test automation for mobile web apps. Over the past year, Selenium 4 has been a pain to build scalable reliable test automation practices across platforms. We’re playing “whack-a-mole” with reliability issues. Look at unifying protocols together to reduce frustration. This is helpful for device cloud providers since you need to support all browser combinations to commoditize the device cloud market. Headless stuff is starting to make its way into device cloud providers. Grab data in real time for automated analytics testing. Interact with UI in real-time by integrating with the device cloud. There are a lot of improvements with Appian. Test outside the pure sandbox of the application. Now simulators and emulators are able to interact with face and touch.

Here’s who shared their insights:

  • Drew Horn, Senior Director of Automation, Applause
  • Angie Jones, Senior Developer Advocate, Applitools
  • Isa Vilacides, Director of Engineering, CloudBees
  • Himanshu Dwivedi, CEO, Data Theorem
  • Antony Edwards, COO, Eggplant
  • Kevin Fealey, Senior Manager Application and Product Security, EY
  • Hans Buwalda, CTO, LogiGear
  • Malcolm Isaacs, Senior Solutions Manager, Micro Focus
  • Madan Mohan, Global Head of Travel and Transportation, NIIT Technologies
  • Jared Go, CEO, OhmniLabs
  • Derek Choy, CIO, Rainforest QA
  • Nancy Kastl, Executive Director of Testing Services, SPR
  • Rishikesh Palve, Integration Product Manager, TIBCO
  • Ray Wu, CEO, Wynd
Testing

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • Key Considerations When Implementing Virtual Kubernetes Clusters
  • The Future of Cloud Engineering Evolves
  • Distributed SQL: An Alternative to Database Sharding
  • How To Use Terraform to Provision an AWS EC2 Instance

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends: