Over a million developers have joined DZone.

Continuous Testing – Where does it fit next to Integration and Delivery?

DZone's Guide to

Continuous Testing – Where does it fit next to Integration and Delivery?

· DevOps Zone
Free Resource

The Nexus Suite is uniquely architected for a DevOps native world and creates value early in the development pipeline, provides precise contextual controls at every phase, and accelerates DevOps innovation with automation you can trust. Read how in this ebook.

On July 11th, Gartner released its 2013 Magic Quadrant for Integrated Software Quality Suites. I would normally think that research did not apply to me, but I received an email from Gartner letting me know that UrbanCode was mentioned. Curious.

As it turns out, Gartner rates IBM as a Leader’s in the Leader’s quadrant** (where positioning is based on completeness of vision and ability to execute). UrbanCode was described as contributing to IBM’s capabilities around continuous test.*

I normally think in terms of Continuous Integration and Continuous Delivery. Where does “Continuous Test” fit in? Paul Duvall, a co-author of the great Continuous Integration book, describes continuous testing as running as many tests as possible with every code change in an article on DeveloperWorks. That’s completely in line with modern thinking about CI and CD. We want to run rigorous tests as frequently as possible to maximize feedback.

Do we need the term “Continuous Test”?

If continuous test is just a subset of continuous delivery, do we need to call it out as its own thing? Unfortunately, I think we do. The emphasis that CI & CD place on testing is too frequently ignored.

We have been frustrated by this for a long time. In 2008, I wrote a somewhat trollishly titled piece “Continuous Integration: Was Fowler Wrong?“. That article focuses on how early versions of Martin’s definitive article described CI as centered on an automated build. However, CI is centered on rapid testing. Most tests require an accurate build to be present. The build itself is an input to those tests and answers the “does it compile?” test. Languages that do not compile still need CI, but they may not need a build. Builds are merely ancillary to CI.

Similarly, deployment automation is a necessary aspect of continuous delivery, but not sufficient. Deployments serve to deliver the application into environments where they may be tested quickly and correctly. They also provide for testing of the production release process as part of normal development and testing activities.

Because so many in the industry focus on the build and deployment mechanics rather than the tests they are for, promoting the term Continuous Test may be helpful in reminding people to drive feedback back.

Maciej Zawadzki, an Urbancode co-founder, speaks to the feedback aspects of CI and CD at length in the webinar “Risking Above the Noise: CI, CD, DevOps and What They Mean to You“.

* “Magic Quadrant for Integrated Software Quality Suites”  Thomas Murphy and Nathan Wilson.

** Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings. Gartner research publications consist of the opinions of Gartner’s research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.

The DevOps Zone is brought to you in partnership with Sonatype Nexus.  See how the Nexus platform infuses precise open source component intelligence into the DevOps pipeline early, everywhere, and at scale. Read how in this ebook


Published at DZone with permission of Eric Minick, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.


Dev Resources & Solutions Straight to Your Inbox

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.


{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}