Over a million developers have joined DZone.

Visual CI - New snapshot compare solution for continuous integration

· DevOps Zone

The DevOps Zone is brought to you in partnership with Sonatype Nexus. The Nexus Suite helps scale your DevOps delivery with continuous component intelligence integrated into development tools, including Eclipse, IntelliJ, Jenkins, Bamboo, SonarQube and more. Schedule a demo today

Software development even with UI automation testing requires manual regression testing, which costs a lot of time and resources. The simple reason is that most UI automation frameworks out there only test the functional part of your UI and not the look and feel of the UI. Your automation can run perfectly, yet some wrong CSS change messes up the entire page. This causes managers to invest a lot of time in manual regression testing before each release. With DevOps and short release cycles on the rise, this is becoming a costly problem.

A new solution?

Lately I found new software that allows to almost eliminate the need for manual regression testing before releasing a new version (assuming you have UI automation), by integrating into your existing UI testing framework. It’s called Visual CI, It will save reference images during the UI automation flow and on next runs will compare and validate the new images according to certain settings. Most importantly it will allow you to select what parts of the image should be compared (Nothing more annoying than a random ID corrupting your tests)

So if you have UI automation covering your functional requirements, and during that automation you take screenshots that are validated as well, the trust you have in automation increases dramatically.

I am very excited about this new solution, there is always some talk on the web on comparing snapshot images during automation, but it’s never a full simple solution, It’s either some concept described, or an over complicated framework that requires rewriting all your automation (Or companies telling us they do it internally, while keeping it to themselves, Google, SalesForce, etc.). Just thinking of the yearly cost in manual testing this tool can save, gives me Goosebumps.

But the question rises, does it really work? Yes, it does.

The application comes with a simple intuitive way to mark regions of unstable or random data, for example clocks, generated Ids, or random data.

While your UI automation is running, you simply take screenshots of the application, and send it with a specific ID, to the Visual CI server, using REST api (Or Java/JavaScript wrappers).

Taking it out for a spin

I tried using it and the sample project runs pretty quickly and shows you the image compare abilities.

In the code, it looks like this:

//Use web driver to get the browser image (during a UI automation test)
byte[] image = webDriver.getScreenshotAs(OutputType.BYTES);
//Use the Visual CI library to compare and validate the image
VisualCI.validateImage("MyImageId", "default", image);

When a mismatch occurs, it will show the original reference image, and validate image, a pixel diff image. It will then allow you to

-  Fix your bug (if it’s a real defect).

-  Mark region/s as ignored to avoid random data interfering with the test

-  Make the new image into the reference image (Intended change)

-  Mark the test as ‘ignored’

Reference Image Validation Image Pixel diff image
 

What's next?

The tool is new and is still missing some important capabilities; it doesn't support LDAP integration and some other enterprise requirements; however it is a serious move in the right direction

For the purpose of interesting discussions, I suggest you try it, and post your response here, with answer to these 2 questions:

-  Will screenshot compare tool reduce regression testing in your app/organization?

-  Does this tool give enough value to use?

Get it athttp://visualci.com 

The DevOps Zone is brought to you in partnership with Sonatype Nexus. Use the Nexus Suite to automate your software supply chain and ensure you're using the highest quality open source components at every step of the development lifecycle. Get Nexus today

Topics:

Opinions expressed by DZone contributors are their own.

The best of DZone straight to your inbox.

SEE AN EXAMPLE
Please provide a valid email address.

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.
Subscribe

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}