Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Visual CI - New snapshot compare solution for continuous integration

DZone's Guide to

Visual CI - New snapshot compare solution for continuous integration

· DevOps Zone ·
Free Resource

Discover how quick and easy it is to secure secrets, so you can get back to doing what you love. Try Conjur, a free open source security service for developers.

Software development even with UI automation testing requires manual regression testing, which costs a lot of time and resources. The simple reason is that most UI automation frameworks out there only test the functional part of your UI and not the look and feel of the UI. Your automation can run perfectly, yet some wrong CSS change messes up the entire page. This causes managers to invest a lot of time in manual regression testing before each release. With DevOps and short release cycles on the rise, this is becoming a costly problem.

A new solution?

Lately I found new software that allows to almost eliminate the need for manual regression testing before releasing a new version (assuming you have UI automation), by integrating into your existing UI testing framework. It’s called Visual CI, It will save reference images during the UI automation flow and on next runs will compare and validate the new images according to certain settings. Most importantly it will allow you to select what parts of the image should be compared (Nothing more annoying than a random ID corrupting your tests)

So if you have UI automation covering your functional requirements, and during that automation you take screenshots that are validated as well, the trust you have in automation increases dramatically.

I am very excited about this new solution, there is always some talk on the web on comparing snapshot images during automation, but it’s never a full simple solution, It’s either some concept described, or an over complicated framework that requires rewriting all your automation (Or companies telling us they do it internally, while keeping it to themselves, Google, SalesForce, etc.). Just thinking of the yearly cost in manual testing this tool can save, gives me Goosebumps.

But the question rises, does it really work? Yes, it does.

The application comes with a simple intuitive way to mark regions of unstable or random data, for example clocks, generated Ids, or random data.

While your UI automation is running, you simply take screenshots of the application, and send it with a specific ID, to the Visual CI server, using REST api (Or Java/JavaScript wrappers).

Taking it out for a spin

I tried using it and the sample project runs pretty quickly and shows you the image compare abilities.

In the code, it looks like this:

//Use web driver to get the browser image (during a UI automation test)
byte[] image = webDriver.getScreenshotAs(OutputType.BYTES);
//Use the Visual CI library to compare and validate the image
VisualCI.validateImage("MyImageId", "default", image);

When a mismatch occurs, it will show the original reference image, and validate image, a pixel diff image. It will then allow you to

-  Fix your bug (if it’s a real defect).

-  Mark region/s as ignored to avoid random data interfering with the test

-  Make the new image into the reference image (Intended change)

-  Mark the test as ‘ignored’

Reference Image Validation Image Pixel diff image
 

What's next?

The tool is new and is still missing some important capabilities; it doesn't support LDAP integration and some other enterprise requirements; however it is a serious move in the right direction

For the purpose of interesting discussions, I suggest you try it, and post your response here, with answer to these 2 questions:

-  Will screenshot compare tool reduce regression testing in your app/organization?

-  Does this tool give enough value to use?

Get it athttp://visualci.com 

Conjur is a free open source security service built by DevOps engineers. With integrations with all your favorite tools and an easy way to secure secrets, it's a no brainer. Come check it out!

Topics:

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}