NVIDIA’s GauGAN Gives Gauguin a Run for His Money

Artificial technology may not actually be turning us into superheroes, at least not in the timeframe posited by this piece in VentureBeat, but it is giving us some pretty sweet new abilities.

“Wouldn’t it be great if everyone could be an artist?” asks NVIDIA’s VP of Applied Deep Learning Research, Dr. Bryan Catanzaro. His excitement is palpable as he reveals one of NVIDIA’s newest AI projects in the above video, a generative adversarial network known as GauGAN that gives everyone the power to create lifelike works of art in minutes.

“This technology allows us to create a smart paintbrush,” Dr. Catanzaro explains, “so that if you wanted to create a new picture you can just draw the shapes of the objects that you want, and the neural network can then fill in all the details.”

And it really is that easy. GauGAN provides three tools for you to create your soon-to-be masterpiece: a paint bucket, paintbrush, and pencil. You just have to make sure you designate the type of feature you’ll be drawing – tree, flower, stone, etc. – before breaking out those MS Paint skills you’ve been honing since the 80s. The network then takes it from there, filling in all shadows and other appropriate details. Having been trained on 1 million images from Flickr, it has all of the knowledge necessary to accurately adjust the sky’s appearance in a snowy landscape, for example, without any additional user input.

For real-time results, the GAN needs to run on a Tensor computing platform – powered by something like this – but with a few modifications, it can also work on a basic CPU, albeit much more slowly. 

While the results certainly aren’t perfect – boundaries between objects are a little too hard to be completely realistic – researchers claim this will improve over time. Regardless, as Dr. Catanzaro explains, GauGAN will hopefully be an extremely useful tool for architects and designers, not to mention anyone who creates virtual worlds, such as the ones used to train self-driving cars.

Although the software isn’t quite yet ready for public release, NVIDIA hopes to add GauGAN to its AI Playground soon, where folks can not only read more in-depth analysis of the project but try out a demo as well.