Images have become a very common medium of human expression on the internet with the coming of social networks. Facebook is the biggest repository of digital images ever. This trend is only going to intensify given the emergence of image platforms like Instagram and Snapchat, also called visual social media. Marketers and analysts generally find it difficult to gather insights from visual social media because it's hard to quantify images being shared. The answer lies in automated image analytics, i.e. visual analytics that can process the visual information and derive conclusions. Until very recently, it wasn’t even possible to do such automated analytics — but times have changed. Now, visual analytics tools are used to analyze images.
With the recent success of deep learning models called convolutional neural networks in automated perception tasks, AI has matured enough to act as a proxy to human observers to document what is being shared. These AI algorithms can actually make sense of what the content of the image is (for instance, it can see that the image contains dogs or cats or an apple) and even assess various quantities for us (Microsoft recently released a demo where they could estimate the age of the person in the image). Extracting such information from the images automates the categorization and processing of visual data.
Deep convolutional neural network
When you input an image, the AI algorithm returns the visual information associated with that image. The output, for example, can tell what all objects are present in the image, how many humans in the image along with their age and emotions, if the image is taken outside or indoors, and even the relationship among the objects in the image (for instance, it can recognize a human and a frisbee in the image and can also recognize that the human is throwing a frisbee and not the other way around).
We decided to run these AI algorithms and check out what we could deduce. While we continue working on our own versions of these algorithms, here we used our own visual analytics APIs on top of social media images to return the image description, tags, and emotions of the input image. Take a look at the following image.
Visual analytics can be used to check the emotions in an image
For this image, the visual analytics API returns these results.
Tags: [“indoor”, “table”, “cake”, “sitting”, “top”, “decorated”, “birthday”, “pink”, “girl”, “food”, “plate”, “topped”, “small”, “front”, “little”, “woman”, “bed”, “desk”, “holding”, “white”, “young”, “large”, “standing”, “bear”, “room”, “phone”]
Speaking of images, Instagram is probably the largest social media platform dedicated entirely to featuring images with hashtags. #gopro is an evergreen hashtag on Instagram almost always. Go pro is known for its action cameras and mobile apps. We crawled all the public images using this hashtag uploaded in last seven days. Upon the image analysis of a total of 727 images using the technique specified above and aggregating results, we came up with the following insights. (Please note there were way more than 727 images that were private, which we did not work on.)
The most common emotion was happiness, which was exhibited in 531 images. This was followed by the no emotion (187), surprise (4), anger (3), and sadness (1).
For the next insight, we have compiled a graph of the top 10 tags with their frequencies that were found in the photo using visual analytics.
Details of images in a graph using visual analytics
As you might be able to decipher, they are more generic image tags, showing demographical and indoor/outdoor distribution of images. Also, white is the most popular color.
The remaining tags are arranged in the following word cloud (the size of the word corroborates with the frequency of tag in the images).
Word cloud for image tags
Judging by the tags we have assembled, it is quite evident that most of the images are taken outdoors either in the vicinity of natural offerings like ocean, grass, rivers, or mountains, or doing outdoor sporty activities. Well, we don’t blame them as the best pictures are clicked when you have a natural light in sight.
In general, such an analysis can help brands, marketers, and researchers understand what's being shared about their hashtags on visual social media without going through the pain of analyzing each image.