Visual Aesthetics: Judging a Photo’s Quality Using AI
Visual Aesthetics: Judging a Photo’s Quality Using AI
When machines become more competent than humans to judge subjective content, it opens up a lot of possibilities that were not feasible before.
Join the DZone community and get the full member experience.Join For Free
The visual aesthetic of a website has been shown to critically affect a variety of constructs such as perceived usability, satisfaction, and pleasure. However, visual aesthetics is also a subjective concept and, therefore, training a machine learning algorithm to learn such subjectiveness presents a unique challenge.
Given the importance of visual aesthetics in human-computer interaction, it is vital that machines adequately assess the concept of visual aesthetics. Machine learning and deep learning techniques have already shown great promise on tasks with well-defined goals such as identifying objects in images or translating from one language to another. However, the quantification of image aesthetics has been one of the most persistent problems in image processing and computer vision.
We decided to build a deep learning system that can automatically analyze and score an image for aesthetic quality with high accuracy. Please check out our demo to check your photo’s aesthetic score.
About the Research
We came up with a novel deep convolutional neural network that can be trained to recognize an image’s aesthetic quality. We also came up with multiple hacks while training the algorithm to increase accuracy.
In our paper published on arxiv, we have proposed a new neural network architecture that can model the data efficiently by taking both low-level and high-level features into account. It is a variant of DenseNets, which has a skip connection at the end of every dense block. Besides this, we also propose training techniques that can increase the accuracy with which the algorithm trains. These methods are to train on LAB color space and to use similar images in a minibatch to train the algorithm, which we call coherent learning. Using these techniques, we get an accuracy of 78.7% of the AVA2 dataset. The state-of-the-art accuracy on the AVA2 dataset is 85.6%, which uses a deep convolutional neural network with pre-trained weights on the ImageNet dataset. The best accuracy on the AVA2 dataset using handcrafted features is 68.55%. We also show that adding more data to our training set (from AVA dataset not included in AVA2) increases its accuracy to 81.48% on AVA2 test set, hence showing the model gets better with more data.
Selecting Best Profile Photo for Your Social Media Account
Social anxiety can be a thing of past while setting a profile picture.
App developers of social media sites can help their users decide which photo will best suit their profile image. We all have faced anxiety when uploading photos on social media sites or changing our display picture. With our API integration, app developers can help their users look good — always!
Smart machine learning algorithms can help you put your best photo on dating apps.
This use-case may not appeal to the zen, non-materialistic folks among us, but to be honest, dating leads to the most social anxiety. The dating landscape keeps changing, as well, and if you're active on dating apps, it’s important to choose your best photos to improve your chances for right swipes!
Dating app developers can easily integrate our APIs to help their users upload their best photos; the visual aesthetics model can also be fine-tuned if the developers want to optimize it on their dataset.
AI-Enabled Camera Phones
Real-time photo quality feedback from phone’s camera.
Recently, Google launched Pixel 2 and Pixel 2 XL, which have portrait mode. This phone offers portrait mode even though it lacks the second lens that many other phones have. For example, the iPhone X, Galaxy Note 8, and OnePlus 5 have portrait mode because they use data from two lenses — one lens captures the image and the other captures the depth information, apart from providing some focal range magic for the blurred background. However, Pixel uses AI to give HDR+ images to users that are comparable to pictures clicked by a DSLR camera.
Similarly, mobile manufacturers can augment the capabilities of their native camera by integrating the visual aesthetic APIs to let their users know in real-time the quality of their photo before taking it! This will enable your users to share their photos with confidence and you will end up creating a great differentiator for your brand at no additional hardware cost.
Virality in Online Content
Content is king, and it has become ever more difficult to write compelling content that resonates with your audience. However, the best content these days often has great images to complement it and therefore, you have to include something that will keep eyes moving down the page.
BuzzSumo did an analysis that covered over one million articles and found that the ones that had images every 75-100 words had more social shares. Using our visual aesthetics tool, you can quickly check how appealing your images are and accordingly improve the virality of your blog post. You can check the demo here.
In this post, we covered some of the use cases of our visual aesthetics API. When machines become more competent than humans to judge such subjective content, it opens up a lot of possibilities to exploit that were not feasible before. You can read more articles on visual analytics here.
Published at DZone with permission of Shashank Gupta , DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.