Automated Survey Processing Using Contextual Semantic Search
Automated Survey Processing Using Contextual Semantic Search
Companies have been leveraging the power of data lately, but to get the deepest information, you must leverage the power of AI, deep learning, and intelligent classifiers.
Join the DZone community and get the full member experience.Join For Free
With the recent advances in deep learning, the ability of algorithms to analyze text has improved considerably. Now, analyzing digital and social media is not restricted to just basic sentiment analysis and count-based metrics. Creative use of advanced artificial intelligence techniques can be an effective tool for doing in-depth research. We believe that it is important to classify incoming customer conversation about a brand based on following lines:
- Key aspects of a brand’s product and service that customers care about.
- Users’ underlying intentions and reactions concerning those aspects.
These basic concepts, when used in combination, become a very important tool for analyzing millions of brand conversations with human-level accuracy. In this post, we take the example of Uber and demonstrate how this works. Read on!
Text Classifier: The Basic Building Blocks
Sentiment analysis is the most common text classification tool that analyzes an incoming message and tells whether the underlying sentiment is positive, negative, or neutral. You can input a sentence of your choice and gauge the underlying sentiment by playing with the demo here.
Emotion analysis can accurately detect emotion from any textual data. People voice their opinion, feedback, and reviews on social media, blogs, and forums. Marketers and customer support can leverage the power of emotion detection to read and analyze emotions attached with the textual data. Our emotion analysis classifier is trained on our proprietary dataset and tells whether the underlying emotion behind a message is happy, sad, angry, fearful, excited, funny, or sarcastic.
Intent analysis steps up the game by analyzing the user’s intention behind a message and identifying whether it relates an opinion, news, marketing, complaint, suggestion, appreciation, or query.
Analyzing intent of textual data
Contextual Semantic Search (CSS)
Now, this is where things get really interesting. To derive actionable insights, it is important to understand what aspect of the brand is a user discussing. For example, Amazon would want to segregate messages that related to late deliveries, billing issues, promotion-related queries, product reviews, etc. But how can one do that?
We introduce an intelligent smart search algorithm called contextual semantic search (CSS). CSS takes thousands of messages and a concept (like price) as input and filters all the messages that closely match with the given concept. The graphic shown below demonstrates how CSS represents a major improvement over existing methods used by the industry.
Contextual semantic search vs. traditional keyword approach
A conventional approach for filtering all price-related messages is to do a keyword search on priceand other closely related words (e.g. pricing, charge, $, paid). This method, however, is not very effective, as it is almost impossible to think of all the relevant keywords and their variants that represent a particular concept. CSS, on the other hand, just takes the name of the concept (price) as input and filters all the contextually similar, even where the obvious variants of the concept keyword are not mentioned.
For the curious people, we would like to give a glimpse of how this works. An AI technique is used to convert every word into a specific point in the hyperspace and the distance between these points is used to identify messages where the context is similar to the concept we are exploring. A visualization of how this looks under the hood can be seen below:
Visualizing contextually related tweets
Time to see CSS in action and how it works on textual data below:
The algorithm classifies the messages as being contextually related to the concept called price even though the word "price" is not mentioned in the messages.
Uber: A Deep Dive Analysis
We analyzed the online conversations happening on digital media about a few product themes: cancel, payment, price, safety, and service.
For a wide coverage of data sources, we took data from the latest comments on Uber’s official Facebook page, tweets mentioning Uber, and latest news articles anpit Uber. Here’s a distribution of data points across all the channels:
- Facebook: 34,173 comments
- Twitter: 21,603 tweets
- News: 4,245articles
Analyzing sentiments of user conversations can give you an idea about overall brand perceptions. But to dig deeper, it is important to further classify the data with the help of contextual semantic search.
We ran the contextual semantic search algorithm on the same dataset, taking the aforementioned categories in an account (cancel, payment, price, safety, and service).
Breakdown of sentiment for categories
Noticeably, comments related to all the categories have a negative sentiment majorly, bar one. The number of positive comments related to price has outnumbered the negative ones. To dig deeper, we analyzed the intent of these comments. Facebook being a social platform, the comments are crowded random content, news shares, marketing and promotional content and spam/junk/unrelated content. Have a look at the intent analysis on the Facebook comments:
Intent analysis of Facebook comments
Thus, we removed all such irrelevant intent categories and reproduced the result:
Filtered sentiment analysis
There is a noticeable change in the sentiment attached to each category — especially in Price-related comments, where the number of positive comments has dropped from 46% to 29%.
A similar analysis was done for crawled Tweets. In the initial analysis, payment- and safety-related Tweets had a mixed sentiment.
Category-wise sentiment analysis
To understand real user opinions, complaints, and suggestions, we have to again filter the unrelated tweets (spam, junk, marketing, news, and random):
There is a remarkable reduction in the number of positive payment-related tweets. Also, there is a significant drop in the number of positive tweets for the safety category (and related keywords).
Brands like Uber can rely on such insights and act upon the most critical topics. For example, service-related Tweets carried the lowest percentage of positive tweets and highest percentage of negative ones. Uber can thus analyze such tweets and act upon them to improve the service quality.
Sentiment analysis for news headlines
Understandably so, safety has been the most talked-about topic in the news. Interestingly, news sentiment is positive overall and individually in each category as well.
We classified news based on their popularity score, as well. The popularity score is attributed to the share count of the article on different social media channels. Here’s a list of top news articles:
- Uber C.E.O. to Leave Trump Advisory Council After Criticism
- #DeleteUber: Users angry at Trump Muslim ban scrap app
- Uber Employees Hate Their Own Corporate Culture, Too
- Every time we take an Uber we’re spreading its social poison
- Furious customers are deleting the Uber app after drivers went to JFK airport during a protest and strike
The age of getting meaningful insights from social media data has now arrived with the advances in technology. The Uber case study gives you a glimpse of the power of contextual semantic search. It’s time for your organization to move beyond overall sentiment and count based metrics. Companies have been leveraging the power of data lately, but to get the deepest of the information, you have to leverage the power of AI, deep learning, and intelligent classifiers like contextual semantic search.
You can also use our Excel Add-in to analyze surveys without writing a single line of code. You can download the add-in from here.
Opinions expressed by DZone contributors are their own.