Sentiment Analysis and it’s use in Market Research

Structured Data 

Sentiment analysisotherwise known as opinion mining – is an algorithm created to determine the emotional tone behind a series of words, used to gain an understanding of the attitudes, opinions, and emotions expressed within an online mention.

In some cases, the analysis can assign a more specific emotional reaction such as anger, laughter, or love, and it does it through machine learning or opinion mining. This increasingly-popular tool is becoming so vital that a huge number of companies — and even government agencies — have gotten on board with it.

Types of Sentiment Analysis:

Sentiment analysis assumes various forms, from models that focus on polarity (positive, negative, neutral) to those that detect feelings and emotions (angry, happy, sad, etc), or even models that identify intentions (e.g. interested v. not interested).

Here are some of the most popular types of sentiment analysis:

Fine-grained Sentiment Analysis:

If polarity precision is important to your business, you might consider expanding your polarity categories to include:

  • Very positive
  • Positive
  • Neutral
  • Negative
  • Very negative

This is usually referred to as fine-grained sentiment analysis, and could be used to interpret 5-star ratings in a review, for example:

  • Very Positive = 5 stars
  • Very Negative = 1 star

 

Emotion detection

Emotion detection aims at detecting emotions, like happiness, frustration, anger, sadness, and so on. Many emotion detection systems use lexicons (i.e. lists of words and the emotions they convey) or complex machine learning algorithms.

One of the downsides of using lexicons is that people express emotions in different ways. Some words that typically express anger, like bad or kill (e.g. your product is so bad or your customer support is killing me) might also express happiness (e.g. this is badass or you are killing it).

Aspect-based Sentiment Analysis

Usually, when analyzing sentiments of texts, let’s say product reviews, you’ll want to know which particular aspects or features people are mentioning in a positive, neutral, or negative way. That’s where aspect-based sentiment analysis can help, for example in this text: “The battery life of this camera is too short”, an aspect-based classifier would be able to determine that the sentence expresses a negative opinion about the feature battery life.

Multilingual sentiment analysis

Multilingual sentiment analysis can be difficult. It involves a lot of preprocessing and resources. Most of these resources are available online (e.g. sentiment lexicons), while others need to be created (e.g. translated corpora or noise detection algorithms), but you’ll need to know how to code to use them.

Alternatively, you could detect the language in texts automatically, then train a custom sentiment analysis model to classify texts in the language of your choice.

How Sentiment Analysis is used in Social Media

The ability to quickly and accurately identify negative feedback in order to react in real-time and make necessary improvements makes Sentiment Analysis perfect for Social Media. Sentiment analysis is able to monitor customer reaction to product changes in order to prevent social media crises. Similarly, it pinpoints positive feedback, allowing you to see what you’re doing right and recruit brand ambassadors.

Sentiment analysis is extremely useful in social media monitoring as it allows us to gain an overview of the wider public opinion behind certain topics. Social media monitoring tools make that process quicker and easier than ever before, thanks to real-time monitoring capabilities.

The applications of sentiment analysis are broad and powerful. The ability to extract insights from social data is a practice that is being widely adopted by organizations across the world.

It can also be an essential part of your market research and customer service approach. Not only can you see what people think of your own products or services, but you can also see what they think about your competitors too. The overall customer experience of your users can be revealed quickly with sentiment analysis, but it can get far more granular too.

Bringing Sentiment Analysis to Business

For accurate, insightful sentiment analysis, you must have a good tool. Many basic sentiment-analysis tools are built into existing platforms, such as Google Insights, Google Alerts or Facebook Insights. Other useful tools include Brandwatch, Hootsuite Insights, Meltwater, and OpenText.

 If these aren’t enough for you, it may be worth building your algorithm that is focused on your specific use case. The crucial thing to remember here is that high-quality, human-annotated data is the key to success. The best algorithms can draw on human understanding of language within their training data, which helps them to better understand the tone, context and subtle nuance of a piece of writing. Since language is unpredictable and highly susceptible to change, your model’s performance will be judged on the small percentage of edge cases where these apply.

 Luckily, you don’t have to label your data in-house. There is a range of online data-annotation services that will be able to provide you with a large volume of clean, relevant data. If you do your due diligence and find a good source of training data, you’ll see a big difference in the quality of your end product. However, there are a few things that you can do to ensure that you maximize your ROI around training data for sentiment analysis. Before ordering your data, consider the following things:

  • Clear instructions: In much the same way as our political examples needed clarification, annotators will appreciate any further guidance you’re able to provide. One crucial thing to consider is whether you require tags in simple positive/negative/neutral categories, or something more complex.
  • Output quality: In sentiment analysis, there is often no right or wrong answer, so it’s difficult to measure accuracy in this way. Instead, it’s often better to use metrics like Krippendorff’s alpha, which look at the consensus between your contributors as an indicator of quality.
  • The number of data points: Often, companies will approach their data providers with hundreds of thousands of data points for tagging. If you only need to train a simple system with limited categories, this is overkill — and the easiest way to ensure your costs balloon out of control. Honesty and clarity around your project will help both you and your data provider to focus on providing you with the best possible dataset for your model.
A trend to keep an eye on

As with any automated process, it is prone to error and often needs a human eye to watch over it. The Owl Strategy allows users to redefine sentiment if they believe that it has been wrongfully categorized.

Beyond reliability, it’s important to acknowledge that human expression doesn’t fit into just three buckets; not all sentiment can be categorized as simply as positive, negative or neutral.