The research is becoming social and effective as humongous amount of information is generated by tapping into minds of people; and no doubts it is driving decisions in private and public sectors. By analyzing online public opinion, governments and companies are accessing a set of insights to shape strategy and better measure public’s experience about products, policies, services and brands. The credit of this success goes to the emerged science of opinion mining. It uses the latest advances in artificial intelligence – AI, to mine public opinion for sentiments. The structured data that they derive to is called opinion data.
The purview of opinion mining is not limited to sentiments, but also the topics that drive those sentiments. This has opened up new avenues in the world of markets, economies and political research as well. Facebook, in the month of Dec 2016, alone had more than 1.15 billion mobile daily active users. This more or less gives a clear picture of the quantity of online conversations. If we couple it with the reactivity of this digital chatter, it makes sentiment-driven opinion data a new revenue generating resource.
AI based approaches & nuances of human conversation
The process of data mining has two steps. The first step is where AI structures the data. The second step is where analysis, derived by processing data from a team of people who verify the data for sentiment and the topics that drive those sentiments, is added to the structured data. The second step is required to appropriately classify the unstructured data, because AI based approach faces challenges in identifying the nuances of human conversations. With social media this has been a persistent problem, as the conversations are typically filled with slangs, humor, sarcasm, emoticons, colloquialisms, innuendo and much more.
“I just spent 2 hours in the queue at ABC bank… best customer service ever!”
Now this is a clear case of sarcasm, but for a machine it is a positive response. Only the fine blend of human intervention with AI has the chances of getting the best of both the worlds – data mining and processing huge data sets with the accuracy in understanding public feelings.
Government – insight into electorates is a game changer for many
In past, these data was only used to for making informed commercial decision making, as it catered companies with deeper insight in to consumer experience. Though it’s penetration in governmental/state/city arena has increased, a part of it still remains clinched to the commercial territory.
Social or sentiment analytics, as we may call it, has proved its worth in both, understanding electorates and driving strategic political decisions. Governments & other state bodies are turning to social media analysis in search for greater clarity. Clinton and Trump election data – social analytics turned the world upside down.
Companies and businesses – agile, pro-active and lean approach would help
Current business landscape of increased competition, the threat of disruptive innovation and on top of it, the changing customer relationships, are making it mandatory for businesses to be agile, proactive and lean. Companies that witnessed tremendous growth stalled in developed countries only are trying to venture out in new and untested markets with help of social data.
The explosion in social media and the increasing use of mobile phones can empower companies with access to huge data sets across mature as well as developing markets. Data mining if done appropriately; is potent enough to not only assist decision making – but also drive product innovation and risk mitigation. For this, they approach market research agencies, database aggregators, marketing agencies, data management agencies etc.
Media – personalized content & on-demand viewing
The focus on personalized content and on demand viewing is changing the media landscape to a great extent.
If we look at it from an opinion mining perspective, may it be a pilot episode, new channel or a major happening; several viewers are honest and vocal about their tastes and to express their feelings – they turn to social media. This information and the drivers behind it, is what media companies have started gaining access to for the type of data they require to tailor their programming strategies, for channels and markets. This helps them deliver the content their viewers want in the form they want to consume it.
News media is struggling to understand their audiences and their feelings about today’s stories. Fake news and “news feed bubbles” can be held responsible for creating significant pockets of public opinion, which are not based on truth. This undermines the legitimacy of their journalism. Opinion mining offers a way for media to accurately measure and report on public opinion, regardless of its veracity.
Data analytic solution providers are a value proposition
Troubled economic climate and disruptive technologies are creating both, risk and opportunities for market research companies. Clients’ approach them not only for data extraction or data capture or data mining as you may say, but with an expectation to understand today’s consumer behavior. It will help them drive innovation, make informed decisions and design and develop their products and services accordingly.
However; conventional offerings and research methodologies are not sufficient enough for the purpose. Their traditional methods which rely on small sample sizes, are slow and expensive, and are full of biases which are sufficient to lead to errors. The recent failure of pollsters to predict Brexit is a pertinent example of this.
Considering these facts, to bring in new clients and retaining them by providing insights based on opinion data along with specialist expertise, social analytics will prove to be a crucial differentiator. Appropriate mining of opinion data would empower researchers to predict what people tend to purchase and understand the driving factors of that tendency.
Consumer to a new model of car or phone, if expresses dissatisfaction by a tweet, as a standalone; might not create that impact. But if a lot of people do so, and after data mining, if the drivers behind their dissatisfaction can be analyzed, quantified and pinpointed – the outcomes can be used to improvise the product lines, and also understand why competitor’s offerings are outperforming.
Way ahead for opinion mining
Till now, unstructured opinion data across social media and many more places was largely inaccessible. The world of opinion mining is set to change the way governments and businesses collect data, conduct market research, made decisions and measure the impact. Tapping into text analysis, overlaying it with refinement of crowd based analysis is the way ahead.