Semantic Analytics – Detecting Context within Social Media Conversations

Posted on by
In many ways, we are at the beginning of social media usage and adoption. Most usage charts and graphs are platform- or country specific. Although, I did find this very interesting social web involvement map in different markets but it’s quite hard to see the breakdown.

Click image to enlarge. Source Global Web Index

However, the point I am trying to make is that social media adoption is still in its infancy. When you consider the number of individuals becoming authors, contributors, publishers, sharing and collaborating it might be worth admitting that few of us know where this evolution is going and what the end-state of this phase may look like. But this blog post is not about the importance of understanding social media adoption and technology but rather how to manage all of this new data, that is growing both quickly and daily.  One of the bigger challenges organizations face with the explosion of social media and consumer-generated content is the ability to extract, process and leverage contextually relevant data in real-time. It brings to mind a few questions, like:
  • What will the social web look like once under-represented locations join the conversation?
  • Do you have a strategy for filtering and managing data that is unique to your brand, your products and services and your industry as the source of information grows exponentially?
  • How do you intend to identify shifts in the conversation as they merge with other topics, of less or more relevance?
  • Have you determined the criteria for when an emerging trend or topic spills over into an entrenched idea?

Today’s Social Media Landscape

Here’s our view of the social media data landscape today, including sources of private data, like: surveys, private communities or call center information. But even we know that this landscape is expanding all the time and our collection may look very different a year from now.

Click image to enlarge

The Search for Truth and Meaning

I’m building a little on the ideas described in this previous blog entry, "Semantic analytics serves the truth & vegetables from a social media diet", which emphasized the importance of identifying all on-topic engagements and then uncovering the truth of social media conversations. Identifying all related and on-topic social conversation will require a sophisticated listening platform that in many ways resembles a more traditional search engine than a simple monitoring tool. There are two key functions of an enterprise listening platform:
  • Identifying and collecting on-topic conversations
  • Applying advanced analytics to surface consumer intentions and preferences, including author information
Both steps are important. At CI, we use clustering algorithms with semantic proximity measures to derive themes from groupings of semantically similar posts. Using this approach we are able to apply contextual relevance to social media conversations. Why would this be important? Because we are able to see groupings of conversations based on meaning. Once you have extracted the relevant conversations from your query, then more accurate sentiment, theme generation, theme trending and term analysis can be performed.  In many ways, you are trying to extract specific aspects of a conversation to uncover consumer perspective on price, quality, customer service issues, etc. You simply cannot get to this point if you are relying on a simple monitoring tool, or (gasp) a manual process. Both approaches cannot scale, are not repeatable and take large amounts of time.

Who Said That?

Once you've identified and analyzed the conversation, you need to explore the audience of consumers, who are generating the content. For your more immediate needs, you may focus your analysis on a specific set of conversations. But how about other areas of interest a consumer may have about which they are expressing an opinion? How important is that context to your organization? CI is able to collect and assign the traits of a post back to a given author, these trait values can be averaged together to produce an ‘author profile’. These profiles are the basis from which CI then describes and segments authors on their demographic and psycho-graphic properties and further defines the true voice of consumer's considerations and preferences.

Do You Have Any Idea of Who I am?

Defining traits and constructing author profiles will become increasingly important as social media analytics evolves on a global scale. Outreach efforts and tactics that work in one locale may not have the same results in other areas, which is why ongoing, scalable listening is so critical. Aggregating and sifting through this ever-growing source of data for relevant consumer social insights is a critical first step for sophisticated business intelligence. Make sure that whatever approach you are considering for managing social media intelligence can scale at the speed of social media usage and adoption. Just updated to provide a link to a great breakdown of social media usage stats from @johnlovett.