Sentiment vs. message in social media – which do you value more?

Scale of justice, close-upRecently I fell into a discussion about specific social media metrics (isn’t that so not like me?) and the concept of sentiment analysis was brought up. There have already been many discussions about the usefulness (or lack thereof) of automated sentiment analysis tools, and I won’t bore you with yet another discussion of whether automated sentiment is worth your while.

What I think is a more valid question: is the sentiment or the message more important to social media measurement?

I won’t pretend to have an answer, and I honestly don’t even know if I feel comfortable giving an opinion at this juncture. I will, however, assert that there is an obvious lack of discussion about messages in social media measurement. And that has got to stop.

While the message may not be as important as sentiment (or visa versa), both pieces still have a place in the mythical social media measurement equation. Before the balance or weighting of these two factors can be solidified, there are a few questions that need to be answered:

  1. How will messages be measured within social media content? How will they be counted?
  2. Will every key message be considered equal?
  3. Will only exact wordings of a specific key message be counted or considered?
  4. Will sentiment and message be considered for each post separately or in conjunction?
  5. How will these factors compare to things like reach and influence?

This topic is still so new and open to interpretation, and I’d be eager to know what your thoughts are. I’m also interested to hear if anyone else has even been thinking about message integration in social media measurement.

Have you thought beyond sentiment analysis? What are your thoughts on the importance of messages in measurement?

Share on Tumblr

  • marksysomos

    It's a difficult question to answer because both have important. In many ways, they so closely intertwined that separating them is a challenge. Companies using social media monitoring need to focus on the message and the sentiment, and identify issues, trends and news that jump out from the landscape.

    cheers, Mark

    Mark Evans
    Director of Communications
    Sysomos Inc.
    @sysomos

  • http://rebeccaadenison.com Rebecca Denison

    You make a valid point, but I wonder what the best way to combine the two factors is? Do you have any suggestions or best practices that you use to take both into account?

  • http://rpulvino.wordpress.com Rich Pulvino

    You raise some very interesting and important points, Rebecca. Both are important to social measurement. The messages convey that the brand is being talked about (an important discovery), while the sentiment provides insights into any type of action/behavior behind the message.

    While key messages serve as excellent guidelines, limiting message and sentiment analysis to only these messages can result in something being overlooked. I think we're still at the point where social measurement tools are effective aggregation platforms, but when it comes to analyzing sentiment, it's best left to us.

    These tools will get better, but the area is pretty new. It'll take time.

    Cheers,
    Rich

  • keithtrivitt

    I'm inclined to agree with Rich in that while yes, trying to use measurement tools to look for key messages and highlight the main message points that are being communicated across a broad spectrum of social platforms that align with the key messages the company was hoping to get across is very valuable, brands may be missing something if they are only looking for those key messages.

    The downside of measurement tools is that they can't account for the human quality of looking beyond just the obvious and beyond just what is right in front of you. So, while a measurement tool can find key message points within social platforms, a real human can also look past those key messages to see what the sentiment was before, during and after that message was delivered. That, to me, has far more value, but it also requires a lot more time and resources, something that given the current massive scale of social media, can't always be accomplished in a given period.

  • http://rebeccaadenison.com Rebecca Denison

    Rich – I absolutely agree with you! Limiting metrics to key messages gives a narrow and skewed pictures of all social media conversations. At the same time, I think it's important to consider the key message. Why not do two separate analyses? One focusing just one key messages, one looking at everything?

    Keith – I'm so glad you bring up the limitations of measurement tools themselves. Unfortunately, even the best out there are still not nearly as good as the average human would be. Narrowing date ranges and taking sample sets can help with the issue of scale, but you're absolutely right. We are still so limited, but I think we're making strides in the right direction!

    Thank you both so much for your comments and thoughts! :)

  • jeffespo

    This post addresses something that everyone participating in the space needs to ask themselves. While tools can get you close, machines are not perfect and to get the most out of the metrics, you need a person to look into them. This is time consuming and not scalable for an agency with multiple clients (but could be a pricey add on or upgrade).

    I would say that both factors you mention should be taken into consideration in context. For the sentiment numbers, look at how you (or the client) stack up against the competition. In terms of the messaging, that falls under reputation management. If its bad, how do you address and fix it? Some companies may not be ready to admit they aren't perfect, but sooner or later they will if the buzz is too bad.

  • http://rebeccaadenison.com Rebecca Denison

    You know, I hadn't even considered the issue of scale when I wrote this post, but I'm so glad everyone has brought it up! The idea of doing deep analysis isn't too daunting, until you think about the incredible volume of social media content out there for a lot of large brands and companies.

    So you suggest comparing sentiment across brands? I like that idea. If you have 60% positive posts, so what? Does your nearest competitor have 5% or 95%? Context is so important!

    I was thinking of messaging more in terms of PR measurement. If your latest campaign for a brand pushed the message “Brand X is cool,” how often do you see this message pop up in social media? Is it a large part of all posts about the brand? Does this matter more than sentiment?

  • jeffespo

    It really goes hand in hand if you want to get really deep in the weeds. Like you said does X move the needle, if so, by how much. Also how much brand impact did it have, did you gain more positive or was it poorly received.

    The sentiment and conversation share of voice can be applied to PR as well (although much more time consuming) but is a metric to look at.

  • http://rebeccaadenison.com Rebecca Denison

    I love “move the needle.” That's really the most important metric. No matter what it is you're measuring, did the needle move?

  • jonnybgood

    Thanks for bringing this up. Sentiment and messages (as well as issues) have been tracked singularly and in combination in media analysis for years now. Like you, I'm not going to get into the human versus automatic sentiment debate. What I will say is that sentiment alone is a useful metric but little more than that. What is really useful to know is not just sentiment towards my brand/product, service or organisation but precisely what people like, don't like and why. If I can know who expressed these opinions (former, present,potential, competitor customers) where and with what impact, then I can start taking specific actions to mitigate or enhance reputation. I believe these insights can only be gained by analysts being in your social media spaces day in day out. From there can come the insights to drive action in CRM, product/service development as well as marcomms.

    Jon Moody
    ASOMO
    Jon.moody@asomo.net
    Twitter: @asomouk

  • http://twitter.com/paulmay Paul May

    Great question. It seems to me that, because they're tools for measuring different things, you can't determine which is most important without first looking at the higher level goals. Jeff has it right with the “does it move the needle” question…determining the definition of “move the needle” for the effort at hand is a pre-cursor for determining which is more valuable. So, for example, if you were putting together an eBook with the goal of thought leadership, sentiment among key influencers might be more important. If you were supporting a product launch, message pull-through and reach might be more important.

    At the end of the day, I think Jeff has it right. When you look at sentiment and messaging in conjunction with the higher level goals, you get powerful insights into what works, what needs fixing and what actions should be taken in the future.

  • http://jackieadkins.com Jackie Adkins

    Pretty interesting discussion you've sparked in the comments here, Rebecca! Sorry in advance for the long comment :) But here we go…..

    Sentiment alone really does only tell you so much. You can look at a pie chart or timeline of your sentiment and sure, you can see that you're killing it with positive sentiment now or that last Friday you're positive sentiment spiked like crazy, but ultimately, you're going to be left asking yourself: Why? Which is where the message itself comes into play.

    It could be positive because people are congratulating your brand on an award. Or, it could be positive because people are praising a new product feature. Both are positive, but each are very different. The first is just a pat on the back, while the 2nd needs to get in front of your dev team so they know what's working (and what isn't).

    This is why I've always been a little hesitant about recommending people use our automated option with BuzzMgr (where I work :) , because, even if the sentiment was accurate (which, well, different conversation…), it's going to be tough for you to have a good understanding of what's behind the raw numbers and how to act on it. That's why we're big on analysis (whether they do it internally, or sign up for one of our media analysts to consult on it), because that's the most effective way to extract key themes and messaging patterns.

    But, as someone else mentioned, the bigger the brand, the harder this is to scale since it obviously takes a LOT more time to manually look at the messaging and everything.

    So, I'm not quite sure where that leaves us, but it gives us some more to chew on :) Good stuff, Rebecca!

  • http://rebeccaadenison.com Rebecca Denison

    Jon, thank you so much for your insights! I tend to agree with you that sentiment is a nice metric but doesn't provide a whole heck of a lot of insights into branding and reputation.

    I like that you brought up not only WHAT is being said, but also WHERE and by WHO! Those are equally as important to most brands and honestly should be given more time than they are in most cases.

    What tools do you prefer to use to track all of these things?

  • http://rebeccaadenison.com Rebecca Denison

    Absolutely! I'm so glad you mentioned defining what “move the needle” will mean because it definitely changes with each campaign or client. That's exactly what made me start wondering about sentiment vs. message, why do we think sentiment is the universal metric?

  • http://rebeccaadenison.com Rebecca Denison

    Dude! I LOVED the insight, and I will say it again: don't even apologize for leaving an awesomely long comment. Amazing!

    You're exactly right. Sentiment is still useful, but the why behind it is almost always going to be far more important. I like that example because often we fall into the trap of all positive feedback is really good and all negative feedback is bad. But in the case of a new feature, negative feedback could just be, “I'm not liking this “Like” a brand thing.” But if you're in the middle of a crisis, negative feedback could be, “You suck! I'm never buying your products again!” Totally different implications and reactions are needed.

    I think you give some great advice, integrating message patterns and other key metrics is very important and more valuable to most clients than just sentiment. But of course, the question of scale will always be an issue. We often do sample sets as a solution, but sometimes that still will not be enough.

  • jonnybgood

    I use and work for ASOMO (About Social Movements) – expert analysts in your webspace extracting insights for action about what, who, where and with what impact opinions are mobilising around brands, products and services. As I said the applications can and frequently should go beyond PR and marketing and into customer service/engagement, product/service development, wider stakeholder relations and even HR. There can be issues of scalability but in the long run, this methodology is best applied to seeking answers to very specific questions about perceptions and what you can do about them.
    Jon Moody
    ASOMO

  • http://rebeccaadenison.com Rebecca Denison

    Duh! I guess I should have known the answer to that question!

    I think you're right that the methodology can be applied to specific questions, and it always goes back to having measurable goals/objectives to begin with. You can't easily measure everything, but you can track message for certain brands in certain places, for example.

  • http://www.rebeccaadenison.com/?p=686 DR. WHAW? – May 18, 2010 « One True Sentence

    [...] Sentiment vs. message in social media – which do you value more? By Rebecca Denison – How could I not include a post from Rebecca?! Because she asked a great question, and brought up a few more in the post, the comments shouldn’t be missed! [...]

  • http://digitalassetmanagement.org.uk/2010/08/06/can-computers-really-tell-what-we-think/ Can computers really tell what we think? | Digital Asset Management

    [...] Sentiment vs. message in social media – which do you value more? (prbreakfastclub.com) Tags: brand management, negative statements, nuances, sentiment analysis, social networks blog comments powered by Disqus var disqus_url = 'http://digitalassetmanagement.org.uk/2010/08/06/can-computers-really-tell-what-we-think/ '; var disqus_container_id = 'disqus_thread'; var facebookXdReceiverPath = 'http://digitalassetmanagement.org.uk/wp-content/plugins/disqus-comment-system/xd_receiver.htm'; var DsqLocal = { 'trackbacks': [ ], 'trackback_url': 'http://digitalassetmanagement.org.uk/2010/08/06/can-computers-really-tell-what-we-think/trackback/' }; (function() { var dsq = document.createElement('script'); dsq.type = 'text/javascript'; dsq.async = true; dsq.src = "http://damblog.disqus.com/disqus.js?v=2.0&slug=can_computers_really_tell_what_we_think&pname=wordpress&pver=2.33"; (document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq); })(); [...]

  • http://edelmandigital.com/2010/05/14/sentiment-vs-message-in-social-media-%e2%80%93-which-do-you-value-more/ Sentiment VS Message In Social Media – Which Do You Value More? | Edelman Digital

    [...] Originally posted on PR Breakfast Club. [...]

  • Anonymous

    I Feeling and messages (and themes) have been tracked individually and in
    combination in the analysis of the media for years. Like you.I share this blog with my family and the friends.
    mobile signal booster

  • http://www.squareberry.com/ Mayan Lee

    Sentiments, it is more close in connecting with other people.

  • http://staging.edelmandigital.net/2010/05/14/sentiment-vs-message-in-social-media-%e2%80%93-which-do-you-value-more/ Edelman Digital » Sentiment VS Message In Social Media – Which Do You Value More?

    [...] Originally posted on PR Breakfast Club. [...]

  • http://virusremovalhelps.com/help-vista-internet-security-2012-virus-removal/ Vista Internet Security 2012

    I think I will consider both sentiment and message.

  • http://www.airmaxstockez.com/ air max chaussures

    At the end of the day, I think Jeff has it right. When you look at
    sentiment and messaging in conjunction with the higher level goals, you
    get powerful insights into what works, what needs fixing and what
    actions should be taken in the future.
     

  • http://www.jordanstockez.com/ jordan pas cher

    Un très bel article qui a été écrit dans ce blog et toute la pensée exprimée par le présent article. Dans cet article, la FA? Consiste à exprimer les pensées sont très gentils et il est court et doux pour moi pla t plus ici.

  • http://www.lunettesporto.com/ lunettes de soleil pas cher

    goood nana

  • SusanDAyers

    It seems to hard to choose which one of this things but I guess business like using the social media will only needs proper management for us to be successful in time.