Unicorn Metrics

Caution: I am not a Social Media “Expert,” “guru,” “ninja” or “shaman.”

When we counsel our clients about what metrics are important for measuring a successful communications campaign, we often lob oral grenades such as engagement, influence and interactions. These kinds of metrics make you feel great.

But they don’t really exist. They have the possibility to exist and some folks have come close to making them a reality, but for the most part, these metrics are figments of our social media dashboard’s imagination. They’re Unicorn Metrics.

Is this real life?

When a consultant tells a client they have “raised the level of engagement,” what are they really saying? When a company that purports to be able to assign a score that assesses an individual’s influence claims it uses a “scientific approach,” what experiments has it done?

To me, these aren’t things that can be shown in words. I’m looking for equations that demonstrate metrics such as reach or engagement. Equations such as the one that philly.com uses to measure engagement. Now, I understand that there is a significant market for these types of services that can accurately measure these metrics (full disclosure, my employer is one of them). And exposing a proprietary algorithm is not a reasonable request.

But applying the same standards the scientific community follows would be a good start.

The Scientific Method

The Scientific Method. Image by Erik Ong, on the Wikimedia Commons.

Remember the scientific method from junior high school? Hypothesis, experiment, record, report. I think that the public relations/marketing/communications industry would be well served by more science and less pontificating by gurus. I’m not advocating eliminating intellectual property.

But we would all benefit from openly publishing of scientific studies that expose the methodologies, results and analysis of the science behind the new metrics that are attempting to define our industry.

Exposing the science of influence would put an end to the debates of which service is better or which metrics are accurate. However you define influence (I like this one from Radian 6), having scientific proof would put an end to these unicorn metrics.

Anybody up for starting the Social Media Analytics Review Monthly? (Yes, that spells SMARM) See you in the comments!

[reus id=”6″][recent posts]

Share on Tumblr

  • I may have just shed a tear of joy.

  • I love it. I think it’s a word issue. People want to label stuff “engagement” and so they just call everything engagement, even though they have no actual KPI to go off of. And even if we decided on a KPI, do we know that KPI measures what we want it to measure? Probably not. Thus the importance of experiments. And if you want help writing SMARM, I’m game.

  • Everyday the “market” is producing tons of bits of papers related to social media strategy: Influence, monitoring, engagement, ROI & other metrics, etc…
    It reminds me, as an old guy from consumer relations industry, the discussion about CRM on early 2000. One of facts that links the themes in my mind it’s about that on both cases all industry is put all industry on the same package, since, for example, the relevance and power of influence for BMW and for a pair of socks are the same, what definitely is not the true. The context of social media needs to be evaluated on the context of each industry segment and not as one fits all.
    CRM was not to all and also all social media approach isn’t.

    • My challenge (I guess that’s the right word) is that these case studies and papers rarely have actual numbers. Perhaps I don’t read the right ones, but I rarely see “we can say with a .005 confidence interval that this person is influential. Their tweets caused an increase in traffic by x%…”

      I’d love for more objective studies behind these metrics. I think that part of it is the industry is still young. And these learnings take time.

  • Rachel Lewis

    Thank you for this post! I’ve attended numerous sessions and webinars promising to help me calculate ROI in social media and I end up with nada.

    I’m in for SMARM (a word I love, by the way)

  • ckvincent

    The scientific approach is proven and I love that you’re suggesting this as part of a measurement tool. It seems to be going back to a trend that I’ve seen a lot in PR – with all of the new social media tools, the way to fight through is really to firmly understand the basics of relationships, listening and observing, which will never change no matter how many tools are out there. And measurement, with a million tools or people claiming to have the answer, we should go back to proven methods.

    Please someone start SMARM and add me to subscribe list 🙂

  • Eric, I think you’re right on the money with this post. If you’re right, the company with the Influence metric that adopts a more open source, transparent scientific method approach will emerge as the winner. No one’s gone this direction obviously as protecting the business model is problematic.

    • I am all for protecting IP. But even drug manufacturers publish studies that support the effectiveness of their products.

  • David Svet

    Great post, Eric! This is the kind of problem that should be treated as raw meat by academic researchers. They need to be funded to conduct open research and publish the results. Then the results can be debated and tested in real world circumstances. Eventually we’ll end up with standards and accurate metrics.

    • Part of me knows this will never happen. It’s hard to quantify free will… But I can at least hope, right?

  • An interesting idea that I would definitely like to take part in.

    Sheldon, community manager for Sysomos (http://sysomos.com)

  • The post is absolutely fantastic! Lots of great information and inspiration, both of which we all need!b Keep ’em coming… you all do such a great job at such Concepts… can’t tell you how much I, for one appreciate all you do!
    Business Loan