Loyalty/Engagement scores don’t predict the future

I took the full list of Brand Keys 2011 Engagement Award ranks and correlated them again sales gains. What a surprise! A correlation of 0.4 but it’s round the other way…..

i.e. the better the Band Keys rank the less the sales gain !

Brand Keys, who sell these surveys, claim that these award scores will predict a brand’s future. Seems like you’ve absolutely got to know which numbers to use and which to ignore, which can only be done after the event. That’s not prediction, it’s weaving a story afterwards (to sell a product).

Pearson’s correlation 0.39 (or 0.11 for just the 5 top ranked brands)

Data sources:
Brand Keys Loyalty Awards 2011 – http://brandkeys.com/wp-content/uploads/2012/01/Brand-Keys-Customer-Loyalty-Engagement-Index-2011.pdf
Business Week Car Sales – http://online.wsj.com/mdc/public/page/2_3022-autosales.html

Professor Byron Sharp

PS it isn’t quite technically correct to use Pearson’s correlation for a continuous variable against a rank order variable, but in this case converting both to ranks and using a Spearman’s correlation produces essentially the same result.

2 thoughts on “Loyalty/Engagement scores don’t predict the future

  1. Apparently it’s that time of year again. Dr. Byron Sharp’s exhibition of “Prediction Envy.”

    Every year Dr. Sharp raises this issue, misstates what Brand Keys does, and exposes his lack of understanding and/or unwillingness to actually understand what Brand Keys engagement and loyalty metrics are actually about. So let’s try to make it simpler: engagement and loyalty metrics are NOT market models (although they can be used as inputs for market models). They are predictive of how consumers will behave positively toward a brand. The Brand Keys diagnostics (far more detailed and granular than rankings) help to shape brand strategy. In the real marketplace consumer tendencies and attitudes toward brands are not everything when it comes to sales and that’s why even though we do see these metrics explaining why brands get treated the way they do by consumers, even we don’t see 1.00 correlations. Dr. Sharp always drags out sales figures, and always in the automotive category, and conveniently forgets that there are other things that influence sales.

    For those of you who are interested in seeing how these metrics play out in the real marketplace, we invite you listen to our own series where we put loyalty and engagement metrics to the test to see “What Happened?”, an examination of 10 categories we also measure where we look back at what the loyalty and engagement metrics said would happen, and what did happen. You can find it at http://brandkeys.com/brand-stories

    For those of you interested in less biased review (and, as we try to be fair, reply), we invite you to visit the Advertising Research Foundation’s site and check out their “First Opinion Review” of Brand Keys. It’s an independent analysis of what research companies do and, in this case, reviews the Brand Keys approach and confirms that what we say we can do, we actually do.

    While you’re there you also might want to see if Dr. Sharp’s own system/approach that puts forward as the paradigm of metrics has been submitted to that same review. Just sayin’.

Please comment on this article if you wish

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s