Net Promoter Score (NPS) Does Not Predict Growth – it’s fake science

“Managers have adopted the Net Promoter Score on the basis that solid science underpins the technique and that it is superior to other metrics.

We find no support that for the claim that Net Promoter is the “single most reliable indicator of a company’s ability to grow.”

The above quote is from a Journal of Marketing article and winner of the Marketing Science Institute /H.Paul Root Award “A Longitudinal Examination of Net Promoter and Firm Revenue Growth”, Journal of Marketing (2007), Vol.71, by Tim Keiningham et al.

The Net Promoter Score was developed by Frederick Reichheld a consultant now well known for making headline grabbing conclusions based on sloppy research and thinking.

I previously pointed out what was wrong with his claim that small reductions in customer defection cause massive profit increases.

Then in 2004 he is said to have had an epiphany describing his prevous work on loyalty as “powerful but useless”. Keeping customers didn’t matter so much, having customers who would recommend you was everything.

So the latest myth he peddles is that asking customers their likelihood of recommending the company predicts company growth. He claims it does so much better than traditional metrics such as customer satisfaction.

Actually, if you read Reichheld’s Harvard Business Review article carefully you can see he employs the same sort of sleight of hand he did in his customer defection work. Pay careful attention to the dates, Reichheld in 2003 writes that starting in the first quarter of 2001 consultancy firm Satmetrix began collecting customer likelihood-to-recommend responses via email survey. Each quarter collected 10-15,000 responses gradually building a small dataset covering 400 companies in a dozen industries. Reichheld then calculated a Net Promoter score for each company and compared this to the company’s growth rate over 3 years (1999 to 2002). Yes, that’s the previous 3 years.

Yes, so the correlation he reports says that firms that score higher now have previously been growing.

Reicheld admits on his website that the statistical analysis in his book was sloppy but says that since then the consultany company he worked for (Bain & co) has done more extensive analysis showing no correlations between satisfaction scores and company growth, but excellent correlation for the NPS. However, Keiningham et al’s Journal of Marketing article perfectly repeated Reicheld’s analysis and they found the same or better correlation between old-fashion satisfaction and growth (hence the quote above).

Such correlations say little about causality (especially when they are backwards in time), as Reichheld tries to use in his defence, but then why on earth did he select these cases to ‘prove’ his case ? He even admitted he’d selected amongst the very best examples!

In sum, this is snake oil, fake science. It’s scary how many CEOs fell for this.  But then lots of people fell for the (completely wrong) zero defection idea too.

www.MarketingScience.info

Advertisements

5 thoughts on “Net Promoter Score (NPS) Does Not Predict Growth – it’s fake science

  1. The case for sounds rather like the case for global warming. Perhaps Reichheld should get together with Michael Mann, a joint paper on “Successfully Grabbing Headlines Through Sloppy Statistical Analysis” would find many ready publishers.

  2. Pingback: Youth and Loyalty 16 March 09 | mobileYouth - youth marketing mobile culture research

  3. Hi, I would like to thank you for your article. Indeed it is scary. I would like an article on the culture of leadership that leads to this mad behaviour. Anyway, leaving that aside one could craft an argument that says, well Keiningham didn’t find a relationship because no-one had focused on driving recommendation so it was not a lever to growth in the past. Of course Reichheld’s work is nonsense, but my point is, is there evidence to demonstrate a positive (not just correlation) relationship between companies who ‘have used’ NPS and consequent rates of growth? It could be that Reichheld started something serendipitously.

Please comment on this article if you wish

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s