The flawed Stengel Study of Business Growth

Here I describe the ‘Stengel Study of Business Growth’ using quotes from “Grow: How Ideals Power Growth and Profit at the World’s Greatest Companies” by Jim Stengel, published by Crown Business 2011. Along the way I point out the fatal flaws in the research design.

The ‘Stengel Study of Business Growth’ started in 2007 when Procter & Gamble’s CEO A.G. Lafley endorsed Jim’s idea to “commission a study to identify and learn from businesses that were growing even faster than we were, in whatever industry” (p. 24).

Initially the P&G team studied “the fastest growing brands over the previous five years” (p.24) identified in collaboration with market research agency Millward Brown Optimor using their BrandZ database. The team “assembled five-year financial trends on twenty-five businesses that had grown faster than P&G over that period. The teams then dug behind the numbers with additional research, including interviewing business executives, agency leaders, brand experts, and academics at Harvard, Duke and Columbia”. (p.25)

“We went in looking for superior financial growth, and only after that for whatever the top-ranked businesses were doing differently from the competition” (p26). Professor Philip Rosenzweig explains this classic sampling mistake as being like trying to learn about blood pressure by only looking at a small group of patients who all have high blood pressure.

Another very important mistake, that we have learnt about as various strategy researchers have made it over the years, is to look for causes of success by interviewing managers and ‘experts’ for their opinions on firms that have been doing well. Known as “the Halo effect” people tend to say that firms they know are performing well possess all sorts of desirable characteristics in terms of culture, leadership, values and more. No one describes a known winner as having “unfocused strategy”, or “weak leadership”, or “lack of customer focus”, or “lack of ideals”, or whatever the researchers choose to decide to ask opinions about. As Philip Rosensweig shows clearly in his book “many things we commonly claim drive business performance are simply attributions based on past performance”.

“Successful companies will almost always be described in terms of clear strategy, good organization strong corporate culture, and customer focus” (p.87). Rosenzweig dramatically shows how when successful companies falter experts abruptly change their assessment. Suddenly the previously described “strong culture” is now described as “rigid”, their previously declared “promising new initiatives” are now described as “straying”, their “careful planning” now in hindsight turns out to be “slow bureaucracy” and so on. In reality large businesses change very slowly, but opinions about them change quickly and are largely based on current financial performance (which is itself is largely due to environmental and competitor effects).

The Halo Effect is particularly strong for subjective, nebulous concepts such as ‘values’ and ‘ideals’. The ‘Stengel Study’ made no attempt to supplement their judgements with ‘hard’ objective measures.

In the Stengel study they ‘discovered’ that their chosen high-growth firms were ‘ideal driven’. The central finding therefore was that “businesses driven by a higher ideal, a higher purpose, outperform their competition by a wide margin”. Yet there is no mention of any systematic investigation of competitors, perhaps many of these lesser performers were also ‘ideal driven’? What we can expect is that because of the Halo Effect less successful performers would be less likely to have been described by interviewees as having a clear ideals well activated throughout the business – irrespective of reality.

Subjective concepts such as ‘ideals’ almost certainly introduce confirmation bias on the part of researchers – when there are no objective measures it’s near impossible for a researcher to stop themselves seeing what they want to see. The “unexpected discovery” of the causal effect of ideals, says Jim Stengel “corroborated what I had implicitly believed and acted upon throughout my career”. Hmm, of course it did.

With this ‘ideals’ hypothesis now firmly in place the full ‘Stengel Study’ was then done after Jim Stengel left P&G by selecting 50 brands based on their excellent recent financial performance over 10 years. As a whole this group (refered to as “The Stengel 50”) “grew three times faster over the 2000s than their competitors…individually some of the fastest-growing of the Stengal 50, such as Apple and Google, grew as much as ten times faster than their competition from 2001 to 2011.”

Promotional material for Stengel’s book says that “over the 2000s an investment in these companies—“The Stengel 50”—would have been 400 percent more profitable than an investment in the S&P 500”. The implication is that this proves Stengel’s ‘ideals’ thesis – but Stengel picked these companies for their financial growth!

If they have been picked purely based on some, ideally ‘hard’ (or intersubjectively certifiable), measure of being ‘ideals driven’ then correlations with financial performance might mean something. Especially if this were future, not past, performance. But as these companies were picked for their financial performance then their stock price performance over the same period shows nothing.

A team of four second-year MBA students being taught by Jim Stengel and Professor Sanjay Sood made the Stengal Study the subject of their required applied management research thesis; “this team crawled all over the Stengal 50 to test the role of ideals” conducting interviews with executives, academics and consultants”. No one should be surprised that they found what their instructors believed. They and the ‘Stengal Study’ both passed with flying colors reports Jim Stengel (page 34).

There is one addition to the Stengel Study which is different from previous similar (flawed) studies of business success. Stengel arranged his leading brands into “five fields of fundamental human values that improve people’s lives” by (1) eliciting joy, (2) enabling connection, (3) inspiring exploration, (4) evoking pride, or (5) impacting society (sic). Millward Brown then used implicit and explicit association measures and found that the Stengel 50 brands are perceived as more associated with their selected ideals than competitors.

Again this is a staggering piece of circular logic. First analyse a select group of brands for what particular ideals they represent, then take these ideals into market research and viola these brands turn out to be more associated with these particular ideals. This is not a test that these ideals drive performance, it is simply a test of the researchers’ judgement of brand image. It merely shows that the researchers live in the same culture as the market research respondents. Jim Stengel thinks Backberry ‘enables connection’ and so does the market, Jim Stengel thinks Mercedes Benz ‘evokes pride’ and so do normal people.

Now one might reasonably argue that there is advantage in FedEx and Blackberry being more associated with a category benefit such as “enables connection” than their competitors. However, leading brands always show higher associations because they have more users, who use them more often. Behaviour has a powerful effect on attitudes and memory, for evidence see BIRD, M. & EHRENBERG, A. 1972 “Consumer Attitudes and Brand Usage – Some Confirmations”. Journal of the Market Research Society, 14, 57. RIQUIER, C. & SHARP, B. 1997 “Image Measurement and the Problem of Usage Bias” in proceedings of 26th European Marketing Academy Conference, Warwick Business School, U.K., 1067-1083. ROMANIUK, J. & SHARP, B. 2000 “Using Known Patterns in Image Data to Determine Brand Positioning”, International Journal of Market Research, 42, 219-230.

There is no mention of controlling for this effect.

Conclusion

In summary, the ‘Stengel Study’ makes the same or similar mistakes as much earlier flawed studies that claimed to uncover the secret of sustained financial success. Jim Stengel, and none of his team appear to have read Philip Rosenzweig’s “The Halo Effect: … and the Eight Other Business Delusions That Deceive Managers” which turns out to be a great pity. As he writes “if the data aren’t of good quality, it doesn’t matter how much we have gathered or how sophisticated our research methods appear to be”. The Stengel Study is yet another study that is deeply flawed, it tries to look like science, but turns out to be merely a story, one that will appeal to many but tells us nothing reliable (or new) about the world.

A final note: Based on the track record of previous such studies I expect the financial performance of these ‘ideals driven’ companies to fall back in the near future. Some such as Blackberry, HP have already suffered very dramatic reversals of fortune.