A short critique of “a critique of Double Jeopardy”

Bongers, M. & Hofmeyr, J. 2010. ‘Why modeling averages is not good enough – a critique of Double Jeopardy.’ Journal of Advertising research, 50:3, 323-33.

A longer explanation of the mistakes made in the above article can be read in:
Sharp, B., Wright, M., Dawes, J., Driesener, C., Meyer-Waarden, L., Stocchi, L. & Stern, P. 2012. ‘It’s a Dirichlet World: Modeling individuals’ Loyalties reveals How Brands Compete, Grow, and Decline.’ Journal of Advertising research, 52:2, 203-13.

Here is a short critique:

The title is misleading, this is not a critique of the empirical phenomenon ‘double jeopardy’ but of the theoretical model ‘the Dirichlet’ – a stochastic model of purchase incidence and brand choice, which predicts double jeopardy and several other empirical laws in choice behaviour (see Ehrenberg et al 1994). The critique is naive, and the “test” of the Dirichlet is wrong.

The Dirichlet is used daily within the marketing science units of corporations to benchmark their brand metrics against the model’s predictions for patterns of buying in a stationary and non-partitioned market. It is useful because the market conditions it models are well understood. It is also interesting to marketing theory because these conditions have been shown to be so prevalent.

Bongers & Hofmeyr use the purchasing of non-stationary brands to test an assumption of a stationary model. Even if they had found something real all they’d be saying was that non-stationary behaviour doesn’t look stationary.  Big deal.

It is, however, good and appropriate to question the underlying assumptions of models – even ones that work very well. In this case there has been more than 30 years of serious investigation of the NBD and Dirichet’s underlying assumptions (e.g. Kahn and Morrison 1989), a literature that the authors of this paper should have read. There is also new work seeking to expand such models to non-stationary conditions and to add co-variates (causal variables).

The Dirichlet belongs is a stochastic models, which is a technical way of saying that it assumes a particular distribution of purchase probabilities (concerning the probability to buy from the category, and the probability to buy particular brands within the category). These purchase probabilities (loyalties) for each buyer in the population are fixed in the model, that’s why we say it’s stationary – and the brands obviously don’t change share – if people aren’t changing their loyalties then brand shares stay stable (which in reality they often do, at least over normal planning periods).  These stationary benchmarks are useful to compare change, when it happens, against.

Bongers & Hofmeyr tackle the Dirichlet model’s assumption that consumers have steady-purchase probabilities (steady loyalties), the paper attempts to refute this by showing a selection of purchases of individual panellists of non-stationary brands. They see what looks like lots of variation, natural wobble in purchase runs, and mistakenly interpret this as changes in loyalties.  The Dirichlet correctly incorporates a degree of random variation in purchasing even for stable loyalties.  Now if I gambled regularly I’d have a steady on-going propensity to lose money at the casino – but some nights I would actually make money but that doesn’t mean the casino investors assumptions are wrong.  All B&H show is that runs of purchases exhibit variation, rather than nice neat identical purchase weights in each quarter period. Similarly, if we had a panel of coin tossers we would see that only a few panel members made nice neat runs of tosses HTHTHTHT. If we looked at small runs of tosses we would see very many where ‘Heads” was far from 50% of the tosses. However, it would be foolish to send off a paper to a statistical journal critiquing the long-standing assumption of coins being weighted 50:50.

Now the Dirichlet models something much more complex than coin tossing. We have the probability to buy from the category mixed with the probability to buy particular brands.

The Dirichlet assumes steady-state probabilities with substantial stochastic wobble (around each individual’s steady-state mean). The multinomial assumption of choices amongst available brands means that a buyer with a 10% probability of buying the brand will buy it in the long run on 10% of their category purchase occasions but this buying will be in an as-if random fashion independently of the brand they bought on last occasion. Put simply when you look at any individual’s brand purchasing for short periods you see a lot of stochastic variation (which the model accounts for – hence its very accurate predictions).

It’s quite reasonable that even though I didn’t buy chocolate last quarter that I still have a on-going probability of buying it 4 times a year on average – that doesn’t mean I buy it exactly 4 times every year, some years I only once or twice, some years 8 times (and how I distribute my purchases amongst brands adds further (predictable stochastic) variation).

The authors don’t understand this stochastic variation and it’s tripped them up. If it is any consolation it’s not uncommon for analysts to misunderstand stochastic variation in purchase data. Common mistakes include taking a group of heavy buyers (e.g. the top 20%) then noticing that in a subsequent period their purchasing is lighter and assuming that real changes in propensity have occurred (rather than regression to the mean). Or simply noticing that a person who bought in one period did not in a subsequent period and inferring that they have defected from the brand. I recommend reading Schmittlein, Cooper and Morrison (1993) for their discussion on true underlying propensities.

References:

Ehrenberg, Andrew S C, Mark D Uncles, and Gerald G Goodhardt (2004), “Understanding brand performance measures: using Dirichlet benchmarks,” Journal of Business Research, 57 (12), 1307-25.

Kahn, Barbara E. and Donald G. Morrison (1989), “A Note on ‘Random’ Purchasing: Additional Insights from Dunn, Reader and Wrigley,” Applied Statistician, 38 (1), 111-14.

Schmittlein, David C., Lee G. Cooper, and Donald G. Morrison (1993), “Truth in Concentration in the Land of (80/20) Laws,” Marketing Science, 12 (2), 167-83.

<a href=”https://twitter.com/ProfByron&#8221; class=”twitter-follow-button” data-show-count=”false”>Follow @ProfByron</a>
<script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0],p=/^http:/.test(d.location)?’http':’https';if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src=p+'://platform.twitter.com/widgets.js';fjs.parentNode.insertBefore(js,fjs);}}(document, ‘script’, ‘twitter-wjs’);</script>

Brand Equity twaddle

I occasionally send some friends interesting (both good and bad) articles from marketing academia.  This is an interesting reply.  I won’t name the academic paper.

Dear Byron,

Thank you for sending this paper. I think the correct response, using the scientific vernacular, is ‘utter twaddle’.

The framework below is very neat. It’s very sequential. But it’s also very wrong.

When marketing academics observe what really happens in the real world, they can make powerful discoveries that help further the discourse around how people behave and make choices. But when marketing academics start with a hunch (disguised as a testable hypothesis) and then find data to back it up, they are, at best, worthless, and at worst, damaging.

I wouldn’t waste my time critiquing each component in this model. What I will do is give you an example of a very real ‘real world’ observation about how people behave, despite what one might think they have in their heads regarding Brand Associations and so called Brand Equity.

I am lucky to live in a very nice suburb of southwest London called St.Margarets. It’s what one might call leafy and affluent. Its residents are, on the whole, fortunate to be significantly better off than the average UK population in socio-economic terms. Lots of doctors and lawyers and bankers and media types.

 The overwhelming majority of my St. Margaretian friends and acquaintances are well-educated and, again on the whole, politically liberal. Generally left of centre, having evolved from the armchair socialism of their more zealous, youthful days. I should put an important caveat in place here; I was never an armchair socialist, nor indeed a socialist of any kind really. Anyway, I digress.

There is a nice sense of community in St. Margarets and I have made many good friends here over the years. And in addition to these friends, there are plenty of others with whom I can enjoyably engage in pleasant and cordial passing conversations. As you can imagine, it’s fertile ground for many dinner parties and for gatherings in local hostelries.

Once the wine has started flowing, and the initial greetings and polite exchanges (such as how the kids are getting on) have been completed, conversations inevitably move on to the more ‘serious’ topics du jour. House prices, gossip about who Sally was seen with last week, standards of schooling, what Charles said to his accountant, how moral standards are becoming polarised between the haves and the have nots, what Carol was caught doing with Bob down by the river. You know the sort of thing, I’m sure.

Commerce will often have it’s place in this cauldron of righteousness as well. I distinctly recall more than a few conversations about business ethics. And a number of these have centred around a well-known retailer which has had the temerity to open one of its smaller store formats slap bang in the middle of St. Margarets. Right next door to the railway station. Outrage abounds.

“Tesco Express … they’re crucifying all our little local traders,” opines Gareth. “They bully farmers into bulk deals with derisory margins … Tesco is ruining our agriculture,” shrieks Camilla. “The way they treat their shop workers … it’s slave labour … they should be taken to the International Court of Human Rights,” booms Barry.

I listen with interest. Sometimes, I must admit, the odd fair point can be heard from time to time amongst the remonstrations and general distaste for having such a purportedly disreputable behemoth impose itself on our little suburban ‘village’ (as the Estate Agents like to describe it). But the over-riding theme is one of deep-seated antipathy. A theme with which, I must say for the record, I disagree. I think Tesco is a great business and great for our economy.

The dinner parties end, the hostelries close, and we all go home to our beds. Watered, fed and safe in the knowledge that the world would be a better place … if only ‘they’ just listened to our wisdom.

When I travel home from work during the week, I frequently do so by train. Most of my friends and acquaintances do the same. We come in to St. Margarets station, wearied by the day’s travails, ready to put our feet up and watch the telly. We trudge up the station stairs to the street. As I start to walk down the street  I remember that Cathy called me to remind me to pick up a pint of milk and some chicken breasts for dinner. Ooh, and I can pick up a half decent bottle of wine too … why not!

I turn in to a shop which is already teeming with St. Margaretian commuters.

Before I can even reach down to pick up the chicken breasts I’m tapped on the shoulder. I turn around to see a smiling friend; it’s Camilla. “How lovely to see you”, she says, “(mwah mwah) feels like I saw you just two days ago at Barry’s for dinner.” We both laugh. “Oh, look, speak of the devil, Barry’s over there with Gareth at the check-out.”

“Anyway, see you soon I hope, Camilla,” I say, “We’re going round to the Greensmith’s next Saturday, probably see you there.”

As I leave Tesco, which is slap bang in the middle of St.Margarets, right next to the railway station (to where thousands of well-heeled St. Margaretians return every evening), I give a little wave to Sally, Charles, Carol and Bob. They have arrived back on the next train. They’re just popping in to Tesco to pick up some things before they go home.

As I open my front door, a question comes to mind; can the need to get a pint of milk, as easily as possible, really trump the most heartfelt attitudes expressed around a dinner table in St. Margarets only a day or so earlier! It would appear so.

‘There’s nowt as queer as folk,’ as the old Yorkshire saying goes.

People may claim to hold firm perspectives about brands. The truth is that there is a world of difference between what someone consciously says and what they actually decide (primarily subconsciously) to do.

So, yes, that paper is truly dreadful.

Cheers,

Seamus

 
On 4 May 2013, at 01:25, Byron Sharp wrote:

A poster-child for everything that is wrong with brand equity research.  If you can’t be bothered reading the article just look at the struggle they had to come up with any findings or implications.

Consideration sets for Banking and Insurance purchases

Dawes J., Mundt, K. & Sharp, Byron. 2009. Considerations sets for financial services brands. Journal of Financial Services Marketing, vol. 14, pp. 190-202.

ABSTRACT This study examines the extent of consumer information search and consideration of financial services brands. It uses data from two surveys of purchasing behavior. This study finds a surprisingly low level of consumer consideration, either by personal enquiry or via the internet. The most common consideration set comprised only one brand, and this was the case for both high-value and low-value services. The managerial implication is that services marketers should make brand salience a top priority, with the competitiveness of their offer not being the primary driver of sales. If a financial services brand is salient to a consumer, there is a very high chance they will purchase that brand, without extensive comparison of the merits of alternatives.

Journal of Financial Services Marketing (2009) 14, 190–202. doi:10.1057/fsm.2009.19 Keywords: consideration sets; evaluation; financial services; loyalty; brand switching

Download PDF.

 

Two type of repeat purchase market, with different loyalty patterns

Sharp, Byron. & Wright, Malcolm (1999) ‘There are Two Types of Repeat Purchase Markets’, paper presented to the 28th European Marketing Academy Conference, Berlin, Germany, 11-14 May.

Abstract

In this paper we report on a pattern in aggregate buying behaviour. We have observed two distinct types of repeat purchase markets with very different patterns of customer loyalty. These differences have profound implications for marketing theory and practice.

The first, and best known, are markets with relatively few solely loyal buyers and with buyers allocating their category requirements across several brands; we call these repertoire markets. Examples of repertoire markets include fast moving consumer goods, store choice, medical prescriptions, and television channel selection.

The second are markets with many solely loyal buyers, and with buyers allocating their category requirements almost entirely to one brand; we call these subscription markets. Examples of subscription markets include insurance policies, long distance phone calls, and banking services.

The distinction between these two types of markets is not a theoretical taxonomy, but is instead a dramatic empirical difference. For example, the proportion of solely loyal buyers enjoyed by a brand over a year seldom exceeds 20% in a repertoire market, but seldom falls below 70% in a subscription market. There is virtually no middle ground between these extremes.

Download full paper as PDF.

 

https://www.dropbox.com/s/7ufcbx1tfpzcfzg/6007.pdf

Brand and Advertising Awareness: A Replication and Extension of a Known Empirical Generalisation

Romaniuk, Jenni, Sharp, Byron, Paech, Sam & Driesener, Carl (2004) “Brand and advertising awareness: A replication and extension of a known empirical generalisation” Australasian Marketing Journal, vol. 12, no. 3, pp. 70-80.

Abstract

From analysis of over 39 categories Laurent, Kapferer and Roussel (1995) found that top of mind, spontaneous and aided brand awareness measures have the same underlying structure. The difference in scores appears due to the difficulty of the measure. We have successfully replicated this work and extended it to similarly structured advertising awareness measures. However, additional analyses then revealed that while there is a good category level fit, modelling a single brand over time is less successful. Indeed, Laurent et al.’s excellent cross-sectional fit appears due to substantially different levels of salience between larger and smaller brands. This suggests that while the different types of awareness tend to vary with a brand’s overall level of salience, this does not mean that the different measures simply reflect a single underlying construct. Further, our finding challenges the previous authors’ claim that knowing the score for one measure allows the estimation of the score for another measure. Instead, the model provides useful norms against which to compare actual scores.

Keywords: Brand awareness, Advertising awareness, Empirical generalisation

Download the whole article as PDF.

Correlations are a poor way of assessing predictive ability

Firstly, I’d like to blow away the myth that correlations above 0.5 are spectacular in the social sciences. On pages 32 and 33 of my book “How Brands Grow” I present some car repeat loyalty metrics and market shares for the USA, UK and France. A quick calculation shows a higher than 0.6 correlation between repeat rates and market share. These sorts of correlations between brand performance metrics are the norm.

Secondly I want to highight the misleading claims of consultants peddling special brand health metrics who often claim correlations of say 0.7 between their special score for brands in a category and their sales – they say how amazing this is, and how it is proof of predictive ability.  Well here are my attempts to predict tomorrow’s temperature in Adelaide Australia, each prediction and reading is taken about a month apart starting in Summer and ending in Winter. I get it right that the temperatures go down as Winter arrives (big deal, it’s a bit like predicting that growing brands will increase sales a bit next period) but otherwise my predictions are miserable, they are always wrong, sometimes too high, mostly too low. The correlation however is very near perfect, 0.99 to be precise.

Actual temperature (Celsius) is in the left column, and my hopeless predictions in the right column:

39   43
32   36
24   22
21   18
17   15

r = 0.99

The moral of the story is that correlation is not a good indicator of predictive ability.

 

Professor Byron Sharp

US brands are not losing their loyal customers – even more misleading metrics

Oops they did it again. Catalina Marketing have announced that packaged goods brands in the US have lost about half of their loyalty customers – AGAIN. Oh no how horrific. It’s a wonder they have any customers left. It’s a wonder that major brands aren’t tumbling out of the market. Sell your shares in Kraft, P&G, Unilever, Coke….

Of course it is complete nonsense. There is nothing wrong with the data, just faulty analysis.

This is the 2nd time Catalina Marketing have made this mistake. They are misinterpreting the natural wobble in people’s purchasing histories as real change in their loyalties.

Marketing scientists have know about this wobble for decades, it can actually be quantified using the NBD-Dirichlet. But Catalina in ignorance instead report that brands have lost nearly half their loyal customers. Oh no the sky is falling!!! Nice headline but it is wrong, plain wrong.

A few years ago when they reported this they said it was an unusual event, due to the GFC. Now having noticed it happens each year they say it is just a terrible indictment on marketing.

All of this is wrong, because this would still happen even in perfectly stationary conditions where no brands are growing or declining, and no consumers are changing their propensity to buy the category nor their loyalties to the particular brands in the category.

Let me explain. Each of us has a tendency to buy from the category, some of us are heavy category buyers and most of us are light. On top of this we each have our own particular loyalties so we buy some brands more often than others. These two mixed distributions mean that there is a lot of diversity between consumers of any product category. Diversity which is modelled extremely well by the Dirichlet.

On top of this we don’t buy like robots. I might have a tendency to buy chocolate bars 5 times a year, and have loyalties so that I buy Snickers 30% of the time, so that’s once or twice a year. But some years I’ll buy chocolate bars more than 5 times a year, and some years less – for thousands of random potential reasons. Plus some years I’ll give Snickers more than 30% of my purchasing and some years less.

So even if I don’t make any changes to my tastes, habits and loyalties I could buy zero Snickers in a year or 5+.

If we classify people into “loyals” or “heavies”, or whatever, based on what they do in one year then a lot of people are going to be misclassified. They aren’t really super-loyals it’s just that in that year they were – perhaps they had a party, some friends visited…a thousand potential reasons. Next year they are likely to revert to closer to their normal purchasing. It looks like they have changed when they haven’t. This is behind the phenomenon statisticians call regression to the mean. Catalina Marketing don’t seem to have learnt their basic statistics.

Catalina categorised someone as loyal if they gave 70% of their purchasing to the brand. If they didn’t in the next year they said they were lost. This means it is largely an analysis of lighter category buyers, as heavier buyers are less to give one brand such weight. So it’s people who bought the brand once out of one category purchase, 2 out of 2, 3 out of 3, or 3 out of 4, or 4 out of 5. So buying the brand just once less, or buying the category just once more means you get classed as lost, defected, no longer loyal. That’s why they get such a high figures as 50% being ‘lost’.

So It’s all an illusion. I explained this back in 2009. Sad that I have to say it again. I’ll repeat what I said then: Catalina Marketing sell targeted marketing services based on using this loyalty program data – which is a bit odd because this fluctuation seriously undermines the capacity to target consumers based on their past buying.

Byron

Professor of Marketing Science

Director, Ehrenberg-Bass Institute

University of South Australia

See the official website for the book “How Brands Grow”

http://marketinglawsofgrowth.com/