Programmatic – don’t believe the hype

Way back last century, management consultants started selling CRM (customer relationship management) systems.  Big expensive ‘transformational’ changes in IT systems and staff training.  The worthy aim was to lower costs and improve customer service.  One of the main ways this would be done was to fix the ridiculous situation where the same customer was listed multiple times in different databases.  You see these old databases couldn’t ‘talk’ to one another.  This annoyed customers who, quite naturally, assumed that the service person they were talking to would know that they had had contact with another service person (or another branch) the week before, and so on. It also led to wasteful activity such as trying to cross sell insurance to customers who already had that insurance product with the company.

So the premise of much CRM work was quite sound and practical.  But the CRM systems were expensive, so these sorts of commonsense improvements weren’t enough to justify the price ticket.  The consultants needed something more, something sexier, something more “strategic” that would win over top management.  Loyalty was the answer.  CRM systems were sold and justified on the basis that they would deliver astonishing gains in customer loyalty.  Customer defection would become a thing of the past, and cross selling would generate huge gains in revenue.  This was all helped along by  absurd claims, backed by shoddy evidence, that small gains in loyalty would generate huge profits.

With the benefit of hindsight, plus scientific evidence, we now know that this was wishful thinking (and snake-oil selling).  The returns from CRM investment turned out to be less than stellar, there was plenty of over-investment.

This story has a modern parallel.  Programmatic buying of media is, at its heart, a bit of streamlining automation applied to the messy, archaic business of trading advertising space.  Computers can talk to one another, which can handily replace the system where two people (or more) sit at their respective desks staring at their own spreadsheets and talking (arguing/shouting) with one another as buyers and sellers of media spots.  The potential cost savings are obvious.  With the added benefit that it might also give advertising planners greater ability to focus on the strategy (the algorithm they instruct the computer with) rather than fretting over day-to-day spot availability and vendors’ idiosyncratic buying conditions and constraints.

Computers (talking to computers) can also handle detail so much better and faster.  Like serving thousands of different ads for the different products currently in the warehouse, or altering the product’s price listed in online ads depending on how many rooms are left in the hotel.

Programmatic is particularly useful for those buying from the pool (cesspool?) of cheap digital spots away from few main properties of Google and Facebook.  This is a truly vast universe of advertising spots, each spot seen by very few human beings (many robots?), but collectively adding up.  It seems obvious, essential even, for the trading of these trillions of spots to be done by computers, human involvement is simply too expensive.

Unfortunately, again, this just isn’t sexy enough for the sales consultants.  So again we have overblown promises based on marketing theory and fashion not facts.  Programmatic will apparently allow deeper relationships with customers.  It will deliver hyper targeting – zero wastage.  Moreover ads will reach viewers just at the moment they are most susceptible to persuasion.  ROI will be fantastic.  On goes the sales spiel.

Stay skeptical.

What is a Chief Growth Officer?

Marketing Week reports that a number of companies  have appointed Chief Growth Officers, e.g. Coty, Colgate-Palmolive, and Coca-Cola.  So what is a Chief Growth Officer?  Well, there are (at least) 3 options.

  1. It can be a new title for Chief Marketing Officer (CMO).  Maybe it’s a better title, maybe not – I suspect it all depends on the person.  Here, the role is to enhance the company’s marketing capability, to make the marketing department better, wiser, less wasteful, more effective.  And to make the marketing function be seen as capable of contributing to growth and being accountable for growth.  This is a tremendously important role, a never-ending one, where success depends substantially on bringing scientific evidence into the minds (and hearts) of the marketing team.  I wrote about this role previously.
  2. It can be the same role as CMO, but with the additional responsibility of the sales team.  Is this a better model?  I don’t know,  I suspect it depends a lot on the implementation.  The idea of marketing and sales reporting to one boss looks attractive, it might help them work together for the good of the brand(s).  Then again, this may be simply too big a job for one person.
  3. The Chief Growth Officer could be a role distinct from CMO.  Marketing capability is at the heart of the competitive performance of many corporations, as they work in increasingly competitive markets.  Mental and physical availability underpin the value of such companies, so the CMO’s role is vital….But, even with excellent marketing, company growth will be stymied if the company isn’t playing in the growing categories, in the growing markets/countries, in the growing distribution channels.  The job of the Chief Growth Officer can be to make the company better at making these investment decisions.  In this case, the CGO and CMO work side-by-side; the CMO builds a better marketing capability, while the CGO works to make the organisation better at deciding where to apply this capability (and resources).

All of the companies listed above are sponsors of the Ehrenberg-Bass Institute, a tribute to how they take marketing and business growth seriously.

Some inconvenient truths about brand image perceptions

A cautionary note….

Marketers spend quite a lot of money tracking perceptions of the brand.  There is some use in gathering this information at least once in a while, because if you know how consumers see your brand you can use this knowledge to craft your advertising (and other things like packaging) to look like you, so it will work more for you and is less likely to mistakenly work for competitors.  But this is not how image tracking is usually used.  Instead marketers look at small changes in particular brand associations, e.g. we are up a bit on “community minded” but down a bit on “a brand I can trust” and try to infer some significance.  What do such shifts mean?

Decades of research has documented how attitudinal perceptions (evaluative ie good or bad) strongly reflect the past buying of the respondents in the survey – so simply our market share (if our survey sample is a good one).  Of course attitudes also affect buying but the effect is turns out to be weaker than we used to think it was, while the effect of buying on brand attitudes is very strong.  So our brand trackers show attitudes improve, but mostly after we gain market share.

Some descriptive perceptions are reasonably straightforward to understand.  If only a third of the population know that we sell men’s as well as women’s shoes then this is going to restrict our men’s shoe sales.

Yet even with these less attitudinal, more descriptive associations, it’s not as clear as we might think, e.g.  supermarket chain might worry about their association with “low price”, because they make assumptions that being perceived as having “low prices” drives sales – but how much? It’s not an unreasonable assumption that perceptions of “low prices” probably affects shoppers’ overall attitudes (i.e. a multi-attribute attitudinal model where improvement on this feature nudges the overall attitude (how much?)).  Alternatively, it affects them in a probabilistic manner, when they happen to think of low prices, or desire low prices, the particular supermarket chain now has more chance of popping into memory as a suitable choice.  But… how often and how much this this affects behaviour isn’t known (isn’t documented over different conditions).

The truth is that we have practically no knowledge of how/where/when much particular perceptions affect behaviour – what is a tiny change worth?  Anyone who claims to know is either lying (trying to fool you), or fooling themselves.

Spider graphs, perceptual maps – none of them tell us how much any perception is worth.

Some analysts use regression type analyses to determine which perceptions are “drivers” of other perceptions, or of sales movements. Sadly this is more pseudo-science than science – fitting models of weak correlations to a single set of time series data, something well known to produce useless predictions (see Armstrong 2011, Dawes et al 2018).  Sales (i.e. behaviour) strongly affect perceptions, so correlations between the two are largely, if not totally, due to behaviour causing the perception.  This powerful causal relationship  makes quantifying how particular perceptions drive other perceptions or sales impossible.  All you get is a bunch of over-fitted models describing spurious relationships. It’s impossible to tell which model might be useful, not without doing many differentiated ‘replications’, the basic work of science (statistical gymnastics is no shortcut).

But we also don’t know how much these shifts in market research response are merely that – shifts in a particular (non sales) behaviour i.e. response to survey questions.  For example, for years Mars used the slogan in Australia for their market leading Mars bar “a Mars a day helps you work, rest, and play”.  So any survey that asks “which chocolate bar helps you work” will record many responses for Mars bar.  And the more recently that Mars have advertised using this slogan, the higher the response will be.  The market really does react to advertising, especially if it is done well – clearly branded, placed in broad reach media.  So perceptual shifts may be useful in evaluating advertising (see footnote).  But how can we interpret a 3% shift in respondents picking Mars bar for “helps you work”?  How much of this is them just parroting back the advertising versus actually believing that Mars bars help you work?  And even if they did believe how will this affect their behaviour?  We simply don’t know.

While we do know that people can learn things and yet never bring these beliefs into play in purchasing situations.

Another related problem is that people learn things about brands largely for identification, not for helping them evaluate, or even recall.  For example, lots of us know that Amazon’s book reader is called Kindle.  That we do is good for Amazon,  but who has thought about the meaning of the word, actually it was chosen because Amazon liked the “start a fire” connotation, that’s why Kindle Fire has the name it has – I suspect you never even noticed the connection.  In the same way that no one wonders why McDonalds has a Scottish name.

My point is that movements in market research surveys are precisely that, and we don’t know what they really tell us about how memories in brand buying situations have changed, let alone how this would affect behaviour/sales.

We have to be humble and realistic about our collective lack of knowledge.

All we have is the qualitative notion along the lines that it’s probably better for a supermarket to improve things like the proportion of people who associate it with “low prices”.  So we watch such metrics to check if they start dramatically trending downwards.  Though the reality is that this virtually never happens unless our sales collapse (when all perceptions track downwards), or that our prices really fall behind (in both cases it’s unlikely we need market research to alert us!).

Footnote: the Ehrenberg-Bass Institute has done research on how advertising affects image surveys.  We show that it does. And that without adjusting scores for changes in behaviour (because of sampling variation and real things going on in the market) the effect of particular messages can be missed or misinterpreted.

Anyone interested in this cautionary note on the interpretation of brand image associations and attitudes can read more in the chapter on “Meaningful Marketing Metrics” in the textbook “Marketing: theory, evidence, practice” 2nd edition, Oxford University Press 2013.

These patterns in image data have been document over decades, and many many brands, categories, countries eg:
Barwise, T. P. & Ehrenberg, A. 1985. ‘Consumer Beliefs and Brand Usage.’ Journal of the Market Research Society, 27:2, 81-93.

Bird, M., Channon, C. & Ehrenberg, A. 1970. ‘Brand image and brand usage.’ Journal of Marketing Research, 7:3, 307-14.

Romaniuk, J. & Gaillard, E. 2007. ‘The relationship between unique brand associations, brand usage and brand performance: Analysis across eight categories.’ Journal of Marketing Management, 23:3, 267-84.

Romaniuk, J., Bogomolova, S. & Dall’Olmo Riley, F. 2012. ‘Brand image and brand usage: Is a forty-year-old empirical generalization still useful?’ Journal of Advertising Research, 52:2, 243-51.

Mistaking statistical modelling for science

Marketing isn’t the only discipline to have been seduced by the idea that modelling can somehow bypass the hard work of developing empirical laws.  Few seem to realise how heroic the assumption is that teasing out a few weak correlations can quantify precisely how much [something of interest eg sales] will change in the future when [various other things, eg media choices] are altered.

Added to this is the ‘Big Data fallacy’ that adding together bunches of weak correlations will lead to more and more accurate predictions – “once we have enough data, a clever programmer, and a powerful enough computer, we’ll be able to predict everything we want”.  It’s as if chaos theory taught us nothing at all.

The basic work of science is making empirical observations, looking for patterns, and then…. once you have found one, looking to see where it holds and where it doesn’t.  This requires lots of replications/extensions over different conditions (eg countries, product categories, and so on).  This is how scientific laws are developed, that give us the ability to make predictions.  These replications/extensions also tell us what conditions don’t affect the law, and maybe some that do.  This leads to deep understanding of how the world works.  Experiments can be used to tease out the causal directions and magnitude, what really affects the pattern and how much.  Again these experiments need to be done carefully, across a range of conditions that might matter.

Yes, this doesn’t sound very glamorous, it takes much time and effort (1% inspiration, 99% perspiration).  Sometimes we get lucky, but generally many many studies are required.  By independent teams, using creatively different approaches – so we can be sure that the empirical phenomenon really does generalise, that it isn’t a fragile result (or a mistake) that only exists in one team’s laboratory.

Unsurprisingly the idea that a computer model could bypass much of this hard work is seductively attractive.

Terribly complicated, yet naive, modelling seems to be everywhere.  In population health statistical correlations deliver precise estimates that if people eat particular foods (or amounts of fat/sugar/alcohol, or sitting around) then their risk of dying early will be such and such.  There is nothing wrong with this, so long as we recognise the weakness of the method.  Unfortunately these correlations often get handed over to engineers who, with a spreadsheet and a few heroic assumptions about causality, produce model predictions that if the government taxed this, or regulated that, then x million lives would be saved, and x $billion saved in hospital bills.  These predictions need to be treated with a high degree of skepticism.  We need tests before legislation is changed and money spent.

In climate science, a rather new, and until recently very small discipline, modellers now seem to dominate.  In the 1970s a short period of cooling led to worry about global cooling, but then temperatures turned around to rising again, and climate scientists started to become seriously concerned about the role of rising CO2 levels.  They rushed to develop models and in the early 1990s they gave their predictions for CO2 emissions to lift global temperature, along with accompanying predictions of oceans rising, ice retreating, polar bears disappearing and so on.  25 years later they are confronted by the overwhelming predictive failures of these models, that is, the models substantially over-predicted the warming that was supposed to occur (given that the CO2 levels have risen – the IPCC, even though they are ‘marking their own homework’, admit this in their assessment).  The modellers are now starting the work to figure out why.  Meanwhile the forecasting scientists who criticised the climate scientists’ forecasting methods, and predicted this result, have been vindicated.

Models that show wonderful fit to historic data routinely fail in their predictions*.  That’s why we revere scientific laws (and the theories built on them) because they have made predictions that have come to pass, over and over.


* *See also Dawes, J. G. 2004. ‘Price changes and defection levels in a subscription-type market: can an estimation model really predict defection levels?‘ The Journal of Services Marketing, 18:1, 35-44.