31st of January 2024
If AI took over making marketing decisions, would anyone notice?
Welcome! At the Ehrenberg-Bass Institute for (*gulp) many decades now, I have the privilege to see a wide variety of great research, some of which is under appreciated because it was ahead of its time, and/or it missed getting the publicity it deserved. My aim for this column is spotlight some cool past research and show how this knowledge can help us with today’s problems.
Hype/angst about AI taking over jobs is rampant, and marketing is no exception. Tasks such as creating digital content, direct marketing emails, or desk market research, are now routinely done by AI products. With tasks that are repetitive or involve synthesizing vast amounts of information, human brains are competing with (and often losing against) the ChatGPTs, Bards, Geminis and Claudes of the world.
While this causes angst amongst the Assistant Social Media Managers of the world, experienced marketers are feeling more secure, trusting their experience makes them less easily replaceable by an AI tool. But how safe should experienced marketers feel?
No better than a coin toss?
When thinking about this question I remembered research by Dr Nicole Hartnett, a Senior Marketing Scientist at the Ehrenberg-Bass Institute. She tested whether marketers could pick the ‘winner’ when presented with two advertisements, where one generated better sales. She recently shared with me that this research was borne from the concern that identifying effective advertising creativity was treated as more art than science, so needed to rely on marketer judgement/intuition. A choice that had a clear winner and loser in sales, rather than just award winners, was a crucial part of this study, as Dr Hartnett said in a conversation we had about this research ‘you talk to any brand manager, the commercial objective is king’.
Each participant saw a randomised set of five pairs of ads, one generated sales, one did not, and they just had to pick which one of the two ads generated sales for the brand. It had an impressive sample size, with over 600 marketers taking part, leading to 1,909 pair assessments.
And the result? Let’s first set the context. When one of two options is correct, multiple coin tosses or random guesses will achieve 50% accuracy. Marketers, with all their wealth of experience, came in at a whopping 52%. The best performers were actually from the insights department, with 61% accuracy.
Forget AI, at this stage perhaps we could replace marketers with a coin toss when selecting sales selecting ads?
A finding that surprised me was that Dr Hartnett found length time employed as a marketer did not affect accuracy. This result calls into question the value of experience as an indicator of expertise. Often these two words are mistakenly treated as synonyms. However experience is what happens to you, expertise is what you (hopefully) learn from the experience.
In marketing we experience two types of decisions:
- Decisions that happen so often or quickly we are on autopilot and looking to get things done as efficiently as possible. We usually don’t take time to reflect upon these decisions and learn from the outcomes.
- Decisions that happen so infrequently and are so situation specific that even if you learn from the experience, instances do not accumulate quickly enough to see the elements that contributed to success or opportunities for improvement.
This makes it hard to ‘learn on the job’ as a marketer, and easier for AI to make better decisions as it can absorb wide swaths of information. However it doesn’t have to be that way — we can take steps to build our own expertise.
Read the full article in MediaCat.
