ArticleMachine Learning + Marketing
Let’s talk about the use of Machine Learning (ML) and Artificial Intelligence (AI) in marketing. We’re approaching 9 months – 3 full quarters – of life in a post-ChatGPT world. And, like most people in the marketing space, I’ve spent way too much time thinking about and playing with various AI/ML tools, trying to figure out where/when/how to integrate them into our workflows – and (perhaps more importantly) where NOT to.
Where ML Shines:
Demand Generation / Prospective Targeting
Candidly, the rapid evolution of ML for automated targeting (Meta, Discovery, YouTube) has been a major surprise. In most of our data sets, using automated targeting for prospecting + demand generation has resulted in CPA reductions / ROAS improvements, ranging from 10% to 35%. That’s not surprising for a few reasons: (1) most platform ML models have access to more data than is available to advertisers via the platform interface or API; (2) the probability functions used by the those models have access to network-wide data, whereas an individual advertiser only has access to the data in his/her account; and (3) most people – including media buyers – are pretty bad at making decisions under uncertainty.
That being said, I do think there are some caveats worth mentioning:
- AI-Driven Targeting Has Raised The Floor – if there’s one thing I agree with both Meta & Google about, it’s that most advertisers meddle too much in their accounts, hindering learning + dragging down performance. Automated targeting has corrected that (to an extent), and thereby “raised the floor” for many ad accounts. I think this has been wrongly interpreted as “AI-driven targeting is better” – when that’s only part of the story. The bigger part is that people aren’t great at lever pulling, and there are still a lot of humans pulling levers. Higher floor = higher returns in the short term. Which brings us to:
- As adoption increases, surplus value decreases – as an ever-increasing share of advertisers jump on the “automated media buying” train, the inevitable outcome is that surplus value (that lower CPA / higher ROAS) will decline. ML is extraordinarily good at uncovering patterns in low-hanging fruit; it’s less good at hunting for fruit everywhere else on the proverbial tree. The result: I think the brands who are able to (a) help machines get smarter via data passback and (b) effectively partner with ML-enabled media buying will succeed; those that won’t (or can’t) will fail.
- Media Buying Isn’t A Commodity – One (unfortunate) outcome of the “AI-driven targeting” hoopla has been the commoditization of media buying. Candidly, I think most advertisers have confused (i) generally favorable economic conditions; (ii) initial surplus value from automated targeting and (iii) the proliferation of the ad buying educational complex™ to mean that anyone can buy ads. While that’s technically true, I do think brands will soon realize that not everyone can buy ads well. In a world driven by automated targeting, ML-savvy media buyers are anything but a commodity.
My philosophy on this is simple: test it. If you can get a short-to-mid-term boost in your ad efficiency by leaning into automated targeting, take it. But don’t fall for the trap that how automated ad targeting performs today will continue forever.
Creative Production
I’ve just started playing with “Video AI” tools (like Photoshop AI), and I’m bullish on their future. While they are far from perfect today, it is not difficult to see how these tools can rapidly displace models + talent, as well as some members of video teams.
I mean, just look at this example:
This is just the tip of the iceberg. AI-generated content will improve exponentially over the next 18 months, to the point where I think it will be able to replace many generic models/UGC creators – all at a fraction of the cost + time commitment.
AI/ML doesn’t just have a future in video creation, either – there are already promising tools for polishing videos (Wondershare’s Filmora), editing scripts (Descript – though I do think there needs to be a better solution for cleaning up the transitions created when you edit out a bunch of content), creating clips (Opus Clip is AMAZING) and even full video generation from text prompts (Visla – though it does need some help + re-prompting more than I’d like).
Bottom Line: AI/ML has earned itself a place at the table among creative teams. But what it will not do – today, tomorrow, next year, or for the foreseeable future – is replace true creative talent.
Content Scaling & Personalization
I’ve come to believe that one of the best possible use cases for AI/ML in marketing is making personalization at scale both possible and accessible for brands. A fantastic first taste of this (at least for me) was Recart’s AI-driven SMS, which I wrote about here. But this is just the beginning – the ability to iterate on brilliant, witty copy at lightning speed is a turning point for personalization.
Data Analysis
This might be the most compelling use case for ML in marketing today. ChatGPT’s Code Interpreter is an absolute wizard at developing excel formulas for just about anything you can imagine, and their code interpreter can double as a data scientist. Feed it sheets (flat files or databases) of your raw data, tell it what you want to know, and watch it go to work. You’ll have to review the outputs, as well as provide additional information on business events, seasonality and/or missing context (remember, all ML models operate on the same core-4 principles) – but what it can do today is remarkable.
I’ve long been a proponent of Automated Marketing Mix Models (aMMM) like Meta’s Robyn, and the combination of those open-sourced models with ChatGPT’s know-how is resulting in a world where marketers can get near-real-time MMMs at a staggeringly low cost. In just a few days of playing with ChatGPT’s Code Interpreter, I was able to get it to perform multivariate regressions, segment customer based on transactional history, identify long-term sales trends, compute the results of with-and-without analysis (holdout testing), perform geographical sales analysis (both historical and forecasted), identify top-performing SKUs (along with low-performers and hidden gems), and determine common purchase patterns/journeys.
In many ways, Code Interpreter is like having your own data science team – it can both suggest avenues for exploration and techniques to perform said exploration (for instance, it suggested a Recency-Frequency-Monetary or RFM approach for determining Customer Value, then K-Means clustering to segment those users into discrete groups), along with writing the code in R or Python to perform the suggested operations, all in minutes (vs. hours or days it would take a data science team).
And once you’re happy with the output, Code Interpreter will go ahead and generate visualizations (and even powerpoint slides, if you need to present it), along with (for instance) CSVs of each segment for upload to your email/SMS/marketing platforms.
A fun second-order effect of this is that many SaaS platforms (especially those without a durable moat) are going to find themselves replaced by the above combination in relatively short order.
Where ML Isn’t Good
Just because ChatGPT/AI/ML have very real (and valuable) use cases does not make it a one-size-fits-all solution to any marketing or data science problem. In what follows, I want to share a few specific examples of where using AI/ML tools has produced terrible (and in some cases, disastrous) results:
True Creativity
The lack of creativity in Large Language Models (LLMs) – the technology powering ChatGPT – isn’t a bug, it’s a feature. LLMs work by resolving complex questions into probability functions, then generating an output based on that probability function. That’s great for things that follow a predictable pattern – like excel formulas and video editing – but terrible for things that turn on the ability to break a pattern (like true creativity).
AI/ML tools can augment your creative process. They are fantastic for accelerating research, better understanding the current landscape and even for informing on what is expected. They’re not good – nor will they ever be – at true, disruptive, scroll-stopping creativity.
Copywriting
This follows directly from (1) – but there’s no bigger favor you can do for your competition than to outsource your copywriting to ChatGPT. At its very best, LLM-generated content is a “C” on the grading scale: it’s average. Passable. Fine but only just. And in today’s information-everywhere environment, that’s nowhere near good enough. Lasting brands are built on a foundation of remarkability, not mediocrity.
If anything, the proliferation of LLM-generated content will only serve to magnify the contrast between it and brilliantly written, sharp, incisive copy. And while LLMs certainly have a role to play in the creative process – from researching content to sourcing to drafting content – that’s all they have: a role. Not the entire thing.
Information Retrieval
I wrote months ago about LLMs being incredibly poor information retrieval systems. That hasn’t changed. Search is perhaps the least interesting use case for LLMs, and I fully expect it to remain that way.
That also means that you shouldn’t be using ChatGPT for SEO research, or for keyword volume analysis, or for writing your content. I’m certainly not the first person to say this, but LLMs are not information retrieval systems. They do not have an inherent sense of accuracy or truthfulness; they work by resolving complex queries into probability problems, then predicting what comes next. Sometimes what follows is true. Most of the time, what follows sounds true, but is not.
Specific Search / Demand Capture
This one will be controversial, but I think it is worth mentioning: most automated systems are incredibly poor at understanding the nuance inherent in complex, high-value queries.
For example: a search campaign I recently reviewed has an exact-match target for “Teeth Whitening” – which Google matched to “Teeth Whitening Kit”. On the surface, that might seem fine. But there’s a massive difference: Teeth Whitening is an expensive procedure (~$500 to $1,500); a Teeth Whitening Kit is a $30 item at CVS. A colleague of mine noticed the same thing in an attorney’s Google Ads campaign: the broad-match keyword “workplace injury lawyer in [city]” triggered an ad when users searched for a specific attorney’s name. The kicker: that attorney’s practice does not handle workplace injury cases.
There are hundreds of examples just like this, of automated targeting missing the mark so egregiously that it boggles the mind. The common thread – and why I don’t think most automated systems are particularly adept at demand capture search – is the nuance. Any marketer worth his/her salt would spot the issue with appending “kits” or appearing for a specific attorney’s name vs. the actual service. But machines – which work by resolving complex questions into probability functions – have no way to do this, so they don’t. And advertisers who aren’t careful pay the price.
While a substantial amount of the AI hype has died down (thankfully!), I think we’re slowly approaching the plateau of productivity. These tools have a lot to offer, and a large role to play in our industry and economy moving forward. The challenge for us – marketers, investors, business owners – is to determine the areas where they can add outsized value and let them get to it. While that’s likely to result in some disruption and role shifts, in the long-term, I think it’s a next step forward.
Cheers,
Sam
