Article

How Should You Evaluate An Ad?

Advertising Ecommerce google ads Meta Testing

April 29, 2024

One of the most common questions any advertiser has (agency-side, brand-side, investor, etc.) has is this: which of our ads are effective?

It’s a problem the advertising industry has been grappling with for a century – and while we’ve made great strides, we’re still not (quite) there yet.

Perhaps the biggest challenge, especially as it pertains to digital advertisements, is how can we disentangle the performance of an ad from the multitude of other factors that contribute to it – the audience, the device, the landing page, the placement, the time of day, etc.?

In most cases, ad performance is evaluated using a few core metrics:

Click-Through Rate (CTR): a measure of how efficiently a given ad translates impressions (views) into clicks (actions) – though it says nothing about the underlying quality of the click.

Return on Ad Spend (ROAS): total revenue attributed (though as we know, attribution is an imperfect science) to a given ad, divided by total spend on that ad during the same time period. This is better than most, since it actually connects outcomes (revenue, leads) to the ad – but as we all know, revenue is vanity.

Conversion Rate (CVR): conversions from the ad divided by total visits to your site from the ad. The challenge with CVR is simple: a non-insignificant portion of the outcome is the result of the landing page itself, not the ad. A brilliant landing page can mask many advertising sins.

Any good measurement framework should do three things: (1) isolate – to the extent possible – the performance of the ad from the other contributing factors; (2) diagnose if there’s an issue and (3) provide the data necessary to explain why something is or is not performing.

My Advertising Assessment Framework:

Hook Attention:

Nothing happens unless the hook happens. The average IG user scrolls the length of a football field each day, flying past hundreds (if not thousands) of pieces of content along the way. If your ad isn’t able to stop the scroll, nothing else matters. You have a better chance at hitting a hole-in-one at Pebble Beach than you do of a person who flies past your ad in their Insta feed remembering your brand.

While it isn’t perfect, the best metric I’ve found (so far) to assess this step of the framework for video-based creative is thumb-stop rate: 3s video views / viewable impressions.

What I love about thumb-stop, calculated this way, is that it assesses only those situations where your ad had a legitimate chance to hook the viewer (i.e. impressions that were 50% on-screen for at least 1s). If the viewer was flying through their feed at warp speed, the overwhelming probability is that they were looking for something specific – and your ad never stood a chance. That’s fine.

Hold Their Interest:

Once you’ve hooked the potential prospect, can your ad pique their interest in your product/service while holding their attention? In general, most ads require at least ~15s of total view time (if not more) to convey the core value propositions of your product/service, along with the pertinent details of the offer. Without those, the user is incapable of making an informed assessment of whether or not to proceed to the next step (visit the shop or landing page).

This matters for a simple reason: if your ad isn’t sufficiently priming + informing the potential prospect, it is significantly more likely to drive lower-value visits to your site, which then translate to lower conversion rates. Basically: we want ads that educate and inform the audience, to the point where they have some commercial intent before visiting the site. If the ad is just getting clicks with no commercial intent, it is likely throwing money away.

One way to calculate the “hold” rate: is [15s*+ Views] / [3s Views]

*The 15s is an average; if you have a high-information-needs product (travel, cars, financial products, etc.), this probably needs to be higher; conversely, if you’re selling a generic service (i.e. lawn care), only 10s might be required to communicate the relevant information.

Sell The Next Step:

The primary role of any ad is to “sell” the next step in the purchase/conversion path. In the case of most digital ads, that next step is the call-to-action button (CTA).

Over the years, Meta + Google have both played games with the meaning of clicks (“all clicks” on Meta refers to clicks anywhere on the ad, including those that only take the user to other parts of Meta, like your profile). If your ad is driving people to view your profile, that’s suboptimal at best.

Enter interested LP View Rate: [Unique Landing Page Views] / [15s* Viewers]

At first glance, this may seem like a weird name for CTR – but there are a few key differences:

This builds on the interest metrics above, which allows for some interesting comparisons (i.e. those who didn’t want 15s* of your ad vs. those who did).
CTR doesn’t differentiate between real visits to your site vs. clicks on a button – there’s a non-insignificant portion of clicks that are attributable to mis-clicks. We want to filter these out when assessing an ad.

Start The Action:

While landing page visits are nice, they only have instrumental value. What good is a visitor who arrives, pokes around, and departs? Nada. We need people who are ready to act, to start the sale:

Ready-To-Act Visitors: [Checkout / Form Starts] / [Landing Page Views]

This metric tells you how efficient your ad is at bringing in people who are ready to act – and is often a useful method to assess overall traffic quality in cases where transaction volume alone doesn’t tell the full story.

Close The Deal:

I can’t deposit add-to-carts or form starts at Chase Bank. Ultimately, every ad needs to be evaluated based on people & profit: did it bring new customers/clients (“people”) at or below an acceptable cost-per-acquisition (“profit”)? If your ad isn’t able to do that, then everything above was all for naught.

To assess this, we want a metric that blends the two:

Contribution Margin Returned From Ad Spend (CMRFAS): [Revenue – COGS – Costs of Delivery – Costs of Fulfillment – Costs of Processing – Ad Spend] / [Ad Spend]

To calculate this, you’ll need to create a custom metric, but you should be able to do so using your product feed. For service-based businesses, this is easier – odds are you know your profit margin per service (50%, 70%, whatever), along with any ancillary costs (if you accept payment via CC, assume 3% fee, as an example).

Yes, this takes more work than simply looking at ROAS and trying to eyeball profitability – but it also produces exponentially more insight than that, too. At the end of the day, it doesn’t matter if an ad is driving a ton of revenue, if that revenue is coming with a $0 or net-negative contribution margin.

The holy grail is ads that create profitable growth for the brand; it stands to reason that the metrics used to assess those ads must hold that – profitable growth – as their north star.

The Truth Is In The Transitions:

The most powerful feature of this framework aren’t the metrics themselves; it’s the transitions between them – that’s what unlocks the insights that lead to creative evolution + revolution.

If your ad does well at Hooking + Holding, but fails to sell the next step (i.e. high hook + high hold rate, low outbound CTR), you know exactly what needs to be addressed in order to improve performance.

Likewise, if the ad does well at everything except the hook, it’s time to find new hooks.

Obsess about the transitions and relative performance across each metric. Find the areas where your current creative falls short, then use that insight to design iterations that attempt to address it.

Advance The Brand:

In addition to the quantitative evaluation framework above, there’s one more component that must be mentioned: does the ad advance the brand? I’ve long believed that the performance – brand divide is a false choice; great advertising can both drive performance AND build a brand. These aren’t opposites, they’re two sides of the same coin.

As products become commoditized at an ever-increasing rate (thanks, Temu + Shein), the last bastion of defensibility is brand. Quality, materials, design, features, functionalities, models, etc. can all be duplicated. Emotional experience? Not so much.

Temu could create a knock-off Birkin bag tomorrow…and not a single Hermes customer would (a) know, (b) care or (c) buy it. The same thing is true for Ridge wallets, or Apple products, or American Express cards – their position in the market isn’t exclusively defined by their product, but by a combination of product and emotional experience (i.e. brand).

To be clear, I’m not saying that every ad should be pre-approved by the Brand Police; I’m saying that every ad should be tailored to create and reinforce the emotional connection your brand seeks to create with its target audience.

You’d never see an Hermes ad featuring a 20-something year old speaking to the camera, with a messy room in the background – because that ad, even if it was effective, would be successful at the expense of the brand’s image and position in the market.

Brand is a long-term investment. It is built drop-by-drop over years, decades, centuries; and it is destroyed in buckets. A single creative miss can destroy years of diligent work (and millions of dollars) in building your brand’s perception among your target audience.

Think of it this way: every ad must balance immediacy (i.e. sell more, generate more leads) with long-term viability/investment (bolster the brand perception). Going too far in either direction can have disastrous implications for your (or your client’s brand).

Here’s a framework you can use to assess if your creative fits the bill. I recommend assigning each category a score of 1 (not at all) to 5 (absolutely). Any ad that scores below a 15? Try again.

Advance The Brand – does the ad create and reinforce the desired emotional connection between your brand and your target audience?
Audience-Focused – is the ad designed to resonate with and appeal to your primary target audience (good), or is it designed to be generic or non-specific (less good)?
Emotive + Authentic – good ads sell using features; great ads move people via emotion and authenticity. Does the ad create the desired emotional experience – using words, imagery, models, aesthetics, etc.?
Differentiated + Memorable – Is your ad differentiated from what your competitors are doing? Does it stand out, or blend in? If you showed this ad, along with 4 others put forth by your competitors, to a member of your target audience, which one would they remember? (if you’re not sure, actually do this test)
Relevant To The Goal – finally, does the ad advance the business/organizational goal – whether that’s to sell a specific product, acquire new customers, generate qualified leads, whatever? A beautiful, brand-centric creative that doesn’t achieve the desired result is just a nice piece of art.

Drop the above into columns on a Google sheet, put each creative into a row, and score. Then – if you really want to get into it – determine the correlation between the “brand score” above and using the metrics at the beginning of this newsletter.

Building Remarkable Ads:

We’ve used variants of the above framework for years – and over that time, we’ve found several commonalities across high-performing ads:

Hook Hard – the first 3-6 seconds of any ad are critical. If you don’t have a strong, relevant hook, it is insanely difficult to create a high-performing ad. Spend 60% to 70% of your effort nailing the hook.
Empathize with the Audience – people don’t join brands; people join people. Your audience should see themselves in your ad. They should be able to visualize their life, their family, themselves using your product or service. If your audience is (for example) young parents, show young parents in a less-than-perfect house.
Make The Ad About The Audience – no one cares about the features of your product/service; but everyone cares about how your product/service can make their life better or help them overcome a challenge that’s weighing on them. Make your target audience the hero of your ad. Put their challenges, pain points, issues, goals and desires front-and-center. Your product/service is just a tool to help them; you are Excalibur. They are King Arthur.
Authenticity Rules – this should go without saying – but if your ad “feels” fake, it’s probably not going to perform. As a general rule of thumb, the younger your audience (i.e. the greater the percentage of their lives that have been spent online), the stronger their fake/BS detector.
Clear Beats Clever – so many advertisements try to be clever, but leave the audience wondering what in the world they just saw / what they’re supposed to do about it. Be clear. Be straightforward.
Experiment with New Narrative Arcs – Traditional narrative arcs are significantly less effective in an attention-deficient world. Here’s an example of what that might look like (image courtesy of YouTube + Motionshift.io):

Compelling CTAs – This is a pet peeve of mine: make your CTAs more relevant, more differentiated and more compelling. No one likes to “learn more” – that just sounds like work. Map your CTA to your offer – if you’re showing me a new spring fashion collection, use “Explore The Collection” or “Find Your Perfect Style.”

If you are advertising a service, make the CTA about the problem – instead of “Request a Demo” or “Talk To Sales” (bleh!), make it: “Start Generating More Leads” or “Let Us Handle That” – those subtle shifts make a world of difference.

Sound Off + Sound On – moving from the philosophical to the practical, this is the #1 issue I see in most ads today: they aren’t designed for BOTH sound-off and sound-on environments. It is imperative that your ad be every bit as compelling and engaging without sound as it is with sound.

Solve For Common Devices – GA4 has a wonderful devices report. Pull it. Identify which devices your audience uses, then ensure you optimize your creative for those screen sizes + resolutions. There is nothing more frustrating than seeing an ad that’s partially cut-off because the brand didn’t bother to design it properly. If you’re not sure, pull a report of performance by creative ID (that’s in GA4, too!) and filter to only your ad traffic, then add a secondary row for “device type” – if you see one a few devices or creatives that are consistently under-performing the rest, start investigating why.

Tokenize Your Ad Creation – One of the biggest waste of money in the creative industry is only creating 1 ad per photo/video session; the bulk of the cost is actually in the setup and coordination. The solution to this is to “tokenize” your ad concepting – break each ad into a series of components (or tokens), then create multiple versions of each component: