The Foundations Of Meta Ads Creative Testing You HAVE To Get Right transcript

This is a podcast episode about meta ads creative testing structures, like deep dive media buying kind of stuff. And it's going to be unlike any episode you've ever heard or seen about Meta Ads creative testing structure because instead of just going to the case studies and the things that are quote unquote working for my brands or whatever it is, instead I want to take you with me into a deep dive into the foundations of how meta ads actually works and the problems and questions that you have to be able to answer to deliver a coherent testing strategy for your brand or for your clients or whatever. whatever it is.

So, we're going to actually think about the foundations here a little bit more. We're going to ask what pitfalls do we have to avoid and what things do we have to get right to maximize the machine that is meta ads for creative testing and then we'll build our actual creative testing structure on top of that foundation. So, let's dive into it. If you understand meta ads at all, I think you're going to like this episode a lot and hopefully answer some really important questions.

This is like a huge question for me right now as I'm thinking about the growth of AJF Growth and how to expand our services. Uh, you know, I've published content, some of my most popular episodes ever are about how you should launch every one of your ads within a manual bid and and you should launch all of those ads against your core winners. And I will just tell you that like all of the things being equal, I basically still stand behind that.

I've released episodes about that like, you know, a couple years ago now. they are if you like go to my channel and support and sort by you know highest views ever that's the stuff you're gonna find at the top of my view counts it's stuff about meta ads creative testing structure that does that for a lot of reasons that I lay out in those episodes I think those are the that's still like a really great way to test uh to test your ads in meta uh and and so that that hasn't really fundamentally changed for me but I've been thinking much more about this issue because of a couple of things that are going to come up and because of how I've come to think about what meta ads is as a machine and how launching new ads fits into like the broad thing and the broad problem that Meta is trying to solve and how you know the problems that your brand is trying to solve fit within that.

So, so I've I'm going to try to develop this for our internal teams and for our you know agency and for the clients that we served. We've been doing a bunch of stuff here uh to do this. And so what I'm doing is just taking you on the journey with me to tell you how I'm thinking about this problem. Like a couple weeks ago, I called like my number my number two guy, like my my my right-hand man, Daniel, who is just a killer, and just said like, "Daniel, I need like 30 minutes, 45 minutes of your time to just talk through some ideas." And this is basically what we talked through on that call.

I should have just recorded that that call and just made it a podcast episode because um because there's there's all kinds of stuff we're sorting through. Okay. So, but anyway, uh I have now come to uh the the the realization or the belief that there are three core principles at play when you think about how meta ads works that are really important to understand and to factor into your to your creative testing strategy and really to your media buying strategy more generally.

And so, I'm going to lay those three principles out for you. And uh and they're all really important. And here's what I'm going to tell you up front. A couple of these actually work kind of directly against each other and that's the reason there's confusion on this topic. I'm going to tell you another thing up front, which is that this is not the last episode that I'm going to record about this. So, wherever you're watching or listening, you should subscribe to this podcast right now.

Uh, and uh, you should also leave a comment here with any questions as it comes up in this episode because I would love to answer them. Any feedback you have, I would love to respond to. I would like to interact about this because it's something that I'm honing my thinking in more carefully. Okay, so uh with that out of the way, let's get into those three different key parts of meta ads that uh that are really important to get right here.

Okay, so these are the three pillars, the three foundational points. Okay, the point number one, the more purchases you can get in each ad that you have, the better. Put another way, uh, a 100 purchases in a week in one ad set is fundamentally better than a 100 purchases per week in 10 adsets at the exact same rorowass and, you know, all the economics being equal for the long-term performance of your meta ads account.

You want more purchases per ad set as best as possible. You want, put another way, a more consolidated approach for your ads. Okay, so this is beyond creative testing. This is just like a general meta ads principle because when you are optimizing for conversions or really any event that you're optimizing for the event for which you are optimizing is currency to meta. Okay, meta is building a probabilistic um forecast of the future of the performance of your ads.

That's the fundamental challenge meta has to solve. Meta has all these ads from all these advertisers, has all these users across its user base, all scrolling a bajillion times a day, you know, at massive speed. And what it has to do is figure out how to efficiently deliver the best ad to the best person at the right time. And not only that, but to do that in the way that actually delivers the most of those for the most brands at the right time.

So the most brands will spend more money and they of course then help their profit and of course their their enterprise value and their stock price and all those things. Okay? So that's what it has to it efficiently. The more efficiently Meta can deliver your ads, the better. And so to do that, it forecasts the performance of your ad before it gets in front of a user. And of course, this is sort of obvious, right? This is Meta says this explicitly like uh with with the with the reference to uh the auction formula including estimated action rate.

That word estimated, if you've ever looked at this, the idea that the estimated action rate of the user, the estimated action rate means it's a forecasted, it's a predicted number. There's an estimate built into what a user is going to do when they get your ad. Okay? So, it is probabilistically forecasting. The machine learning is probabilistically forecasting what will happen when your ad gets into somebody's feed. That's what it's doing trying to do.

And it's forecasting that based on all kinds of signals. Very clearly, it's not only using conversion data to do that. This is sort of obvious in every possible way, right? Obviously, Meta uses engagement signals to forecast conversion possibilities, okay? In in any given ad. That's why an ad will spend $100, $500, $1,000 without getting a single purchase, depending on your AOV. That all might be totally fine. And actually, the ad will perform fine.

And and that's also why an ad sometimes will spend $100 and then it will burn out. That's this is like this is uh you know, some people get frustrated with when that happens or $50 and then and then it doesn't spend anymore because Meta is using you it's not getting tons of conversions at those kinds of spends. is just using upperfunnel signals to uh estimate what's going to happen further down the funnel towards the conversion event that it's optimizing for.

That's all very very clearly true that the probabilistic model works this way. However, the more conversions that Meta gets, the more confident that probabistic model can become and therefore can then spend your money more freely, more stably, and probably at more scale because as it's getting positive feedback for the event that you're optimizing for, that is the ultimate currency for for what is going to perform best next.

And so, Meta is probabistically doing that. And the more conversions therefore that you get per ads set, the better because that learning happens at the adset level. And that is a critical point. Learning happens for Meta at the adset level. So the ability for Meta to get more conversions in less adsets means more adsets have more learning and therefore uh more uh because there's more learning per adset. The adset can deliver more effectively, more efficiently, etc. the the most stable accounts that I see reflect this exactly, right?

Which is that essentially more conversions per adset um ends up generating more consistent performance. Now, that's sort of a tautology. It's a truism in the sense that like more conversions per adset also probably means a larger brand and larger brands generally have, you know, um less uh noise and more signal. There's less small sample size kind of problems in there, etc. Um but that still nonetheless seems to be true to me. includes brands of similar sizes with different setups.

More consolidated approaches tend to deliver more consistently over time. Now, there's some limitations to that and there's a whole bunch of details there, but just all other things being equal, that is basically the case. All right? And so, Meta gets more confident as it gets more conversions per adset because learning happens at the adset level. And therefore, the more purchases you can get in each adset, the better.

And I want you to notice that. And then by the way, right there, stop and pay attention to the stories you hear in the uh in the Twitter verse and in any forums you're in and any any place where you're seeing people talk about this sort of thing and just think about the question uh when somebody is telling me about how good their creative testing method is, ask yourself the question, how many conversions are they likely getting per adset uh relative to the reality of their AOV?

Like let me just say that way more simply because that was a stupid mouthful. Okay, lower AOV brands generate more conversions per ad set uh and it's particularly in highest volume setups, lowest cost setups because uh the CAC on a low AOV product basically by definition is lower than the CAC on a high AOV product. Like like a 2 to1 rorowass on a $50 AOV is $25 in CAC. Okay? And so that means if you spend $1,000, right, and you get uh and you get uh uh and you have a $25 CAC, then there's 40 purchases right there.

Okay? A $1,000 AOV uh a 2 to1 rorowass on $1,000 AOV is $500. Okay? So if you spend $1,000 and you have a 500 uh and you have a $500 CAC, then you get two purchases on that $1,000. And that difference is critical because 40 purchases on $1,000 versus two purchases uh is the same in rorowass. And if the unit economics are the same, it's all the same to the advertiser. Okay? It doesn't matter. But to Meta, it's actually very different because more conversions per adset is currency.

It it is the optimization event that Meta cares about and therefore uh it can optimize better. And so when you hear stories about brands that are performing extremely well in their creative testing setup, I want you to factor that in and think about that. This is actually a major disadvantage of higher AOV brands in meta. People think low AOV is sort of only bad in meta and then actually at a low enough AOV, it's a really big problem.

Actually, this is a huge advantage that meta advertisers have at a lower AOV. As long as you can make the CAC within reason, uh you get more purchases per adset and it is a really really big advantage for using that tool. I'll give you a really specific example and one that I think I myself have missed this nuance on in the past and that is my friend uh Jack Rubin from Pretty and Fig in the UK. Uh Pretty and Fig is at home like cleaning supplies, right?

Like like cleaning spray, cleaning solution. And uh I you know I don't I don't even remember what his exact AOV is or exact CAC is, but it's it's not a particularly expensive product, right? I mean, you're not going to charge somebody $1,000 for cleaning solution. Okay, so let's just call it like I don't know 30 bucks or something like that. That's what that that's that's the AOV. I really don't remember, but that's let's just call it something like that.

Okay. Well, Jack has scaled really really fast with a whole bunch of adsets all running manual bids. And I have talked about that story a lot of times is a really good example of how manual bid media buying can work. But one thing I wonder about is if Jack, first of all, I mean, I still think that, right? I still think it works. I still think it's a really good thing. I've had Jack on here doing that. But I wonder if I've undersold and underappreciated the advantage that Jack has by having a low uh AOV because he can pile up conversions per adset and they're launch therefore launch tons of creative and get tons of conversions per adset in the process.

That I think is a pretty pretty big deal. Uh and so so conversions are currency. Uh more consolidation is better in the sense that more purchases per adset is better. And it's really clear that Meta believes this. The reason why it's really clear is the introduction of ASC uh as a way of having more ads per adet and then the collapsing of BAU and ASC into one thing so that everything becomes ASC. So like think about it.

In the old days, Meta used to actively give the um the input that you wanted four to six or five to seven ads per adet. But then they moved to ASC where you could put 150 ads in an adset and they would limit you originally in the early days of ASC to one ad set with all those ads in it. Essentially tying your hands into a super consolidated account. Now I think the current guidance is not clearly it's clearly not to put 150 ads into one adset.

In fact, these days you can actually only put 50 ads in an adset. So they've they've backed off that that much of it. But there is still nonetheless this big idea that more ads per adset gives meta more data per adset and that allows its machine learning to work best. That is why they introduced that and did that. It's a very strong signal of that reality. And connected to this is the notion that the learning phase which they also put a lot of emphasis on and queue you on a lot and give you a lot of signals about is really bad.

It's really bad to be stuck in learning. And I don't think that's because your ads are going to get um like bad performance in learning and then good performance when an adset gets out of learning. I think it's the wrong way to think about it. Instead, I would think about it in terms of volatility of delivery that your delivery is going to range really really widely in terms of scale in particular and sort of which I would take as a proxy for meta's confidence.

When you're in that learning phase, you'll get these like big swings and meta in fact even I've seen it give up sort of really early on ads that look good in an adset that will not get out of learning. And so getting out of a learning phase is a really really big deal too. Meta is telling you that all the time. The best performance happens via lots of purchases occurring in each adset because meta is a giant math problem solver and the more data it has to solve that math problem probabistically the better it can perform.

Okay, and conversions is the currency. Okay, so that's the key idea. You've heard me talk about rich panel before. That's because there's a really obvious value proposition here where you're going to save 30% on your bill from Gorgeous or Zenesk guaranteed by switching to Rich Panel. So, you're just going to save money just like that by switching. Uh, and that is awesome. You're also going to reduce your customer ticket load on average by 30%.

That is also awesome. That means your customers are having a good time. That's why multiple of my clients are using Rich Panel to handle their customer service help desk. Uh, it's why people are so incredibly happy with it. People all across the e-commerce space have basically never heard a bad word about it. It's um really, really awesome software. But let me tell you one of my favorite things that Rich Panel does that is relevant to this particular episode and that is an AI comment moderator.

Okay. Um Rich Panel has a feature uh where you can go and make it so that uh when you're running meta ads and people are commenting on your ads uh Rich Panel's AI comment moderator will go in and interact on behalf of your brand for you. And it does an incredible incredible job of doing this consistently and accurately and in a way that you're going to be proud of to where like huge brands like Ridge has like basically taken all their comment moderation over to Rich Panel's AI machine uh last I checked uh because it works it works really really well and allows them to interact with customers really fast.

Um like meta ads comment moderation has been a problem for as long as I've been in Facebook advertising which is like over 10 years now. And uh and it's because there's this always been annoying to handle, but people also know that it's important to interact with your customers and the places that they interacting with you. Uh but uh but it's just like a huge mess like staying on top of the comments and on top of all the threads and everything.

So AI doing that for you with rich panel is one of those use cases for AI that just like makes so much sense. Uh and it's one of those places where you think if AI can help anywhere, it can help with something like this. Look, Rich Panel is an incredible piece of software that's built with AI first from the ground up and exactly the area of your business where it makes sense to be built AI from the ground up customer service and you really should check it out.

They're also going to transfer you to a rich panel from whatever your other customer service help desk software is in like two weeks or less because they know that you don't have time to blow um to transferring customer service software. They're really really good. It is more affordable than the other options. It is an awesome piece of software. My clients are using it. You should use it too. richpanel.com tell them that I sent you.

This is also I think probably why meta at times although I've you know I I haven't seen very strong guidance here one or the other and I've actually seen mixed but this is at times why I see people talk about the idea of maintaining separate scale and test campaigns because there is this idea that you don't want to interrupt scale campaigns where your adsets are working they're getting plenty of purchases per per ad set and they're out of learning and you don't want to disrupt that and keep screwing with those campaigns and kicking them back out of learning over and over and restarting the learning process etc because you want the stability of that delivery.

And so that ends up being this really, really important point in all of this. It gives Meta more options and more control uh because you get more purchases per adset, more ads per adset. And I would also file this in with uh with uh a CBO setup in your scale campaign so that Meta can actually not only pick more ads per adset for more adsets per campaign. Put the budget there. Let Meta distribute between your adsets, especially if those adsets are out of learning.

This is the ideal setup. and manual bids, bid caps, target rows, cost caps, these are all various tools to also do that same thing where you give meta control particularly for day of week effects. Manual bids allow you to scale up really aggressively on the weekend and scale uh back down on weekdays when daily active users are lower and therefore meta can capture volume according to its predictions. And these certainly work better, manual bids certainly work better, more consistently when you have more purchases per ad.

Okay. Um, what I've heard some people from Meta refer to this as is the idea of creative liquidity. The notion of like Meta having liquidity like liquidness to the creative options that it has. So more ads in one CBO, more in a consolidated setup, more ads per adset, it can make more choices and gets out of learning to do all of those things. And here's the thing I want to say about all of that that is really, really important.

Okay, that the meta having maximum amounts of data and then control to run its probabilistic engine in your ads is what you want. You want to get your human decision-making brain out of the way. The ideal meta ad setup certainly is this includes this as a big part of it. Okay? Because like the moment that you have given a dollar to Zuck for your ads, you have already consented to this reality that Meta is going to use machine learning to solve an unsolvable math problem.

Like it is un just consider for a second the scope of the problem I laid out above that Meta has to solve. So many advertisers spending so much money on so many customers across so many formats like every minute, every minute, every day in real time. all these live auctions happening at just incredible pace. It's it's it's it's such a crazy problem that actually when you talk to Meta about meta people about this, they will they will reference these people who work at Meta who are like auction experts or like people who study the machine learning the algorithm because because nobody even knows anymore exactly how it works because it's too big of a problem to solve with too many people working on it.

So you have to go commission internal studies on how the thing is working relative to how it was designed which by the way is why I don't think meta is perfect, right? is like I don't think it can perfectly predict the future. It's impossible. Like nothing can be perfect at best. Okay. But I think I think it is way better at solving that than me. And so every time I every time you run an ad, you have already consented to the idea that that is the right way forward that Meta's machine learning is going to solve this problem.

And then you get in there and start tinkering with all these little decisions that you make in the account because you think you're going to be able to solve this problem better and in a more sophisticated way than the than than a machine learning based probabistic based approach to this. I mean that's just like insanity to me. And and this seems to me to be one of the most obvious things about this whole thing. The more uh the the more you reduce Meta's creative liquidity, the more you insert yourself in the way of especially your like core scale ads, the bigger the problem actually is.

Okay. So, I'm going to now in a little bit I'm going to tell you the counter to that and the thing that goes wrong there and the reason why that isn't the once and for all final solution. It's going to be my third pillar. Okay? But I just want to say upfront. So, if you're frustrated, if you're like, "Come on, Andrew. I've seen Meta make plenty of mistakes many times." I understand. I really do. And I'm going to address that in a little bit.

But at the baseline level, you want a highly stable version of Meta with lots of conversions per adset, delivering your ads to the right people with confidence based on all of that information as much as possible. That is the ideal version of an account that you are going towards. Okay? And so um so that is that is number one. That's pillar number one. Let me repeat it. Pillar number one, the more purchases you can get in each ad set, the better.

Okay? And the more consolidated setup, the better. Conversions are currency in if you're optimizing for conversions. Okay, that's the key idea. All right, so that's number one. Exactly in the opposite direction of this. Okay, I just told you you want all the stability. You want to keep adsets in learning or out of learning. You want more conversions per adset and you want a stable setup where you're not introducing all kinds of problems and taking conversions away and and all this stuff.

But number two works directly against this, okay? And it is this creative volume is good, particularly diverse creative volume. So you don't just want volume for volume sake. Actually, I think volume is really important. I've put out a lot of stuff on this. We try to generate I, in fact, I think we probably do generate more ads like than the vast majority of agencies working for clients. We are working on that all the time.

The creative supply chain idea is right. Um, but I really especially am even more moving towards more creative diversity in our volume at the same time. They're both really really important I think. So, so diverse creative volume, let's just say it like that. Diverse creative volume is good. And the reason I say this, this like is the fundamental challenge of this whole problem to solve. Okay. And the reason I say that is because what I am telling you is that on the one hand, you want all this stability and on the other hand, you want to be constantly launching highly diverse ads relative to the other ones that you do.

And if you do that, you will likely create some winners that detract spend from your previous winners. You will outperform previous ads of yours. And therefore, your adset that's getting a 100 conversions per day or whatever over here, okay, is going to lose conversions to your new adset over here that you put in the same CBO or that you run a manual bid on or whatever. And uh or there's a lot of ways this could work.

And you start doing that and now it starts getting more conversions. And of course, hopefully the total conversions are higher than when you started. So it's not just detracting from one ad set and coming out in the wash and staying in the same spot. It's actually scaling to get you more spend. That's what you want. But at the same time, there is a reality to where meta is going to rep prioritize different ads in that process.

And so what this is like this is I think the reason why there is people there's debate about this subject of creative testing. This is the reason why it's hard to sort out what the best answer is. This is the reason why I'm actually sympathetic to people disagreeing with me about all kinds of things in this because these two things are fundamentally at odds and I heard them both from Meta a million times. Give us stability, stay out of learning, more ads per ad or more conversions per adet is really good.

Number two, all the best advertisers have a whole bunch of volume and variety. Okay, how do you do both of those at the same time? They work directly against each other. You may not have a strong opinion about creative testing setup, but I bet somebody in the Workspace 6 community does because Workspace 6 is a closed private Slack community for operators of eight and nine figure and and high seven figure e-commerce businesses.

It's it's uh operators, founders, executives in those businesses working out problems like this one together without people like me, agency owners, uh pitching them on their exact method. Um, and because they want to make Workspace 6 a private community where operators can talk really honestly with one another, not only about problems to solve, but about um about software companies they've worked with, about agencies and other service providers that they've worked with and all those things so that you can get into uh tap into all of the knowledge that exists in the DTOC community around you.

One of the great things about the DDC community in general is that people are really willing to share information with one another. Everybody talks. Everybody talks and therefore getting into community where workspace 6 like workspace 6 where that talking is lubricated by the simple mechanism of a great and well moderated Slack uh community is really really valuable. There's just a million questions that you can and should be asking while you're running your business that somebody else has already answered for you and you can save tons of money and tons of time by getting their answers in that community and you'll feel great as you provide those answers back to others.

There's Workspace 6. It's also totally affordable and there's no long-term commitment to it. A dollar for your first month to get into the community and check it out for yourself. So, go to workspace 6.io. Workspace 6.io. Workspace 6 is also run by some of my favorite people in e-commerce. Just people, one of the only newsletters I consistently read is from Workspace 6. They just do a great job of staying on top of what's going on.

They're great. Go check it out for yourself today. It's really, really hard to not have the monthly membership cost pay off for you, even in the piece of software that you don't buy because somebody told you that it's garbage or something like that, right? go to workspace 6.io, get up for yourself today, tell them I sent you. And I've asked people at Meta about this same question and and they have been said all kinds of things and and none of them in my view have been like super super confident, but they have both said like what your setup needs to do is hit both of these.

Okay. Now, um I do want to say here this is where I think uh one clear way to sort of solve this problem a little bit is that um is to try really hard to hammer that creative diversity point. Okay. Um, there's two reasons for that. One of them is that two ads that are basically the same are very likely to perform at similar levels to where if you turn one off, then the other one that is turned on will just capture the spend of the one that you turned off if they really are like essentially the same ad.

And that's actually pretty important if you think about it because uh because the uh the that would allow you to actually have like each adset have a a broader diversity of ads and you don't have this risk of like these tiny little differences making a big difference. And this is something I've really done about face on. You know, I've released content about how we used to do multiple hooks per ad and there's a reason for that and there's logic to that and there's still I think a potential case to be made that you could do that for a couple of reasons.

Um, but I've really moved away from doing ads that are very much too similar for multiple reasons. Um, one of them is sort of like is that like this kind of thing that I'm describing happens where basically the same ad um produces uh produces apparently unique spend but actually it's not really that unique of spend and that if you turn off again ad B that's very similar to ad A that ad A will just suck up the spend from ad B very likely.

Okay. Um and and so there's that, but actually there's even a deeper layer here, which is like one of the reasons some of your ads don't spend is because Meta cohorts them uh together. And what that means is um two ads that are extreme that Meta recognizes as 70% or more the same, it will actually just stop entering one of them into the auction and instead will enter uh only one. So if there's ad A and ad B, meta's machine learning recognizes or AI or whatever recognizes um ad A and ad B is 70% the same based on what it reads of the ad.

Okay, if that happens then ad A uh may get entered into auctions and ad B will not. So it'll just get a ad B will get a couple cents of spend and you go why is this ad getting like no spend? And the answer is because Meta is actively giving up on it and putting all its attention on ad A because Meta is saying uh well these are basically the same ad so we don't need to test both. we can just test one of them and and come out uh with the best possibility.

Now, there's some problems with this idea and again, wait for pillar three. I promise I'm going to get to pillar three in just a second. Okay, that's going to talk about one of the other issues in the midst of all this. But again, most of the time that's actually true. And that's because I think a lot of people think that meta ads is like a sort of giant like slot machine or casino or something where it's like the the performance of a given ad is just like totally random.

Uh and it will and like you know everybody knows at this point that creative is targeting and Meta uses your creative to target users and so you met Meta people sort of think like okay two ads that are basically the same will end up targeting very very different users in very very different ways sometimes and it'll do that um because there's there's just a lot of randomness in the machine. There's a lot of noise amidst the signal and and so you know you just sort of take more shots and you do that sort of thing.

I tend to think that actually in most cases it's not a casino. it actually is machine learning based and if the ad has basically the same message uh the two ads and it's presented in basically the same way what's most likely to happen is that those two ads are likely to reach basically the same person and they're likely to reach them at basically the same efficiency and so there actually is redundancy in that setup and you should reduce that redundancy as best as you possibly can um and instead again because because all that's going to do is work against point number one that I made before now let's say let's say your ideal number of ads in an adset is 20 I don't know what the actual number is but let's just say it's 20 okay Um, right now you've taken up two ads in out of that potential 20 instead of just one ad out of that potential 20.

And if you really want to get more ads per adset that are spending and more stability in the process, then well, you probably uh ought to turn off the second version of that ad so that the top one spends. And by turn it off, what I really mean is never have made it in the first place. You should just not make ads that are that similar to one another in the first place because um they are not going to they're going to be um again basically reaching the same person.

And what you really want to do is reach the most different people the most effectively as possible. That's the real job of advertising. Okay. In any case, there is this problem nonetheless. So what do you do? Do you launch all of that creative volume that you are pursuing against your scale ads? Do you launch them in a separate test campaign? Do you launch them in an auto bid campaign? Like all of these are different ways people try to solve this problem and and and have all kinds of um justifications for those things.

As I said before, all other things being equal, I still think there is a a great version of this setup and basically the one that I still use where you launch all of your ads with manual bids against all of your winners until you have sort of closed adsets where all the ads in your adset at x number of ads per adset and I don't have a clear number there are spending something and you just take your new ads and you and you and you get all of your spending ads in launch all your adsets against your winners with manual bids and just let Meta sort it out over time.

Sometimes meta is slower than you'd like. Sometimes Meta scales new winners extremely fast. But that's basically the way that I think you should do it because that allows you to to maintain number one while at the same time pursuing number two. I that is you maintain more stability, more consolidation, more ads per ad set and at the same time uh number two you you launch a whole bunch of ads over time um and let Meta sort out the winners.

Okay. Uh, and the simple way to do that, right, is to take your new winners or to take your new ads and launch them into a new adset in the same CBO as your other adset, uh, as as your scale adsets. So, so let's say you've got two adsets that are delivering scale ads set one, scale adset two. They're both delivering. They're both delivering stably. They're both they've got both got the number of ads that you want in those adsets.

Maybe it's the full 50 in the adset. I'm not sure. Again, I don't know what the ideal number of ads in an adset is right now. I've heard mixed things about this, but but just say you're there. They're out of learning. That's great. Okay. So now now you launch scale ads set number three which is technically like a test ad set and in fact you probably don't even use the language of test and scale at this point and you you do that and you say all right well I'm going to launch this alongside that and maybe it won't spend on it immediately but after a little while if those ads are good it will be um additional it will be incremental spend in my account so meta will gra will figure out spend money there it will it will determine which of these ads across all of these ads not only in the ads set but across the CBO are the best ones.

It will do all of that in my manual bids. And in the end, what will we end up with is a really stable account that that may scale, sometimes a little slower as meta takes a little bit of time to get more confidence, sometimes a little faster because Meta will get really strong signal really fast, but all but uh but most most of the time we'll scale as best as possible. Um and that's that. So, I hope that makes sense.

But that's the way that sometimes I've tried to solve that problem in the past and and broadly speaking is the way I solve it now. What I really want you to see before you even come to an actual solution as you evaluate this question for yourself is the fundamental tension of those two problems. Okay? More stability, less more ads per ad, less less changes per adset, getting out of learning, staying out of learning, those things, launching a high volume of highly diverse ads.

Okay, those two work against each other and that's the problem. And now there is a third pillar that is also a problem in this whole thing. And this is the one that I think actually makes people super crazy and probably rightly so. All right, and it is this. You need to avoid false negatives. Okay, so let me tell you what I mean by a false negative. A false negative is an ad that Meta does not spend on that could be a winner.

And this is like a real problem. In fact, you probably should this is this is like first of all overwhelmingly the number one frustration people have with the creative testing method that I talk about a lot, okay? Which is launch everything in a manual bid, launch them against your winners, don't bother with test campaigns, etc. Uh people the thing people hate about that is they launch so many ads that do not spend and it's really frustrates them and they do all this work and then they get like $5 in spend or they they like hire a creative agency and that creative agency gets really little spend on their ads.

Okay. Now uh and and so then you're like well wait a minute why if I'm going to produce this money if I'm going to spend all this money producing these ads I'm really going to let Meta give up on it on five bucks 10 bucks and spend like that makes people absolutely insane and I totally understand that. Now, the answer to that, I think the idea that the answer to that is to force spend to all of those ads is just another way of describing the cost fallacy, right?

Oh, we spent all this money on production. We better distribute even if it doesn't look like it's going to be a winner. No, you already spent the money on the production. Don't waste more money if the ad doesn't work, right? Like like that literally is a sunk cost fallacy. So, you should be careful of that in your logic because it's a big huge waste of money. And I've been said repeatedly that I think people waste tons of money on creative testing really unnecessarily uh because of basically this.

They're really nervous about false negatives. Having said all of that, if false negatives actually are a gigantic problem, meta giving up on winners when they shouldn't. If that is actually a huge problem in uh ad accounts, you probably should be willing to pay to avoid it because the biggest most valuable thing you can have in your ad account is a massive smash hit winning ad. Now, I have basically never seen a time, and people will disagree with this because I've heard a million anecdotal stories, where in my setup that I'm describing, a true smash hit winner ad has been passed over by Meta and like given no spend.

Like I I have one account right now where I have like rebuilt a series of ads that that like me and the team who I'm working with like completely love. We've rebuilt these ads a million different ways, tested them a million different ways and have had the same result every single time. We launched them in my manual bid, you know, uh, launch them against your scale ads setup. They got very, very little spend. Okay, by the way, I didn't develop these ads.

It was with a thirdparty creative agency. I love the thirdparty creative agency's work. They are they they like they were beautiful. They were awesome. Um, I because I've talked about it publicly, I will say it is not my friends at Fire Team, just so you know. Um, so, uh, so, uh, the this other agency was was working this account. I I thought the ads were awesome. Like, they were really good. They were thoughtfully done.

They were creative. They were clear. They were interesting. I liked a lot of things about them. I thought they were really, really good. So did the team. So did the client. The client loved the ads as well. We launched them on a manual bid setup. They basically didn't spend at all after we had spent a bunch of money on the agency. We re we lowered the or we made the manual bids more aggressive. So basically we put our thumb on the scale and said okay meta just let's just make sure these are winner these are not losers go ahead and spend on these a little bit more still didn't work like still didn't spend either spent very little or spent below target uh a bunch relaunched them in an creative testing setup exactly the way people talk about all the time still didn't work still performed well well below target relaunch them in a fourth setup that I haven't even talked about before but it's a whole another like potential workaround play around with this thing still hasn't worked still didn't do anything.

So I like I've given them every possible setup, a bunch of ads that are subjectively good and yet nothing. And that I think is actually the most normal experience that when Meta see like if we had just listened to Meta in the beginning, quote unquote, listen to Meta in the beginning. Then uh we would have just never run these ads really aggressively. we would never would have spent that money on these ads because Meta was saying no our probabistic forecast says that you are not going to hit your manual bid target here and these are going to be a cost and not uh distributing these will be a cost and not a value so you should stop this okay and we retried it retried it retried it retried it and just nothing and that was it okay so yes so that like that is I think the normal situation in mo most cases however I've heard enough anecdote stories and I do think it's a really important thing I mean I have seen recently ads that have spent zero dollars initially, okay?

Like literally zero, like something was broken or something, something was wrong. And I relaunched them in an auto bid setup and that just like got them going a little bit and then was able to transfer them and get them going. And they listen, they weren't like mega smash hit like destroy the world super scale account ads, but clearly Meta had missed on them. And I think it's it's fair to say that that Meta had given up on them too early because Meta is not perfect.

And I've never said nor do I believe that Meta is perfect at predicting all these things. And so but that false negative problem if you think it's a real thing and I think there's at least some consideration that it might be a real thing at some level of cons some level of uh problem that's another problem to solve in all this right so as much as I I I I'm just saying I understand that you really really want to avoid false negatives and finding some way in your creative testing setup to make sure that you don't miss out on ads that could win is a really really big deal.

Um, that's that is uh that is that is the problem. And I think this is probably why Meta at Times has said you want to get an ad to 50 purchases before you put it in your scale campaign when they have like I've seen that be public advice at some point in their process. And the idea there is like make sure that Meta doesn't give up on on an ad too early because false negatives can happen at the ad level. And further false negatives can happen at the ad set level.

And this comes back to point number one, which is this notion that if Meta doesn't get an adset out of learning, it will if it's just getting um too few purchases at a time, it will uh give up on an adset or a bunch of series of ads too early. Basically, it will just sort of stop spending and dry up when it really actually could have spent more be not because the performance was bad, but because it was not confident in the performance.

And um and again, that's because conversions are currency. And so um and so this is the other challenge that you have to solve along the way. Okay, avoiding false negatives. Is there some way to do this? This is I think the reason why some people love creative testing setup. When you hear people talk about the idea of I'm going to test I'm going to launch all my new ads, do a minimum a minimum spend on some ad set or or even an where it's like, hey, we're going to launch all of our ads in an creative testing setup.

We're going to find winners as fast as possible, scale them right there because the notion is we're just going to make absolutely certain that we don't get false negatives. And false negatives are for sure a bigger problem with manual bid setups than auto than automatic bid setups. And the reason why that's the case is because uh in an automatic bid setup, Meta still has to distribute the money and it will tend to distribute a bunch of money in the early phase of your ad spend across a bunch of ads uh with where like most ads get at least some spend.

In a manual bid setup, not only does your ad uh have to sort of beat the other ads in the ad set to get delivery, uh so it's, you know, a bunch of ads against each other in an ads set where Meta is force ranking the best performers and the ones that she spend the most money on. But it actually has to do that while clearing the manual bid. And if it if Meta does not have confidence that it's going to clear the manual bid, then uh then Meta will will stop spending on the ad.

And therefore, it's sort of two places this can break instead of just one. So Meta may lack that confidence. meta may uh see that and then ultimately give up on an ad. And that's why manual bids would have a higher theoretically false negative rate than than automatic bids. Okay, so this is why people like automatic bid setups because they want to really avoid that. They want to get test dollars to each ad on the assumption that Meta needs those conversions and cannot properly predict this at scale with just upper funnel upper upper funnel signals.

And so that's the reason that you would want that kind of setup to get those going. Get no false negatives. And yes, it's going to cost you some on some ads that don't work. But if you get really good at making at most at least some decent number of ads work, and if you can do that while at the same time avoiding false negatives, you will end up in a place where net net it's going to work out to be a better place with more scale for your brand.

I think that is a reasonable case that somebody could make. I don't think it's the ideal setup, but I think it's I understand why people get there. Okay, let me just run that back really fast. all three of these issues that are the things you're trying to solve at the same time. Okay, number one, more purchases you get in each adset uh per ads set the better. Okay, more consolidation is better for meta machine learning.

Number two, create a volume and diversity is good and that presents a direct challenge to number one which is the fundamental challenge in this. Number three, avoid false negatives. False negatives are a problem to be avoided and that presents a challenge to basically every setup and especially uh again number one uh in particular and of course in some ways becomes almost more of a problem with number two maybe less of a problem because uh because if you launch enough ads then even if you do get a false negative you also have enough ads to paper over it.

Okay. All right. Those are the three aspects of this. So what should you do in light of that? I'm not going to tell you today. I've told you what I think is a good starting place setup. launch all your winners or launch all your ads against your winners in manual bid scale campaigns. That is the place I would start. I am playing with a couple of other ideas that are trying to be a little more nuanced in some of this.

I don't even know if it's going to work yet. So, what you should do is subscribe to this podcast because I'm going to do an update about this as I continue to test this and build it out, write it out with as much clarity as possible as I try to sort through all of these. What I really want you to do today is just think more deeply. I don't want you to be satisfied with somebody telling you a case study about why their setup works or why their CBO setup works or why their manual bid setup works.

When somebody like me goes and tweets, I have a client that 10x our ad spend in five in four days, okay? And it's because of manual bids, right? Take that for whatever it's worth, which is one story among others, but but think down to the level of foundations instead and in the midst of that and think, okay, wait a minute, how am I trying to solve those problems at the same time? Do I agree with the foundations that Andrew just laid out?

Do I understand and can I think through that? If you're a serious media buyer or a brand shopping an agency, I want you to think about whether or not um somebody is thinking about the problem with this kind of rigor. This is the kind of rigor I want you to apply to this. It actually is a really big deal. I know it's tempting to turn off this conversation and to say this is like for nerds to argue about and um and and to care about it all and like you should care about all these other things instead.

I understand why you think that. It's actually again, as I've said in a lot of places, meta ads represents the biggest line item in your business. Understanding the machine is really important. And for a DTOC brand is an important part of operating. And so having perspective here and point of view is worth your time or someone on your team who's well equipped to think about this their time and it's worth your agency's time to be thinking carefully about that's really what I want you to do with this episode today.

And like I said, subscribe because I am playing with a setup that I think gets all like has answers for all three of these in the setup and the most probabilistically best way that I can come up with and I'm going to lay it out in a future episode when it is finalized. You're not going to want to miss that. Um, so that's what you should do. Subscribe today. We'll come back to this more in the future. All right. I'm really curious if anybody actually stayed with me through that whole thing.

It is exactly the way I've been thinking about this problem in my business and for my clients. And so I'd love your feedback. Like I said, please do leave a comment arguing with me, yelling at me. Please tell me I'm an idiot. I would love to hear all of those things in the comments because I actually want to know what you are thinking about this topic and what I'm not thinking about and all that kind of stuff. Of course, like I said, also subscribe wherever you're watching or listening uh so that you don't miss out on future episodes both on this topic and many others that I have coming up that are going to be great.

You're not going to want to miss them. It's going to be really good. Thanks to my friends at Rich Panel and Workspace 6. Go check both of those out. The links are in the show notes. and I'm so grateful for them. They are the reason I'm able to do this show and keep it of course free uh for you. So, big thanks to them and I really do love the people who are running those businesses and their their um their services and software.

Great. Uh so, go check those out today. Sign up for my newsletter afgrowth.com. If you would like to work with me uh and with AJFrowth, uh we're not taking clients today, but we will be again at some point here. And so, you should reach out and we should start a conversation and check that out by going to ajfgrowth.com. this episode catches fire and has a very long shelf life. If it's the future, maybe I'm taking clients.

Afgrowth.com. Fill out my intake form. Would love to talk to you about doing that. That's everything. You know how to reach out. podcastfgrowth.com if you want to email me. This intro, this outro is going too long. Let's be done with it. Thanks for watching, listening. See you next time. [Music]