Meta Ads Are NOT Volatile (And Believing They Are Is Costing You Money) transcript

I can't see what's happening in your ad account, but I can tell you something that is true across the board, and it is this, Meta ads are not volatile. Meta ads are not volatile. That is not the problem with your ad account, no matter what's going on there. The problem with your ad account is something else, and I'm going to explain to you what it is in this episode right now. It is probably hard to believe when I say that Meta ads are not volatile because you know that performance changes a lot day-to-day in a Meta Meta ads account, and that is true.

And I want to say that up front. What I am not saying right now, and what I will not say at any point in this podcast episode, is that Meta ads will deliver stable outcomes in terms of purchases, ROAS, those sort of things, day over day, spend level, any of those things day over day. That is not what I'm saying when I say that Meta ads is not volatile. Um those things all happen. In fact, they happen a lot, and they range really widely.

I have a client that had an almost uh it was probably like a 40% spend reduction day over day from Monday to Tuesday of this week at very similar performance, at very similar ROAS, all on manual bids. We didn't change any budgets, any manual bids, nothing, and yet we had a 40% drop. So, that's true, in fact, performance got better um as it as it spent less. So, if that's true, how is it possibly the case? And that's before you talk about the wide range of ROAS outcomes, revenue outcomes, purchase outcomes that are happening for individual ads, ad sets, campaigns, and accounts for all kinds of advertisers.

Those things are happening all the time for everyone. So, if that's true, and I recognize that, if that's true, why and how can I possibly say that Meta ads are not volatile? Because I am saying that, and I am saying that with um a lot of conviction, actually. I think that what I'm about to say is one of the most misunderstood things across the board in Meta advertising, and really it's true across all kinds of advertising platforms.

Anything, any advertising platform that's programmatically delivering your ads attempting to get results per you know, whatever the results are, whether it's yeah, anything you're optimizing for. I don't need to list them. Anytime you're telling a machine to optimize for a certain result and you're having it delivered. So that could be meta ads, TikTok ads, Google ads, Snap ads, anything you want. I think that they are not volatile necessarily.

Now, I'm going to hold my observations here for meta ads because that's the platform that I know and understand Um but almost certainly the things that I'm saying in this episode are applicable across the board for those other platforms as well. Um and so and here's the thing. I believe that misunderstanding this point is one of the largest things costing advertisers money right now across D to C. I think brands are wasting truckloads of money or missing out with huge amounts of opportunity cost by which I mean money not spent on high performing ads.

Truckloads, and I mean it, truckloads of money by getting this point wrong, okay? I think they're getting this point wrong and that's a critical thing. And so what am I saying? What am I saying? At the core level, what I believe almost everybody gets wrong is they think they are being data driven with their decision making but in fact are being almost 180 degrees opposite of data driven from in a lot of thinking as particularly media buyers and really anybody touching the ad account.

This applies more broadly than the ad account but especially in the ad account. Even if they think they are making decisions based on data because the vast majority of people as far as I can tell drastically misunderstand the distinction between probabilistic forecasting and the actual outcome of results. Put another way, process versus results. So that's a common idea, right? The idea that you that there's a gap between process and results.

But I think I think this point is so counterintuitive that I actually almost doubt whether or not this episode is worth recording because it is so hard for people to get this into their minds and I've actually started to wonder if some people will sort of never get it. Not because they're dumb or something, but because it's just so tricky to understand this point. It's so counter intuitive to the way that we look in the world.

We all are looking at the world all the time, analyzing patterns and seeing them and spotting them everywhere. People have pointed this out all over the place. We are seeing patterns and responding to those patterns. And that's really hard to turn off and to understand the gap between process and results, you have to really shift your way of thinking about pattern analysis and and pattern analysis, that way of viewing the world is extremely natural to humans.

So, it's really hard to make that shift, right? So, again, it's not about being dumb. It's just I'm just saying it's really hard to get this right. I'm going to give you a couple examples that I'm going to try to illustrate this with over the next couple minutes here as I work through this episode, okay? The ability though to separate process versus results leads to a number of really practical implications about how you think about distributing ad dollars and really how you think about doing a lot of things in your business.

I My hope is actually that you could take the insights from this episode as far as they are useful and apply them across not only more of how you approach your media buying, but your total business and honestly lots of things in your total life, okay? I really think that that could be useful to you, okay? And so, let's start by illustrating the most the clearest example of this that I can think of, which is which is poker.

Poker is a great way to think about the gap between process and results because it's a sort of self-contained system with 52 cards and you play the game and there's this weird element of making bets, of course, on different poker hands and human behaviors interacting that throw things off. Blackjack in some ways is actually even more self-contained game, but I don't know Blackjack very well, so I'm not going to say anything about that.

But, let's just take the most famous version of poker, right? Texas Hold'em, where it's you get two cards in your hand, there's five communal cards in the middle of the table, and you match your two cards with the five cards on the table to create the best five-card hand you possibly can. And if your hand beats everybody beats everybody else's hand, you win, okay? Now, of course, the way those five cards are put out on the table is you start you start by making a bet off the two cards that you have, and then and then three cards go out that everybody can see, and then one more card called the turn, and one more called card called the river, okay?

And you make bets every round as different cards come out. So, the cards come out at three different times. Okay, I'm not going to go any more into the rules of poker. Hopefully, you know it. If not, go look it up. Uh, the basic concept here, though, is that, of course, you have two cards in your hand that nobody else can see, and you have to bet on what is most likely to happen and how good your hand is based on information revealed to you step-by-step.

So, of course, at first, you have only the information of your two cards. You can't see anybody else's. When the first set of three cards come out, now you have more information. And now, you have a better sense of the most likely outcomes of the hands in the game. And as the turn comes out, the fourth card, you have more information, and you also have less future information coming, right? So, you have more information, and now there's only one more card left.

And so, you can change likelihood. At each step of it, there's mathematical likelihoods about the results. And and of course, great poker players, and actually, I don't play much poker, you know, except goofing goofing around with a friend's birthday or something like that. And part of the reason I don't play much poker is because I know I do not know this world well enough. Great poker players understand all of the math involved here and the probabilistic outcomes, and it's a major factor in their decision-making.

They can tell you what cards they have, how good they are relative to the possibilities from there. They just have this very mathy approach to it, and they can play that. Now, it's not the only thing in poker, of course, because again, there's this human dynamic in some of it, but it's a really crucial part of it. And the basic point here is that um what good pokers are able good poker players are able to do, and the reason, and this is a this is a great point that I think maybe it's in Rounders or or something like that.

I don't remember which poker movie it is, but somebody's pointed out, why is it that the that uh the final table of the World Series of Poker often has the same players in it if it's just a game of chance? Well, of course, the answer is it's not just a game of chance. Chance is a big factor in poker, but the ability to think in a process versus results way means that there's a distinction here. It is it is not just about chance.

If if it was just about chance, then the process versus results gap wouldn't really matter. But what great poker players can do is they can play aggressively or conservatively, bet aggressively or conservatively based on the mathematical outcomes likely based on all the situations at hand. And so, what they do um is they say, uh okay, uh let's say I have uh two good cards in my hand, and I the the very likely outcome as uh four cards on the table, and I'm waiting for the last card, right, the river.

I'm waiting for the last card. I'm going to bet my chips super aggressively, okay? Uh let's say they have two kings in their hand, all right? And uh and there's a third king on the board. Um Uh okay, great. Uh there's no aces on the board. There's one king on the board. There's two kings in the hand. So, they know they have three kings, um and it's a really good hand. And let's say uh let's just say for the sake of it, there's also two fives on the board.

Um Okay, so I I think I said that right. So, they now have a full house, three kings, two fives. They can play that uh well, no, let's even make it better, three kings, two jacks, okay? And so, they can they have a really good, really strong full house in that case. So, they bet extremely aggressively. Uh I don't know if I'm just doing all this math right with this theoretical poker game, but um but they bet extremely aggressively, okay?

Uh yeah, we'll do that. We'll do three kings, two jacks on the board, okay? They've bet extremely aggressively and they know that they probably have the best hand, okay? What they don't know is the person sitting next to them has two jacks in their hand and they have four of a kind, four jacks, and they're about to get crushed, okay? Now, here's the thing. A good poker player bets aggressively when they have a full house with three kings and two jacks.

They bet aggressively every time. They bet aggressively every time they play the game. So, even if in that particular hand they lose cuz the person next to them has four jacks, they know they made the right bet. And if they play a hundred more games of poker, a hundred more hands, and that hand comes up every single time, they keep making that bet over and over and over and over and over again because they know that the statistical odds of them winning that hand, if I just did all that right in my head, okay?

Um are very, very good. And that gap, even if they lose all their money in that first hand, right? They walk away from the table and make the exact same play the next time. That is the gap between process and results. They had a very bad result, but a very good process. They made the right bet given the information they had on hand, and if you keep making the right bet over and over and over again, you are going to win.

Do you even know how profitable the last decision you made with your website actually is? Uh look, you probably don't. You might have made copy changes, image changes, added a piece of software or something like that. And you should be measuring the things you're doing on your website with Intelligems. Uh Intelligems at intelligems.io is a great sponsor of this podcast and I'm so grateful for them because all they care about is operational excellence so that you can drive profit in your business.

Uh testing tools make it easy to measure tweaks in all kinds of different ways, headlines, layouts, button colors, whatever it is, right? Traditional CRO testing tools. But business decisions go beyond the page, they go beyond the basics of those testing tools. Intelligems brings CRO level clarity to things like product pricing, shipping rates, average shipping free shipping threshold, discount offers, yes, even new SAS tools.

You can actually test live whether or not the software you're adding is actually doing the thing that the sales person told you it would do when you added it. It shows you exactly how each one impacts profit. It's easy and fast to install. You don't need a developer and it's also reasonably priced and you can get 20% off it for the first 3 months by going to intellijems.io and using the code Ferris 20 f i r f a r i s 20.

I took intellijems as a sponsor because my clients use intellijems to test all kinds of stuff exactly like I just said. I've got a brand right now that's running price tests, that's running offer tests, like their stacked offer or the new customer discount. They have used it for regular AB testing. They measure all of it on profit per visit just like intellijems tells them to and it's really made a difference in the business.

Go to intellijems.io to get started with intellijems today. Smart analysts approach data that way all over the world, okay? And therefore, what they do what they do really is they think there is a probabilistic likelihood of what is going to happen next. This is a critical point. All prediction of the future is probabilistic. All of it, okay? The future is fundamentally unknowable. You do not know what's going to happen next on a lot of different levels.

The nice thing about a game of poker is that actually there are no unknown unknowns in poker. You know, that is really there I mean basically there are not, right? It's a self-contained game with 52 cards and with a certain amount amount of chips on the board at a given time and players. And so, there's no additional factors besides those things. At some point, you know, there's a closed number of outcomes in a in a game of poker relatively speaking, probably not literally because you can bet different chips or whatever.

But because of that, it's a self-contained game. So, it's a really good place to think this way about process versus results. The actual world is quite a bit different than that in the sense that the range of potential outcomes are extremely wide. And so, when you predict the future in the rest of the world, it gets really tough. And so so probabilistic forecasting of the future requires you to think in terms of the probable that the not not whether something will or won't happen, but the probability that it will happen, how often it will happen.

And so maybe in that poker hand I said I don't know, maybe maybe that person wins that hand 96% of the time or something like that. I don't really know, okay? But you can't I don't know the math well enough to play much poker, okay? And so they bet over and over and they just happen to hit that 4% that one in 25 chance that that it went wrong, where they should play it that way every time because 96% of the time they're right.

And so they bet and they bet and they bet. Another great example of this is if you ever watch Jeopardy when James Holzhauer was on. He was the guy who essentially broke Jeopardy by basically going for a daily double every time. I'm not going to go through the rules of Jeopardy if you know it, but he realized that the expected value of the bets that you would make on a daily double because of the success rate on getting daily double questions right in Jeopardy was so high and he he in fact was a professional gambler, by the way.

So he thought this way about lots of things was so high that he ought to make it go for a day he ought to first of all hunt down the daily doubles on the board. He ought to do his question and answer selection on the board based off of that and he ought to and he ought to bet really big over and over and over again because the likelihood that he would get it right would win. And he would put up these gaudy numbers, huge numbers of how much money he would win on Jeopardy.

He sort of broke and changed the game by applying this mentality to that and expected value calculation based on a probabilistic outcome, process versus results. And of course at some point maybe he would get the question wrong and it would work out poorly for him, but but he would end up doing okay. All right. So so if you don't know what I'm talking about, go look it up. It's a really fascinating story. Like just Google like guy who broke Jeopardy or changed Jeopardy, James Holzhauer, whatever.

You'll find it, okay? It's that same kind of thinking. And so why am I belaboring this point? Because it is the critical point in thinking about the future. And and that is what your media spend is doing. It is an an to predict the future. And And if you don't understand this, it's it's critical that you do because that is what you're trying to do. You are You are allocating dollars to Meta in this case. Let's call it a thousand dollars, okay?

You're allocating a thousand dollars and we'll call it a thousand dollars a day to Meta and telling Meta, "I believe I can get a return on this money if I give it you you this money." And so So, you're you're ask the predict your prediction of the future is that it will generate more money than it cost you, okay? And And Meta is saying, "We are pretty confident that if you give us a thousand dollars, okay, that we will be able to allocate that money in a way that generates value for your business." If that's true, um if that's true, then what you're doing is predicting the future.

And in fact, every im ad impression is a prediction of the future based on a model. You actually tell Meta a goal you want. Let's call it conversions, purchases. And when you tell Meta that's the goal that you want, Meta then delivers the ad with some kind of probabilistic prediction of the likelihood that you will uh get a return on that ad impression. And of course, it does that at a well calculating the potential cost of that that impression, calculating the potential cost of the conversion, and doing that at a massive scale billions of times a minute, uh you know, probably per second all over the world.

It's a crazy system, okay? But all of it is based on this probabilistic forecasting machine. That is what the thing is. And I'll just tell you, the moment you have given a dollar to Meta to allocate towards something, you have already accepted the premise that probabilistic forecasting of the future is the way to think about forecasting. You've already accepted the the process versus results orientation because that is the engine of the whole thing.

If you think Meta ads is good for your business, it's because you think it can do this effectively because the machine learning is really good at predicting this, okay? So, you've actually already pot committed to the basic concept of that Meta can do this really effectively. Okay, but I want to now show you why I believe that even though your results are volatile, um you are probably misinterpreting that to say that meta is volatile when that is not wrong.

I'm going to give you another illustration beyond poker that is from Kurt Bullock on X. It is the best illustration of this I've ever seen. I've been retweeting it constantly, coming back to it, re-quoting it over and over because to me it so perfectly illustrates the problem at hand here. And the problem particularly at hand is small sample size noise because in any prediction of the future, there's a gap between signal and noise and weird things happen in small samples.

Here's the really tightest, simplest version of this. If you flip a coin three times, there's a pretty decent likelihood that you're going to get heads three times even though you know that actually, let's call it four times, even though you know that that it should come up two heads and two tails because the sample size is small enough, okay? Only four flips, there's a pretty crazy stuff happens. Even though even if you flipped it 10,000 times, you'd probably get 5,000 and 5,000.

In four flips, we all know that nothing would be crazy. You wouldn't suspect the coin of being rigged somehow if it flipped heads three times, okay? Even though it's a 75% rate of heads, which is, you know, 50% higher than the 50/50 rate you're supposed to get, okay? And and so so that's the idea of a small sample size, right? When a small sample size, there's more statistical noise. There's more variance, which is to say more ways in which the results are different than the process.

That's what I mean by statistical noise. Where the results are different than the process, okay? Weird outlier type things happen, all right? The larger the sample size, of course, the less statistical noise there is. That's the idea of the 10,000 flips, okay? It's very borderline impossible that you'll get 7,500 heads and 2,500 tails if it's a like legit coin, okay? And that's because the sample size at that point is so large that it would be an incredibly crazy outcome to have heads flip that much more often.

In fact, at that point you would almost certainly believe that that that heads has gone wrong, okay? Or or that that the tail is rigged and or that the coin is rigged in some way, excuse me. So, that's a critical point. So, um so, this means that the smaller the sample, the more noise it has. But, what many people get wrong is that the small sample it or that there's a larger numbers than you think still make a small sample.

That is, small samples are bigger than you think. What if your customer service software costs could get cheaper by like over 30%? What if your customer ticket volume could go down by like over 30%? And what if your customer service satisfaction scores from your actual customers could go up all at the same time? That is essentially the promise of Richpanel. Richpanel is used by multiple of my clients, uh multiple of other huge brands in the space, and I'm a really big fan of the way they are thinking about their product in relation to how to use AI to make customer experiences better and to make your software stack cheaper at the same time.

That's the promise, right? That's the promise of AI, that it reduces your operational costs and at the same time makes things smoother, easier, and faster for you and for your customers, and that's what Richpanel does. They also promise like no more than 2-week install and like migration process, and literally a 30% guaranteed cost reduction if you switch from Zendesk or Gorgeous. Uh I just love the way they think about AI in general.

I've had conversations with their CEO who has resisted the temptation to over-promise with AI uh with his tool, and I really, really appreciate that because that's what everybody's doing right now. They're all telling you AI is going to solve everything and do everything for you all at once. It's not, but it can do a lot of things really, really well, and if it's uh leveraged thoughtfully and carefully in a great software product, it can do the things that I just said.

You should be exploring it. Um I'll just tell you right now that when my brand launches, we are fully planning on launching with Richpanel uh because of all the things that I've just said. It's really impressive stuff. Go check check out Richpanel right now at richpanel.com. It's richpanel.com. Very simple, just like that. richpanel.com. To get started, just get a conversation going, see if it's going to be right for you.

I think you're going to really love what you see. I have loved it. I've loved working with them. Great people. richpanel.com. And I'm going to put this on the screen now. If you're if you're watching this, you should see um Kurt's chart. There's a whole bunch of numbers here. I'm going to talk you through it. If you're listening, I'm going to explain it, so don't worry about it, okay? Kurt basically um says it like this, okay?

Here's a dumbed-down way to think about it. Here's a spreadsheet where I simulated the rolling of a 20-sided dice 1,000 times a day, 7 days in a row, okay? 20 sides simulates an average conversion rate of 5% on 1,000 clicks a day. So, it's 1,000 dice rolls, okay? Which is like we'll call it 1,000 clicks, okay? It's like again, if you spend $1,000 a day, it's like a $1 CPC. So, 1,000 clicks, okay? And um it's a 20-sided die, which is essentially it simulates a 5% conversion rate cuz each number it has a 5% likelihood of coming up on any given dice roll, okay?

The number one, 5% of the time you'll get the number one, okay? 5% of the time you'll get number two. Now, again, to use the the coin analogy here, uh to you know, the it would be a 50% conversion rate on heads versus tails, but on a 20-sided die, if you rolled that 20-sided die 20 bajillion times, okay? Right, just pick the largest number you can, you would get really, really close to all 20 numbers coming up 5% of the time cuz the sample size would get really, really large.

What Kurt points out is that at 50 like {quote} {unquote} um expected purchases per day, okay? So, 1,000 1,000 um rolls per day, at 1,000 uh at 1,000 dice rolls per day, the actual range of daily outcomes for the dice, and I promise this all going to relate to Meta Ads in a second. The actual range of daily outcomes for the dice is extremely broad. And he just randomly picks the number two here and highlights that and says, "Okay, over 7 days on 1,000 dice rolls, how many times did the number two come up?" On day one, it comes up 57 times, okay?

So, 57 it should the expected outcome the probabilistic forecast here would be that it gets 50 that number two comes up 50 times, okay? But 57 times means that it's actually coming up 14% more often than that. Okay? It's it's a higher conversion rate than than 50. Okay? The next day, day two, it comes up 49 times. But day three, it comes up 33 times. Okay? So, the expected conversion rate, again, 5%. You expect 50 of those 50 times the number two comes up.

It actually comes up 33 times. Again, this is just a random number generator simulating this. Okay? He didn't actually do all these dice rolls. The number two comes up The number two comes up 33 times, not 50. That means it underperforms the 50 conversions we're saying, right? The 50 the 50 times it comes lands on that number. It underperforms that by 17, which is a which is a 34% underperformance. Okay? It's It come up 66% of the time as often as it as it could.

And if you And And then the next day, by the way, the next day it comes up 38 times in Kurt's simulation. Okay? Which is still a 24% underperformance. I am just telling you right now with 100% certainty that basically any media buyer who looks at those two days back-to-back, okay? 33 and 38 like purchases, conversions here, right? On on those outcomes, okay? On 1,000 clicks per day, 1,000 dice rolls per day, whatever.

I hope hope you see the point here. Okay? Looks at those and says, "Something is wrong. Meta is volatile. Something is broken." Right? And on the other hand, if you get to some of these where it comes up like, you know, the the the dice roll number, like the number nine, for example, on day five comes up 67 times. The number one on day four comes up 72 times. You are going to go to your client and say, "Everything is going awesome.

Send it. Push in. Push harder. Spend more money. We are crushing." When in fact, this is just random dice rolls in Kurt's illustration. This is the critical point. The critical point is Kurt's illustration is just about process versus results, signal versus noise, okay? The idea is there is built-in randomness into larger samples than people think, okay? Even when you get up to 7,000 dice rolls for some of these numbers, like one of the numbers um only comes up 319 times in the sample, which is nine 9% below the expected conversion rate of that number, which is to say that even when that sample size gets larger, there's still some weird outlier performance where probabilistically you would expect it to perform quite a bit better.

Now, 9% is not as big as, you know, 35% like I said earlier or whatever it was, but it's still a real number to where at some point if that came up in your meta ads account, you would probably have a conversation say we're we're almost 10% under target. We got to change what we're doing. Maybe make make targets a little more conservative or spend a little less money or whatever it is. Even though, and this is the critical thing, right?

Even though you just roll you just should just keep rolling. You should just You should go and roll. If you wanted a 5% conversion rate, you should roll it that way every single time because you are going to get that over a large enough sample. Okay? I hope that makes sense. The idea that there is random statistical variation in these kinds of outcomes so that even if you have the right process, you get different results.

Um so, if you are analyzing data based off of a thousand clicks and a 5% conversion rate, and of course the lower the conversion rate you you need even more clicks to do this, okay? If you are doing that and you're looking at any set of 50 purchases, all right? Then it's on 50 purchases, 50. You are very likely still or you are in a small enough sample size that the range of statistical variation even on something as controlled as a dice roll.

Again, there's no unknown unknowns here. The mac macroeconomics does not affect the way a dice rolls. Even with something that self-contained, okay? Where there's so little volatility in the potential outcomes, even in that scenario, there's actually a really wide range of potential outcomes. And this is the thing you got to get under your head. The actual range of potential outcomes is really wide even if you do the right thing over and over and over again.

This is um It's and and crucially, what happens if you layer in something like human behavior, okay? What happens if you do layer in uh macroeconomics? And what happens if you also layer in another thing that almost everybody under rates, which is the daily volatility of daily active users on Meta's Excuse me. On Meta's platform, okay? There's actually a really wide range of who uses the platform when and how. And uh what's happening at different times of day and what's happening on weekends versus weekdays and holidays and relative to payday and whatever, right?

There's all kinds of things happening all over the place. And so, if you are not factoring that in and and just again, the baseline level of the unpredictability of human behavior, you are going to get way more noise, statistical variance in the outcomes that you are pursuing than a 20-sided dice roll, okay? And if you are looking at things like 50 purchases, okay? If if 50 purchases, even in that large of a sample, has a lot of volatility.

Like so few advertisers that I see on Meta are are making decisions based off of turning on and off ads and moving creative testing winners into other campaigns, right? There's so few that are making decisions based on sample sizes that large. They are making their decisions based off of much smaller sample sizes. You and I both know that at some point you have looked at an ad account if you have been involved in an ad account and been like, "Dude, this ad has five purchases uh on the first day.

It's ripping." Like I think I did this yesterday. This is part of the point, right? I'm not saying I'm any less prone to this as any body else. Like I I have all the same human biases that everybody else does, okay? So, this is not about good and bad, smart and stupid. I'm saying this is a human behavior issue. I think it does help to identify it and try and get it into your head as best you can, but still a challenging thing.

So, if you, um, you know, yeah, you you and I have both looked at an ad account on based on five purchases seen a three-to-one ROAS and been like, "Send it, Meta. Go scale that ad. It's definitely working." And that's on five purchases. We've We you get to 20 or 25, you know, like, "Got it. That's definitely it." In fact, again, to use the example that Kurt gives, on a thousand clicks, on 50 purchases, there's still an extreme range of statistical variation.

And what I think is fairly clear is that Meta's model is probabilisting probabilistically forecasting based off of a whole bunch of signal that we can't even see and that, uh, is much better at predicting those outcomes probabilistically than you and I are. It is reading all of the engagements on the ad, all the clicks, all those things, and making a forecast of what's likely to happen next, not just looking at these tiny purchases.

Because the thing is, if Meta was only reliant on recent purchase data to make the decisions that it makes, then it would it would also be stuck in small sample size land and it would have very low confidence, um, very low confidence in being able to forecast what happens next for your ads all the time. Right? All the time. But, if it can actually ladder its predictions, its probabilistic modeling based off of lots more signals of the way people are interacting with ads, then it can it can ladder those, uh, that signal up towards the outcomes that you care about.

And that's really important, um, because that's the stuff that you and I are going to have a really, really hard time seeing and weighing properly. It's happening, of course, across multiple different placements, across multiple different demographics, multiple different income levels, all these different things that are very, very hard to see in the data. But, Meta can see all of that and and again, if you've, uh, given your money to Meta at all, you've already bought into the idea that it's really good at sorting out signal from noise and all these different things.

And it's and it can start to push, um, in those directions, okay? And it can start to make probabilistic forecast based on what's likely to happen next. I.e., that is it's not making the decision based off of a few purchases, it's making the decision while while a few purchases are informing its model, certainly, uh based off of much more information than that. That's what makes it powerful and a probabilistic machine that that it is, okay?

So, that is why I am saying that uh the separation between process and results is so important. And um I'm going to give you now some really specific uh examples of what I mean, some tactical examples of what I mean, that to to and some implications for how you should think about this dynamic in your business, okay? Because I I there is a point to this Jeremy that I'm on here, to this uh to this uh soapbox, okay? And it is to try to get you to use your money in the best way possible, okay?

I'm going to give you five of them, all right? Uh four of them are going to be highly specific and one of them is going to be broader, okay? It's going to come back to something I said earlier. Number one. Beware of daily reporting. I actually think that one of the reasons that people make the decisions they do is simply because you manage what you measure, and many people measure based on daily inputs. Particularly if you have smaller spends uh and smaller purchase volumes, daily reporting, and again, small can be like 50 still uh purchases, okay?

So, um but but daily reporting can really mislead you because you can look at a day or two of results and think you got to go change things, when in fact, you don't have to go change things. You should just keep staying the course. There's just a natural variance, natural uh noise in the midst of the signal of what's happening in your business. And uh you can under-react, for sure, but I think most people, especially once you get to the level of daily reporting, are over-reacting, not under-reacting, a lot of the time.

Uh to Now, that um that that could be wrong. It probably varies a lot by media buyer, by agency, by setup, like um yeah, but uh but the the the baseline principle that I want to point to is just be aware of daily reporting because daily reporting are by definition making the sample size smaller. That's the point. And so if you look at your your analyzing small samples and it's also a very arbitrary cutoff, right? It's an arbitrary endpoint of analysis.

There's no reason to pick yesterday's performance as being more descriptive of what's most likely to happen today or tomorrow than the day before in most cases. So you're just arbitrarily selecting a day, a 24-hour period and saying, well, how does that impact the way I think about what happens next? You shouldn't do that. You should take a larger sample in your doing any analysis because instead of arbitrarily picking one day and and and and extrapolate what you think is happening in your business based off of more information, larger samples, etc.

Okay? All right. So be aware of daily reporting and the impact it has. I we do daily reporting for our clients at this point. I have resisted it for a long time for exactly this reason. I just know that people react to the data they see and I'm concerned about it. I've found it mostly useful, but this is still a concern I have about it. Okay? Um number two, don't turn off high spending ads. This implies a lot of things about your media buying setup, but above all, what I want you to understand clearly is that assuming you're running ASC, assuming you're running CBOs, assuming you're doing something that gives meta the power, especially especially if you're running manual bids, assuming you're doing something that gives meta the power to turn off or to spend more or less aggressively based on the outcomes that you are telling it that you want, okay?

Assuming that you have a basic setup that's roughly correct. If that's the case, okay, then when meta spends on an ad in the vast vast vast majority of cases, it's because it is probabilistically forecasting that your ad is going to perform well based on those inputs. Now there's some ways that it can go wrong and I understand that, okay? And And not for a second saying that it's perfect. Okay? I'm not saying that. Um but uh and I'll again I'll talk more about that in a second.

But I do think that one of the number one ways, when I look at ad accounts, that brands are just costing themselves tons of money, is that media buyers see ads that are spending a lot of money, but are like underperforming, quote unquote, uh on ROAS, and they turn off the ad. Okay? There are justifications for this at times, but they're extreme outliers. Like, for example, if you have an a super clickbaity ad, okay? And you know you've built this clickbaity ad, and that ad has uh is getting a ton of engagement, but it's not leading to purchases, it's possible that you would want to isolate that ad or something like that.

But outside of that scenario, or and sometimes there's like a weird delivery mechanism where Meta will get stuck in a loop in delivery ad to like a weird subset of people. You'll see like a vastly like I have one client that has an $80 CPM usually, suddenly it started delivering at a $5 CPM with no purchases, and whatever. It's like it's not a perfect machine, right? So, I turned those off or rebuilt them in a new campaign and relaunched them or something like that uh to try to get out of that.

So, that is such an extremely weird thing, and it was and it was spending enough money that it would be outside of the range of statistical noise that okay, we should probably turn this off and do something else with that. But besides those scenarios, on the overwhelming um and and those are rare. Like, that's not where the reason main that people are mostly turning off their ads. The reason people are mostly turning off their ads is because they see an ad spending, they don't like the results, and they go, "Of course I should turn it off.

The results are bad." But they are stuck in small samples. They're rolling the number two 33 times, even though it should come out 50 times on a set of 1,000 clicks, to use the example I did earlier, okay? And if they just kept rolling it, it would convert at 5%, okay? By the way, AOV's another factor here. It's also possible that people like, for whatever reason, you get a weird batch of very small or very large orders, which is another bit of variance in this.

Um and that that creates strangeness. So, you got to know what your numbers are, what the outcome is that you want, and this is why I I a manual bid is so helpful. Let's talk more about that in a second. But in general, if you're doing anything to allow Meta the freedom to scale your best ads, um even if you're auto bidding, uh and if you're telling it to scale your best ads, if it is spending on that ad, the rule of thumb should be that unless you have very, very strong evidence otherwise, you should let that ad spend, okay?

All right. And again, assuming you've paid attention to unit economics and all those things, okay? All right. Number three, don't run creative testing campaigns where you analyze the outcomes of your creative testing by purchases. If you are looking at a creative testing campaign and saying an ad got five purchases, I'm going to move it to my scale campaign, or an ad got one purchase on $1,000 spent or two purchases on $1,000 spent or whatever, right?

I mean, let's call $500 spent, and you're like, "Ah, turn that ad off, it's not a winner." I mean, you you are costing yourself tons of money. You just are. You just are. And it's because uh it it's because there is simply no way you have reached large enough samples to do this. And this is why I say almost nobody is doing real creative testing in their ad accounts. They're just not, because you have to spend too much money uh to uh on your ads.

You have to spend too much money to get purchase-based results. Now, if you are analyzing, I've seen some people talk about doing creative testing where they they only analyze um upper-funnel metrics like uh CTR or hook rate or something like that, because they think that ladders to better performance, and they move those into winning campaigns. Um some people I've seen talk about the idea of just using it to force some spend to some ads at a baseline level, cuz they're really concerned about Meta turning off an ad uh too early.

Like, I can see some justifications for doing those things. I still I still think it's not really the way to do it, and doesn't make much sense, but that's fine. Um you know, there's there's there's worse ways to go about things, okay? But But on the whole, if you are launching creative testing campaigns and analyzing the ROAS of those ads, you are never going to get to a point where you have a statistically reliable sample of ROAS, okay?

Of of of purchases without burning like hundreds of thousands, maybe millions of dollars on those creative testing ads. You just aren't. Um and and the notion of doing that that way, I think is so fundamentally flawed because people are constantly not not being able to see the difference between signal and noise processing results in this situation. Um creative instead, just launch all your ads, as I always say, in manual bids.

We we've played around some with like having a separate creative testing campaign as opposed to a scale campaign, where um it's isolated new testing cuz there's some evidence and some did um framework from Meta saying basically like launching an ad against in an ad set or in a CBO with a whole bunch of winners already in it is maybe bad for getting spend to that ad. Um but we when we do that, we do it with a man the exact same manual bid as we do on our scale campaign.

So, we're still telling Meta it has to achieve a certain performance level. You know, if you want to do that, that's fine. I'm I'm not saying I don't really care about like the test versus scale setup. What I care about is wasting money. That's what I care about. And when people are forcing spend and doing bad analysis and rating the performance of their ads based off of um based off of creative testing campaigns, that's where I think it's a real problem, okay?

All right. Uh number four, do trust manual bids. Manual bids, bid caps, TROAS, cost caps, those kinds of things, are in fact just ways to take the probabilistic forecasting engine that Meta has, which already is forecasting like what your bid is going to be in an auction. It cannot deliver your ad in an auction without doing that, and just saying, "Don't spend if your forecast for that bid is too high or too low or whatever it is, okay?" Um and and and that's it.

That's the whole point. You're just you're just putting a governor on your spend based off of its probabilistic machine. Now, um I don't think these are perfect. I don't think these are perfect. That's because nothing is perfect. Nothing is always right. Always right about the future is impossible. What I think is they are more right than you or I are about what's most likely to happen next and how to allocate your spend.

And therefore, it is a probabilistic approach to the problem of how to deliver your ads. And that's what I'm getting to. Uh is playing that hand that I described in poker earlier when that when when um when those cards come up every time, even if sometimes it's wrong. Now, at times it may have some false negatives, okay? But that's what I think you should be doing. I think you should play it the right way uh or you should play the hand that way every single time because that is the way to maximize the probabilistic uh machine working on your behalf.

Uh so, yeah, run manual bids. They work. They just do. They work. Uh now, there's some range. Meta has the ability with cost caps and target ROAS ads to bid dynamically, so it can actually go well above your bid or well below your bid as the case may be, above on the ROAS side, below on the cost cap side. Uh I mean uh no, I think I have that wrong. Anyway, you get the idea. It's to bid sort of more aggressively than you have until it finds sort of what the right bid range is.

Um I think that has some upsides and some downsides. Bid caps it doesn't. Bid caps you just give it a threshold and that's that. But that's the basic concept, okay? Um the basic concept is that is is using the probabilistic machine to deliver your ads and that and that's the idea, okay? Um Okay, number five. And this is to come back to the thing I said earlier. Meta is not volatile. This is what I mean. Meta is not volatile.

The future is and humans are, okay? So, when you say Meta is volatile, you're putting the blame in the wrong place in most cases. What's actually happening, assuming you have a good basic media buying setup and you you listen to my podcast or watch it and done what I'm doing, I hope you do. I think you do, okay? Um what I'm saying is future behavior of people is unpredictable. This is what happened with my account earlier that had a 40% spend spend decrease day over day.

I think the very most likely explanation of what happened there is that there were much less daily active users who were likely to buy or people were overbidding them that day or something like that. It was a random weekday. Maybe everybody was at work. I'm not sure. Everybody's Nobody's calling out sick that day cuz they're all going to go to Memorial Day weekend in the next weekend and take a four-day weekend then or something like that.

I don't know, right? Um and so for whatever reason Meta didn't see that opportunity. And so it looks like Meta is volatile, but actually what's happening is that human behavior in the future are volatile. The The future's fundamentally unpredictable. Sample sizes are small even as they grow. They're small for longer than you think. There's more volatility than you think. The dice roll illustration is what I'm getting at there, right?

There's more volatility in over more time periods than you think. And therefore, the point is not that Meta is volatile. The point is not not that Meta's doing something wrong. And when you see anybody tweet something like, "Man, Meta's great today." or "Meta's bad today." or "It's been good for a couple days." or "It's been bad for a couple days." they are all falling prey to this kind of noise-based problem. They are They are falling prey to the idea of overextrapolating from a small sample nearly all the time, okay?

And there's all kinds of external factors for why that might have happened. And probably none of them have to do with Meta being volatile. What they have everything to do with most likely is the realities and the vagaries of these changes in human experience and the the the inability to predict the future exactly. Instead of thinking that Meta is volatile, think that humans are volatile and therefore have to take a probabilistic approach to the future.

I'm going to trust machine learning to know that better than I do. I'm going to use that to my advantage. This is incredible machine thing that Meta has built and handed to advertisers and say, "Here, grow your business with this machine we built that probabilistically predicts the future, okay?" And then watch how it plays out and watch what happens. And then take that and watch what happens in the rest of your business.

Start trying to to have a process over results approach. When you do your influencer seeding, individual influencers may or may not perform well. When you do your affiliate program, individual ones may or may not perform well. But you could create a probabilistic expected value outcome based off of 500 seated influencers, 500 affiliates, and something like that and have some idea of what every affiliate or whatever influencer is likely to be worth you, even though the actual tail of distribution is really long in terms of in terms of how different people perform in that, okay?

And it's just a game of getting numbers because some people are going to drastically outperform the rest. Like probably how MLMs think, right? A few people crush everybody else, but they have outsized value. It's the Pareto principle kind of idea, okay? And so what you want to think about is how do you play the best hand the most often and leave the results aside within broader frameworks assuming that you aren't tanking your business or something like that.

You may also want to think about which side you're willing to miss on, right? So you may need to be more conservative and you may want to be more aggressive based on the economics in your business, based on where the risk lies. Like there's some businesses, if you're a supplement subscription business, you're almost always at more risk of underspending than of overspending, okay? Um if you're a business that captures all the value from a customer up customer up front, you're self-funded and you don't have that much cash in the bank, you're more at risk of overspending than of underspending, okay?

Um and you probably should think about all those things that way. Your goals play into this, but ultimately you want to separate process from results as best as possible. Play the best hand the most often and the results will come after that. If you do that, that's how you win the game. Thanks for watching or listening. As always, you should subscribe wherever you are doing that. And while you are doing that, you should also go to ajfgrowth.com, subscribe, enter your email address in the pop-up or in the footer.

You will get my four free essential e-commerce resources. That includes like cohort forecasting template that I use, my unit economics sheet that I use to calculate the profitability of each product that I'm working with uh for each brand that I'm working with, and my my weekly reporting document that I use with my actual clients. It's been a really helpful document for me. Uh some stuff there. It's going to be really helpful to you.

Totally for free and I'm not going to spam you with email, I promise. Uh thanks to my sponsors, Richpanel. Save 30%. Uh reduce your total number of tickets, increase your customer satisfaction at richpanel.com. Go to who else? Intellgems on this episode. Intellgems.io Ferris 20 gets you 20% off when you work with Intellgems. 20% off for the first 3 months for CRO testing that is really built around profit maximization.

I love Intellgems. Really excited about continuing to work with them. And like I said, subscribe. You know what to do. Email me podcast@ajfgrowth.com. You know, I don't need to say anything more. You know what to do. I'll talk to you next time.