How Meta's Forecasting Makes Bid Caps & Cost Caps Actually Work transcript

All right, I put together some quick thoughts on how bid caps and cost caps actually work as best as I understand it from a combination of what I've seen in the past plus Meta's documentation on this sort of thing. And I think it really helps to understand the mechanisms that are powering manual bidding so that you can understand why they work so well most of the time, which they do, and then why occasionally there is a challenge uh with a particular ad or particular ad set that that kind of breaks the model.

So, um basically, if you start at the top here, what powers bid cost caps and bid caps is two basic things, right? Which is ECTR and ECVR. ECTR is the expected click-through rate on an ad. Um now, as once you start spending on an ad, impressions of the ad are very, very cheap, and so you get very quickly, you know, hundreds of clicks on your ad, and therefore Meta can update its expected click-through rate on your ad incredibly quickly.

It'll It'll understand very, very fast, even even distributing the ad across a lot of different placements, it'll understand very, very fast what click-through rate can happen or what quick click-through rate is likely to happen. Imagine um a coin flip. Imagine you don't know how many sides are on the coin, uh right? You You have some kind of weird coin, and you just You can't tell how many how many sides it has, okay?

Um and so maybe it's a maybe it's a a weird three-sided coin or something like that. You just have no idea, right? So, you start flipping the coin, right? If you flip the coin six times, and you get three heads and three tails, it's probably a two-sided coin, but you don't have enough data to know that it definitely is. But imagine that you flip that coin 150 times. If you get 150 uh flips, and you get some measure about ha- about half heads and head and half tails, it's very, very close to that.

Now, uh you know, and zero of a third thing, um right? Now you know pretty for sure that you have a two-sided coin, and that most of the time, 50% of the time, roughly, it's tails, 50% of the time, it's heads. You'll get very, very close with a set of 150 coin flips um to 75 and 75. And if you get a thousand, then you'll really know. And so, you can update with a very high level of certainty what the expected outcome of of a coin flip is the more times you flip the coin.

And so, because um every ad impression potentially gets a click, Meta can update the expected click-through rate on an ad very quickly because it gets so many clicks. It spends money so fast and and go on. So, the ECTR, I think it's very easy for Meta to to predict. But, the tricky thing is um ECVR. Now, that number gets a lot smaller, right? Imagine in this case, uh for every 100 coin flips, right? You get one purchase.

And now I'm breaking the analogy, but hopefully you get the idea that you have to get a lot of clicks for Meta to understand how many conversions you're very likely to get. And so, the ECVR, um if if you were to just run a um if you knew nothing else about this coin or if you knew nothing else about what was going to happen, then you would have to start from scratch with this information and say like, "Okay, we need to build a model of ECVR." And a lot of people actually think this is how bid caps and cost caps work is that like you start getting conversions, that informs Meta, and then Meta updates its model in real time, starting from scratch, and then going from there, and going like, "Okay, now now Meta can predict how many conversions you're going to get based off the clicks because it starts from zero, and as it starts getting conversions, Meta learns, "Okay, you're going to get a 2% or 3% or a 4% conversion rate or whatever it is." But, to start, Meta knows nothing.

Okay? Now, I'm going to tell you why that's wrong in a second. But, the basic idea of what is informing these two things uh what's informing the bid cap and the cost cap is ECTR and ECVR because the expected CPA, which uh cost caps and bid caps have no reference for the value of the conversion. They only have reference to the cost of the conversion. That's why you set it as a lowest cost or highest volume bid as Meta says, right?

Um and in fact, auto bids do the same thing. Auto bids are are bidding for the least for uh for if you run highest uh highest volume or if you run lowest cost bids, it's just trying to get the lowest cost possible bid. Lowest cost possible conversion based on the information that it has. And the way that you get that, right, is a really simple calculation, which is the amount spent times the conversion rate, okay? And so or the expected conversion as the case may be, which is the expected CPA, right?

So if we just quickly look at this and said $1,000 spent times 1% conversion rate, okay? That's going to get you a $10 CPA, okay? Or if we do it like that, now you can see here, let's see, $1,000 spent times 1% conversion rate um equals uh well, anyway, it's not going to show it as a dollar amount, but you get the idea. 1% of 1,000 is $10. And so that would be in a $10 expected CPA. And what Meta is doing with its bid cap or cost cap is building in an expected click-through rate, so it knows how many clicks you got, how many of those clicks converted, and then how much you spent, basically.

And with that information, it can predict what the conversion rate actually is, all right? And um and so as it as it predicts that conversion rate uh you um then from there then from there it can then say here's how much money we can spend because here's how many folks with our expected conversion rate we expect to convert on your ads. And this is if this and if you set a cost like, "Hey, keep the conversion rate at $15 or below." and it knows that it's got a 1% expected conversion rate on the next set of ads that you run and you spent $1,000, well, it's $10, so it can keep spending.

Okay? So that's the basic idea. Um so there's that, okay? So but the question is where do the ECTR and ECVR come from and where does this thing get powered, okay? And here here's what I think is is happening, basically, okay? Meta, and I've written some stuff out here, but I won't read it all. Meta has at the Meta can read the engagement of an ad very very quickly. It gets signals for how users interact with the ad in platform.

And those signals are really strong. So I've given some examples of this, right? If your ad gets a very high volume of shares relative to the impressions that it gets, then your ad that probably sends some kind of a signal to Meta about the value of your ad and about what is likely to happen next at the purchase level. It can read the engagement signals and then forecast from the engagement that's happening what's likely to be happening at the purchase level of the ad.

That That is basically a built-in forecast so that as your ad serves to users, it doesn't actually wait to see how many conversions you get and this is the key distinction. It doesn't wait to see how many conversions you get to decide what your expected conversion rate is. Instead, Meta uses its giant boatloads, truckloads, incredible, like virtually, literally limitless data set and always expanding data set of the relationship for the optimization that you've selected, in this case conversion, for the relationship between ad engagement in platform and the likely conversion rate that is relative to happen that is likely to happen on your website based off of that behavior.

And of course, it's updating that based off of the information that it has from from your ad account as well, etc. Okay? Um but all of that information informs for Meta a forecast so that when you get to the level of serving the ad, as you get shares, as you get likes, as you get comments, and I think as you get people stopping scrolling and just looking at an ad, like if you just interrupt their scroll or watching a video for 5 seconds or something like that, those versus and I bet you I bet you Meta has a forecast that changes if somebody watches 5 seconds of your video versus 4 seconds of your video or something like that.

Like this is an incredibly well-tuned um algorithm and it probably it also has all this other user information about if this person is like the right kind of person to buy it, right? So, it's they also can tell like based on who the user is if they're likely to purchase based off of their other behaviors and based off of what else they're looking for in market, all of these kinds of things. So, it has user information and it has this boatloads of of engagement information that can inform, roll those all together, in this incredibly complicated algorithm, machine learning, whatever, um where it's going to take all of that information and say, "Here is what we expect to happen next at the conversion rate level." Now, the whole thing that makes Meta so powerful is that this is such a good forecast, right?

But, those that idea that Meta has all this information that it uses to predict your conversion rate before you ever convert or don't convert a customer, right? On that ad, on that ad. Is called is the notion of priors, okay? In in Bayesian probabilistic forecasting is the idea here, right? The idea is that Meta doesn't come in to the Meta doesn't come into your ad account blind and say, "Let's see how many coin flips we can get and then predict the conversion rate based off of a bunch of coin flips." Meta's saying instead, "We know what coins are like.

We already know. So, we before you ever flip the coin, we can just tell by looking at it and holding it and seeing what a coin is like, what the expected outcome is, and we'll go from there." Now, in the vast, vast majority of cases, this works incredibly well. It works incredibly, incredibly well because the data set is so freaking rich. It has so much information, etc., that it can do this really, really well. But then, what happens is those priors, those assumptions that that inform this EC ECTR and ECVR forecast, those assumptions then get updated at the level of an ad set.

And this is where Meta tells you that that this information gets stored at the ad set level is the notion of the learning phase, etc. Um so, the ad set um at the ad set level, it updates your it updates the forecast relative to the particular ads that you've got in that ad set. And so, Meta starts with those very broad priors to inform its forecast and can do very, very well with that. But, what happens if your ad's actual conversion rate is wildly different than its priors, okay?

What happens if you get all of this user engagement at the ad level, but then your ad doesn't convert very well because something crazy happens on the website or whatever, right? Well, for a little while, your ad's probably going to spend too much money. That's probably what's going to happen. Now, the thing is it's going to update that information over time. The The actually The example I see most often here where this kind of breaks is you get um you get a an ad that gets a huge amount of engagement on it for some reason that's unrelated to purchase activity and that engagement signals intent to Meta and therefore Meta starts spending it even though you're actually not going to convert those clicks.

So, an example I've seen of this recently is an advertiser who ran uh who ran an ad that unwittingly, they had no idea that in the ad and in the product there was something that was a signal of uh potentially of related to Palestine, okay? And so, the ad um the ad started getting all this engagement around the Israel-Palestine uh war and because of that all this engagement came in that had nothing to do with people trying to interested in purchasing the product or not.

It just was a whole bunch of engagement on the ad. Uh and therefore, the ad ran and it didn't convert very well. Now, it didn't convert nothing, right? Like it it started spending a bunch of money cuz Meta goes, "Oh, look at all this engagement." Starts spending a bunch of money, but it didn't convert very well at all and therefore um it overspent for like a day, you know, not not for super long but for like a day, okay?

Um now, I have done this enough to where I could see that that was basically what was happening as best as I could tell and and made a simple decision here which was to to pull that ad out and in fact we we ended up killing the ad because we were not trying to get involved in that conversation, right? But I've seen this in other places where we just take the ad, put it in its own ad set, reduce the budget, and basically give Meta time to do what it does which is update its ad set level metrics.

And then over time, if the ad um starts to perform better, then maybe we start to scale it and make sure that everything is working, etc. Now, this is risky. I don't really like to use my judgment in favor of probabilistic forecasting or in place of probabilistic forecasting, but there's times where I think qualitatively you could probably make a best guess that that's the right decision, okay? So, that all happens, okay?

Um and then what happens over time, right? So, even this ad with this with these potential Palestine comments, if you um if you think about that, given enough time, Meta will update its priors at the ad set level. So, it will figure out that the conversion rate is lower and it will stop promoting the ad. It will stop spending quite so much on the ad. Um and that's really um significant because it doesn't do it that fast and mostly that is a very very very good thing for you.

You actually don't want Meta to update its priors too quickly because the whole thing that Meta does that's so good is that it doesn't overreact to statistical noise that happens at small samples. So, let's go back to our coin analogy, right? Imagine that you have a coin. Meta knows it is a coin. Meta can read that it is a two-sided coin and you flip that coin and you get six heads in a row. It's a very small probability outcome, but it can happen, okay?

You get six heads in a row. The thing you don't want Meta to do is update its forecast so that the um expected heads conversion rate is any different than 50/50 on the next one. You actually want it to stay assuming, okay, that the that it's 50/50 on the next one and to update its priors very slowly, okay? So, that's a feature not a bug. It's a really really good thing because the next 10 flips, very likely, it's 50/50 heads and or five heads five tails or at least it's likely it's possible and the next thousand flips um it's almost definitely going to be right around 500 heads 500 tails.

So, you don't want it to update too quickly uh because of that. Now, if it flips the coin, you know, 20 times and it's 18 heads and two tails or 18 tails and two heads, then maybe Meta starts to go, "Hmm, maybe there's something weird with this coin and maybe this coin doesn't act like other coins for some reason. Maybe it's weighted funny or something like that, right? Something about it is different." And then it might it might move its prediction now to let's say it's 18 18 tails and and two heads, maybe it will expect 55% tails going forward, okay?

And let's say you flip it another 20 times and you get another 18 tails and another two heads. Now Meta's going to keep updating its forecast until you get to, in that case, 90% tails, 10% heads. Okay? Um and so So, that's the idea. And so, the place where this gets thrown off is these engagement signals. And the engagement signals are really, really significant. So, um that's the basic idea of of how these caps work.

And this is the reason you can trust them because on the whole, it has all this It goes into the It goes into the ad bidding process and into the auction with all this information. All this background information, you want to use that. And you and I are very likely to overreact to small samples, right? This is why it's actually kind of a little dangerous even to like assume that the Palestine comments in that ad were actually screwing up the engagement for it.

Maybe. Maybe not. Now, you you may change your approach to this depending on, you know, sort of your aggressiveness or conservativeness in general. Um I would almost never shut the ad down. I would maybe lower the bid cap, put it in its own ad set, you know, tighten the budget, do some things that if we want to validate um that the spend can work, then we can do that. Or if it's, you know, it's just a global conflict with very uh strong opinions, then maybe you want to stay out of it, you know, who knows.

Um who does an ad brand trying to trying to sell a product that you didn't mean to have anything to do with that, right? So, it's a different conversation. But, that's the basic concept. Um and yeah, hopefully that helps you understand it as best as possible.