Cost Caps Spent At 20% Over Target. Here's Why. (With Actual Ad Account Screenshare) transcript

all right so I'm looking at an ad account here where uh there's been a consistent uh actual CAC above the cost cap this is one of those things people are really frustrated by the Run cost caps you're supposed to get the ca that you want it's just not working quite correctly so you got a $70 uh CA here against a $60 cost cap uh in the ad account um there's some details here that are a little bit hard to sus out but last month the the cap was changing a bunch in the previous month so this $77 for the previous month maybe makes a little more sense at points in the month the the cap was actually set a little bit higher um but if you look at this month the same thing you know last couple weeks basically um you see you know really for most of this time that cap that number is $74 for most of that time we can just check the bid was you know around $60 at different times uh at times it was 65 a little higher than that as you can see so um so you know it had a little bit more room uh earlier on and uh and but earlier in the month you know you're at $57 you can see the Caps here between 60 and 6 5 over that time as well so it was working much better then so um but you know for the last couple weeks at least it's just not doing what it's supposed to be doing it's really frustrating when this happens so the first thing I I'll note here is that for this brand there is no weird uh overspend day there's no single day throwing this off I've gone and look day by day I won't make you walk through it but it's not like it's like oh it's spending $2,000 a day but then uh one day it spent eight and it was a really bad 8,000 it throws off the average there's nothing like that going on here it's just sort of consistently missing and so what's going on and this is really frustrating when this happens I totally understand it makes people very frustrated with the tool it's not missing by like an insane amount but it's missing by you know 25% here on on the ca Target so it's it's a big deal and um and and so what do you do well um I think there's possibly two things going on one of them is that for some reason the the the basic probabilistic forecasting that is driving cost caps as a tool is not working correctly uh some data input when you say like not working correctly it's just important to know it doesn't mean that something is like broken like the tool broke somebody broke it I think uh the much more likely explanation is that some data is it has been fed is throwing off the forecast and so um that's one possibility for what's going on here sort of in a general sense the data is doing that um but I'll tell you what I think is probably more likely going on here and that is this adset is actually a DPA adet a Dava adset so it's a huge bunch of products all mixed in and if we go to the last 30 days what you'll see is that there's a really wide breakdown of products um so if we go to whoops uh delivery product ID there we go what you'll see is that over that time period the highest spending product is actually only at $4,400 so um so one product is getting you know what a little over 5% of the spend or something like that um and then down from there right and if you if you actually look at these like last seven it's these products here that are spending a whole bunch more but actually earlier in the month let's see it was not those products it was different sets of products there's some of them maybe in there linen relle linen let's see yeah these are different and these products are also um on pre-order and so uh so far as I can tell from the ad labeling and so if you look there's all these products and so what you're actually asking meta to do here is um across all of these different products you're asking meta to predict a with reasonable accuracy a conversion rate um and and really each of these products is like its own ad and its own Lander right because it's a different product along the way different price points for all of these products right I'm I'm assuming that crop tanks and miniskirts and dresses all have different product Point uh have different uh uh uh holy cow price points uh and so so what I think is basically Happening Here is that you're actually asking meta do something kind of impossible and if the product set is changing all the time then it's um going to be especially impossible normally when you're running caps right you're running at the adet level with a set of ads where you're getting consistent outcomes over and over and over again but here it's just really really different you're actually there there's all of these variables going on and so by Nature by definition the probabilistic forecasting mechanism is going to create it's going to have wider error bars because it's way more things to predict in one one place and so um the way I would think about how to solve this problem probably would be to take some of my top products separate them in a separate ad set and set a cap on those uh differently or maybe even simpler now this this wouldn't allow you to use shops but would be maybe to run highest value ad bids because then you should get a range where there's a more aov based kind of thing now in that case it's still the same problem you're going to be asking it to predict a lot of different order values over time so that might actually not work very well it's probably not what I would do what I actually would most likely do is just change my cap relative to the actual results I'm getting so if you're getting a $75 CAC and you want 60 then for at least a little while I would as long as the product set staying relatively the same i' drop it down to 45 or 50 or whatever and see if meta will get you 60 by doing that um and if you're going to have more consistently the same products in this product set then then you would expect that maybe um it would sort of stabilize a little more you could raise it back up um but yeah probably the most likely thing here if you really want control you really want to create the same Financial outcome over and over would be to take products that you know are going to be in this product set for a really long time put them in their own ad Set uh because now you're going to have the same kinds of traffic going to the same kinds of landing page and in that case a a PR model may work better at predicting this but I think what you're doing here is basically say uh uh saying predict all these things for us please predict all these things and uh and it's just uh too many things to predict and and so sort of an impossible task and so you could sort of manually adjust the cap relative to the outcomes that you're getting and see if that changes things again especially if this product catalog is changing a lot which is usually the case in apparel Brands so that's my assessment of what's happening and what I would probably do next um I don't think cost caps are broken here I think it's just uh the challenge of using the tool for this particular outcome