Do Cost Caps Work? Taylor Holiday Brings $200M Of Data transcript

You want to start on CTC data reveals? We got some spicy media buying data. You have actual data driven answers today to is it okay to turn off an ad? Yep. And to do manual bids to work. Yeah, yeah. The two most most discussed topics in this room DGC Twitter. If you were like here's like forget it. I'm I'm done arguing. We're just going to go find the answer. Yeah. Do you want to start there or do you want to you want to yell at me for being bad at business first?

What do >> No, no, no. Um we could start with that because I think I think it's our Give me your data. The research cuz I think I think this is a good one for the two of us and I I was why I even think it's a good place for us to We have been waiting to publish this, but I think, you know, you've you've probably, I'd say, taken over as the cost cap maxi lord. I think so more so than me. I think so. I think I've backed off of it so much cuz I just hate the argument so much now.

I I'm getting there. I dropped the flag and I think >> The reason why is I'm so close to accounts that I see like it just still makes me so crazy to look at somebody's business and look at the dollars being lit on fire. And it just breaks me. I don't actually care about what is the best media buying method. That's not what I care about. What I care about is how to grow your business without lighting money on fire, you know?

And I think it's just a big thing that The actual position I hate the most is this doesn't matter. Unless there's a data driven reason to say it doesn't matter. Right? Then it's like fine, but uh but like this doesn't matter like go make more creative is just like I don't understand why we can't care about both. So, anyway, you're right. I do think I've been become that to some degree. >> And I I'd be curious like what do you think the industry disposition is?

Now, I feel like the Overton window moved a lot in our direction. Like most people conceded eventually to the idea that the manual bids >> Yeah. >> Uh I actually don't know if I think that. I still see some pretty loud voices. I think the auto bidders stopped arguing about it at some point. >> Okay. Um well, so what I want to say is that uh one of the things I try to do despite maybe my more dogmatic public presentation is that I just really want the idea to work.

I don't actually care about the merits of the idea. Like I don't want to ruin people's ad accounts if it's a bad strategy to do that, right? >> Exactly. Exactly. >> yeah. So, the the the the way we got to this was not to begin with dogma. It was to begin with like, "Well, what would be the best way to protect their money and to >> That's exactly my point. Is that that's the reason I care about it, too. Is like Yeah, it and yeah. >> So, so with that, like I I think that one of the experiences, one of the things we've been struggling with cuz we've been working really hard to try and we call it the canon, like operationalize a defined set of ideology further than we've ever gone before. >> Yeah. >> And you get into really hard questions about setting bids. >> It's funny.

I'm I'm in a stage at Ahrefs Growth right now where like our big goal right now is document more things. >> Yes. >> Make them more repeatable. >> Yeah. >> Um and the thing this one of the struggles I have with it is that in the act of writing something out, >> Yeah. >> you start trying to explain what to do in a situation, and then you start thinking through if you've got real experience and thoughtfulness here, you start thinking through all the details of what you would tell somebody to do, >> Mhm. >> and then you're like, "Well, wait.

And And now you get to like you get to details that you have to make a you have to put you have to put your pen on paper on, >> and there and I know when I'm doing it that I'm I'm more or less confident about parts of that at a given time. So, it's just really it becomes really tough and where I'm like, "Oh man, I wish there was a study. I wish there was a piece of data." >> And you you force yourself into the further you go in this, the more you encounter that exact feeling you're describing, which is why do I believe this?

Like and how would I actually define it algorithmically such that somebody else could apply it consistently as well. So, when we say like even something like use manual bids, well, there's three different types. >> Yeah, which ones? >> Right, which ones? And then, okay, let's say I pick one. How do I set the bid? >> Yes. >> How do you build this count structure and all of that? And what happens if it fails? Well, what does it mean to fail?

Like you start pulling on the threads and you realize there's so much language that we use that is actually absent of any meaning at all or at least any shared meaning. >> The other day I told somebody in my company okay, you have to recognize there's going to be a larger amount of noise in smaller sample sizes. This is the the the coin flipping analogy I use all the time, right? If you flip a coin three times, it's very possible you're going to get heads all three times.

That doesn't change what you would expect if you flipped it 3,000 times. You'd expect 1,500 1,500 basically. Right? So, you need to build a larger sample size. Okay, how large of sample size? >> That's right. >> Right? So, like when I tell a media buyer, you should care about, you know, before you change your manual bid, whatever, how you know, it's just and and like when I answer that question, I'm like, uh 50 purchases. >> What? >> That seems reasonable to me.

And so, that's what you have to do. The the position I end up in is saying make a decision that you think is reasonable because you're not going to get the perfect truth on all of these things all the time. But then but then over time, my hope is that you could be more reasonable. So, you guys did two studies on this stuff. And I was I when you told me you guys did these studies, I actually had two reactions and I think they're both they're both interesting.

One of them is I was like grateful. I was like my my one of my thoughts was like, oh, this is what an agency should be doing. >> Yep. >> Theoretically, an agency the whole value proposition is that we have more access to more data than anybody else. But a lot of agencies are actually not providing that good of a service because they never do the work of trying to actually operationalize thing or even worse, they say it's different on every account, which is the worst possible thing you can say.

So, I that was my first thought was like this is like why I still look at CTC as the gold standard in the industry about like what it means to be a good agency and why I happily refer you guys business when I can't service it and when it's or just when you guys are a better fit or whatever. But then secondly, another reaction I had that I think is worth noting before we roll out these studies. One of them saying does do cost controls work, the other one saying should you ever turn off an ad?

Getting data-driven answers with real studies across billions of dollars spent on this, okay? Um I realized something before I looked at this this stuff you sent me was I thought, "What if I am wildly wrong?" >> Yeah. >> Have you Did you think about that when this happened? >> For sure. >> Because you you and I both have had have made serious public hay out of both of these positions. >> Yep. >> And I have like it you like you said, it's like become a thing.

Like people associate me with me, the bid cap guy, you know, or whatever. Um and so even when you told me like bid caps versus cost caps versus tROAS, like you I have made the statement a lot of times that I prefer bid caps to cost caps. Now, I've always tried to hedge that a little and say like it's actually not the biggest thing I care about, but I had this thought of what if I'm about to go on this podcast and a large data scientist say, "You are actually really wrong about this." Did Did you Did you have that experience with this data at all? >> So there's a weirder thing.

In some ways, I feel freer than I ever have about the outcome. And like And I don't mean that I don't know how to make that not sound lame, but like I'm post concern. Like in some ways, that's part of >> because you sold the business? >> Yeah, like like and literally like if I'm wrong, the cost is actually almost negligible. Like I If If it turns out that highest value is like then uh okay, but but it's sort of beyond the pale of direct life impact where the thing I'm risking is like not that great, you know?

Um so that then that's could could just be a total narrative structure in my head more than it's anything pragmatic or real. Maybe the cost is devastating, I don't know. But I just don't feel that concern. It's like even the way I feel like comfortable talking to customers now is this sense by which I just feel like I can tell you what I genuinely believe and devoid of the constant concern that I know I carried for so long about what it might cost me if I do it, you know? >> Yeah. >> Um >> Yeah. >> So that's just like a little thing.

But also, I just genuinely care about being more right if I can be. >> So, okay. So, so, so I do too. And I I This is This is one of my things that I thought about was like, well, if it turns out I'm wrong, then I'll have definitive evidence and I will adjust my position accordingly, which I've always said I would do. That's right. Quick interruption to say you should subscribe wherever you're watching or listening to this video or this podcast ad.

Subscribe. If you like this content, you're going to like all my content, I promise. All of it, 100%. That's probably not true. There's probably something you won't like, but you're going to like a lot of it. Subscribe. I have Taylor back pretty regularly. Subscribe. Go do it right now. And then maybe share this with a friend, like it, do all the things that help me, you know, expand my reach. That's what I'm really asking for here.

Help me. Thank you. Subscribe. That said, I would feel a little bad about having like maybe misguided people. Though I I think the principle Anyway, actually I found myself doing that a lot, which is like I I'm As I started thinking about the possibility this would reveal me to be very wrong about a lot of things that I've been public about, I I started trying to figure out why the things I was saying were actually right, you know, even if they were wrong.

Like I started kind of playing like spinning them positively in my head. Like, oh well, it still would have been good to like think financially about your business, you know. >> You have to think that I've also navigated this feeling a ton as it relates to the other parts of my life, where I feel like I did this for a long time, right? Like so so >> Oh, interesting. >> I already worked through the idea >> Can you tell people what you mean? >> that I misled people down the path of choosing a religion that I no longer ascribe to and I like vehemently attempted to get them to commit their life to it, right?

Like so, I think there's lots of ways I've worked through that feeling a ton for myself. >> What did you answer? >> Again, it always goes back to this idea that in the moment, was I acting in integrity with what I truly believed? I think deceit would be I didn't believe it and I didn't do it. >> And so I don't worry about that. >> Yeah, this exactly. So, to me I feel okay that what you can expect from me is that you will get an earnest attempt to present what I truly believe in the present moment and that given new information, I would readjust my position to you.

That could mean like what I tell you is wrong and you should filter that accordingly, which I think we all hold that obligation ourselves anyway, that I receive anybody's information with the obligation to verify it and fact check it for myself. >> This is actually This is actually one of the things I've thought about here. Even when I refer business, I feel a little bit of that. I'll refer business to you guys or to somebody else or whatever. >> The outcome. >> And then I feel a little responsible if that experience doesn't go well, and then I have to actually release that and say that's actually I can make I can make the best referral I can given the information that I had. >> That person still has to make the choice. >> They have to make the choice.

And sometimes I'll even say to them like this is as far as I know, but listen, I'm not in the day-to-day of their business. And so you should still run a good vetting process for this, and I would encourage that. And I I think that is right, which is that you speak the truth as best as you can, and then you adjust. But the interesting thing also about that, what you're saying, is that uh when you are in count Like this is a real organizational problem.

So So first of all, if you were to give me this study >> Mhm. >> and then I like denied it >> Yeah. >> or if whatever, even though I was convinced it was true in order to keep doing my thing, that would be like really bad. >> Yeah. >> Because that would be like sort of non-integers. But what it has made me think about is something that I saw you point out a long time ago, and I remember I had a conversation with Olivia at House about this.

Um which was like one of the problems with selling truth um is that people actually don't want it. >> Right. >> They don't want Like many times they don't because the truth often is costly. >> Mhm. >> Um and so So at the level of like incrementality studies, right? Like one of the things that can happen with these kinds of things is it can say, "You've actually been hundreds of thousands of dollars, and nobody wants to hear that.

Or you should spend less ad dollars and grow slower." >> Right. Nobody >> You were the person who did it. >> Exactly. >> Right. >> And this is the other This is the other element of that. So like the organizational level, I think this is a real problem. Um and and you have to like figure out how to solve that. But But you pointed this out to me a long time ago with some times when we would go in to clients, and we would be the we would be the people going to like a VP person or whatever saying, here's why all of this ad spend was done poorly.

Maybe they were auto bidding and they were foolish about the targets or whatever, you know, or their measurement was bad or whatever. Then somebody like it's really hard if you are going to the to that VP who then has to go to their CEO and say, "Oh, we've been wrong about how we've been spending all of this money." >> What's the incentive to do that? >> And so yes, you pointed out like that will actually that will kill deals because that person actually can get away for a long time.

One of One of the things so it actually has made me feel more sympathetic to that person. Cuz I thought like, "Oh, this could be costly for me." And so and so it's made me go like, "Oh, okay. I understand that." That person's That person's in a hard position. It's hard to commit to the truth. But the other thing I believe is that if you are the person, let's say actually as we roll these out, it it goes against somebody somebody listening to or watching this narrative. >> Mhm. >> And they then have to turn around and be that person.

My encouragement is the truth actually does always win in good organizations and you will you will it will almost always feel really costly in the moment to do that. But actually I think it is one of the biggest and like trust builders there possibly is. >> I was thinking about this like what what actually builds trust between humans and I think about this with my kids is that you know, there's the classic thing that like your dream is that they make a mistake and they bring it to you proactively with honesty. >> Oh man. >> Like that that is like the ideal state of trust relationship is you would do something wrong, you would break the glass in the kitchen or you would you know, whatever didn't do your homework or lose your folder or water bottle and rather than hiding, you would come and say, "Dad, I broke the water bottle." And knowing that and actually knowing that there would be a consequence, accepting the consequence but trusting that the end result of that relationship that would make it more productive, not less. >> You You're saying as a parent you you are you are hoping for this.

One of my One of my best One literally one of my favorite moments with my kids who are four and six, right? Is my six-year-old coming to me and telling me like like at first he like hid that he did something wrong and then and And he quickly turned around on it and it was like I was so proud of him. I felt like he had been so courageous as a little 6-year-old. >> And so I think I think that the truth is is that as an organizational as a leader, having a VP of marketing or somebody else who would do the same would actually probably build reinforce trust in the same way. >> 100%. >> Now, you would you would hope that they would scrutinize their decision and build some sort of structure around it that makes you confident that they're not going to make those kinds of decisions again.

Yes. But at the same time, the willingness to admit being incorrect is actually like an impetus for change. It's it's it's a prerequisite to improving your thinking. >> Yes. >> Um and so I think that the kind of people that are actually capable of that are like the kind of people you want to be around. >> Yep. My creative team at AJF Growth has been built entirely with the help of my friends at Behind the Scenes Studio.

Behind the Scenes Studio is a Philippines-based Meta Ads creative agency, an extension of your creative team that can generate for you video ads and still image ads and more. [music] More. I don't know what more there is. Those are the only two kinds of ads I'm not a But they can generate those for you at real scale at a reasonable price doing that with performance instincts. So they're not just like making beautiful images and videos or whatever.

These are performance creative people based in the Philippines who do a great job. And I know they do a great job because literally my team at AJF Growth is staffed out of Behind the Scenes Studio. So I work with them to staff my entire team. They do an awesome job and I just love working with them. If you need more creative, if you're trying to increase the output of your video ads, that kind of thing, go talk to BTS about what the opportunities are for you, what the service looks like for you.

They have creative strategists as well who can be part of your team or you can use them as a more traditional extension of your team, editors and and designers who can work with your creative strategists internally to go make more creative for you. Get on a call, see what their creative looks like. It's really good. They do an awesome job. I'm a huge fan of theirs. I'm I'm to keep working with them for a really long time.

BTS studio.co. If you need Facebook ads creative, you should be working with them. BTS studio.co. >> So, are we about to change our minds on cost controls? >> Okay, yeah, let's roll it out. So, let's let's let's do it. Why don't you It's your study. Do you want me to show Do you want me to show I can screen share some stuff here. >> Yeah, you could you could >> What do you Which one do you want to do? Cost controls? >> No, let's start with the cost >> Okay, so so what is what is the question that this that this study sought to answer? >> So, the headline is do cost controls work, which is more clickbaity than anything, but the question >> I'm trying to actually I was >> I was like, that's not really what this is about. >> The question it's really asking is do they deliver the outcome that you've set? >> Yeah. >> So, if you're trying to The idea of a cost control is define the expectation of the outcome that you want either in a return on ad spend or in terms of a cost per acquisition or cost per result goal. >> Mhm. >> Does setting the goal deliver the outcome?

Now, there's a separate question about volume and all the things that you want to you could do as a supplement to this, but strictly we wanted to ask is does Meta deliver against the bid that you set? >> Okay. Yeah, okay, so I set a ROAS target of two to one. >> Yes. >> Do I get a two to one? That's right. I set a cost cap of $100. Do I get $100? >> That's right. >> Right. One of the things that's challenging in this conversation we should get out in front of is that uh it is it is bad to perform below ROAS but above CPA.

So, we're going to for sure get confused there, but right the bad goes in two different directions. You want your ROAS to be better than it says. You want your CPA to be Yeah, anyway, you get the idea. >> And so, one of the reasons I think we're in a unique position to ask this question is because uh almost all of our spend is on cost controls. >> Can we link this, by the way, before So, this will go in the show notes.

Okay, so you can get this study somewhere in the show notes for this episode. You can read the whole thing, and there's another one coming as well. >> Yeah. You'll You'll have access to all of it. Um so, we're in a unique position because we have a really large set of spend. You can see that it reviewed 1.5 billion. The specific analysis on 200 million dollars of spend. So, um >> Across 253 accounts. >> accounts. >> It's a lot. >> So, and not only is it a lot of spend, but all of it's because so much of it is on cost controls, I don't know that there's many samples that are much larger on specifically this issue.

So, I think that's why it's exciting to go after. >> I don't I've never seen any other agency publish it, especially because I don't know if any other agencies buying so uniformly on cost controls like you guys are. And uh I would think the only larger data set is meta. >> Yeah, right. I I And um so so with that that I think we can ask this question pretty effectively. And so, there's an outline here of what the report is trying to learn.

Um and there's a few different things that we're like after here uh in terms of looking at this. We're trying to understand all three different types of cost controls. And here's another acknowledgement is that the vast majority of our spend, if we look at what Andrew has on the screen now, he's actually on min ROAS. >> This surprised me. This surprised me a lot. So, that this is an interesting thing. So much of your spend is on is on minimum ROAS, which of course, like for people listening, means it's actually on highest value optimization, not highest volume optimization.

So, I think that's very interesting, and I have something to say about this. Some people I I got there's some attention on a podcast episode and a couple tweets about incremental attribution between switch to, which may materially impact this, and I do want to talk about that at some point. >> Yeah. >> Um cuz I've I've seen some different behaviors a little anecdotally here with incremental attribution than standard, but >> Yeah. >> but um but uh anyway, so much minimum ROAS spend.

So, anyway. >> And and not very much bid caps, Andrew, which is your favorite. >> Yeah. Yes. Yeah. >> So, um let's talk quickly about why. This is actually one of the things that's fun about doing these studies is you get to see the actual true behavior of your organization and how the the the cultural trends and narratives have shaped towards that, cuz this This always been true. >> say like yeah. 2 years ago, this was totally fortunately cost per result.

But what we believe, and I think it's part of what manifesting the study a little bit, is that the problem with cost per result goal is that it's incredibly difficult to actually set the bid. >> Yeah. >> Um especially for brands with tons of SKUs where the pathways at purchase are many, that actually defining the correct target is so, so challenging. >> Yeah. >> Um >> Well, I'll give you an example. I have a client right now.

So, because we've been using more cost cap, even with bid caps, we've had this problem for forever. I've got a client that sells like um uh jewelry. >> Yes. >> And so, their account structure is like uh sort of a consolidated CBO. And each ad set I would say is what I would call an economic unit. So, it's not always an individual product, but if it's like the let's say there's three products that all have roughly the same AOV and the same margin profile and the same LTV profile, which this in this case there's a a fair amount of these products that do this.

And do that. So, now so in the end you've got a CBO with like right now it's like probably 12 or 15 ad sets, and it's kind of expanding all the time. And each of those ad sets represents a slightly different AOV or something. And so, now as a media buyer, if you're setting a cost cap or a bid cap, you have to pay attention to the AOV in 15 ad sets. And this gets back to the sample size thing, where it's like what is at what point does an AOV become reliable?

What happens if one of the first three purchases somebody buys five pieces of jewelry, and now your AOV's like twice what it normally is. Do you believe that's because something about this ad produces monster or you know, what I mean? So, so so then so the media buyer is stuck doing this. I've actually been frustrated at points with my media buyers who are going cuz we're not paying enough attention to this. And what what I hear you reflecting is that it's cuz it's really hard. >> Well, and it forces you to guess and change, guess and change, guess and change. >> Constantly. >> And so, and then you run into these situations like I I'm going to show I'm showing Andrew something that we maybe we could pull a screenshot later, but I'll just describe what I'm looking at.

It's an AOV histogram for a brand. This sort of illustrates the complexity of the problem sometimes. So, this is a brand with a very large SKU say it's home goods, the lighting >> Oh, gosh. Everything from $20 to freaking $3,000, I bet. Yeah, right. >> So, you can see the AOV histogram order distribution and this is I encourage brands all the time to look at the three measures of central tendency, mean, median, and mode order values, right?

We talk a lot about this with setting free shipping thresholds and things like that. But, what it illustrates to you is the actual distribution of order values such that you would think about in an ad account what you're actually trying to accomplish. Um and so so often what I see brands do is they take their average order value for their website divided by their ROAS equals their CPA target. >> Yes. >> Right? But, in many ways it's totally wrong.

It's totally wrong. >> mislead you dramatically. You'll end up dramatically overspending on low AOV things and underspending on high AOV things. You'll the distribution of your dollars will be a mess >> doing this. Because because like I've got a another client right now where we just did this and it was like they've got one product that is like something and it's like probably produces 3x the AOV of the vast majority of their catalog.

And if you lump those together, you're going to really underspend on that product even though it's probably the best product to spend on. You have to give it separate treatment. >> That's exactly right. And and to be clear, this is true without cost controls if you run on low versus high value anyways. But, so the example I'm looking at, this brand has an average order value of $1,000, $1,027. The median order value is 512 and the modal order value is 141.

So, all three measures of central tendency wildly disparate. And so, the question becomes if you have a limited budget, where do you set the cost control? >> Right. >> And I don't I don't know that there's a right number. Um and and in reality what VEO Min Ra solves for is it solves for having to optimize to answer that question. >> Because the AOV's baked into the control. >> That's right. And so, it uh minimizes the amount of work that humans have to do.

And one of the preferences I have all the time is to reduce the dependency of lots of numbers of people making individual decisions about where to set things. >> Now, we we should say Meta's rolling out I don't know if you guys have used this yet. >> Max conversions on VO. >> Yes, they're rolling they're well they're >> even know what it means totally. >> And I think the idea is that you can set a so they so they're rolling out highest volume optimization so CAC optimize instead of value optimize with a ROAS control and I think the idea is that the is that it's sort of >> it was the other way.

I thought it was still value optimized for max conversion. >> It's volume optimized it's volume optimized but you can set a ROAS target. So I think the idea is that they are solving this problem which is like >> It doesn't doesn't really make sense to me. I don't really understand what the input would be there. >> I think the input would be that Meta just reads the AOV and says like we're still going to try and get you a certain a a certain ROAS even though we're optimizing for the for the highest volume of purchases. >> But the highest volume of purchases would if you just >> trying to actually get higher value spenders which is the problem with highest value. >> Yeah, but that's but I go if you think about this distribution, if I'm pursuing if I have a fixed budget and I want the highest number of purchases, I'm always going to end up with the cheaper product. >> Yes. >> There's just no way around that. >> Yes. >> So in that sense it doesn't really seem to make any sense to me that it would solve for this problem other than it would allow you to run towards a different distribution part part of the order set. >> That's that's it.

I think the point is it would allow you to reach a different customer by uh the customer who's going to spend like in your example of home goods, right? The reason the modal order value is 141 and even though the mean is one a thousand dollars is because of course there's a lot more people buying a hundred forty dollar objects than three thousand dollar objects or whatever it is, right? Just because it's just it just is the reality of how like money gets spent in the world, right?

So um so in that case like you'd you'd be able to sort of target that customer without actually optimizing up the value chain which is what the highest value optimization does. It's the reason why I recommend running both volume and value, right? Is in every account. We literally in our setup, we duplicate every ad and run it in a value setup and a volume setup. And in some accounts, we've gotten more value spend. In some accounts, we've got more volume spend.

It just depends on the products and the customer and all those kinds of things. So, yeah. So, that's the reason I recommend it is cuz you're actually reaching people on different parts of the like behavioral distribution. >> you and if you actually look at the reach, it'll it'll be so so I totally agree. We can argue that our structure is start V O. We we call it bid surface expansion to get more volume, you would expand it.

Actually right now, partially because of the results of the study which we're dropping ahead, we actually are now the default is bid cap, not cost per result. >> Really? >> Yeah. We'll talk >> Wait a minute. Wait a minute. What just happened, Taylor? >> about why. Like I I think that I don't actually know that the answer is >> don't have enough bid cap spend for that to >> Yeah, but but I feel fairly confident that CPR goal being like just not as useful as I would like it to be.

We'll get So, jumping a bit ahead. But this gives you a bit of the distribution of the spend is just where we spending the money. Um and then the next is like how bid targets were set. So, the thing this is just an interesting illustration of what that weighted average goal was, uh what the median was, what the middle 90th percentile, how many data points in the spend. So, this just shows you where we're setting our targets.

So, this is always kind of interesting because I think people um are always wondering sort of what's the the, you know, average result on Meta, that kind of thing. And the other thing we'll get to and if you scroll just quickly down >> So, you're Is there anything I want to pull here? Like >> Let me come back to this cuz this next part is really important. But get on to the attribution setting split cuz this is something I think you brought up.

So, >> Yeah. >> So, we'll show you by placement and then we also show you how much of the spend is on each attribution setting. Um >> Yeah. >> So, our default most often is min ROAS 7-day click. And you'll see that that houses the vast majority of the spend. The second most would be cost per result 7-day click only. But there is some spend on 7-click 1-view, 1-click only, um etc. etc. So, you can see how much spend is on each attribution setting as well, and how those delivered to target individually. >> Yeah.

So, >> um the the TLDR, if you scroll back up, or actually that we can just stay here. >> one this is helpful. Sure. >> is that min ROAS uh is an incredible product. I am actually like I think we need to stop, and the whole world needs to acknowledge Meta's unbelievable capacity to deliver this outcome across a broad enough data set. It's um >> It's astounding, and and it is the single dumbest counter-argument to all of this to me is like Meta's out to get you, or like Meta's or what or Meta's not reliable.

It's like it's it just like not only is it obviously wrong, >> Yep. >> but like for a bunch of reasons theoretically, I think uh it it has always seemed fairly clear to me that was the case. But, when you point this data set out and saying that it is 97% actual to target. >> Yes. So, across 100 >> I'll say something. This result actually really surprised me because what I have found is that we find that our tROAS bids are uh typically uh less reliable. >> So, you should run it.

Like, you should you should Well, you should check. >> Well, I I'm going to again for exactly this reason. And but um Well, anyway, I So, this is the most surprising result in this whole thing to me. The idea that tROAS actually performs really close to target because in our experience it's Do you think this is because you're spending so much money here that essentially the model gets really good? >> experience this.

Like, I actually think that it's so different than our experience. is is that it just worked a lot better more often. Um and so, we ended up with that result. >> Spending more money. Yeah, interesting. >> So, so now I don't know that for sure, but I I know that we didn't have some sort of like organizational pressure to put people into VO, but um so, you can see that the target was 236, the average delivered was 228.

That's a 96 and 1/2% of target. 7-day click one day view is actually at 101% of target. Uh one day click um missed target more often, which I think is like uh not totally surprising either. >> It's more It's going to be noisier on a one day window, right? Yeah. >> Uh and then where where though it is I think the thing that I was actually more surprised by was how poor cost per result goals were at delivering to target.

And this is actually much more consistent with prior expectations as well. >> Yeah. >> So seven day click was at 140% of target. >> This is part of why I have not used cost caps very much historically. >> Yep. >> Because I have when I have when I have tested this in accounts and I use tested in a general sense. Like this is real testing what you're doing here. What I'm doing was like anecdotal testing. But it's part of the reason why is it because of this is yeah. >> So 100 so an average bid of $78 delivered $108 as an outcome.

So 140% of target. Um now, what ends up happening is the way that it gets managed in an account, and this is my experience too, is that you just keep changing the bid. >> Right. >> keep lowering it and lowering it and lowering it and lowering it. That's how it you drag it into the performance. >> Right. If you have a baseline idea that it performs about 50% worse than you expect, then you just set the bid 50% worse. And this has always been the case with cost controls to me, which is like generally speaking like it doesn't matter to me if they're actually accurate as long as they are relatively consistent. >> That's right. >> And in an account, you just set you just adjust.

We do this all the time. >> Yeah. And and so what happens there um is I think about this not too dissimilar from the way we're going to treat incrementality factors, which is that what you want to do is you have a starting point of expectation of how you can teach people to set the bid. >> And you want it to it be algorithmic in the sense that there's a formulaic input to it. Okay, what is the cost control that I want?

And so we can use these factors to begin the process of analysis. Now, bid cap was better than cost per result, 123% of target. It's a small sample. It's only $3 million. So to me, the reason we've moved to say, "Okay, well, I have a signal that bid cap is more accurate in delivering to the expectation that it wants." >> Yeah. >> Um it's not a large enough sample. I'd like to increase the sample there to see if I can the study in however many >> cost >> per result I feel fairly confident is not going to deliver to the exact outcome and I'd have to make it a factor adjustment to my bid. >> Yeah. >> Um and I think that that is just related to the difficulty related to the distribution of possible end nodes to purchase. >> right. >> Um it it's just a more complex task. >> This is a great point because if you take the AOV out, of course the cost cap is going to work worse in the sense that to use your home goods example, there's just a broad range of potential purchases.

And so like >> When people people get mad of about this sometimes as if there's like as if Meta's being like like weird. It's like I I actually saw somebody from Meta point this out in a tweet a while back, Kristin Kapke, shout out. She said like, you're asking Meta to predict human behavior across your entire skew set, across every brand with all of these things, and it's actually incredibly good at it. And but but is it an inherently volatile exercise?

The point is like the forecast is inherently volatile. So yeah. >> Yeah. So um and this is the thing is that there's there's aggregate data here that shows the power of a modeling product in large samples. >> Yes. >> Okay. And this is goes back to the coin flip example. Is that this is what you should expect to be true. And and if we scroll down now and we look at the actual distribution of individual results, >> They're all over the place. >> we can recognize that any individual set of flipping the coin >> Yeah. >> can produce a wide array >> Yeah. >> of potential outcomes. >> Yeah. >> So what's on the screen now is the distribution of the individual accounts whereby the dotted line in the scatter plot represents the delivering exactly to target. >> Yeah. >> Okay.

So you have the delivered result on the Y axis and the target on the X axis. >> Yes. >> And so you can see that distribution uh same thing for the bid cap or sorry, cost per result goal. And then you can see the distribution plotted a different way below. Now, what's important to recognize, um if you look at the dots, you'll see that each of them more of the dots coalesce below target in the Min Ros sample and above target in the CPAs.

Yes. So, if you look at that it's easier to see maybe visually in the distribution below where um >> Yeah, it's true. >> It is also fair to say that the majority of accounts deliver below target on Ros goal and above target on CPA target. In fact >> by below and above you mean worse than? >> Yes, so in fact, specific data, 67% of accounts on Min Ros up the other way. um Right there. 67% of accounts are greater than 5% below their own target. >> Yeah. >> And 75% of accounts are greater than 5% above target on CPA bidding or Casper's all goal.

So, again, the average in the distribution is different than the majority. It's different than the individual outcomes. They're all different ways to consider the potential result. >> Yes. So, what we have always done about this and what I think we'd still say is run manual bids, be conscious that they they are especially the smaller the sample that you're dealing with, the the more likely it is that they're going to deliver a range of outcomes.

And you And the way we've handled this is we've just like looked at it with clients where it's like is this client is it better for this client to err on the side of overspending or underspending? >> That's right. >> Like And so like your super high LTV supplement brand might be like it's actually more important for us to get the spend out than it is for us to protect the CAC target exactly. Cuz if pushing out our our um our uh break even point to month four instead of month three, we're financed enough, it's not actually that big of a deal.

We as long as we get more customers. >> That's right. >> That's that's one way to think about it. Whereas your brand with low LTV and needs to make the money on first purchase might be like, "Mhm, we're going to constrain this a little bit more. We're not going to push the growth super hard here, etc." Um the other thing I'll point out here that I think is really interesting is this like the clustering of data points on the lower end of the ROAS and the and the CPA thing.

Some of that is because the percentage is maybe the same when you're lower like the percentage range is similar like uh you know, what I'm trying to say is like a $50 cap uh or if it's $50 uh cap and a $55 outcome is a 10% difference. >> Yep. >> Right? Uh whereas like at at a $200 cap it's like a two a $20 is going to anyway it's going to they're going to be it's going to look further from the line. But what I also wonder if the thing is happening here is that the larger the sample size for Meta >> Mhm. >> the the closer in an individual account, the closer you're going to get to actual target.

And there's a reason that I and I would actually be curious to see if there's a way to follow up on this for you guys. Um like essentially more conversions and less ad sets might produce results closer to target. And the reason that I think that in part is probably cuz it's logical that like Meta having more data inputs would make it better at predicting. But also because when you understand how these products work what they do is that um mini target ROAS bidding and and um customer result goal or like cost caps they both do the same thing actually which is they um they are they are able to bid dynamically above and below your target to explore audiences more aggressively and then come into your average in the aggregate.

But when you look at the way this works and I'm going to use my hands to illustrate this point, they start by bidding really widely like uh my understanding is they can bid like like 9 to 10 extra target or something like that like in the earliest stages, but then they condense over time as as the more data comes in. And it would therefore be logical that like if you could sort of isolate the you know, the first 10 grand in spend on an ad set, I bet you the range of outcomes is wider uh on a on a cost cap or a or a min ROAS, uh I bet the out range of outcomes is wider than the next 10 grand, and then the next 10 grand, and the next 10 grand, and so on.

Especially if you're getting like especially the more purchases you're getting per ad set, right? So, um because it's all happening at the ad set level. So, it would make sense to me that what you're saying is true here, which is that especially considered in the aggregate for minimum ROAS or for target ROAS, like that would that they would all come together. But but that maybe in an individual account you would expect some condensing of the outcomes the longer the campaign exists.

And I wouldn't be surprised if that's the reason target ROAS works so well for you guys, you guys are spending you guys have a lot of accounts spending big time dollars. Um you know, and that that like if you're spending three grand a day or two grand a day, you might expect a wider range of outcomes. And one of the problems with bid caps, actually, that I've experienced and what people sometimes get frustrated with is that they don't dynamically bid, and therefore you can sometimes over constrain your spend because Meta will sort of lose confidence, especially with low conversion volume.

And so, that's the place we haven't used it. That's a that's one thing to consider here. The other thing is that I'll point out is that um I am really curious what happens if you rerun this with incremental attribution. Um and I I'm sure you will in 6 months or whatever when you guys are on it right now. I'm sure. Um because what I've experienced with that is that so far performance has been much closer to target >> But but target is an interesting question here because you're not setting a bid. >> Well, no no, we're running incremental attribution with a cost cap or target ROAS. >> Okay. >> Yeah. >> Okay.

So, yeah, I think the but um I think the the problem with the the question one is um well, so we're designing this right now. We only have a little bit of of spend on this way right now. Have We're running CLS on all these to try and answer the incremental effect to determine if it's worth rolling out broadly enough to answer out to the target. The question is what I'm trying to say is what is the target? Meaning I don't know the incremental at the incremental factor of IA. >> Yeah, I think you would just run it given cuz these are all relative to the optimization method you set, right? >> Yeah, but I'm saying that the target is set with a consideration for the incrementality factor.

So, right now the struggle we have is we're trying to set the target. >> So, you're saying the actual here in this report, like the actual CPA, is it is incrementally adjusted? >> the way that they set the bid with was the consideration for it, but right now >> I see. So, you can't you're just saying you don't know where to set the bid. >> So, right now the step one is we're running CLS studies of of individual holdouts to try and understand what the factor is on IA.

Because >> Then you can set the bids and then you can see. >> Exactly. >> Okay, great. >> So, there's like a step process to getting to it to get there because >> Your early result your early studies on this and I think I think Houses as well. >> A couple of that Houses more. >> It's very early, but I think the idea is that the is a higher incrementality factor. That would be the goal, right? I would hope so. Yes, and and and logic and what you had said I think what you first said and your I think it's called like let's call it a beta or something like that, right?

Like it's it's like grand assault on this, but you what you first reported on this was incremental attribution was producing a higher incrementality factor. So, you have to adjust the actual performance up relative to what Meta reports. Meta's under reporting more on incrementality, but even net of that standard attribution was beating it. That was what you first rolled out. And what I believe is that this has all gotten better in the last few months. >> Which is what the House study showed and why we're why we're willing to dive back into it.

That what I have seen and it's funny is that like incrementality both the geo holdouts as well as if you decompose incremental attribution and you were to sum up 7-day click, 1-day engage view, 1-day view, is that it's all sort of coalescing around 7-day click plus 1-day engage view, which was the historical 7-day click. >> I can't possibly do an episode this data center without talking about my friends at Intelligems.

Intelligems are the kind of people who are You're to love this episode. And if you're the kind of person who loves this episode, that means you're the kind of person who's going to love IntelliGem. That's because IntelliGem is an AI-driven, data-driven approach to optimizing your website to generate the most valuable possible traffic from every click that comes to your site. What I mean by that? First, core product from IntelliGem as a piece of e-commerce software is their testing tool.

They've been doing this for a long time. CRO beyond the basics is sometimes the way they talk about it, which is which is because it is CRO type work, split testing type software, but it goes way beyond the simple headline, button change kind of things. You can test the things that really move the needle on your website like price, bundle offer, theme test, free shipping thresholds, shipping charges, all kinds of things like that.

The first purchase offer you're giving to customers when they get to your website, all of that stuff is testable with split tests with IntelliGem. One use case that I haven't heard enough people talking about is testing different subscription offers and even seeing the number of subscribers per page. Duplicate your subscription offer page into another page with IntelliGem and see does one of them actually generate meaningfully more subscriptions than another?

And what's the trade-off on all those things and how does your CAC change and and really understanding that for businesses where subscription is an important thing. You can do all that and more with IntelliGem. It's a really awesome tool. Also, they've built in all kinds of other ways to optimize the value of your traffic. That includes post purchase upsells, in cart upsells, in checkout upsells, shipping threshold nudges, you know, status bar type things.

All of this kind of stuff is now built in product level upsell like product page level upsells. All this stuff to make it so that you can make traffic more valuable on your website with one tool to rule them all. Go check it out. intelliGem.io If you're serious about turning website visitors into more value for your business, IntelliGem is a critical tool. It's that simple. intelliGem.io with the code Ferris 20, F A R I S 20, you get 20% off your first 3 months.

Go check it out today. >> Yes. So, it's like we all pressed meta meta into the 7-day window. Yeah. >> [laughter] >> Fair. Um and I think that the end result is incremental attribution might be very similar to that. But, we'll we'll see. >> Yeah, the engaged through thing I I think this this has been a lot of my take on this is that like the switch of click attribution really screwed up a bunch of our accounts. Like it really really even even when we adjusted our optimization method.

And so, the reason we tested incremental was because of that. And it seems to have been a really good solution. >> our videos campaigns that are video heavy or this is another thing theory I'm working on is this >> Interesting. Yeah, that the change >> that that when you and if you break apart like the amount of engaged view that you get, you know. Now, it's hard cuz they added a different definition which is the watching 5 seconds that doesn't require any click at all.

So, it's not exactly apples to apples, but anyways, I just want to I wanted to >> Was there anything else I was going to say on this cuz we're going to run out of time? And we're going to I want to wrap up a couple one last thing. >> Okay. So, so the end result I was going to share or something. >> Yeah, the end result is just that um we're using a factor when setting our bids. So, just like an incrementality factor.

For Mineros, it's 95.9 and for cost per result is 154. And part of that is it because of if you wanted to get to a 95% confidence range, so if you take that distribution and you wanted to try to create a bound where the predictive value was 95%, >> Yeah. >> then you can see what that bound would be in each case. It's 90 to 101% or 145 to 164%. Covers a pretty large percentage of the potential viable outcomes. So, using that factor is a good starting point for people before you run that.

But, then the last thing I want to show is just this table below cuz it kind of touches on this thing which is like to give you some real examples. >> Yeah, yeah, yeah. >> To illustrate like we have this one account as an example that is just it might be brand 14 in this example. It just it is so off all the time no matter what. And it drives me insane. I can't figure out why and I'm going through the events manager and the EMQ score. >> yeah. >> Is it the subscript like we're trying to figure it out.

But, we recognize and acknowledge that there are brands for which the individual outcomes will deviate from the mean. >> But, the interesting thing about that, right, is you're not abandoning cost controls. No, not at all. >> No, you just said you just make an adjustment. This is what this is how I've always handled this and I've never known how to say this to people exactly cuz it gets tricky. But this is also why by the way all of this is now maybe you're going to take the opposite stance here but it's also why I just don't think this is going to be very easy to automate anytime soon and why media buying I think it's still really hard.

People will say to me like like something like ah come on media buying is just commoditized or whatever. It's like I I just think that's that's >> that's yeah that's very very wrong. Now I actually think this is the foundational piece that allows us to automate because >> Yeah. >> Like if you think about how to set a bid is just a operationally challenging idea. >> Yeah. >> Um and so you have to have some starting point for it.

But um so those factors just like incrementality factors are important components of automation. >> What is the spend period? When it says like spend $6.9 million for brand six in the top of this over what time period do you know? >> I think that the the day range sampled here is March 12th of 2025 through July >> Okay, so a little over a year. >> 6, yeah. >> Yeah, interesting. Okay. >> Um so these are just examples of yeah accounts sampled and the deviation between them.

So it can be pretty wide. >> Okay, we're so low on time. You're we're not going to have any time for you to yell at me today. >> That's fine. >> We're going to have to save that. No, but you wanted to also talk about do you want to save this for something else? >> Yeah, let's let's stop loss. >> Yeah, yeah. >> Okay, this one's like it's more convoluted. It's just it's harder to What I what I want to say is something >> Should you turn off an ad? >> No, the answer is no.

The answer is there is no discernible impact to making ad level decisions. It doesn't make things worse, it doesn't make things better. It has no impact. You think you're doing something but you are doing nothing. And so it is it is a giant waste of time to spend a bunch of effort trying to manage things at the ad level. Um >> So when you say manage things you mean turn off and on ads? >> Yes. Uh and we have like and and to be very clear I know this because we've done it 16 trillion times or something.

Like because we do this endlessly and it drives me nuts and we do it at the compulsion of clients, we do it at the compulsion of ourselves, we do it for so many different reasons. So um Uh now the that the tail end research that we're trying to get to is a stop loss rule that says like well okay but Taylor what about the outliers cuz everyone tells me about the story of an ad that spent $20,000 at zero >> Can I tell you how I handle those stories?

I think they're wrong. >> Yeah, yeah. Well, so I and I just go >> Most times. >> okay, but you have no counterfactual uh what would have happened is it's impossible to >> Exactly. >> Yes. And um you have no broader set of information about other circumstances like that cuz the beautiful thing about if anybody gives me a rule. So if you're out there and you think you have a stop loss rule that makes accounts better. So I see it all the time like there's people that say like at 3x CPA if there isn't one purchase we turn it off.

The beautiful thing about if you're willing to stand up and put your hand up about a rule like that is you can analyze >> test it. >> every ad that crossed that threshold over time and what happened after. Yeah. So I can put any rule like that to the test across a very large sample and what I'll say is that we had a bunch of internal people with rules. It used to be 5x the CPA at zero purchases auto kill. It doesn't work.

It is it is a non-beneficial structural effect on the ad account. Um and if you have a rule, send it to me. We'll run the study and you can analyze it. We can even talk about it on the pod. Um So we'll The point is is that this kind of stuff like we have a research directory that we're going to be publishing that has things like this all the time um that we're trying to do the MCP at CDC and the database structure that we have established combined with Claude has made this just like unbelievably helpful to do.

Like as an example, you just gave me a question which was does does an increase of spend increase the accuracy of bid, right? So while I'm sitting here, we can just like take that research study and build on top of it across the database that has the question. It's awesome. So it's like so fun. So more to come. If you have more research questions, >> So maybe the maybe the thing here is do you are you going to do a separate podcast about this? >> Yes, we have a whole we have a whole set of things that we're going to be publishing >> here's what you should do. >> It just doesn't get the answer.

We'll link exclusive. >> We'll link this in the show notes so you can go check it out for yourself. Um go make sure you subscribe if you're if you're listening to or watching this on my feed. Let's go go to Taylor's feed and subscribe. What which how How do people go find you for this? >> I mean this comment >> Yeah, you could do the e-commerce playbook podcast. That won't be me in most likely, but I'll I'll be publishing on X a little bit.

I'm in the middle of a incrementality study myself there. But uh >> [laughter] >> I've noticed you're quiet. >> so but >> How's it feel to be quiet on X? >> Uh >> Is it fun or is it annoying? >> It's a little bit of both. There's lots of days I'd like to to jump to that. >> Cuz it's fun. X is fun. >> I agree. >> You You You told me how bad you want to go away, but it's fun. Um I go I go through weeks where I don't post much and then weeks where I'm like, "Oh, this is fun." Um Okay.

So So links in the show notes. Follow up with Taylor's stuff. Go get the Common Thread Collective podcast e-commerce playbook podcast formerly hosted by Andrew Ferris. >> Yep. So formerly hosted by me. >> [laughter] >> All right. Yeah. Um and go do that. The um the uh the thing I want to point out here is some I think sometimes people think that I am like uh I don't know. And I think people think that especially about you.

But >> And they're like it I'm like >> um dogmatic. I'm unwilling to be moved on things. >> Right. >> And the answer is not that I'm unwilling to move. It's that It's that uh this is the kind of stuff that actually moves me. And And the vast majority of like This is the thing that kills me. The vast majority of like, "Oh, we've tested this." >> Yeah, yeah. >> in media buying land is nonsense. It is nonsense. It is nonsense.

It is nonsense. It was not tested and measured and studied and then those things It's like where I appreciate this um the CPM R guys who did Yeah, Phil. Right. Phil, you dis You ended up disagreeing with Phil on some of that stuff. But he made a real attempt >> Publishing allows you to scrutinize in a way that's helpful to everybody. >> Yes. I love that. That's why I invited him on the podcast. I allowed somebody to go and say, "We're actually trying to discern truth here.

It's not just some media buying like I tested this, you know." >> And it's risky. You subject yourself to scrutiny. Publishing data means that someone is going to take it and they're going to try and tear it apart with intention. And if I I actually They're going to do that with these. Yes, exactly. It is beneficial to the community that we do so and that we You refine that. That's kind of what we do internally. It's like, "Here's a proposal that we go through and read and it's like, that doesn't make sense.

What about the like so, all for it. >> Yeah. >> Um 5 minutes, one thing um uh about you. >> Okay, hold on. >> I've talked about like >> Andrew and Patrick's podcasts themselves that they're doing that they're just like subjecting themselves to the world of like just >> Build in public, yeah. >> Yeah, yeah, yeah. Taylor's talking about >> In a way I >> What what Taylor What Taylor's talking about is that Patrick and I, we record my my business partner Patrick and I could do record every every month.

We do my podcast. And what we're just doing is airing conversations that we're having internally about what to do at our agency. Sometimes it's like cool stuff we're rolling out. Uh sometimes it's like our last one was about like I sort of feel like we should go chase a creator business, but I'd also feel like it would be secondary to our thing. So, we do it we do it every month. >> So, there's this there's this brand of podcast where I've said this about the SaaS operators, too, cuz it it it feels and sounds similar to me.

It's very dissimilar to the operators podcast, and this isn't a knock on those guys. They're very smart. But, they're just telling you what to do the whole time. And they're giving you advice from a place of knowing the answer. And and that is super helpful, don't get me wrong. But, I love being invited into the wondering, uh the curiosity, the the lack of answers. It makes me want to jump in the conversation way more than when I feel like you're just talking at me. >> Yeah. >> Um like I want to be in the room with you guys when I listen to >> should do one with us. >> Yeah, it'd be well, yeah, I mean I I think that in some ways your guys's journey is your own and I I I think it's really cool.

So, I I I really appreciate that. But, I think >> Okay, go ahead. >> I get to ask you now >> Great, great, great, great. >> So, you have you brought up this whole thing of like, is our goal the right goal? And you and Patrick Patrick is so like he's funny. He like you you uh you I think would be so much more willing to throw things out and start over. And he is like seems like ruthlessly committed to just know >> Stick to the plan. >> Yeah, yeah, stick to the plan.

Yeah. >> Um and we have like a EOS VTO. We've done the whole thing. We've got like a 5-year We did it on a 5-year window, not a 10-year window, but basically that whole thing of like, here's how much money we want to get to EBITDA-wise. We've got all of our goals really clearly defined that we have this year. Here's the Here's the services we want to have like rolled out. Here's what we care about, all that kind of stuff.

So, we got down to rock. So, so yeah, we have a We have a pretty well-defined goal. >> I Yeah, and I think the plans are awesome because they will increase the likelihood of achieving that specific thing. I think they are also limiting in that they will help you to achieve that specific thing. >> Yes. >> Um Yes. And so, if you wrote down a number twice as big, you would probably increase the likelihood of accomplishing that specific thing.

Um And so, the number you write down becomes very powerful. It's a very powerful driver of your behavior. >> This is really helpful. >> Um And so, and the other thing that I've learned uh is that the macro environment in which you exist is a unique variable that should be a consideration in the plan. Um >> Interesting. >> That is hard to to predefine. >> you mean by that, the macro environment? >> I think that it is a very good moment for service businesses. >> Yeah. >> Um And I don't know that that will be true 3 years from now.

I don't know that it won't. I I think there were years that were very hard for CTC, and there were years that were very good for CTC. Um >> That were not about you guys, it was about the environment. That's right. >> And I think that you could potentially accelerate the end result by recognizing those windows, >> Yeah. >> acknowledging >> Yeah, acknowledging that in the in the good or bad spectrum of environments, if you're in a good one, that that should at least uh beg the question, could we gather more now while the weather is nice? >> Yeah. >> Um to potentially recognize that that won't always be true. >> Yeah. >> Um Now, again, I I It's really hard to tell you for sure w- the weather will be bad later or it won't be >> Yeah. >> Like, but I just There's this moment um I think you're uniquely in a position to to do that. >> Yeah. >> X matters in the world.

Podcasts matter in the world. >> Yeah. >> The specific service function is still in demand. AI hasn't disrupted it in any substantive way. Uh if anything, it's been additive to the expectation. >> We've only hired more with with More AI. Yeah. >> So, um all of those things, I think should ask the question, like when we set the goal, did we uh underestimate uh any of the inputs? Were we wrong about them in any way that if we if we were recreating that plan today knowing what we know now, would we create the same >> that's a really good framework.

If I was to recreate the plan today, would it still be the same? And I think that's yes, then stay the course. That's a very good That's a very helpful way to answer that question. >> If you're cuz like when we build our input for new for our business, one of the inputs is like new customer acquisition expectation. And it it if if that was just an input because we had a model that said we were going to do $10 million in new business, and it turns out 15 was possible, then sticking to the model would be dumb because it wasn't about some lifestyle thing that we wanted >> You've You've gone all over the place on this question over time.

And I And I That's fine. This is exactly what we're talking about earlier, right? Like you just >> I just think creating a plan is a really powerful shaper of behavior. >> But at one point you said you actually said you think brands should spend less to achieve the same amount of revenue, take the profit if it's going to meet their plan. >> Well, so so because I think that when you define the plan, you should behave within the context of it.

I think there should be moments where you scrutinize or evaluate the plan. I don't think it should be every day. >> Yeah. >> I don't think it should be every week. I don't think it should be every month. >> And I I I do have a tendency to almost be every week with it. Like I like like we're And it's because I'll see an opportunity. It's One way of looking at it is being opportunistic, and another way of looking at it is is shiny object syndrome. >> freedom if you actually define the space at a midpoint or a quarter point to say like Andrew's allowed to change the plan here.

And you could >> You know what's funny about this is I remember sitting on the other side of this conversation with you as a leader. >> Yeah. >> I'm realizing right now you're you're doing this to me. >> I changed I said don't change the plan. >> No, no, you I just I remember You've heard me say this as a as a critique, but I don't really think even a critique is right. Is it like I I think about this with your questions around faith.

I think like this is what Taylor believes today, but Taylor believed a really different thing about a lot of things 2 years ago. And And so I'm And actually once I accepted that about you, then it made it really fun to work with you cuz it was like okay I never think you're lying to me I always believe you're telling me the truth about what you believe today I also know it may change and that's okay we're going to try and do what we think is best in the in that moment but you also didn't make those swings every week it was like a year or two >> that I've tried to do and this is probably what has created a little more stability for CDC is to try to have at least some Horizon to the set of behavior like it to say like okay quarterly's is the business rhythm where for this quarter these are the kpis and we're not going to change them and we're we're going to behave with these and so we're going to do this and like right now we just did this at CDC is that in part because we got off planet in like this in in a portion of it but it was like okay I want to try for the second half we're going to add this specific set of actions as an alternative opportunity to see what it unlocks and we're going to behave within it the context for a while I think it's very hard if you change it too much but I do think that the scrutiny of the inputs in some period of time this is where like when clients >> come to us I we we do every month forecasting at CDC if they tell me they have a budget that they wrote at the beginning of the year and they force me to behave within it it drives me insane I don't actually know how it's possible if you're have a two trillion dollar new customer acquisition gap for me to go next month and act like your August forecast is right like it's very demoralizing right now there's certainly a balance like I don't know I do think and I've gotten this critique from people that like the biggest limiter of CTC is actually the plan that I wrote in the sense that it's just a number now it's a really good number but why didn't I write five million more and it's just because I didn't have a mathematical model pathway to it that I knew the answer to but rather than it being a discovered Frontier I just I operate within the bounds of a thing I can build a bridge to in my brain >> I'll say we built our plans basically on like month one of really building a business and I so I think it ought to be a wider set of error bars on there, right?

Like there's just you just know less about how it's going to go. And we just learned a lot in the last 7 months or whatever. >> Well, and and part [snorts] of it is I hear you saying things even the conversation you had with Matthew and it just feels like your resourcing model's wrong. Like I just don't I don't really understand why you want to be in a circumstance where you're doing all this value creation on the front end demand side and not being able to satiate it. >> Oh, I don't want that. >> Yeah. >> I'm working on the resourcing problem. >> Okay. >> It's just I'm just not there yet.

It's just the problems are hard. >> Yeah. >> You know? So like my hope I I never I want to stop referring you business as soon as I possibly can. >> Yeah. >> But right now I I'll take the referral fee >> Yeah. >> over nothing. >> Yeah. >> You know? And so and I'll build a wait list. >> you're insane for not offering more services. Like I just the fact that you don't do Google media buying is just absolutely crazy. >> We will. >> Uh >> We just suck at it still. >> But like spend 5 minutes on the internet.

Come on. You're You're too intellectually capable for that to be the excuse. Like being good at Google media buying is literally like a 3-hour task. >> That's not true. >> Yes, it is. You have a foundational set of knowledge about media buying. >> It is for me. >> Yes. >> Yes. Well, what I'm saying is that like yeah yeah I mean I Yeah. We I think we we will offer more services. It is all part of it and right now we're above our plan doing the things we're doing right now and our attention is elsewhere.

It's just not It's not a It's not a valuable enough thing for us right now to go put the time into building a service and then documenting and all that kind of stuff. But it will be and it won't be that one forever. You know? So >> I think it's a weird That's a weird definition of value. I think it would be literally financially valuable to do it. >> It would be, but it wouldn't be as valuable as other things. >> I don't know. >> It wouldn't be.

And you you It was way more valuable for me to go find more growth strategists than to go build it than to >> No way. >> than to build a growth marketing >> No way. >> Yeah, okay. All right, well that's interesting. Okay, well next podcast you got to go. You have to work out. You're not going to get jacked if you don't >> Hiring new growth strategists is like the lowest value task you could possibly do. I just never do it again. >> [laughter] >> How's that?

End and and on that. Don't do not do that. >> Thank you. >> All right, bye. >> After we finished recording this episode, Taylor said to me, "I think that might have sucked." And I said, "I don't think it sucked. It's just not our usual random show style episode because we had really interesting data on something that Taylor and I have both been really public about for a long time, thanks to Taylor's team. So, um we are going to record another episode as soon as we can, more random show style, if that's what you come to us for, if you like it when we do more storytelling conversation type stuff about life and meaning and work and whatever is on our minds, baseball, whatever.

Uh parenting, all the stuff that we usually talk about. Subscribe wherever you're watching listening. We'll do another episode like that soon. I always love that. And and we both were lamenting that we didn't get to spend enough time on that stuff cuz we just like doing it. So, Taylor's a great friend. And yeah, email me podcast@ajfgrowth.com if you have any questions, any thoughts, or if you just want us to work with you.

Uh even better, if you want to work with AJF Growth, go to ajfgrowth.com. Fill out the intake form on the website. Tell me a little bit about your business. I'll get back to you whether or not we are fit for you right now or in the future. We can get a conversation started and see what's there. ajfgrowth.com for all that information. Subscribe wherever you're watching listening, of course. And don't forget to follow up with my great sponsors, Intelligems and more uh and excuse me, Behind the Scenes Studio.

Uh both the links for them are in the show notes. All the links for all of the things that we referenced here, the studies that CTC is putting out, are in the show notes as well. So, go do all that stuff. You know what to do. If you made it this far and you're not subscribed, what are you even doing? Subscribe. >> [music] >> Thanks so much for watching listening. I'll see you next time.