DTC Brands See 38% Better Meta Ads Performance When They Switch To Incremental Attribution - Olivia transcript

[laughter] Olivia Corey, CMO of House Analytics, just said something funny to me right before we started recording. And I'm so glad that you are on this podcast to validate everything I believe about incremental attribution. You you came out right after I released one of my most popular episodes in a long time, which is still not that big all told, but uh I told the world that we've switched our media buying strategy over to incremental attribution on meta ads.

I will link that episode if you have not seen or heard it. Um, it it is uh my take on what we're doing since the time that I released that episode. 100% of my clients are running manual bids with incremental attribution. The bulk of our spend now is cost cap incremental attribution on meta ads. It has been really really good. We have zero clients for whom it has taken us backwards. We have some clients where it hasn't made a huge impact but we have zero clients where it's taken us backwards.

So um so you after I started tweeting about that posting about that immediately went back to the amazing data machine at house and and came and and tried to adjudicate publicly whether or not I was an idiot. Um and uh I stand uh I stand confirmed what's the word validated. Tell tell people uh what you guys studied about incremental attribution on meta ads. Well, we we owe you a thank you because we have been wanting to come back to this.

We published some data last summer on incremental attribution and uh and we were like we need to come back to this at some point and we were kind of putting it off and kicking the can and then I was in that that uh that thread. I think it was you and Barry and some others and we were like I really wish we had fresh data and so I went back to Tyler her the house data machine and he has been doing a lot on his end to make this more scalable in terms of running these analyses and he produced something for me in like an hour uh and that ended up being similar to you.

It was like our best research report of all time. So I owe you a thank you for the ination. It was your best research report of all time. >> Yeah. by far the biggest LinkedIn post we've ever done. Uh, and it was it was massive. I mean, Simon from the the VP of sales at Meta, he posted about it. It was it traveled pretty far. I think this is >> you can we call it trending topic content. You can feel it in the market when like everyone has the same question and it was just it was good timing.

I think everybody has been wondering. I got a text this morning from somebody who thanked me because they switched their account over and they said it's doing much better now. Um, so the timing was right. Thank you for for pushing us. >> People make fun of me sometimes uh for like everything always comes back to media buying tactics and like oh it's just another cross cap thing or whatever. Except the counter-argument to that is like is like it is the stuff that does the best.

Like and there's a reason for that which is that it's incredibly practical and it makes a really big difference and it's where all your money's going. So like it really matters to people that you get the stuff right. And I just continue to reject I said this in my recent podcast with Taylor about cost controls which garnered some of that hate from a couple people. Shout out Rock Ladnik. I don't know how to say her last name. who's actually a nice guy.

But uh uh I'm just being I'm just poking fun at you, Rock. Um but I was saying on there like the actual position on this stuff that I hate the most is it doesn't matter very much. Like it definitely matters. It definitely makes a difference for people's businesses every day in terms of how they're trying to grow because they are spending so much of their money on meta and so getting it right really matters. And you don't have to get one thing right.

It doesn't there's no there's no there's no rule that says if you try to get your media buying right you can't also try to get your creative and your offers and your products and your calendar right you know like you should try and do it all well this dovetales into my thoughts in a roundabout way about Terk Scoo coming to the Dodgers Olivia so we'll have to footnote that in a little bit but uh but uh in any case I'm I'm interested to hear that you that I I didn't realize that you had that that post had done so well for you and garnered so much attention.

I'm going to quickly screen share it. Um, I did see that Simon posted it. That was cool. Oh, that's the wrong thing. Mike, please uh edit out that screen share. Um, >> do you want the blog post? >> Yeah, I got it right here. >> You got it. Okay. >> Yep. Okay, Mike, start screen sharing here, please. And just cut out that little bit. Thank you very much. Okay, so here's your post. Uh, the link for this is in the show notes.

So, if you're watching on YouTube, it's in the description. If you are listening or watching on podcast feeds, it's there. Um, but that the title is is men is meta's incremental attribution outperforming standard attribution. And let me set this up a little bit. Um, because you just referenced it some, which is that when meta announced incremental attribution last year, uh, people tried it, including me, and it just didn't work very well.

And, uh, and I think your tests suggested the same thing. The tests I had all seen basically said two things about incremental attribution on meta. They said uh they said it has a higher incrementality factor than standard attribution. So you have to adjust it up more relative to what it reports in platform. The the true performance is a higher percentage of the reported performance. Uh but even net of that adjustment it performed worse than standard attribution.

That was everything I had heard before. Uh so here's here's your guys' study. Uh there's some lovely charts in here and people should look at it. Do you want me to show anything here while uh while we're doing this or you just want to tell us what you what we learned from this study? >> Yeah, let's let's stay on the visual. We can talk through the the highle visual and then we can talk about dice versus omni. I think that's interesting.

But I can can I set can I also just set up why I think this product is so >> groundbreaking. Yeah, tell me what you want me to share. Go ahead. >> I am IA's biggest fan girl. Uh I think that we so we at Netflix tested incremental that it was a different it had a different name but this idea of meta optimizing your campaigns for incremental outcomes based on the results of conversion lift studies. Uh we tested it in like 2017 or 2018 and it sucked.

It was so bad and it was a, you know, I think we were bummed. But it makes sense in hindsight that they just didn't have enough data. Like if you're training a model on conversionless studies and it's 2017 and Netflix is the only company running conversionless studies, there's just no signal there. >> And so it made sense that it didn't work. And >> they kept going. Uh, I remember like iOS 14.5 broke conversion lift for a year or so >> and then they rolled it back out with Cappy and they made it free for all advertisers.

Anybody, no minimum. This is so different than other platforms. No minimum spend. Uh, no prerequisite. you can. >> It's actually one of the It's one of the great arguments in favor of Meta being like a more trustworthy partner than others is is like Taylor pointed this out a long time ago, but just the idea like if you want to know which ad platforms are really delivering according to the reported rorowass, just look at which ones will will fund the studies for free for you about about their ad performance.

Um, and it's like yeah, Meta will will happily let you run geo holdouts on their product because it really works. >> Yep. And so they kept going. They made it free. And again, it's all it's it all makes sense, but at the time you might think, why are they doing this? They're sacrificing short-term revenue. Like why would they be encouraging advertisers to go run lift tests? Uh but what it enabled was this huge mass of conversion lift data that then they went to use to train a machine learning model that enables advertisers now to optimize for incremental outcomes.

And if you're interested in advertising like and you're in this world, you just have to appreciate the the um the product strategy here. like they are so far ahead because if you think about if any other platform wants to do this they have to go build conversion lift they need to go make it self-s serve they need to make it free they need to build up all the data and then they need to go train a model they're like >> they're like three to five years ahead here I don't see any other platform being able to catch up and so >> I uh I just like shout out Meta product team for continuing to to move and to go on this product even when it didn't make any sense from a you know what is best short term and I think they're they're going to run they're going to run away with it long term because of this decision. >> I agree and I I want to come back to something that I have tried to point out when I've talked about this publicly >> which is that the old way to get at this was to use click as a proxy for incrementality.

Right? So we would all say like this is why you I would constantly tell people get rid of your view attribution. I would still tell them that get rid of your view attribution for meta ads dashboard and then meta introduced this engaged view and then they changed to engage through but all everything in some ways was there was like a longer click window with click reported conversions being the thing that sort of validated that meta was really incremental and the logic is pretty simple which is if somebody clicks on your ad in an outboundbased system like meta where I'm putting an ad in front of you not it's not a searchbased system like Google search or shopping u then that's a really strong signal of intent that was created by your ad and therefore it's probably incremental in what it drives.

But the critical thing in that is that it was always a proxy. The idea of a click was always a proxy. It was not the full truth. Everybody also understood that some people watch a video and then come back and buy three days later and that doesn't get captured by a click that people will comment on something or share or whatever or just something is hard to track. And so and that some clicks probably are not really incremental.

And so the the notion that Meta would say, "Wait a minute, why use the proxy if we have like potentially millions of conversion lift studies that we've run and we can instead look at userbased data >> and then build a machine learning model that has better proxies basically u than just a click, then that becomes a much better way to do it." And at the baseline theoretical level, that's completely right. That's exactly exactly right.

Everybody already understood that that click was a limited way to view this um and that it worked really well in meta but that it was limited and so anyway so introducing this tool as a way to solve that problem uh is a brilliant solution to exactly your point and it required all the conversion lift studies first and all the user data and all that kind of stuff. So anyway, >> and you know, we don't I don't think we talk about this enough, but like you all in your segment, it is so obvious that Meta is incremental.

Like these businesses were built on Meta. >> Yeah. >> But when you go into the enterprise and I look at some of these ad accounts, >> it's it's not as clear. It's it's it's a lot fuzzier. Like when and you if you go audit what a agency holdco is running sometimes they have that the account is like primarily retargeting these are bigger brands with a lot of organic demand like there are some very it's very hard to drive incremental outcomes on meta and I think we take this for granted in in the DTOC segment of just like >> usually like there's there are no question marks around meta's incrementality it's more of a question Yeah, I was so confident in releasing my podcast episode because I people even asked me like, "Well, did you do any incrementality studies?" It's like, I don't need to because I see that we're spending three times more money than we were before for this client or maybe not that much, but like we're spending a bunch more money than we were before and there's a bunch more revenue with the bank account and those happen at the same time and it's happening for all of my clients.

Like the increment, it's the five incrementality test. It's like the bank account is the ultimate attribution tool kind of thing. But what you're saying is exactly right which is like it's not nearly so obvious in many many brands you know >> and it's and it's a huge problem like this this you know you're saying like click is a good proxy for incrementality in your world >> in you know for Airbnb they're not a client but like as an example click might not be a good proxy you know and so >> this is uh yeah this is just it's so cool but should we should we get into the data?

Yeah, let's do it. I've said this so many times now that I bet it sounds like I'm making it up, but I swear to you, it's true. I just opened up another job search with my friends at More Staffing. More staffing, as I've told you a lot of times, are a headunting firm is going to help you, recruiting firm is going to help you find great talent for your e-commerce business or uh or related business, service business in the e-commerce space like we are going to help you find incredible talent in the Philippines.

And they're going to do that because they were actually built off the back of building US-based e-commerce businesses. So, they know the world of e-commerce really, really, really well. If you want to go to them and look for somebody specifically related to e-commerce operations or supply chains or design or ads or whatever it is, uh, if that talent exists in the Philippines, they're going to help you find it and they're going to they know what it looks like.

They'll give you good quality job descriptions. They'll help you form the job description so you don't have to do that work yourself. They will, uh, pre-in candidates. They'll make it so that they pair down candidates so that there's just uh there's just way less total resumes to go through in a world of monster massive AI candidate job applications with a lot of slop. They're going to help you sort all that stuff out.

They'll pre-in people like I said so they can place the right candidate. They're just really helpful, really good. And the basic uh value proposition here remains really simple, which is that you can find incredible talent in the Philippines for dramatically less than it would cost you for that same talent in the US. And therefore, you should consider it because running a lean opex is really good for your bottom line.

You also find really loyal people because you giving a great job opportunity, hiring at, you know, high end of the market rates in the Philippines creates a great opportunity for somebody who's going to love working for you and stick around and work hard and do a great job. That's been my experience completely with more staffing. It will be with yours. Go to morstaffing.co/af. Get a job search started with them. See what kind of rums you come back.

You're not on the hook for doing anything. Just go check out what rs you have. You'll be amazed at what kind of talent you can get. Staffing.co. >> Yep. So, okay. So we ran this study as I mentioned in July 2025. Uh incremental attribution had just come out. We had probably a dozen tests. So the sample size was small and we definitely um caveed that as we shared the meta report last summer. >> We said it's still early uh very early in in terms of this product's roll out, very early for us in terms of tests that we've run on it, but it's not there yet.

That was our conclusion. Not there yet. Um, >> and the way we're expressing the data here is let me just kind of set up this idea of the the ratio. So, uh, the calculation here is IA over standard. And so, if incremental if the IROZ of incremental attribution cell is two and the IROZ of standard attribution is one, then the ratio is two. >> If the IROZ of incremental attribution is one and the IRO as of standard is two then it's 0.5 so >> and we should say here >> yeah yeah >> well I was say so IRO as though that that that word just for people who haven't heard you guys use that or whatever can you also talk about what you mean by that >> yes this is a great point thank you for for stopping me here you can look in platform at these um at these campaigns and you can try to glean insights we've taken it a step further We are actually looking at these products impact on incrementality as measured by a geohold out test.

So what this means is the way these tests were conducted is they have one cell where they're they're running incremental attribution targeted to say half the country for simplicity and they have another cell running on standard attribution targeted to the other half. two mutually exclusive cells running different optimization types and we're looking at the incremental lift to the business as measured by a geoh holdout study.

So we have moved off of attribution entirely like we are not looking at the inplatform metrics for any of this data that you're about to see. >> Yes, it does not matter at all what meta says here or what Northbeam says or anything else. It all that matters is what is the actual contribution according to the hold out test. Yep. And and and the way these also were conducted, I should say, is we we have head-to-head tests.

We also have if a client had a standard attribution test run and then they followed up with an incremental attribution test. That is what this data set encapsulates. What we don't have is a huge sample of tests where >> yeah, >> the customer like David Herman mentioned this. He was like, did you look at my test? that's not included in here because he made some changes to the account, incremental attribution being one of them, and then he launched a meta allup holdout.

Uh, but he didn't label it as incremental attribution because he was just testing meta as a channel and that didn't pull through into this study. So, um, we're probably missing a lot of just like people who are making changes to their meta accounts over time and retested meta. We didn't pull this in. They explicitly were testing IIA here. So it's also why the sample isn't as large as as as you might think. >> Yep. >> Um okay.

So so going back to the ratio here just to to explain these numbers. >> Uh under one is is a win for standard. Over one is a win for IIA. So just think about that as we go through this. >> Very simple, very helpful. >> Yeah. And we use this geo mean. If you want to talk to Claude, ask it to explain like you're a 12-year-old. Um, but we use this geomine concept because we're looking at ratios and it's like the right way um to uh it's the right way to average out ratios.

So you see in uh the 2024 through July 2025 data that that that ratio is8x which means standard attribution had a slight edge over incremental attribution. when we pulled the data looking at uh let's see July 2025 up through present day which was July of 26 you see that number jump to 1.26x to 6x which means now incremental attribution has the slight edge. So there you go Andrew. Uh mic drop. We can end the podcast here.

Incremental attribution is winning more on average now. And you can see in the visual I think this is a really helpful chart. There are still a handful of tests that are where I where standard is performing better than IA. But for the most part you see those green you see those green bubbles. um uh obviously far more than than than the red and so on average IIA is winning more than standard. >> Well, yeah, listen, I mean this I'll tell you how this happened for me, which was like it was a desperation thing.

I had a client that was just like uh struggling and um and I was trying to figure out like what can I do that's different to try to help this client figure it out. And one of the things I really experienced over the last few months was that the the and again I said this before is that when Meta changed their definition of a click in March as somebody who was optimized for click attribution for smaller you know like let's call it like uh early 8 figureure e-commerce brands early mid 8 figureure e-commerce brands that's like basically where my clients are uh for those clients like I just we just found that like moving away like that that changing definition of a click seemed to really hurt their delivery, particularly on manual bids.

Um, and so we didn't have a lot of clients where like the account was on fire or something like that, but it was just like it felt like we was getting less volume than we should even we were making some adjustments and changing definitions of standard attribution. And I had one client in particular, it just seemed like we couldn't get anything going for them. And it felt weird. And I thought, screw it. I'm just going to try incremental attribution again. and they were spending little enough that the easy thing to do is just like move everything over on their bidding to there um as a test and um on the belief that I've always had about this sort of thing that this reflects which is meta tools get better over time.

And so like I bet if you ran this again in a year it would actually be even better, right? Like that's that's that would be my prediction for what's going to happen next for IIA. Um and I thought I hadn't tested it for a while. And so I launched it and it it was exactly this like it had done it had done poorly before but suddenly now it was like oh this is actually performing a lot better than it used to on every measurable outcome.

And um and so yeah what you see here reflects very much my experience. Um I want to ask a couple questions about this quickly. One of the things I want to say ask is like it looks like it's still not a huge volume of tests. Um, you know, I think enough to feel pretty good about the result, but still not that many. Are you guys doing more of this testing still? >> I mean, this is a call. If you're a house customer and you're listening, this should be on your testing road map.

Um, you know, we we guide our customers on testing roadmap. I think this has inspired a lot more customers to test this, but most of our experiments these days are still at the channel level. So they'll run like an all up meta test and it's it's um it's a little less common to run like an attribution or an optimization test within meta. And so part of that is you know the channel level tests help to calibrate the mm.

Um the channel level tests are much easier to set up. They're easier to power. The structure for a IA versus standard is a little slightly more complex. So like it's not like our you know if if I talk through like the most popular house tests it's mostly channel all up but you see some of these um you see some of these from time to time and this I think this data set has probably inspired like a lot more of these experiments.

So, we should now that Tyler has this built all, you know, in a way where it's easy to pull. Like, I'd imagine we'd have, you know, more every month for the next few months. But, it's like >> incremental attribution as a testing priority really hasn't been >> sure >> uh hot button. >> Yeah. So, do you think that will change though? Is that what you're saying? >> I think so. >> This is getting this attention and stuff. >> I think so.

Um, you didn't really hear we I wasn't really hearing much about this until recently like everybody kind of wrote it off um >> like you know in the based on the initial results and so I think we're going to see more of them but yeah to your point >> there are not a lot of IIA specific experiments in the data set but I don't know we have enough here that >> yeah oh yeah yeah it's yeah for sure and I'm assuming you're skewing also larger accounts as a general rule do you want to talk about um do you want to talk about uh what you saw with DTC only versus omni channel brands Yes.

And this makes intuitive sense as well where we looked at DTOC only versus brands who uh have an omni channel KPI uh to define that it's either Amazon or retail as a secondary KPI in addition to DTOC. And we saw much bigger gains from the 2025 study to 2026 on DTOC only. So DTOC only brands saw bigger gains. And that makes intuitive sense to me because it's signal, right? It's like the signal you're passing back to Meta and Meta's probably like if they have visibility into the dotcom purchase, they are better at optimizing that versus being blind to the omni channel purchases happening and thus they're not able to um to optimize as effectively.

So it again it's just all about like the signal you're giving back to Meta and if >> these DTOC only brands are feeding Meta all of its purchases versus just a subset it's probably better at finding more of them. >> So in some ways this finding is like especially for a lot of my audience is actually even more significant. There's a couple things I want to sort of ask about and point out that I see in this data. First is about what you said the intuitiveness.

I actually had the opposite reaction at first. My first reaction was like, why would this be more intuitive? But I think what you're saying makes sense. And then also what has occurred to me is if you think about what Meta is basing this off of, it is the steps that lead to a purchase that they measure and they they measure the conversion on. So it's a really particular kind of purchase, which is like a a purchase that registers in meta.

And so it's like signal is one way of putting it, right? Which is what signal they have and don't have, but also it's the incremental steps that lead to a Shopify purchase. in most cases Shopify as registered in meta you know whereas uh whereas perhaps actually standard attribution you think about Amazon as as a thing here like Amazon might actually be something where uh you know lowerfunnel audiences let's say would be close to making a purchase and then they just bounce over to Amazon to go complete their purchase and Meta would have no visibility to that and would actually kind of take those people out almost you know and would would optimize away from those people in an increment incrementality uh in an incremental attribution method whereas in a standard attribution method it would sort of be in there and you would want those people and I you know I don't know so I I think it's I think there's a couple explanations you could come up with here but I think something around signal and sort of what the actual action that's being optimized for here is the first one is there anything else you want to say about that >> if not it's okay >> I'm I'm I'm thinking through um is there anything about the way incremental attribution works that would make it any worse or better than standard attribution with respect to this problem?

And I I think my answer is no. I think it's it's just it's um it this my point here also would apply to standard, right? So like if if um so maybe that doesn't explain it >> uh >> maybe that doesn't explain it fully because if if you >> if you if you play this through like the being able to feed it only DTOC signal would also plague standard attribution. >> It would Yeah, that's actually a good point. Uh >> so maybe I'm wrong. >> Yeah. >> Why is this happening? >> Maybe.

So I don't know. I don't know. It is a weird result. But more to the point, it actually doesn't matter that much. Maybe maybe somebody come up with a good theory here. Um, but more to the point, this is the thing that's so fascinating about this result to me is like one of the questions to hit really practical application here that people have asked me is like, well, how should I test this in my account? And what this data shows is that you should not test it.

You should just do it. Um, and that's probably no >> still would disagree >> for the DDC only person. Uh uh no because what this data shows is that you should at least test it more and more more aggressively. I know I I I said that sentence and I thought Olivia is going to hate this but I'll tell you why I get there. Because there is there is there is basically no test here that is materially worse in its result than running standard in the DDC only crowd. uh like essentially what this test shows that the very bottom of the performance for GDC only brands is still basically even with standard uh attribution and and so there's just extremely little risk in making the test.

And so what I would say is of course you should test it but you should test it you should test it more aggressively than you would test other things. You shouldn't feel the need to go and run a careful 50-50 test here. What what I would do in most cases if you're spending let's call it less than 10 grand a day. I would I would take my highest volume spend at least and I I'm going to leave highest value to the side because I have some questions about that.

I'm going to take my highest volume spend and I'm just going to kill my volume spend and move over uh my standard attribution highest volume spend. I'm going to move it straight over to incremental and and see what happens. And I think you are actually running extremely low risk of doing that. That's that's my take from this data. And and I'm I'm really thinking of these specific pool of data which is like the low end of these tests is basically at par with the standard attribution tests.

And in fact the high end is so much better that that it's like maybe maybe performing 40% better 38% better than standard attribution. Like that's just a lot of upside and not that much downside according to this set of tests which is not a huge sample but still >> I I agree on the the DTOC point here in looking at this. There is >> that's what I'm saying. >> Yep. If you're a DBC brand, you Yeah. And what Andrew is showing here, if you're not watching, if you're not seeing the slide, is um is a really good representation of this point is that even the worst results on IIA are better than um than the good. >> They're at least they're at least they're at least even with the standard results as a general rule.

Now, the counter-argument to that is that there are some standard attribution results that are better than some of the results. So, you know, I like I I say that tested aggressively and maybe maybe that's overstated to some degree, but I think you should at least be doing it. And the other thing I'd point out here is that the trend line is up and to the right in favor of incremental as we've discussed, right? Which is which is the notion it's like it's better now than it was a year ago.

My guess is it'll be better in another six months. Um, and so yeah, >> I just got a text from a friend, uh, and he said, "I've officially switched over to incremental attribution and I asked him if he ran a a lift test to validate." He said, "No, but I needed to give the account a change." I could tell Meta was just going after the same users over and over. So far, I'm seeing it is doing a better job spreading ad spend across creatives than standard. >> Yeah.

Is >> that does that in line with what you're seeing? >> We we've definitely seen some of that. that it has not been super stark so much in that respect as it has been more about the total volume of spend increase has been has been the really the more standout thing and that sort of implies again because we're running all manual bids that implies also more ads getting spent. Um so yeah I I wouldn't say um I wouldn't say we've necessarily seen that.

Barry Hop makes a big point of this. He says like when you look at the ads that it spends on in incremental versus standard you often see differences and there's maybe something to that. I haven't I haven't it's not been the standout result for us. So, it may it may happen for people. So, >> yep. The the reason why I said that you shouldn't just roll this out is because it's still very close. >> It's anti it's anti antithetical to everything you think in about how the world works. >> No, it's you see it's still close.

I look this ratio is >> uh 1.26. It went from point it went from >> it went from standard has a slight edge to IIA has a slight edge which means that it's probably >> that's dramatic that's not slight and for DTOC only it's 38%. That's a that is a gigantic amount. If I could tell everybody on this call, I could give you the same row or on on from this podcast, I can give you something that on average with one little change gives you the at the same exact rorowass you have, but 38% more spend or the same exact spend you have, but 38% more rorowass.

And that's not exactly the right comp, but it's it's relatively close to what we're saying here, I think. >> Like, everybody would be like, >> "Pause, go, pause, go." Everybody stop talking, Andrew. Shut up. I'm I'm changing it right now. That's what they would do. >> You would prefer the sample set be larger. That's also why I'm saying don't immediately roll it out. Like you I can tell you're slightly uncomfortable. >> I am >> with the amount of tests included in this study. >> Yeah.

I think it's definitely the best set of tests I've seen and I think we want more of them. >> Yeah. >> So that's why that's why that's my case for why you shouldn't just switch. >> Yeah. I think you should probably mostly for most brands just switch. you can always switch back. That's the thing. Um but uh if specifically if you're if you're sub five grand, please don't run two cells and test it. If you're sub five grand a day and spend, just switch.

Just switch. Like it it's like it's just Yeah, just switch it over and see what happens. So there you go. >> Andrew, what do you what do you say to the people who are I'm seeing a lot on X about how they made the switch, it went really well, and then things got bad over time. >> Block said the same thing. Yeah. >> Yep. I haven't seen that yet. So, I don't know. I don't know what to make of that. We don't have any accounts where that's the case.

And we made the switch, I don't know, a couple months ago. So, yeah, we haven't seen any where it has it has done that. I mean, I think it's possible that what's happening there is that it's exposes your lack of creative volume or something, you know, like maybe fatigue happens quickly or something. I'm I'm not sure, but uh but or there's seasonality things that is getting reinterpreted, misinterpreted as that. I think I think it would be Yeah, I I just I haven't seen that.

So I I don't know what to say about it. I'm definitely keeping an eye out for it though because uh I've definitely heard that a few times. So >> cool. >> Um okay, let's talk about a couple things that I want to see next. So when we have when we have switched over to incremental attribution for our highest value optimization, our t rowass ads, it is a much less dramatic impact and sometimes seems to have even taken us a little bit backwards.

So I'm really curious if the tool works as well for value optimization as it does for volume optimization. And when you think about that, it is a different optimization. So it would make sense to me that it's and I also know uh historically there's less total value optim optimized spend on meta than volume optimized spend. So if you think about this, right, it's a machine it's a model. It's machine learning. So if it has less conversion lift studies that are value optimized and less less data to feed that model, I could imagine it being slower to be great there.

Um, I'm also curious about it for international ads. I've sort of seen less compelling results there. Uh, where you might have some similar things, just maybe less CLS's to based this on, which is what Meta needs to power this tool, right? They need the CLS's. >> Um, and so I'm cur Yeah, go ahead. I'm I'm curious what you guys have seen about that, if anything, and and if not, what you could find out about that. >> On international, I looked at the breakdown of this data we published, and it was like 90% US, 7% UK, 3% Canada. >> Okay. >> So, >> yeah, >> very small on the international.

Yeah. >> Yep. And then that's interesting. It would be interesting that it's not ready. I mean again it just goes back to like volume of conversion list studies and maybe they're a little bit behind in Europe and AMIA on incre adopting incrementality and running these studies. >> Yeah. >> Um value verse volume. I mean I heard you talk about this with uh with Taylor on the pod which is that a lot of brands are running both simultaneously.

So, like would I need to get the customer to to um to change their BAU in order to run that experiment? That's usually like a more difficult ask of like if they have both value and volume running than to ask like turn one of them off in this cell might be a uh a tough ask. But like is that is is that what you see as well is that are are customers like are these brands running mutually exclusively on one or the other? >> Um I don't know anybody many people at least who are running value only.

So yeah I think what you're saying would make sense that it would be sort of hard to design the experiments in some ways. Um but >> we were struggling with because you asked me for this >> done. Yeah. >> Yeah. You asked me for this ahead of time and we were just having trouble pulling that because most of these lift results have like some combination of both. And if we want to get it broken down one or the other, we have to isolate that strategy in a set of regions. >> We just really can't talk about data and being data driven in an e-commerce business on this show without me telling you about my friends at Intelligence.

Intelligence is the postclick website optimization testing tool of record in e-commerce. One of those tools that basically should be in every e-commerce brand stack. Most of my Shopify based clients are using intellig at this point to test their website across a bunch of different dimensions. That includes things like pricing tests uh where they're actually split testing the price of certain products. Simpler AB tests of course are part of that.

You know, design changes and copy changes and that sort of thing. [music] You know, PDP changes. You're talking about free shipping thresholds. How much you're charging for shipping. And of course, they're also using it now for in cart, inch, checkout, and regular PDP type upsells and post-purchase upsells now are part of the product. You can measure the output of your performance on a whole bunch of different stuff in the exact right way, which is by tying into your COGS data.

You can see the actual profit impact of every test that you run with Intelligj. So not just revenue, not just conversion rate, whatever, but actually how much if you run a discount in cell A of your test and no discount in cell B. Intelligence will help you to see what is the actual profit impact of that, not just revenue impact because of course it costs you some margin to run a discount. So you need to be able to run those tests and see that output at that level.

Or if you're running a test that is that is impacting a page as subscribers, you can see which test, which offer drove more or less subscription activity for you, the subscription take rate. So all of that kind of stuff that you'd really need to make really good decisions across your website to make it so that every click is as valuable as it can possibly be for you and the experience of your website is as good as it can possibly be for your customers, which is often the same thing.

So intelliggeems.io is the place to go check that out. Intelligence.io use the code ferris 20. F a r i s20 to get 20% off your first three months. Really awesome tool. You should be using it. Go check it out. Yeah. Okay, that makes sense. And I I uh yeah, I we'll see. We'll see how that plays out over time. I'd be curious to see if there's some way to design test around that or something like that. I don't know how you >> This is my pitch for you.

If you want to come join house, Andrew, come join us and you can help us figure out how to test that >> design value studies. I don't want to do that. Thank you. I'm gonna just instead have you come on the podcast sometimes and tell us what you learn. Um, okay. So, let's talk about anything else about IIA. Oh, incrementality factor of IIA relative to the incrementality factor of standard. That was another thing that we talked about before.

Do you have anything on that? Maybe not yet. It's okay if not, but like not just how they compare to each other for IROS, but how they do compare to platform reporting as far as you guys see. Do you have any anything on that? No, we don't have factors um across many of our experiments because we have moved toward um doing this in an automated way like we will we have a a causal attribution product I can talk to you about where we're like debiasing the pixel and the platform data using a machine learning model because we found that the factor approach is like a very blunt instrument and like things about the business and the account and the the um and increment mentality are just changing so dynamically that like slapping like a static blunt if on top of >> uh platform reporting has been challenging.

So we've moved away from from factors and we use a machine learning model now to kind of like debias attribution data. We'll maybe we'll talk about that when we talk about attribution but we so we don't like require customers actually >> pull in if um like we used to. >> What you're talking about is your guys' mm right? We have so no actually we have MM for macro allocation decisions and then causal attribution is our newest product and that is like microlevel decisions like incrementality at the daily level down to the ad or adset level and that is where we where we found that a model is better at just like a blunt kind of if a top platform reporting.

Can you talk about that product a little bit more here and and tell people who should be signing up with you to use it? >> Sure. So, this actually it's it's um something I've been wanting to talk to you about, which is why do brands use thirdparty MPAs. So, I'd love to have this conversation with you. >> I would love let's do it. Let's talk about it. Tell people about yours and then let's talk about it. >> Okay. So, we um we've been on this journey of operationalizing incrementality.

I talk about this all the time of like it is not enough to just run the test. You have to action on that test and this is how you get ROI from testing. And so as we've been on this journey, experiments are incredibly helpful and there are a lot of brands who have been successful on experiments only. There are a couple issues that we've needed to build products to address. Number one is the question of like all right, you get a good test result.

How much budget should I add? Right? like I I just ran a YouTube test. The result was good. It was better than my goal. >> What do I do? And it was like well add 20% and retest used to be the answer. >> And then um the second issue is channels you can't geo segment. So influencer affiliates, you know, like some some big channels for these brands, you can't run a geo test. And then the third issue is like being able to kind of use it as a planning tool of like what if I moved a million dollars from meta under YouTube?

What might I expect to happen? So the MM solves all three of those issues. Like that is why we built MM was to address all three of those concerns. Now the next logical question is if I'm moving a million dollars from Meta into YouTube, where do I put it? Like YouTube is a very big channel. Meta is a very big channel. which campaigns do I put it into and how do I monitor the performance dayto day u so it's like I call this the microlevel decisioning the microlevel kind of operationalizing of incrementality because the mm handles the macro >> so the micro decisioning that needs to be made at a you know day-to-day at the adset level at a more granular level that is where brands use attribution >> and so a couple years ago we embarked on this journey of okay we can use incrementality factors as a way to adjust and debias platform reporting.

And what we found when we launched that product, we use ifs kind of similar to the way CTC was is is using them is that like, and this is where I want your take, >> fundamental rejection of platform reporting. Like when you say we're adjusting platform reporting, marketers should shut down. >> And I don't think it's rational. Um, but there are a few reasons why. I think number one, >> you can't really make sense of the numbers. not adding up to 100.

You know, like when you're looking at platform reporting as your source of truth, you add up all the conversions and it doesn't equal >> the amount of conversions in Shopify. So, like that's one reason I think why why brands don't like platform data. The other reason is there's this love um for like new versus returning breakdowns and the platforms are not good at providing that. And then the third reason is I think it goes back to just like distrust of platforms in general.

Yeah. And so >> we found that they weren't using it because they were like this is calibrating platform data >> is like way too far away for my workflow. And so we actually ended up kind of rebuilding >> and we launched and and then there are just issues with static ifs in general. I mentioned uh like you know seasonality and your um your incrementality changing over time and this idea that we can kind of leverage more data points to adjust these incrementality factors more dynamically than just like one experiment you ran two years ago.

Like an example is if I have an experiment from three months ago and I have an experiment from last year at the same time, which one gets more weight? like is it the experiment from three months ago or the experiment from this time last year >> and it's just like hard to work through as humans. So we build a model that leverages all of our cross customer data to like again kind of debias um attribution and we have our own pixel now but the purpose of the pixel is only in so far as we want a daily stream of data.

We are adjusting it for incrementality but we need that like daily stream. >> Yeah. Um, and the and it and the again like the reason why we built this product is because we want to help marketers operationalize this. We want to be able to say here's where we should here's where you should allocate these dollars at the campaign level and you not have to go log in to another dashboard or another platform that is not adjusted for incrementality because it that causes like it causes confusion and then these marketers are like doing kind of mental gymnastics >> um across dashboards.

Now, I think you could poke you you did poke holes on all of all of this at dinner when we were at Meta Performance Summit. >> Um, >> but but you also you are fine with platform reporting and so you're not probably like you would have been fine with the original version of our product >> um is is kind of what I what I've gathered in talking to you. >> I've never used Northbeam or Triple Whale for a single client ever.

I've uh I mean like I've I've referenced it when other clients have had it um and like looked at it, but I've never actively used it. I've pretty much moved all of them away from ever >> keeping it going. >> So why why do like why do smart people like why do you not need it? >> Um but like really smart people who I really respect feel like they need it. >> Like why does Conor McDonald use it when his business is so much bigger and he's so much smarter than I am and so shouldn't I listen to him?

Um, so first of all, yes, right? Like that's like a great counter-argument. Like, uh, Connor and Cody and people like that who are really good at this, um, they use it. Um, so I think there's a couple things happening here. The first is, uh, the first is that I think you guys and your well actually let me ask one clarifying question about everything you said. when you talk about your pixel and sort of the adjust like the the scenario you gave like do you use the seasonally adjusted incrementality factor or do you use the the recency adjusted incrementality factor right like that that question you're say like the three months ago if versus the seasonal one from the previous year um it sounds like what you are saying houses the answer to that question is is to say actually you probably ought to have a modelbased approach that regresses both of them in some way uh so that you're not so reliant on your own individual outputs and that's what we are going to give you with this data stream via our pixel.

Did I say that right? >> Exactly. Nailed it. And one other thing that's really cool about it is >> the cold start problem like what do you do for channels you haven't tested? Yeah. >> So we have what we call our incrementality index which is the full cross customer database of thousands of experiments that we have used here to build the model. >> Yes. Yes. and um and that helps you with cold start and then once you run experiments of your own the model brings it in but we just think machines >> but when the model brings it >> okay but when the model brings it in Olivia this is exactly my question when the model brings in my own tests >> it's not going to completely overwrite the thousands of tests you already have right it's that your tests are going to regress my tests correct >> yep and it will weight them appropriately >> okay so to meas based on what we've seen like because We're running these tests in the background to say when we provide an estimate here, how often are we right?

Like then we'll customer will run a geo test and we'll say how often do we get that right? And so we've been building this model. This is it was like really hard to build. I just talked about this with the marketing operators. This is this is two years of building a model that we felt comfortable shipping. >> Yeah. Well, so I'll tell you one of the things about this whole conversation. I think the thing that we just talked about, the idea of regressing my individual results instead of just interpreting my recent test to some kind of mean based off of all of these other tests that you're talking about.

That notion is a very hard concept for people to get their head around. I think it's not super intuitive at first because it really feels like what you're saying is you're not factoring in my data very much. You're just giving me this random set of data that is not related to my business at all. What you're actually saying is that there is a lot of noise in any given test and that you ought not over interterpret an individual result even that happens.

You should get more confident in your own tests the more of them that you run and on the more dollars that you run and the more recently you can compile all them etc. So if you know I've heard you talk about um the Ridge team and just how much testing they're doing. Just judging by what you've said, if I'm Ridge, I'm gonna be pretty comfortable waiting, and I'm sure your model does this, waiting ridges specific tests more aggressively than my individual or that than than the sort of regressed tests, right?

And again, like I'm I'm talking about that like two different things. I'm assuming the regression model does that exactly, right? That the more tests it puts in from an individual brand, it weights them more heavily in the model. Um, but that con that what I'm saying this concept that we're talking about I think it's just like what is one challenge with this conversation is it's it's confusing at first like it does it's it sounds weird you know >> so confusing it's so many products we're trying to like simplify the mental gymnastics and the math that people are doing in spreadsheets of like hey you probably as a human shouldn't be doing this anymore you know what I mean >> um but and this is I think what's lost in >> in this ind in this specific segment of the industry is like the other players, the other vendors have like no nuance in the way they talk about these conversations, you know, and so I think it's doing the industry a disservice to like not talk through these very hard challenges around how you actually make this work um practically. >> Well, so one of the things I really believe here is that eventually the truth wins moneywise, but often not in the short term.

And you know, again, you and I have talked about this and and but yeah, that just like like it's really hard to swallow truth that strongly disagrees with your take, but that eventually that truth tends to win in in great organizations. And if you want to be great, you have to be relentlessly committed to that. And sometimes that actually requires you to work back through the conversation we're having right now and make sure you understand it, you know, um and or or whatever, right? as as your business grows, like yes, when you get to $100 million in revenue and a bunch more channels, stuff like that, all of this is dramatically more complicated and you have to level up your skill set and the ability to to handle these things, you know?

So, to me, like I think um I think it uh yeah, I I think I think it's just what's required of the challenge. the challenge changes to being a more complex challenge because you're deploying more dollars in more places and it's harder to measure and then the statistical concepts are harder and all of that stuff and you got to get the right person on your team who can help you do it and and yeah that's that's just what the challenge requires you know um because eventually again great organizations um at least uh they eventually gravitate towards truth.

So yeah. >> Yep. >> Um this this answers your question. Go ahead. >> Yeah. Why why don't so why don't you use them? because my clients are not spending enough money on other channels besides Meta as a general rule first of all and secondly because Meta's platform reporting is really reliable and people just don't like to believe that and I can give you my reasons why they don't like to believe that. I think there's a couple of particular ones like root like historically particular even uh but uh but but Meta's platform reporting especially tied to a click historically and now tied to incremental attribution uh is is really really reliable.

Uh and I mean you another way of putting this is that the reason that I don't use them is because of you guys in CTC putting together meta analyses uh which is a confusing term here. I mean that in the statistical sense uh like meta analyses of meta tests uh showing that meta is under reporting if you are optimized for a click under reportporting at least historically 7-day click uh optimized campaigns are are under reportporting their true contribution by 10 to 20%.

That is not maybe not exactly what your test showed, maybe not exactly what CTC showed, but it's something about right. Another way of saying that is that 28 day click attribution in meta is about right. And I'm just not going to bother trying to get more precisely right than that. It's good enough. You can see it very clearly. If I just look at my meta revenue and my Shopify revenue, there's basically almost no clients that I'm dealing with, including at my larger, you know, we're talking about like mid eight figureure clients who even for them, uh it just seems pretty clear that my metapend is impacting uh accordingly.

Um, so that's part of it. Uh, that I just I think that meta reporting is basically right and I don't need a third party to show me that and that is actually a regressed mentality like like that is what all of the tests show. So I'm going to regress my understanding to that and I don't have anybody who really disagrees with that. Secondly, at the same time uh I do think that there is a place for those tools when you get to those larger more complex levels they become really important which is why Cody and Connor are using them and I'm I'm not really. it's because they're just running businesses with more complexity to them.

That said, uh I think one of the weirdest things is this idea that they are definitely right. Uh like I just think anybody who's around these tools should un like it's it's a strange it's a strange like why do I believe Northbeam or triple whales like rorowass number is correct? Why do I believe that any more than I believe meta is or Google's is or whatever it is like uh and and so incrementality studies this is why I've always loved talking to you I think are great but MTAs especially are like I don't know useless is useless too strong of a word they are maybe actively bad because they are sometimes pointing you in the wrong direction for a bunch of reasons M&M's I think are more interesting but can very confidently tell you a wrong answer it's like talking to AI it's like sometimes AI will tell you here's all the stuff but it's wrong and then you walk out worse for having used it because you have now believed something that was told you with a lot of confidence it's wrong.

So I just don't I don't trust I trust Meta's reporting way more than I trust North Beams for what Meta is doing. Like it's just and I think that's a data driven decision. So that's the reason why I don't use them and why I don't use them for my smaller clients. >> I think the reason people do is because of iOS 14. Uh, so they used them as a replacement for meta during that particular moment and they were probably right too.

They're probably more accurate at the time, but that problem got solved. Um, and then I actually think there's some people with a very long memory of like the crios and ad rolls of the world. Uh, like this is like 10 years ago e-commerce where it was like, you know, you'd be seeing these insane view attribution windows and I think ad roll had like a 30-day view attribution window that was like natural for for display ads. um like it was like uh it was like these insane reports and I think it it sort of gave a bad taste.

Google is also like um you know a harder tool to work with in terms of their built-in reporting for a whole bunch of reasons. So I definitely don't have this attitude towards Google reporting like I do with meta. Um so all of those things together means people sort of have a lot of bad taste in their mouth between those moments and particularly at iOS 14. Triple whale was really smart to introduce their tool right then as a as a as a solution to that problem and it just like caught fire, you know. >> Yep.

I I have I've I was looking back I have an email to my founder Zach from 2022 just asking like why would anybody use this over G4 and at the time I like don't think I understood how much people hate just the the G4 UI. I think that that's what did it like I don't >> I I can't think of any use case >> for MTA for one of these attribution tools that I couldn't get out of G4. >> Yeah. Yeah. Yeah. But people just hate using it.

It's crazy to me. It's free, you know? >> So I But I'm with you. Like it's it's um it's it's kind of similar to your MM point, which is it's just like >> it like trains you on this idea of false precision where it gives you a number. >> Um >> and in like but there's there's um >> it can be wrong. I it's it and like again there is there's this element of like is the number good and that may be why people like it is it looks good and it makes you feel good that the number is good but um >> I don't know I I just it's kind of a you know I again I really trust a lot of people in this industry in this segment in this market and >> I still haven't fully gotten to the bottom of like what I'm missing on like why they might trust that reporting over uh G4 or something else.

It feels very irrational or platform. Platform is a great example. Like I'm on I'm in your corner. Like I I I would trust Meta to do attribution more than I would trust one of these companies. But that sentiment is not shared by the industry. And I don't understand why. >> I don't understand why either. Doesn't make any sense. I um Yeah. I mean I think also speaks to just like statistical uh challenges that people have thinking through these things.

Like I think it's, you know, there's a bunch of stuff there, too. You know, uh it's it's like hard to to un unwrap all of these. I don't know if somebody's selling you truth and they're and they're good at selling it, then you're going to believe them a lot of times. You know, >> people say all kinds of things. Um do you want to tell people again? I don't feel like you did a good enough job telling people who should buy your product. if you um this is why Andrew this is why I love talking to you so much because I feel like we're very open and honest with each other about like why some of your brands are not a good fit for house like >> Yeah.

Yeah. >> You got to feel like you have an incrementality problem. Like you need to be like spending in more channels than just meta. >> You you need to be at a size and scale where like you have enough organic demand where you think incrementality testing might unearth something that you don't already know. >> Yeah. Yeah. >> Um, and so that's why I tend to think that brand spending like more than 10 million in ads, this roughly equates to 50 million in revenue.

Like this is when it starts to become a problem. And then like >> when you you know and and you like you no longer trust that Meta is delivering new customers versus are they latching on to organic demand that was out there anyway via partnerships just via like momentum like what Grun has right now right >> um and so like I bet Grun's platform attribution looks incredible like they are probably crushing >> that's the moment where you need incremental ality. >> Yeah. >> Uh and so again, like I've been brand side, I've been an operator, I've been in companies where it's just like >> I can tell like everything we're doing is incremental, you know, um there's no question mark.

And so I don't want to like I don't want to sell house to a brand who doesn't actually have a need for it. >> Um and so that's where I tend to say and if you're selling in in um in Amazon and retail, then it gets fuzzier. But even then, like if you're >> You guys can help with that though, right? >> Yeah. Yeah. That's a good use case for house. Like that's another like trigger moment for like, hey, we're launching in Target.

We need house. Like that like half I hop on. >> Can you guys help people measure owned retail? >> Owned retail is easy because it's all on Shopify, >> right? Yeah, that's Yeah, we work with a lot of brands who are passing owned retail as a KPI. >> Yeah. >> Um but that's like a really good use case. But even then, if you're on the smaller side, and I say smaller just like, you know, by um uh just by that definition of less than 10 million in ad spend and you're and you're doing like 10 or 20% in of your business on Amazon, like there's not like it's not like we're going to unearth a whole bunch of sales that you didn't know you had.

Like if only 10% Amazon only represents like 10 or 20%, then we're not going to like >> it's not going to radically like transform the way you think about your media mix. So whereas like we work with these brands who are, you know, they're they'll be spending 20, 30, $40 million on a tactic and we'll find no incremental lift and it is like it is just >> um transformational for their business and they'll move it and we'll see immediate improvement in that.

It's hard to >> like >> it's hard for them to actually see that because it's there's so much noise happening in their business dayto day. >> Yeah. Yeah. It's funny. I have a brand right now that is like exactly what you're laying out is so right there. The brand is like yeah sort of approaching middle eight figures. They're probably spending like I don't know six or seven million on ads and they don't need house yet.

But if I look at three million or so more in spend and another 20 million or so in revenue, which is not that far away for the way they're going, like they're going to get there. Um I'm like, "Oh, you'll need it then." Uh yeah. and and it it makes and it's exactly all the reasons you said the the channel the sales channel mix gets more complex the the ad channel mix gets more complex there's organic demand for them there's all of these kinds of things going on and then it's very easy to see how the cost of house is much less than the value possible for just just in terms of ad dollars deployed you know just the intelligence will help um >> the my favorite my favorite uh illustration of this is the Javi coffee case study they they They they did a really cool meta test where they put like their uh lowerfunnel like more static promotional type assets in one cell and then their like founder story brand videos that they weren't getting any spend on in the other cell and those ended up being more incrementally efficient than the lowerfunnel promotional static ads in meta in their MTA loved the the static ads.

That's the best way I can. But but if you're small, like if if you're if most everything you're doing is incremental, then like in the beginning as you're starting off, then you won't really need a hold out test to tell you that. >> That's right. Yeah, that's right. Um, okay, we have three minutes and we're already over. Do you want to end the podcast? Do you want me to be your CMO for a minute or do you want to talk about Terco and the Dodgers? >> Oh man, that's so tough.

When when are you publishing this? The school news might be old news by the time this comes out. >> It'll go out really soon. >> It'll be out on Monday. >> Okay. >> I mean, the actual evergreen topic is are the Dodgers ruining baseball, but go ahead. >> I think you probably have a lot of um you probably had have a lot of local LA listeners, so let's let's hit that. >> Okay, let's do it. >> Are you you're born and raised in LA, >> Orange County, but yeah. >> Okay.

All right. So you you like most of the you know living in LA it's like a lot of transplants. So like you actually have allegiance to this team. >> Yes. I'm a big I've been a Dodgers fan for my whole life. >> I am well for for listeners I am born and raised in suburban Detroit. I left for a while and then I came back so I'm I'm back here now. Um my family loves baseball more than life itself. So this news hit us pretty hard.

Terble, best one of the best pitchers in the world, traded from the do Tigers to the Dodgers this year. Yeah. >> And so I wanted to talk I wanted to ask how you're feeling about it. Maybe I don't blame the I don't blame the Los Angeles Dodgers. Like I don't have there's there's no ill ill feelings, but like take your Dodgers hat off maybe quite literally. >> Like >> as a baseball fan, >> yeah, >> is this right? like is B and one of the things I heard my family talking about is just that every other sport has a salary cap like >> every other major sport has a salary cap why doesn't baseball >> what do you think >> uh the first there's a I have a lot to say about this so it's gonna be hard to do it in a little bit of time I should do this with Taylor as well because we're going to talk next week um uh okay the >> should I send him a Marco should I send it should I you should you should yeah you should >> you guys can have the full combo you care No, no, it's okay. >> The first point is that do the parody in baseball is better than it is in any other sport or at least about the same.

So the idea that the salary cap sports have better parity is just nonsense. It's just it's just completely and obviously wrong. Like tell that to the Patriots dynasty. Tell that to the to the uh Golden State Warriors and Thunder over the last 10 years in basketball. Like basically every sport is dynasty prone. And part of the reason for that is because basketball and football in particular, I mean, the Chiefs and the Super Bowl the last bunch of years, right?

Um, it's not that they win every year. And of course, the Dodgers haven't won every year either, but there are good teams built around good players that make it back to the playoffs year in year out and make deep runs year in and year out in every sport. And actually, across all three sports, there's no indication that there's a parody difference. In fact, if you look over the last 10 years and just say like how many teams made it to the championship, how many total teams as a marker of par, the baseball has the most, I think.

So, um, so baseball has the most teams that have made it the farthest over the last 10 years. Now, the Dodgers and Astros have are over represented in that sample because they've had really, really good franchise runs over that time. So, baseball doesn't have a parody problem in that respect. further. Uh, the Dodgers were the most improbable World Series home run away from losing the World Series last year to the Toronto Blue Jays.

It's like it it was like this close. It was they were they were they were two outs away from losing the World Series and they were needed a guy to come back on zero days rest to get them out. So, it's just like they were an inch away multiple times from losing. And so, yeah. So, that's the first thing. The second thing is it is sad that players move teams, but I just don't know another way to do it because otherwise the players get really really screwed.

So, I get why all Detroit fans would love Terco to be a Tiger for life and that is the fun of rooting for baseball. I'm a The only professional athlete I relate to like a child anymore is Clayton Kershaw from the Dodgers and he was a Dodger for his whole career and it's like it was special to me. It it meant something to the city. I was there for his 30,000 strikeout at Dodger Stadium. People cared about it, you know, and I think it was sort of lovely.

U so there's my short take. Okay. So, you don't think there's a problem? >> No. And the reason the Dodgers are so dominant right now is twofold. One of them is Show Otani. He is a oneman cash machine. And I think people do not understand this. The average World Series game, I think, had like 16 million viewers and like 20 million for game seven or something like that last year in the US. But in Japan, I think it was like 50 or 60 million.

It was like it like the Dodgers are just raking in sponsorship dollars because of Shoe and to some degree also Yoshino Yamamoto their other Japanese star. Um and so so it is not and he is a he is he is a generational player. So so that is like a crazy thing on its own. Um secondly uh the Dodgers organization is excellent. I think what actually everybody's mad at is that they're really good at it. Um, but nobody complains about the Mets payroll even though they have the number two payroll.

That's because the Mets are terrible. They suck. And so nobody thinks that if you need a salary cap to stop the Mets from spending all that money. Um, here's my hottest take. Parody is a stupid idea. Um, I don't actually want the Brewers to have the same chance to win as the as uh the Dodgers. And that's because nobody lives in Milwaukee because it's too cold and it sucks, right? I'm sure it's a nice town, but it's too it I'm sure it's a lovely town.

I'm sure it is, but like 15 people live there. So why would you want the Brewers to have the same chance to win the World Series as the Dodgers where >> 20 million people live or whatever? If you want to maximize >> It's all we have. It is all we have. [laughter] >> No, you should move then. You just told me you moved back. >> Fabric of this city revolves around sports. We need it. What you're saying is you want like 45 people to be happy in Milwaukee so that 20 million people around Los Angeles can be sad.

That's what you're telling me. So like why do we care about uh about those teams having the same chance? The big markets should win more often and it should be proportional to that. The biggest if a perfectly designed system would have the biggest markets winning more than the smaller markets because I want the most people to be happy, right? So I don't really care where they live. I just want the most people to enjoy baseball and that means the bigger markets should win more. >> All right.

We We'll have to do a follow-up because we have to [laughter] go. I want to talk about how the Dodgers have this ridiculous television deal based on their bankruptcy that is essentially cheating. >> I want to talk about how you guys caused >> Yeah. The Tigers would really win a lot more if they got the$2 million more dollars a year from that deal. Probably >> it's not 2 million. It's not 2 million. Uh >> spread out among all the teams.

It would be Yeah. >> How you guys have single-handedly caused next year's lockout. Yeah. >> And I want to talk about how >> just buying what the owners are selling. >> Uhhuh. >> This may totally backfire because I think the Dodgers have lost like five or six straight. So we'll I'll leave you there. >> Yeah. Okay. All right. Thanks, Olivia. All right. See you. >> No, I I I um >> It's great. It's great. >> I I think this is continue that continue that conversation with uh with Taylor. >> Okay.

Okay. I'll do that. We're gonna meet next week. This is a great random show topic. >> Okay. All right. >> All right. >> Thanks. Bye. Bye. [music] >> Big thanks to Olivia for joining me once again and do follow up with her and with House if you're the right brand. I think you probably know her and what she's doing. [music] I just think they're really great. They offer a great service for brands that are in their space where the intelligence really does matter.

They'll talk you through the statistical concepts that are challenging for you. They'll help you think through what the right thing to [music] test next is. They're just a really good team who's really going to help you. So, go follow up with Olivia there. Don't forget to subscribe wherever you're watching or listening. I'm going to try and have Olivia back every couple months. We'll see if she has time for it, but I want her to be the data correspondent for the Andrew Ferris podcast where she's bringing the best and newest of the studies that she's releasing and working on uh for at house because she just has access to like she said thousands of studies and >> [music] >> uh so much data.

It's really helpful to understand. So hopefully she'll be back in a couple months as well. And don't forget to follow up with me. If you want to work with with AF Growth, you can go to afgrowth.com, fill out the intake form there, and as soon as we have availability, we can um talk about getting you working with us. You can also email me podcastfgrowth.com. I'd love to hear from you. If you have any thoughts or questions or anything like that, any questions you want me to do a podcast episode about, shoot me an email there.

You can also leave a comment on the description of this episode or in the comment section of this episode. I read all of those, so do leave that there. Bunch of great episodes coming up. Like I said, do subscribe. Not only Olivia in the future, but Taylor's coming back soon to talk. I've got Patrick again. Uh, in the near future, I'm going to have Mab coming back every couple months who I've had on recently as well. Just a bunch of a bunch of really good episodes.

Like I said, thanks so much for watching or for listening. I'll see you next time.