YouTube Ads Perform 342% BETTER Than Google Says – Olivia Kory Shares The Data transcript

holy crap this is an incredible episode Olivia Corey from house analytics came with incrementality data about YouTube that is not findable anywhere else on the internet how much value do YouTube ads create she's going to show you right now with tests run like across 190 different experiments serious incrementality testing across every aspect of how YouTube ads work if you are serious about advertising in your business you are going to want a strap in this is a great conversation with a whole bunch of real data shared she's got slides uh it's incredible let's jump in we have actual data with incrementality studies that is like we're we're like legitimately breaking news right now on this podcast this is a really actually I think a really important conversation um Olivia Cory hello how are you hey doing well how are you good you are from house analytics um I I think fair to say the leading um incrementality analytics firm in our space in DDC and e-commerce running serious holdout testing incrementality testing for Brands about the effectiveness of their ad spend um in my view incrementality testing the best thing that there is in terms of attribution type thinking obviously slightly different than attribution but um anyway you you're breaking news you have told me in preparation for this call that uh you have data about the effectiveness of YouTube ads um and so tell me so I I don't know do we want to just jump right into it because this is like the question right like everybody's spending a bunch of money on meta I'm like most of us like me are sure that YouTube does something none of us trust Google um like let's talk about this a little bit can you tell us like maybe to start Olivia how you got to the data you got to and then we can jump into like what what do a wide range of incrementality tests say about the effectiveness of YouTube advertising for DC Brands I'm so excited uh in terms of the timing of this podcast that it worked out such that this is hot off the press um we we just did this analysis and and uh you'll be we'll be breaking it for the first time on on this podcast and I'll I'll set up the data in a moment but just this might help in terms of of introducing myself u i was a you know former growth Market our uh brand side at companies like Netflix and when we were running incrementality testing and quby you said right andby I tried to skip that part uh thanks for remind yeah no no yeah we because because before we started we were talking about pre failures and we both had some so you you uh yours was failed with much more money behind you than mine but I think probably equal in terms of the pain so no I saw in your your advice like lessons for Ecom is that that failure is important and I I very much stand by that yeah um but we can cut that out if you want Olivia no I love it I love it yeah um at Netflix we so I was I was on the the US Performance Marketing team we were spending as much on YouTube as we were on meta um we were very heavily concentrated in Google and meta but within Google we were primarily spending in YouTube and so I had this prior based on my experience in incrementality testing um that YouTube can can really like hang in there in terms of performance and over the course of a few years and hundreds of incrementality tests uh We've we've been able to to to see this play out and so I have uh data from 190 tests that we we ran at house across um you know vious categories and industries but I'll go ahead and just kind of okay wait before we do it before we do it let's let's set this up a little bit more let's get the let's get the people salivating a little okay let's talk about that basic thing you just said which is like your prior was that YouTube can be valuable and so I I want to say something about this like my clients are mostly um well my clients right now for my very Boutique agency right are all eight figureure Brands um but they're pretty generally low eight figure or if they're mid eight figure they're multi- channel so the D Toc side is like Loy figure so um and they used to be seven figureure Brands and so they're growing meta spend it's going well they're pushing most of their money there it's the first and best place to put your dollars in a d2c business I I I have a feeling you agree with that maybe you don't maybe you think people should be investing in YouTube earlier I don't know we're going to find out in a second what you say about that but in any case like just my prior also is that YouTube is just videos like like just sort of if you just consider what the ad is what the ad unit is there is no a priori reason why YouTube ads should not sell products to customers like um it's not like there's something magical I think about the newsfeed and especially when you like get the way the the payment works on YouTube and you know you only pay for actual videos viewed and some of those things like it just has always been strange to me that it has been so hard to validate the effectivess of YouTube I've been interested for like at least five years and taken little foray into YouTube advertising because it's like this has to work and the only reason it wouldn't work so far as I could tell op prior would be the a matter of essentially um conversion optimization engines like that the idea would be that like Google can't allocate your dollars towards the right person is effectively and efficiently as meta could and I think that's definitely the likely explanation for it but there's this other problem which is that nobody trusts Google's reporting and if YouTube works as a lot of people have pointed out it is there's less clicking happening right so like essentially if YouTube works this is another thing that I think has been pretty well established if YouTube works it is not it is it is not measurable on a pure click basis the way that meta is I think so those are like my prior coming into the conversation okay so YouTube has to work like it just has to secondly uh if it doesn't it's because of the con it's because of the efficiency of distribution of things uh of conversion optimization and third if it works it's harder to measure because it's not based on the click okay that's that's where I'm coming to it so I don't know if you want to say anything else before we start to talk about what you've seen and and sorry one last thing before we do this which is um I have wanted aggregate data like this most people in who are smaller DDC Brands cannot afford yet what you guys offer at house even if we would like to do it um and so I've wanted like a baseline heuristic to work from you know just essentially some some range of outcomes of like how incremental is YouTube relative to what you know Google says or something like that so um okay so you've got data you've got some slides if you are listening to this and not watching it we will publish this on my audio feeds too um Olivia is going to show some slides right now we will talk through them as best as we can um but if you want to see the slides go to YouTube or um Spotify which has video um and uh and you can check them out there so okay let's do it awesome okay you can see the setup slide yes all right I want to hit on some of the points you just made in terms of of of the prior that you come in with um YouTube uh ad Revenue $10.5 billion in Q4 of 2024 just a fraction of meta ad Revenue at at 46 billion um which is up 21% compared to to YouTube being up 14% so like and and of course meta's user base is larger which which makes sense and that accounts for some of this but but yeah like YouTube um meta is monetizing its user base much more effectively than YouTube and you hit on it do you know Olivia I'm sorry on the point about larger user bases do you know about aside from user base about like essentially time spent because this is another thing people say about YouTube is that people are like much more deeply engaged with individual pieces of content on YouTube than they are with meta like they spend way longer per sessions I don't know if you have any data about that and it's not if not it's okay but there there is publicly available data on that I think even Tik Tok and YouTube are are much better on time spent than meta interesting um but like meta is just better at monetizing you you hit I think number one here is like their their algorithm is just better like their conversion optimization is better and we'll actually see that play out in the data um so fully agree with you there I think there's a higher barrier to entry for Brands who want to go into YouTube just due to the creative cost and the fact that you need video and you need you need like slightly higher produced video than you might be able to get away with in in meta so let me let me let's pause on that really fast I'm sorry to interrupt you the the um I did an episode not that long ago with Jack Rubin who I've had on three times he is great from um pry and fig monster business in the UK so we did he was he got to $100,000 a month spend on YouTube and he talked through how much money he basically had to blow figuring out which creative worked on YouTube um in a way that was distinct from meta and I think this is a huge thing the bar to entry factor is huge that includes production cost although Jack would would say at least and I'm sure other people would say it's actually not as high of a bar entry as you think uh in that respect like you don't necessarily have to make the most beautiful campaign you've ever seen uh let's take for example Taylor holidays uh breaking of X with his ovas you say that brand name ad it doesn't have to be that I don't think um and that was certainly what Jack discovered but I agree with you there's a barrier to entry problem and essentially like a you have to be you have to have enough money to start wasting some money at least on like traditional horizontal YouTube yeah anyway go listen if if you're watching listening to this you should go check out that episode with Jack it was a really helpful conversation for me on this I'll put I'll try to put in the show notes here so there's a link to it yeah that team is awesome they've been working with us to crack YouTube and it's just been amazing like seeing their journey through a series of incrementality test and how much they've been able to to unlock in YouTube um we're going to I have this as an open question at the end of there like four or five slides but at the end it's like do advancements in AI make that barrier to entry like far lower yeah I wonder yeah I wonder I had the same thought if you are spending serious money on ads and landing pages or any part of your strategy you ought to be using firmont firmont is the Ferrari of landing page software because it is Way Beyond Just landing page software there goal is to make it as easy for you to create post adclick experiences like Landers and enhanced pdps uh to make it as easy for you to do that as if you were actually just making ads themselves like and and it really is unparallel and ability to do that in in my experience I used I've used forat for one client of mine for a long time now probably going to add it to a second one very soon because the ability to create the fastest loading best performing landing pages I've ever used like they're really smooth you can do it extremely quickly you can measure everything in platform really really well I've got a brand for example that is subscription focused and you can see the percentage of subscribers versus onetime purchasers on every Lander where you give both options like right there in the interface it makes it really easy for us to report and optimize and build all the things we want to do and there's AI tools built in to generate the content extremely fast on top of that you can build Landers specific to different ads and funnels you're running really easily including even enhanced pdps and custom upsells again specific to each ad you're running so let's say you've got the product that is has a lot of appeal to both men and women but in different ways to men and women you could build a men focused Lander in PDP and a women focused Lander in PDP or a different age one or different use cases or whatever it is again very quickly very easily and at real scale it's an incredible piece of software I really love it I love working with them in fact at some point we were not working together for a little bit after working together uh just because of the way things go and I beat down their door and said we've got to work together again I really love from and what you guys do so go to formont commerce.com forat commerce.com because I believe in it so much uh you can check that out and get started with Vermont today meta has a meta is just it's it's easier to attribute performance with meta just based on the way the product works everyone's logged in um there are clicks as you mentioned whereas with YouTube most of this exposure is actually happening on a TV screen like nobody's clicking on YouTube ads um and and also in in it lends itself to being more of like a kind of longer term brand play in terms of of of a view through Channel where you're just getting more and more exposure and just less prone to click So Meta has the advantage here just in terms of its its product being better suited for direct response advertisers and like easier to attribute performance to meta ads um this was the thing that blew my mind recently somebody pointed out is just how much YouTube happens on TV screen like I still think of it as computer and phone because that's overwhelmingly where I watch YouTube but that apparently is not true like most people are watching on a TV screen now or at least a huge portion of them is that right yes I think it's it's two-thirds uh uh is on is on TV screen yes is that because of YouTube TV like is that are the is it is this is this analysis going to be covering ads that are sort of like interstital in YouTube TV is that part of it or something we have YouTube TV in the analysis but it's small um okay all right these are like this is like YouTube on this is just generally this is just generally how people are consuming YouTube YouTube on TV is just like the the best connected TV platform out there um that's fantastic okay great and just to to again to set this up um the reality is that nobody's spending as much on YouTube right now as they are on meta so when we when we go through this analysis remember that these brands are spending about 30% of the per day Budget on YouTube as they are on meta so they're spending about onethird what they spend on meta in YouTube which probably has some effect on like that means that among other things that people who are watching your YouTube ads are also seeing you elsewhere at three times the volume and maybe a maybe a cheaper CPM I don't know you'll probably able to tell me that Y and it's I mean you know I'll be the first to say that it's not a a very fair comparison if you're spending onethird the amount you're spending on meta like and so just take it with a grain of salt is that that the budgets are not equivalent here but if you could spend 30% more effectively and we'll find out if it's effective or not in a second here um that's a really big deal especially if it's growing and getting better yeah especially if you think about how like these first dollars in YouTube are likely more efficient than spending that budget in addition to everything else you're spending in meta um it's like marginal efficiency right so that's right that's right um okay this is the fun stuff we we don't often share this data so again I'm excited to be able to to bring it here um but it's just so striking and and so powerful summary Point number one YouTube holy conversion campaigns drove a 3.42 X incrementality Factor just on D Toc so what this means is YouTube in platform metrics are Under reporting the incremental impact by the tune of about 70% um and that's Notting for that's just uh just to say that's not even for like Amazon or retail impact so the if here the incrementality factor of like what the platform is reporting versus what you're actually getting is 3.4x that is crazy that is so much okay I have a couple of questions right away I'm going to let you go through this this slide that way I don't keep interrupting you but then I'm going to come back with questions one of them though is uh let's just clarify that metric a little bit more and you tell me if I'm repeating this back to you correctly so that everybody's tracking what you're saying what you're saying is if YouTube says that I'm getting a one toone row ass like if in my Google ads reporting dashboard right so I put in a dollar I got a dollar back in Revenue what I'm actually getting back on average is $342 now this is an average averages are evil this is a range of things let's have all those caveats here Olivia is not saying I don't think that everybody should immediately go and multiply their revenue by 3.4 and assume that's definitely what's happening but but but but um am I stating that correctly $1 in Revenue reported in Google is actually $342 in uh in incremental value absolutely yep okay and the the average is important here range of incrementality factors when we looked at this data and again we have a a large sample size here we have 190 tests um the range here of incrementality factors is 1 to 10x so it could be as much as 10x whereas if you think you're getting a one you're actually getting a 10 um but it also could be as low as one so so one to 10 is the range 3.4 is the average okay I have so many questions keep going I'm so tempted to just keep interrupting you go well you I'm GNA come back to them yeah you mentioned this at the top and we can spend a lot of time here but you you mentioned mened um that this this platform is like notorious for Under reporting what's surprising to me is that like Google has allowed that to happen like I know how could like the this this money-making machine be so bad at reporting the impact of its ads on YouTube like it's it's uh it's surprising to me usually we see the platforms send over report and like take too much credit and so this is just well that's what everybody says and I think they're partly that's because in part Google has like abused them all with branded search reporting which is like oh you're getting a 15 to one or a 10 to one and actually like the incrementality is like I don't know do you have a off top of your head metric for Branded search uh incrementality Olivia I I don't want to go on record with that fine that's fine um it's much lower that's the point you're not actually getting if you were actually getting $15 back every dollar you put in or whatever then you would spend more there um okay so but the um the but so that's that's the reason why right everybody is a scarred from that but I have long believed long long long believed that in fact meta certainly is Under reporting um in the windows that it gives you especially if you are paying attention to click based reporting um and I've heard this comment from YouTube for a long time what I actually Wonder though is if it's actually true that YouTube Google's actually Under reporting or if it's that people are only looking at click-based roas essentially they have selected their attribution settings to be that and if you were to actually factor in view conversions it would be a different story yeah that's a really good question maybe when you when you have when you're opening up to view through and meta your you ought to be highly skeptical yes you want to be skeptical but maybe in YouTube it's a little closer that's that's the way I've thought about it for a little while but but I'm such a meta background person that's my time that it's still hard for me to like break the habit I still don't think that would cover it though because I'm sure exposures happening on TV and like it's not the case that users are like logged in to YouTube as much as they are logged into meta I don't know if if even Google would be able to to connect the purchase back to the exposure even in a viewr model yeah okay yeah I'm sure that's true I'm sure that's true this episode is brought to you by my friends at more Staffing more staffing.com is the place to go get started with more Staffing very simply more Staffing gets you access to the best Filipino talent for your e-commerce business and that is a huge win for you for a couple of reasons the first is that um is that you can access incredible Talent working in your DDC business like at higher levels than you think if you think of Phil Talent as virtual assistant only you know $5 an hour or less you are missing out on the opportunity for what is available for you you should be hiring up Market in the Philippines to get experienced people with Deep D Toc resumés across every part of your business whether it's marketing operations supply chain anything like that at manager levels and up who can make a real impact everywhere you go uh or everywhere your business is working and everywhere your business is operating um and the the beauty of more Staffing is they connect you not just to Filipino Talent which is definitely less expensive ensive than hiring that same talent in the US but they're going to also connect you to Great high quality talent and the way you know that's true is that they give you a one-year guarantee on anybody they help you hire which is way longer than what most recruiting firms do so if they don't if a if you hire with more Staffing and that new employee doesn't make it for a full year in your business they will rehire that position for you at no additional cost that is an incredible guarantee and it speaks volumes about what they are able to accomplish in terms of connecting you to incredible Talent my business has been transformed bying with more and their sister companies and I really mean that everything about I was just in the Philippines a couple months ago um I can't tell you how much I love working with these people they are great high quality humans who really care about doing a great job and connect you to incredibly talented people um every time I think to myself um about something that I could accomplish better in my business I now think how do I go higher in the Philippines to make that happen uh if it's something that I need to hire for and anytime I um I'm thinking about what is possible there they exceed the possibilities I just I just can't say enough good things about Mor Staffing they've been a sponsor of mine for a long time for a lot of reasons and I'm a huge fan go to Mor staffing.com staff. or find the link in the show notes to get started with them today you're gonna like this one um okay this this this stat was was was striking so halo effect I know Andrew you talk about this a lot of omni Channel Brands who also sell on Amazon or in retail in addition to D Toc YouTube drove an average halo effect of 99% across Amazon and Retail versus just 53% for meta so to to to um to kind of play this back to you if your example you were seeing like a one incremental ioz on D Toc what we're saying here is that you're actually getting another oneish of like iroas on Amazon and Retail so it's actually yeah twice twox better than you thought based on the fact that that these ads are driving Amazon and Retail impact to like to the tune of D Toc impact and meta is also driving Halo effects but just not as not not not as much like 53% so in this example if you're seeing a one iow as in D Toc it's probably another .5 irow as in Amazon retail I mean that's actually I actually think that meta stat would shock a lot of people um and this is a mistake I Taylor actually recently recorded a great podcast episode about this just like the impact of your meta ads on your Amazon account the siloing of these different channels as their own pnls is a gigantic mistake um it's it's it's a it's actually a huge mistake for Brands because obviously um you know these are these are actually having um your ads are having a really big impact on your other channels and um yeah I think I think it's a it's a sort of fascinating thing I'll tell you one thing I would do with this data if I didn't have the if I couldn't run this test for myself um and I had a brand that was Omni channel that was like primarily Amazon and then some other retail as well and looking to grow I would what I would do with this and this this anticipates a conversation we'll have in a little while but um the first thing I would probably do is I would forecast um an impact based off of this this data as an average and then I would just try to measure the revenue over time against against my forecast and see what was happening um and I probably wouldn't make a huge bet on this but like for example what I mean is like if I was like okay I'm gonna spend $100,000 more on meta um and I'm going to spend that money at a two to one let's say I exactly achieve that on my D Toc number then I would just basically forecast out an additional dollar the 53% you just said I guess A1 and three cents on every on everything I um do6 cents on on everything that you just did uh I would forecast that into my Amazon spend growth as well and if that ended up happening then I would keep spending it on that same assumption and of course if my Amazon growth actually underperformed that number then I would just adjust my forecast and adjust my spend accordingly because because and that's that's I I mean again you may think that's a bad way to do this but that's the way I would initially think about this I think either way this is like this is a huge impact and people have got to take this seriously because uh because it is actually one business your in your uh The Impressions you're getting across the board with customers the awareness and brand Equity You're Building as your ad runs as somebody sees your ad sees your value propositions whatever sees your brand building all those things and they they respond they are going to go to the places that are the most convenient for them to shop and they don't care about you know about providing you back clean metrics on your DDC site or whatever you're offering there right they are so happy to go to Amazon and get it in two days or less for free you know so um so once you've built the impression it just makes all the sense in the world they're going to go collect value in the places that they want to you know or provide that value so to speak it just it totally changes the calculus on like whether you invest in a channel like this because if you if you need a two to one to to you know break even and you think you're only getting a one to one then turn it off you know like yes but you know if you if you're actually able to understand this then you can really drive like meaningful growth to the business by continuing to invest so it just yes really changes your decision framework which is is something that um I've I've um like I've opened my eyes to because I also used to think and I wrongly assume this um and again like I've I've learned um through this series of experiments uh you know in adjusted my my thinking is that like if you just like drive people to D Toc that they'll buy there I used to think that like if you link out to your D Toc site that they're gonna buy on your DC site and that's just to your point just not at all how consumers work um and so again the data has like continuously proven me wrong on that assumption with Amazon in particular this is just such a like as a consumer right you know the category matters a lot here too but like with Amazon in particular just think about like this is what I always tell people about this Amazon is the greatest shopping experience in the world for things you need you click a thing you get it for with with uh with incredibly sometimes same day shipping and you get it with no shipping cost same day shipping to your door for free why would you do not do that especially especially if the offer is the same in both places if you're going to charge for shipping on your site versus elsewhere right there's like all these little things that skew it more towards that direction so yeah this just makes all the sense in the world to me and is exactly what I would sort of expect and that's before you talk about other categories where like you know like I used to work with um with I don't think she'd mind me saying this with cirri uh deodorant brand and you know they they've they they launched every Walmart door at one point which Sarah had said on my podcast at one point they recently just announced that they launched um a bunch of Target doors it's it's a deodorant like yeah if you're running that ad people are going to want to smell it you know what I mean so like it just makes all the sense in the world that your ads would impact in stor sell through like somebody's going to click on your ad and then go to these other channels and buy them where it makes the most sense you know if they're already in the personal care section at Target like why wouldn't they go oh I've seen that somewhere I saw their website what does it smell like and then go take off the cap and smell it or whatever you know just it makes all the sense in the world so people just have to kind of like keep that straight I think and if they if they don't I think they're they're prone to make very big mistakes about underinvestigated influences kind of behavior over a longer period of time and YouTube's impact is is is certainly I think more pronounced the longer you run so the stat here um if you're not if you're not reading the slides with me is that iro has improved on average by 79% during the post treatment window versus only a 37% Improvement for meta campaigns which means that over time if you include the post- treatment window which is often you know two or three weeks that iroas improves by almost 80% from the end of the test through that post treatment window for YouTube and it improves about 37% for meta so the the long-term effect here and it's not even longterm it's like medium- term it's a few weeks after the test ends is like significantly more pronounced with a YouTube test than we have seen with meta that 37% number for meta fits almost exactly the estimate I make for nearly every brand I work with on like the sort of impact over time we I just call this with my clients the the delayed purchase Behavior or the delayed purchase multiplier um and I I build this into our targets um very much on purpose and the again the idea here is really simple and is super intuitive if you think about it but I think people have a hard time with this if they get too much into sort of click-based measurability which is that people take time to make a decision like I don't know maybe they see something they want today and they wait two weeks for payday or maybe they need to they talk to a friend about it or they research or they're just lazy or they forget I know who knows you know all kinds of reasons people delay their purchase and um and so yeah this this is fascinating that fascinating to me that YouTube is so much better I I have a question about both of these these two that maybe is a good that we can kind of lump these together both the larger halo effect and the larger delayed impact that you're talking about here um why why do you think YouTube outperforms meta on both of these I have a theory but I want to hear yours well you've gotta you've got to say that like the the fact that you don't click on YouTube has something to do with it where it's like it is just inherently view based so you're you're getting exposed to the these ads but you you don't really do anything after and so I imagine like you need frequency there if you're not actually um like prompted to take an action um and we we've seen this we've seen this play out anecdotally with like Ott as well where we see a larger impact on Amazon and Retail than click platforms like meta yeah it just has to me it like has to come back to this idea where you're like you're just not clicking here and you're not researching and thus it's going to take longer or like the second I guess second theory and that's going to come to point number four here when we get to it is that YouTube is prospecting harder like YouTube is potentially targeting like a a less warm audience and thus it's going to take longer to move them through this funnel then it's going to move somebody who's a little bit warmer in meta that's interesting um my theory is just that people are having a deeper impact uh a a deeper a deeper engagement with the ad that essentially like this may be this may not be right but just that like my understanding of YouTube ads is like you don't you get like essentially 30 seconds before you have to pay for it like somebody spending 30 seconds on your ads a long time and so um so yeah I I mean the the the counter argument to that is that a click normally results in like a minute on your website and that's a really deep brand impact as well so the click also produces that but just that I wonder if the impressions are essentially just higher quality impressions in terms of the amount of attention somebody has I also know people talk about with YouTube again it's sort of like less of a like sit and scroll behavior and more of like I am deeply engaged with the content that I'm consuming behavior and that that may also lead to like a brand impact um for for folks so those are that I think I think your answer is also uh reasonable like I think all of those are speculation but they're reasonable but it's it's a really interesting it's a really interesting thing I know we haven't disagreed on anything yet Andrew we we'll get there we'll get there um other thought I had when you were talk do you want me to do you want me to antagonize you more that make a better podcast um no I just I was I was expecting it but this is this is this is fun too um no did did you think that I was anti YouTube coming into this no no no no I just I um I I like your contrarian takes I my contan what do you have a sense of what my contaron takes are I would love to know what my I think some people have accused me of hot takes at times but I actually feel like I'm constantly couching everything and nuancing everything I I sort of like I don't think I've ever tweeted a a take that I internally thought was hot in my life I was and and we can cut this out but I was listening to your interview with Barry where you were just oh yeah very much convinced that um that cost caps like don't over don't don't deliver or overd deliver to warm audiences yeah you're like no I'm I'm definitely right on this yeah yeah yeah well Barry's a good example Barry like comes on and knows that we're going to do that and he came on specifically wanting to argue with me that's a fun episode if people want that I'll put that in the show show Notes too if you want to hear me argue about that yeah you mentioned something though here that I think like I talk to our house clients about this all day which is not getting too hung up on like the exact iroas or cpia number from a two or three week test for these two reasons that we just talked about of like this is short-term efficiency like there is definitely latent impact of ads that we can't captur In a three-week experiment and so I try not to focus too much on like the exact number and more so how do we improve that number over time like if it's a 1.5 how do we get that up to a two in the next experiment that we run or you know like a three over time instead of like just anchoring on the exact number because they're just it's just so obvious especially for these you know like higher aov longer consideration products that there's there's value here you're not capturing In a three-week experiment and that's just the reality yeah yeah I think that that makes a lot of sense okay let's do the new customer impact you you referenced it a little bit but maybe just restate it just so that we're yep for folks who aren't on the slides this is um summary Point number four which is that YouTube is primarily driving new customers and new customer Revenue so 76% of the DDC impact of YouTube was on new customers versus repeat that is an astounding stat I like love it because we know how important it is to be prospecting and and bringing new customers in and other platforms have struggled with this especially in the world of Google search where it tends to over IND index on on repeat purchase so yes YouTube is is definitely a a um a prospecting Channel and where we do see lift it is primarily amongst new customers this is where uh this is the thing that Cody plofker has has harped on a lot with YouTube um which is that like he is frustrated with his meta span because he's just not reaching you know sort of percent new visitors something like that is something that he hammers a lot let's kick this around for a second I think I've heard you say or maybe seen tests from other people that that doesn't necessarily mean it's more incremental that essentially like retargeting can be very incremental um maybe I haven't heard you say that maybe I'm making that up may I've just heard Taylor say that but um what do you think about that well here this is incremental new customers like all of these results that I'm sharing here are basically incrementality tests so this is incremental and new customer but people fetishize this idea of reaching new people as if it's the as if it's definitely the next best place to put their dollars but it's not clear to me that's the case that in fact sometimes retargeting dollars is is exactly where you should be spending your money for actual incremental sales lift like essentially now retargeting I should Define I don't necessarily mean repeat customers although it could mean that I mean people who have at some point already you know people who are warmer or whatever it is so I do you see what I'm saying does that distinction make sense yeah well I think there's like I and I remember this debate from Cody's app Lov and test where I think it was you like well why is this bad it was it was incremental there's there's you can be incremental at driving both new and repeat Revenue yeah and it's not necessarily a bad thing if you're driving incremental Revenue that would not have happened anyway amongst returning users but I think say is that it kind of breaks the model if you spend this money to reacquire but it breaks it positively it breaks it because you make more money than you forecasted because your model should be built on the fact that you weren't spending on that channel before that's the whole idea of incrementality right the idea would be that that like essentially these customers now that I'm spending on this channel I'm getting custo I'm getting purchases I was not previously getting therefore if it breaks your model it only breaks it in the positive direction because now you're adding purchases at theoretically at a cost index that's uh or value and excess of the cost yeah this is where I tell Cody he's like playing on advanced mode I was like this is you're kind of being unfair to app in a way because most brands don't even care about this distinction like if this is revenue that would not have happened otherwise then we should be happy to like that're spending to acquire but um but I it's it's like there's this idea of pull forward like how do you know you didn't just pull forward so that's the better counterargument that's what I struggle with of like if we waited another month or two would this Revenue have come in without paying for it this this is the better counterargument I think and it would be interesting to see if there's some way to design a test that would actually do this where you could like have like a three-month treatment window or something like that where it's like like Could you actually do something that allowed you to do this because in my view that that and I know exactly which sort of Twitter debate you were talking about but like I that comment mostly didn't come up from people arguing with me and I was kind of waiting for it because it's the actual thing is like is it actually truly incremental or did it just happen faster um now that still may have value for all kinds of reasons but it would probably have less value than if it was truly incremental I'll tell you in my experience one thing I do is I allocate at least a little bit of money this is the way I handle this this is a sort of a side note at least in meta I I allocate a little bit of money to specifically targeting returning customers um in my ad account and I just do that at a much higher row ass Target than I do um and definitely click only I definitely don't want view conversions in there um but I I am running click based conversions at a much higher Target than I would my um than my new customers and I I mean again just like judging by the bank account and by I track new customer customer returning customer Revenue religiously for my Brands and um and I mean it's fairly clear that there's a lift of returning customer Revenue when you spend some money there again I just I'm doing it at like probably 3x at least the Target that I'm that I'm doing new customers at allocate a little bit of budget I'm not running bid caps I'm just like couple hundred bucks a day goes towards this see if we can generate some additional money stay on top of mine for people it's a pretty small overall spend and but I think is as long as you go in like understanding the the intention there and like the purpose of those campaigns are to drive repeat I think where where I see these Brands get stuck is like they have a campaign designed to drive new customers and then we break out new verse returning and they're just like always surprised by how much um they are serving to returning customers because like you think that if you're doing all the things in terms of excluding audiences and pointing them toward a new customer event that you're like you've completely filtered out these new these um these existing purchasers but like just the reality is like meta can't do that anymore and so um I think it's always just like it's just surprising to folks when they come in and they see that and they weren't expecting it um but it's still again I think it's still n positive business if you like the way that we are talking and thinking in this episode you will like admission from Common Thread Collective admission from Common Thread Collective is their sort of self-contained course style content where you can get all of the knowledge that you need to grow a profitable D Toc brand on the back of meta ads I mean the really simple way of putting it is that if the way Olivia and I are talking about um metrics and Analysis in data in advertising and in d2c Brands if you like the way we are talking and the way we are thinking and you have not joined admission and you're trying to manage your ad spand um towards a profitable growing business as effectively as possible with the best tools possible backed by the most data possible and you have you need to join admission um you can do that by going to your admission. but really what you want to do is do it through the link that is in the description or show notes of this episode because when you do that you not only get access to all of the closed courses the monthly exclusive webinar with Taylor holiday who's been mentioned a few times in this episode um Q&A with Taylor but you also get a free coaching call or if you sign up for a year in advance four free coaching calls with media buyers from Common Thread Collective whove been there for years running manual bids on you know midsize and and smaller and larger accounts every kind of account and know what it looks like to run an effective ad account that way so you actually get somebody looking over the shoulder of your ad account helping you work out how to apply this stuff with your actual business admission is an unparalleled resource it is it is D Toc University it is the place to access all the intellectual capital on the DDC balance sheet as I've been discussing in this episode um go do that go follow the link in the show notes to join admission or just tell them that I sent you you go to your admission. and tell them that I sent you um and you'll get access to that offer it's really awesome go check it out today I mean there's not even really a right or wrong here it's just it's it's just understanding understanding the role that each thing is playing in your business and then like sort of trying to come up with a Target that makes sense relative to your goals for each part of the business I think and what kind of business you I mean Cody right is is consumable cpg like you know he's running a a makeup brand basically right um and so like that might be really different than like I think about simple modern here who I used to do some work with as well where it's like there're or an apparel brand where you're constantly releasing new products you're not I mean certainly consumable cpg uh would have some of this but what I'm saying is there's a really big difference between getting somebody to refill a product that they already loved or to buy a new version or new color or new something of a product that is not a consumable and so how you relate to the question of how to spend on returning customers and the and the incrementality of new versus returning is going to have some things that um you would probably need to think about uniquely for your business because they're just so different you know like one thing we're starting to do for my my subscription supplement brand or or one of the ones that I'm working with is like we're starting to spend on some returning customers specifically towards the goal of generating additional subscribers so we're trying to exclude everybody who has subscribed and then put a big offer for people who have bought but who are not subscribed and try to generate new subscribers to me that's like well worth it because subscribers are just worth like you know three to five x what a non-subscriber is and yeah so anyway there's just a there's just really different approaches here depending on what you're trying to do and um yeah so yeah that makes sense th this always um just makes me chuckle because like we we are talking about like the most advanced kind of like sophisticated way to do this and most brands aren't even measuring incrementality so when we start arguing over like the incrementality of new versus returning on Twitter I'm always like this is amazing to see just like how quickly we've we've matured as a like as a community because most brands I work with are like not even CL they're just like still trying to understand incrementality in general um Olivia I can't tell you how important this point is to me I I just I the way I always illustrate this is I go back to like the way I always phrase it right is that the the intellectual capital on the collective DC balance sheet that's the way I think about it the balance sheet that we all own together okay um that you you and I both have a part in this balance sheet okay um the intellectual capital on that balance sheet is like so much higher today than it was when I started in the industry right I just wrote like a 10 things I learned after 10 years in e-commerce post I didn't even put this here but like I still the way I illustrate this is the first piece of content I remember recording at CTC when I was there early on was how to define your CPA and why it's important your C like it's like I mean can you imagine releasing that piece of content today like I mean I guess somebody out there would probably be helped by it but like it is just like it's crazy yeah it's just illustrating that point that was the world we were in and that's because DDC was a newer Channel and a newer way of thinking from a bunch of people who had been in other kinds of businesses where they just didn't think that way down to the level of customers and so yeah I I think you're completely right about that this is a a hobby horse of mine it's part of the reason bullish on e-commerce the industry is just getting way smarter um go ahead sorry did you want to say something about that oh no just that that like and that's when again I it's it's kind of just been accepted as like common practice to to remove like view based um yeah like conversions from meta but when I'll start with a brand like they're it's not that's not just default like they're they're reporting an 8 to1 or like a 9 to1 row as because they're including viewbase conversions and these are like larger businesses and so a lot of these things like that that I think this community just like kind of just um like inherently understands is is like not at all the case and I work with like some pretty modern sophisticated smart yeah right yeah but this is this is uh this is the way influence Works in communities right is that like essentially I think these conversations are good for that reason like people people who are like you who have access to this data set that so few people have can then disseminate that content we can have a conversation about and I've been in the industry for 10 years and have learned a lot since my videos about CPA days like and you know Cody I'm sure would say he's learned a lot and Taylor would say he's learned a lot and all these things and then the conversation happens and can make it to people who who got into the industry last year or two years ago or or their brand grew 100% And I mean you know 10,000% or whatever in a couple years and they're just opening up to all kinds of new ideas they didn't even have time to learn you know whatever so um yeah I think that's I think it's important to to do it because even though these situations are different it is it is a significant thing hey I want to ask you about something with this data set really fast which is um which is to come back around to the thing you mentioned in the beginning which is if the 3.42 X incrementality Factor against you Toc sales is YouTube I have two clarifying questions the first is is that YouTube spend including both sort of quote unquote traditional YouTube and shorts or is that YouTube only not including shorts because I have a lot of questions about shorts okay great you're going to get there you just changed slides okay if you're going to get there don't answer that question for a second I want to ask a different one first okay um so go back uh go back to the previous slide if you would really fast we'll come back around to my question in a second because you've got slides about this okay um you are at house analytics house analytics is expensive um relatively speaking I don't know if you are cool with me saying it that way but um but I just mean it what I mean by expensive is not that it's overpriced your Al to the value just that it it requires a large investment you got to be spending a lot of money with you guys for it to make sense because you guys are doing really sophisticated stuff and I should just pause here and say despite you telling me I didn't have to do this I'm going to say if I was running a larger brand uh I would absolutely be working with you guys to figure this out especially if I'm multi Channel like if you are a larger brand listening to or watching this you should be getting on the call with Olivia or whoever on your sales team you point people to to say like okay is this something that I should be investing in and if you're at probably at least two ad Channels with significant spend and probably especially if you're at three you really absolutely should be working with house um they're great I've seen a bunch of tests uh the stuff you guys are doing is mind-blowing okay um so there's that first of all follow up with Olivia the links are in the show notes DM in the relevant places go to house analytics. or house. and do that anything you want to say about that really fast no let's get into it in the plug portion of this okay okay here's my question the range of in incrementality factors you say is 1 to 10x okay so if it's it so so um 3.42 is the average um is that range a bell curve uh essentially do most of them end up in the in the middle uh of that and then there's outliers on either side that's a fantastic question I I need to look at the data need to check I'll check and uh by the way this is a great moment to to shout out Tyler Herer who actually put together this analysis he's inred like brilliant mind um so appreciate it and that's a fantastic question in terms of the the range I I I would assume that that that that's that's fair to say but but we'll confirm yeah okay that would be great maybe we'll have Tyler on for a future one of these or something um but the um the the reason I ask is that one of the reasons I'm interested in this kind of data is that for Brands like mine right like that are getting to a point in their spend where they would like to be investing in incrementality but can't um quite yet not with you guys the um like one of the questions I have is like should should we assume the average as a starting place with some error bars or is it so different across so many brands that they actually it's actually not that helpful and my assumption my prior here would be that they should assume the average because most brands are average um right and that's not bad or good it just is that's just what an average is um and that basically or at the very least you ought to regress whatever happens in the individual test to some degree mentally towards the middle because outliers are outliers for a reason um so I say anything you want about that really Olivia like how how I should think about this you know I've struggled a bit here because I've been a pretty vocal advocate for YouTube and then these smaller Brands sign up with us and they launch YouTube and out of the gates it's it's not a success and so it ends up being like not even close to that average um and it's it's much lower and you could I mean my hypothesis here is that the larger Brands just have larger creative budgets like they're used to producing these types of assets for TV and and for other channels and like this is just a new muscle and if you can't really like to your point about about prie and fig like you need to invest like some serious money to actually produce creative that's going to work here and so it hasn't really been in my experience like anecdotally home run out of the gates like for these sign up with us and it really like I have an awesome slide that we can link where a brand signed up with us they started at a one iroz which wasn't profitable for them with YouTube but they were like we are committed to this channel we are going to make it work and over the course of two two or three more experiments over a few months they got it up to a four and now it's like profitable and working hard for them but they had to they had to work at it and so that's where like maybe we can cut the data in different ways to show that um for certain C ories or for certain like stages in terms of maturity maybe you're you're you know further to the left or further to the right yeah okay yeah I think one way to reframe that would be you have to break the meta muscle that you have the idea of like throw a million things at it and it'll work and you're somewhere close to it like you actually have to invest a little bit more in some different ways so um yeah that's interesting okay great um are also I'm just assuming this sample set is mostly larger Brands uh because they're the brands that can work with you guys is that true so we have a we have a wide spectrum of of customers our like sweet spot is you know like 100ish million dollar like kind of D Toc darling Brands who are who are um you know really Savvy growth teams like Aura ring um yeah you know ag1 um so that's like our sweet spot like Bullseye and then we have customers who are much larger in the Enterprise and and and smaller in terms of like Brands it that you're working with as well okay so this represents a decent range of those probably Skuse bigger but not but not crazy bigger that's what you're saying yep okay okay that's good to know that was another question I had about this in general because one thing I thought is like is this is this day a bias towards brands that have already built up millions and million you know hundreds of millions of Impressions on meta by the time they start YouTube and therefore the incrementality factor might be different for them essentially because they're actually coming into it even if they're reaching new customers they're coming into it actually with a little bit more brand awareness in the space Etc I I think there is some truth to that like based on our customer base like they are you know like Sean from Ridge talks about this often where it's like they've been spending on meta for a decade like it would sense that they might be starting to max out there and they need to expand into other channels where like that is you know it's say a large percentage of our customer base are like more mature companies okay you I'm gonna I'm GNA play a game with you and then we'll go to your next slide I'm putting you in charge of a brand let's say that is doing $10 million in Revenue on4 million ion in meta ad spend uh their margins support that so so they're still profitable okay they really high margin brand let's say 4 milli on ad spend um those are the that's the only thing you know about the brand they've spent four million on meta they're growing they they are at 10 million in Revenue would you put your next doll next Dollar on meta or YouTube in light of this you don't get any other information so oh I don't get one test because like H because there's a perfect test to to answer this question which is you just spend up on meta and you put that in a hold out and then you can see the marginal efficiency of that added spend yeah we um Cody just talked about this experiment as well where like that last Tran of spend in Meadow is doing nothing and so he moved out out of meta but like meta was extremely incremental for them um you know driving material no like their spend is just like met meta is um a material like driving a a material perc of sales but like yeah that last tranch of spend is in terms of marginal efficiency is doing nothing and that's where there's this idea of like if you move that last fifth of spend out of meta and into YouTube the first dollars into YouTube could be significantly more effective than those last dollars in meta that said like I wouldn't make made a make a a blanket statement that that size brand like 10 million four million in spend is there yet I don't I don't I don't necessarily think that's true and like I would very much caution against like actioning on on Sean's wisdom of like Hey we're diversifying out of meta yeah this year I think that there are a lot of Brands who are not even close to saturated on meta this is a crucial Point Sean's brand is different than your brand like this is a crucial point it's so I'm so grateful for guys like sea and Cody who spend their time on X and you know them operators guys they it's an incredible it's again this is the DDC balance sheet of intellectual Capital right like they are they are they've answered questions that so few people have gotten to 5 years ago nobody nobody had gotten there like basically it didn't exist the nine figureure brand like this and now they're out there giving advice for free it's incredible but the caveat is you've got to recognize that you are not in that spot if you are not in that spot if you're a $50 million brand I mean listen to every operator's episode religiously you know listen to that over my podcast like for sure but like if if you're um yeah if you're if you're a $10 million brand it's it's probably not going to be that helpful like I'm not saying don't listen to it I'm not saying that at all I'm saying you just got to put that advice in its right place because just a very different situation and if you're $1 million doll brand maybe don't listen like you know like I don't know it's like I just think it's so different so yeah I've had to talk a few Founders off the ledge recently who are seeing all this um all of this discussion on Twitter and they're they're wondering if they should diversify out of meta and I'm like hold your horses like let's not do that yet seems drastic given your size you've been around for three years like you're you're you know you're very young brand um I don't think we should call it in terms of being maxed out on on meta yeah great okay let's go to your next let's go to your next slide about campaign types because this is a big question for me um this is so great by the way Olivia thank you so much for sharing this this is like I'm like I we scheduled way longer than I normally schedule for a podcast for this and I was like a we'll we'll knock it out in an hour we're like getting there already so uh this is not not too surprising because this is really fascinating I the team I told Tyler that I felt like a kid in a candy store this morning because it it just again it like reinforces a lot of what I see and talk about an but we have like again very large sample size here so uh excited to back it with some some real evidence U so campaign Types on YouTube this is this is a common question really interesting stuff here this first point so we we we looked at YouTube vac or video action campaigns that's like the equivalent of of a meta conversion campaign YouTube reach demand gen YouTube TV and shorts so this first stat here is on YouTube reach campaigns which interestingly we found are like similarish to vac campaigns with only which is vac is what video action campaigns it's they're like conversion conversion optimized campaigns um so only 12% worse incremental Raz for fascinating conversion now this is very different from meta like this is strikingly different where these campaigns are kind of similar and you know only 12% Delta between reach versus conversion campaigns I think this supports your thesis that the conversion optimization engine on YouTube or on yeah like on YouTube is is nowhere near where meta is right now what happens if you layer in a halo effect there because you're saying D Toc iros only if you add a halo effect I bet I wouldn't be surprised if reach outperformed like you know that's a really good question really good question like like is is YouTube reach actually more effective if you work in the Halo effects if if you're Omni Channel yeah yep yeah um so yeah this is this is again just very different from how meta operates where not too much difference between YouTube reach and YouTube conversion campaigns yeah um okay moving on to the next step have you have you heard of um like how YouTube is moving all of the video action campaign yes advertisers were not very happy about this change felt they were being like forced into a a pmax like black box where YouTube is is just by default they're moving all vac conversion campaigns into this demand gen bucket of inventory where you could buy on YouTu but you can also serve on other inventory sources other sources in the Google ecosystem so this was a very encouraging stat which is that demand gen is actually outperforming their like Legacy video action campaigns um so average D Toc iow has 27% better on demand gen than vac and a bigger boost in the post treatment window um from like final through to post treatment window so so better lagged effect as well this is encouraging and so I think this is a good reason to be skeptical but it's not doomsday for like these changes that Google is making by pushing advertisers into demand J I mean there's nothing more bankable in the world than advertisers being annoyed when you change something in their dashboard so um they they just always assume the worst and I with Google there's sometimes they've earned that reputation a little bit but uh yeah it's that's that's really interesting okay great so demand gen has been positive that's good uh let's go to YouTube TV you had asked about YouTube TV uh and how much of this kind of phenomenon is actually driven by YouTube TV we actually saw worse performance on sales on YouTube TV than on just like standard YouTube campaigns which makes sense like my guess is just super expensive like really hard for the economics to to back out there 65% worse iroas on YouTube TV than BAC and again this is like reservation inventory this is like truly delivering on a very much like a TV like experience so makes sense to me that it's just like too expensive to back out into like short-term outcome also is this a problem of like who you reach I've heard people talk about this like if your YouTube TV um is there an adree version of the membership um I wonder well anyway I we if you don't know it's okay but the point is like one of the problems with some forms of advertising is that by uh on ad supported uh TV type buying by definition like if you're buying ads on like the Hulu version where the customer has elected into ads or something like that one of the problems is you're you're targeting customers who by definition are not willing to spend an extra $5 a month or whatever to not watch ads which means they probably have less spending power so like it's it's probably just like a worst customer to reach at the Baseline level in terms of spending power if you want to Target rich people you know with your ads I don't know if there's something something similar here or not but I think I agree with that I'd be curious on like the average income of a YouTube TV subscriber versus like the average YouTube it's probably like if there it's a pretty expensive subscription and this is at so my guess is this is a much wealthier consumer yeah but my guess but what I'm what I'm wondering is is there a non is there a commercial is there sort of an adree version of the subscription oh I don't think so because it's like linear TV like how do you skip the yeah we'll have to fact check that I don't know if you do a fact check at the end I'm not sure though I don't think there is because like I have Hulu live and I skip the ads even if it can skip the ads on the streaming part but like not not the linear sure sure if you're both a YouTube premium and YouTube TV member you can watch your favorite YouTube videos on Youtube tv ads free but ads will still appear on live TV so that makes sense which which of course makes sense yeah okay great okay so YouTube TV is a luxury I feel like that's but that's but that's limited testing too very limited yep and this this this last Point here on shorts ALS also limited tests we only have a handful but shorts have been driving similar outcomes in terms of of incremental efficiency but much lower scale So 20% lower IR so so close to like kind of the general YouTube inventory not too far away in terms of Delta but like half as much spend so not able to scale these budgets to the level that the more standard YouTube is and I'm I I haven't been in the in the ad Platforms in a moment but like I don't know why I would break out shorts like I feel like you could just you could just run it as like part of your YouTube mix and not have to worry about like forcing spend of shorts and like it just seems like an area where I would let the algorithm handle it I can tell you the answer to that question and the reason that shorts is actually really compelling to me right now um which is that it is very different as an experience for the customer there's two reasons it's different as an experience for the for the user than traditional YouTube it is a feed-based experience right the same way that Tik Tok or Instagram or whatever is um and so so at least on mobile um so for that uh shorts um is going to have a little potentially more of the sort of like user characteristics in terms of the way they engage with the platform like Instagram or whatever for example the clickability thing that we've talked about um it's just like I don't know that people are watching shorts on YouTube TV or on TV right like maybe they are but it just doesn't it doesn't feel like the same experience at all um G to be mobile first uh and I I um I think that's really interesting and I actually see click based numbers on shorts that get into the range of meta ads now at less scale but I'm you know I've Got a Brand right now that's you know getting towards you know 10 to 15% of their meta spend over on short specific ads that's pretty interesting now again YouTube is moving or Google's moving all this into demand gen so that might might hurt the ability to separate this exactly but there's that but there's another really probably even more important factor which is that it's vertical video so if you're running meta ads the like the the sort of like creative barrier to entry is so much lower vertical video I've never seen work well on um traditional YouTube placements which is a horizontal placement it just like is a tiny part of the screen basically and so um it just feels non-native to the to the um to the placement in a way that on shorts it is native to it and so the most compelling thing about shorts is that barriered entry thing we've talked about which is that you can Port your meta ads over your best performing meta videos over straight into YouTube shorts it's part of the reason app 11's compelling right is like you can take this kind of content that has worked in other places you can you've already validated the message in the creative like quote unquote works if you can find additional distribution for it um that you're not currently accessing it's it's compelling so that's those are the reasons why I think it's it's it's interesting um and it's sort of interesting it's not surprising to me that the iros is a little lower um than what you're seeing for the other ones for all the reasons we've said which is like this sort of user experience is maybe um a little different a little less like devoted and focused a little bit more click based less view based like kind of quick hit kind of things relative to the longer YouTube experience TV experience all those things yep yeah you're just you're you're going to hit a ceiling in terms of scale like if two-thirds of the exposure is happening on TV screens you're just gonna you're going to max out in terms of scales which is kind of what makes YouTube so so powerful but I that's a great point that like this this is a lowc cost low barriered entry way for a brand on the smaller side to actually get started in YouTube it would be a fascinating thing Olivia to get some of this data for Brands like in like I said kind of in the space that my clients are like lower eight figures who are starting to do this they want to diversify outside of meta they can't quite stomach all The Upfront YouTube costs yet but this is like a step into that it would be interesting to get some specific testing around this because um I think Zack suck has said that he was spending like 15 grand a day on shorts at some point like real money you know um I don't know what the inventory availability is like I don't know what the user adoption is on shorts I think it's growing but I'm not sure there was a second there where I thought maybe the Tik Tok um yeah the Tik Tok traffic would go there among other places I mean I don't think we'll all go there but so yeah it's sort of an interesting thing if they can if they can make that platform really work um I I would be I would love to know because if it is possible I think it's potentially a sort of first step into this world for a lot of Brands um that are in that space that I'm talking about yep so we've had some some campaigns at click-based numbers spending you know multiple thousands of dollars a month or whatever not again not huge for brand that is like click based two to one or better you know and they don't need those Brands don't need a ton of scale you know like yeah yeah just need I don't know 20K 20K a day like yeah like more than enough right and so oh yeah to scale those types of budgets um yeah that that makes a lot of sense but I I'm GNA take that as we should we should look at like maybe this is like your starter tier for that would be a really interesting set of tests like to sort of group together some brands that are in that same size range kind of entering into there and if you guys could could put some data together it' be fascinating I you know I don't know yeah um okay okay takeaways let's let's do it yeah I'm just gonna we've talked about all of this so I'm just gonna I'm gonna run through these but YouTube Can Be an Effective direct response Channel it can drive short-term outcomes if you're measuring this in the right way and you've kind of moved off of click-based attribution due to its view-based nature platform reporting is Under reporting significantly Under reporting somewhere between 1 and 10x YouTube is even more performant for brand selling on Amazon and Retail and new demand gen campaigns are on par with or better than VC open questions here we I don't think we'll have time to get to like the rise of AI video here might make this more accessible I love your point on shorts also making this more accessible to to smbs and then how scalable is YouTube like can we punch that number up higher than 30% of meta like I I imagine right now YouTube is a very strong number three after Met and like Google search like Can it can it hang in with meta at scale and like how much can that number increase and I imagine it's going to be a function of of how much creative you're producing but those are those are my open questions I'm curious you know went through a lot of questions I'm excited to dig on with you but yeah anything else here before we we switch over I think the next question can actually be a nice bridge to the to the next conversation which is sort of like um at what point this I think is an open question um at you sort of said like how scalable is YouTube where do brand start to find diminishing results that's sort of like that back end of your YouTube experience right but I'm interested in the front end of it like which is like where where do we think about what a brand ought to do listening to or watching this what should they think and do with this information and how do they sort of step into it you know we talked about the shorts thing a little bit um which I've definitely is kind of the way we my you know I have appr PR with my clients is like shorts is sort of the step into this um but like I maybe I'll just put the question to you Olivia you are again we we'll give you that same brand right you are a $10 million in Revenue brand let's say you're eing out uh how much profit am I gonna give you right now I'm gonna give you I'm gonna give you six% okay margins growing growing business okay um so net margin 6% you're growing you're reinvesting into growth something like that all right um I'll give you more I'll give you more money than that you get 10% okay we'll make the math easy $10 million brand $ million on ads and 10% net margin so you got a million dollar in profit every year but you're growing fast so you're plowing a lot of that right back into cash for inventory as you're trying to get the 20 next year whatever it is what do you do with this information how do you start how do you where do you start taking it next you know uh I mean I I like putting it that number on it like maybe you're like Ah that's the point where you should hire your house like and if that's what you think that's okay you know talk your book but like uh like what's what do you do I I have to kick this back over to you um because I'm super curious like at that stage do you have an incrementality problem like this is I've I've worked I've worked at large businesses I've also worked on ads for small businesses and growing businesses um I like I never wondered like if meta ads or if if these different channels at that size were working because I could just so see I could see it so clearly and you say this like in the bank account like if if I ended like messed up something in meta in my campaign turned off accidentally like we would see the business come down by like 60% overnight and so that's where like I don't think the answer is is to hire house at that stage I think yeah um I'm I'm super curious like from your perspective if this is actually like a hair on fire problem of like if it's challenging to tie media investment to business outcomes at that exact stage that you describe okay so I think two things about this first of all you referenced my conversation with Barry hot about like cost caps targeting people too much and this is the reason why I reject every time I see somebody say manual bids don't scale I'm just like what do you mean like I like I run 100% of my prospecting spend on manual bids 100% And my brands are growing and the only way that's literally even possible is if they're reaching new customers I can I can actually see it in the Impressions and reach data like really clearly and the reason I learned about you just real quick tweeted this about cost caps and I was like I was very suspicious I was kind of in Barry's Camp of like oh this is definitely like picking off all the lowest hanging like non-incremental fruit we've tested this a handful of times yeah it's it's it's not it's um like there's not a large difference between cost caps lowest cost and there's no no meaningful difference in incrementality between the two but the thing is of course that's true I didn't think so I was like I'm not trying to pick on you and I'm not even picking on Barry and I don't actually want to misrepresent Barry's opinion go go ahead say what you're gonna say oh I seemed so like in theory it seemed obvious to me that like meta is just going to deliver to like all the people who are buying anyway but I heard your point with Barry which is like like at that size like that's great like that's actually what like I'm paying meta to do yes so if your business is primarily driven by paid then like that makes a lot of sense I think where it does become challenging is like if you're Netflix and you're like business like maybe 10% driven by marketing then it might point you in weird directions but like I I've come around F the I don't care that's the thing I like I'm just not trying to talk to Netflix as a growth team like I'm just not you know so and that's the that's the thing like every customer in the world you know Taylor and I both have used this analogy for a long time exists on a spectrum from least likely or most likely to buy to least likely to buy right they all exist on this spectrum right here all seven billion humans or however many there are how many are there seven billion okay something like that everybody exists on this spectrum from most likely to buy to least likely to buy the most likely customer to buy do you know who it is Olivia there's a person if you started a business right now there's actually an individual who would be do you know who the most likely to buy is it's your own mother it's your own mother that's who the most likely to buy is okay yeah okay I don't actually know what relationship with your mom is like sorry if I just stepped on totally yeah great okay so for almost everybody right now around Michigan right exactly so for for almost everybody your mom is the person your mom is the least expensive person to convert therefore in the world she actually is probably negative CAC right she'll she'll probably pay you twice as much as your product costs okay um so or as your is your product retail sport okay on this side is somebody who is a subsistence farmer who doesn't have access to the internet okay so you can reach that customer but you have to pay a lot of money up front to reach them and that's the thing that the Spectrum represents is they all represent least cost to the most cost okay so the least cost is your mom the most cost that's CAC CAC goes up as you go down that Spectrum um the point is I actually want my ad machine to deliver to the people with the least cost First now if everybody is in the my mom range where they were actually going to buy anyway then I don't want that and at some point a business does have a real problem there right where again like you said organic demand is so high but that's so few people and the other point about this on the manual bid side and this is the point I was trying to make to Barry is like that's the whole point of the of the bid that's the entire point of it is at some point there's a threshold and again on this spectrum that exists for everybody like let's say it's right here across this part of the spectrum it becomes too expensive to reach the customer and you don't want to pay it anymore and so what you tell meta is that the number where I don't want you to go any further down the Spectrum anymore I don't want you to go towards the subsistence farmer anymore um and that's the whole point and so like the you know the cost cap or the T Target or the bid cap or whatever what those all do is set that limit for you and they say keep going past that so yeah a lot of those customers are going to cost less and I don't know maybe there's some optimized way but that's that's that's the whole point the re the reason that's relevant to your question is because um what I can then do is just correlate that with the bank account exactly as you said for a long time especially if I have less channels I have no trouble seeing this data at all I don't need incrementality testing I just use click-based meta as a way to optimize and I just go look at my bank account and I see is this driving more or less business there's a couple little pulse checks you can do on this the one that I like to use the most is used to last click last click data and this is going to beep like mocked by many other people but last click data does not tell you the whole truth but it never lies um it's it's extremely tight uh in terms of what it tells you and it's consistent over a long period of time so you can use that to say like if suddenly meta is reporting that I'm reaching five times or making five times as much money as before but my last click data is the same something is probably wrong there um and and I should probably be skeptical of it so that's that's the way that I think about it having said all of that there is a point where and this is where I've reached out to you and dm' you and tried to ask you questions about exactly this thing which is um Google it's the it's it's the addition of Google and the expansion of Google which is an incremental which is a a mess it is an incrementality mess branded search is an incrementality mess categorical demand that is basically in Market is is an incrementality mess YouTube is an incrementality mess you just told me they're Under reporting by 70% um pmax is a nightmare you can't even see what's going on and so as Brands try to layer that in that's the point where it's like I just want some at least some heuristics also the halo effect so it's the addition of Google and the addition of serious second channels which is usually Amazon first those two things are the point where um it starts to be more complicated and we have to find some solutions and what I what I'm like desperately looking for I don't know if how can create the product for this to go SE this side of the market and then keep them as customers into their $100 million journey or whatever but what I'm like pining for is like what's the intro to this product to to what you guys are doing for those two problems if we can solve those two problems for Brands I think they could grow a lot faster I think I mistakenly thought that like Google had self- serve incrementality the way meta does like meta has these self- serve incrementality test that I always recommend folks who are just starting off like go run a few of these I'm learning that like I was spoiled in the like as a large brand at at you know Netflix companies of like oh yeah like we got we could just do it free through through Google and so I'm always wondering like oh why don't these these Brands just just run directly like using n Google's native tools I know I know now um like that that they don't actually offer these widely especially to like the mid-market and SMB and so that's the Google ecosystem is a big question mark I agree with you it's often a fraction of the spend of meta but just like you want to grow it like YouTube as an example you know this is it's it's getting up there if we want to be spending like you know onethird the amount that we're spending on meta and and I I think a lot of Brands feel like they're Flying Blind here in addition to Google Amazon retail effects so we're like we're thinking and and like kicking the tires on like maybe this isn't like an annual subscription but maybe this is like an allart model where you can just like pay per test um there's a heavy kind of services component to our business because these metrics are so new again we we talked about it but like you are the upper echelon in terms sophistication here but like a lot of Brands need support and like service kind of incorporating these new metrics like what does iraz even mean um I'm not to this you know this nomenclature and so maybe there's a like an allart testing model it's like pay as you go and like somehow we can get the um the services to make sense so that it's it's we're like we're we tend to be a fairly expensive SAS tool like relative to the stack um but that comes with like a lot more service than you're going to get from like just a Shopify app that you're plugging in and and and using so I'm like very committed to figuring this out I think Google Amazon retail are like very big question marks that that that that probably warrant um incrementality tests but you just you want to make sure um you you really want to make sure you're like powering the experiments in a way where you're going to come out the other end with a result like what we see often is if you just try to hack this together or you like you know pay a random vendor they're just going to um like they're not going to actually show you how much you need to spend and like how long you need to test for how large your hold do group needs to be in order to like sufficiently power the experiment so there's ways to like ways where just like you can do it wrong and that's actually like more destructive than not doing it at all uh I I mean so I've never I've never for any brand I've ever worked on used a single MTA tool or anything and and I've been against them the whole time because there was a minute there where they were probably really useful because of uh iOS 14.5 nuke met ads tracking um and so for a second there they they sort of solved that problem faster than meta did which was I think a useful thing the triple whales of the world Etc and I'm not trying to pick on anybody it's just the reality what the tool offered the the um but I I think what you just said is so important which is um uh confident advice in the wrong direction is like the worst thing for you um because if if the whole point is that it gives you an answer it tells you the answer to the problem that is vexing you and then and then it's wrong oh my gosh it's so hard to like get walked off that ledge and I think this is actually the this is fundamental to the problem of like of the Amazon thing which is sort of like you know there's there's like there's nothing telling you that the ad is working or it's the problem with branded search branded search telling you that like you're making this huge return it's like it's right there here's the report it says it you know and it's it's wrong like it's so yeah I think I think if there was a way for you guys to do that my guess is it would be like hard for you guys to make much money on it because of what you guys do and I don't know maybe you could figure it out but like it I I do think those are the particular problems that need solving the meta problem I just don't think needs solving that much um like you said you can actually run meta conversion lift studies right um uh and and just use that tool and you really like that tool I I like it like the the it's it's a really helpful tool and understanding if you actually have an incrementality problem like if in in like at your size that you mentioned that 10 ion doll brand like in my experience like most of the business is being driven by paid they're just so young like 80% if 80% of your of your ad spends incremental like move on like there are other things to focus on like there are other you know I think other priorities and you you really don't need like you you have this luxury of like not needing to worry about measurement it's it's not that complicated yet yep yeah I think that's really good well anything else you would tell brands that are that like aren't with you guys yet that you would tell them specific specifically to do uh yeah um if not if we covered it it's fine did meta conversion lift um is is like I I always recommend folks do a couple of these like free tests in the in the tool to understand like if they have again if they have an incrementality problem um it's I kind of say like congratulations like it's a luxury to have an incrementality problem because you've grown your organic demand so much um but I just again I just wouldn't wouldn't worry about it if you run one of these meta lift studies and you you are happy with what you get back and it it's efficient that said I know Google and the Halo with with Amazon and and Retail like if Amazon is only 10% of your business like don't worry about it yet like I see it start to kind of creep up as an issue when it's like starting to represent like 30 40 50% um but my general rule of thumb is like how much outsized impact could media be driving if Amazon in total only represents like 10% of share yeah yeah and normally this is the thing about the Amazon thing is you can actually see this too this is the reason why like I've always sort of tried to push people towards at least assuming some halo effect because it's the same thing as like the bank account never lies like it's like if suddenly you're making a bunch of money on Amazon that you weren't making before once you launched there and you're not spending a lot on Amazon ads like well those people came from somewhere um the other little thing people do to use this or the other people the thing people use also uh which I think is a nice proxy metric for this is brand search on other platforms so like uh you you could actually use it for Google too but if you're if you're running you know let's say you're uh let's say you're a brand adding Amazon okay you could check your brand search listings and check like your brand search volume on the channel That is a good leading indicator that you are driving awareness in other channels because that came from somewhere that that demand didn't just pop up out of nowhere you know it came from something so that that can be a good leading indicator so if that's growing you can use that um and you can use that same thing if you're like on Walmart.com if you're on chewy if you're on like some of those kind of other online retailers um it can be that but there actually is a point here where where um I even would say like I don't know forecast your p&l and make sure your ad spend is a certain percentage of the revenue and like you know it just like chalk it up that way so but th those I think are things that I would generally tell people um if the p&l is working it's working y one last piece of advice with the Google stuff like people are so afraid to shut it off um I've talked to so many people of like if you really want an answer to this question to shut it off and then I'll have a brand shut it off they won't see any impact especially on the brand search side um not always the case but sometimes the case and and then I see like the um people come out and say well like it's going to be a delayed effect like it's going to impact your business longer term and I just like I fundamentally reject that because if you turn meta off tomorrow you will see your business come down so like right why are you holding like Google to this like totally different standard if it's driving no immediate impact then like it's probably not yeah yeah right so that's like I encourage folks so just just turn it down turn it off like see what happens and and usually you can you can start to I'm doing that with a brand right now we're we're really suspicious and there's there's another this is a really specific Google thing um but other people have this for sure I'll give you an example when I was at um 4x400 um we had a brand called FC Goods I've mentioned it a lot of times we made wallets out of old baseball gloves okay um and uh and that brand so therefore as we drove um as we ran our ads we were there's some other brands doing that but they were much smaller than us we were we were definitely spending more on ads than others and so as we were doing that we um we could see in our search data suddenly the the increase of a term of the term baseball glove wallet in our Search terms um that term is might as well have been a Brand term we ranked number one for it organically you know and uh and people were looking for us they just forgot our name you know and um and so I would say like that's another thing that happens for some of these Brands so I have another brand that that just did this we basically just turned off all those terms we just thought unless we're getting like a a crazy in platform number we just don't think it's very incremental like we're just going to stop and so that's what we did we just turned off all of our Search terms made them account wide negatives for a sort of range of terms like that that are sort of like I would call them like almost brand terms you know um like that and I think that's another thing you can do is like go look for those whereas like if somebody was looking for like gift for Dad I'm like well I mean what am I ranked for that 100 on Google like it doesn't matter you know so probably not even that high so that is like that's that's true top of funnel and so you can just sort of use some intuitive thinking about like what is what does the search term represent the person thinking on the Google side um and where are they in the funnel uh I think a lot of that is kind of intuitive so yeah um okay you let's let's argue you um you told me you wanted to argue with something argue with me about something that I think I've I probably argue with you about we didn't even probably know each other but I maybe I could just tell right away that you were game for uh for some uh some arguments and you weren't gonna be offended um but um I one of my least favorite Fe one of my least favorite pieces of advice in the world in e-commerce right now for most brands is test everything and here I am talking to Olivia podcast episode about it did I yeah you like a solo pod oh yeah yeah yeah yeah yeah yeah yeah just preaching just just full pontificating about this your box episode yes I have a lot of those um uh and this also fits with you know you mentioned my comment to Barry but just saying like at some point like no I'm right about this um the reason I try to things actually like this really strongly at times is because I actually want to be pinned down on an answer the the my actual least favorite piece of advice is not test everything my actual least favorite piece of advice is related to test everything but it's a cousin of it it's it's um it's different for every brand um it's not quite the same as say test everything it's close though but um but you know if it works for you then it's different I reject that I think I think manual bids work or they don't and there are ways in which they work differently for different brands depending on your scale but like uh I think in general you should approach it with that kind of perspective right this is kind of what I think about like YouTube incrementality test or something is is like it there's something to be said here at the Baseline level and then you have to go out from there a little bit to how it applies uniquely to your brand and and that's that's definitely true so I try to say things really strongly in part because I want somebody to tell me I'm wrong if I'm wrong that's fine you know like um but I want to do that so here's so one of mine is that brands should not test everything that they what should they should do instead uh because I think what they do is they make the mistake of testing a lot of small things and sort of swimming this is the problem with the the uh intellectual capital on DDC balance sheet is there's too much of it actually and it's hard to sort through who is worth listening to and who's not and so sometimes they actually need to turn off the podcasts and just like do the three things that matter the most and and all that so anyway uh that is here there's there's my opening statement tell me tell me your opening response to that or what whatever you hate about my advice which is fine you are literally the head of a testing company so I I'll tell you what I agree with and what I what I disagree with what I absolutely agree with and I think is like essential is having a set of strategic principles like otherwise you just end up like you end up saying yes to everything and I think like the lack of focus um especially in an agency where you can just get like pulled in a million different directions is like crippling very it's just it it it um i' I've seen it like so many times where you just end up saying yes to everything because you don't have a framework through which to like evaluate anything and so so when I was running like when I was when I was overseeing growth teams like we had a set of strategic principles one of them was like we are heavily concentrated in meta and Google that allowed us to say no to like all thesection not get hung up in Pinterest and all these other channels and and we like aligned on that with the exec team when we got started so specifically for agencies I think people need to I think people need to find a Playbook the the worst thing you can hire in an agency is an agency in general who's going to say uh you the thing that we do is we like test it all for your specific brand like I think that is like a gigantic mistake the whole value of an agency is collective knowledge um that's the that's the whole value right and I've like very much come around to the Common Thread like Collective approach here of like yeah our Playbook and and you either buy into it or you don't and like it actually helps weed out bad fits and I think every agency probably needs this to differentiate that's right this of like you need a core set of strategic principles that you live and die by and then it will attract some set of a Brands and it it will you know it'll help self self- select so I agree with that I tell me where I'm wrong that well I also agree that people test small things and it's it's not worth it like I always try to run these through like the effort impact framework of like yes this is high effort low impact skip it and like that's what we like when I mentioned there's like a heavy service component to house like we're helping them work through that and and really like evaluate these hypotheses through that lens yep where I disagree um is like two so what if your business stalls like that you have these like core principles that you anchor on like how do you improve like I've been in this position where like we've been stuck in terms of growth and like best practices can only take you so far are you just like to your client are you like H you're out of luck like I don't know yeah I'm not I'm not willing to adjust or I'm not willing to to to to try new things okay great question and a really important distinction it is precisely in this case where I think this advice is the most important because what happens when for brand for operators when they do this is then they put every possible test on the table and then they have they have major decision paralysis about what to do next and they start throwing things at the wall what I think is the value of eliminating test everything is it actually Narrows the band of things to try to accomplish next right because there's no question I mean I don't want anybody to ever listen to or watch my content and think that I only win all the time that is wrong I like there's plenty times where we hit a plateau or something and inevitably what happens is like an operator goes like wow should we revisit this but what they want to revisit a lot of times instinctually is manual bids uh creative testing campaigns like all of these things I'm like no that's an open and shut question that is not where you're going to find the answer you're just going to spin your wheels all all we should do instead is Think Like okay do we need to generate new creative do we need to think about like is this a business phase problem or whatever like try to narrow to like what are the actual high impact things that that are the principles that you can stick in and then yes you're going to have to test those for sure right so like um so like within that narrow band you can do it but the thing I really want to avoid in that case is is retiga the questions that are least important this is how Brands get on the agency Carousel because they're like oh somebody else can do the Creative thing somebody else can do the media buying thing their way is better whatever it is and what you actually need is a principle around this kind of thing to go okay this is what I believe is going to work most likely and I'm going to commit to to that and I'm going to eliminate that test instead I'm going to try and say like what is the actual larger impact thing that is different than what I'm doing right now because you do you do need to do that right so like right now I'll just say like business phase earlier earlier Sub sub uh sub eight figures um generate a creative machine that you run through manual bids on meta ads to grow your DDC business uh ruthlessly make your supply chain more efficient because I just guarantee you if your sub eight figures you've got all kinds of mistakes in your supply chain um that are costing you a whole bunch of money those two things uh lean X those three things are the three things I would effort if you hit all of those and you're still not growing then I'm willing to talk about what what else you need to test but do all three of those first okay over eight uh once you cross that eight figure Mark then it becomes probably for most brands a product development problem where it's like okay now you need to think about where your products are do you need to launch new products whatever it is um now I just spoke to what every brand is doing everywhere there's going to be a different thing for a subscription brand versus an apparel brand versus whatever there are real real differen in those categories I'm I'm definitely not trying to say otherwise but that's where I think that's that's where I think that that this like uh this kind of test everything approach actually just sends people spinning their wheels on the wrong things and um and and is really counterproductive in those cases because you're GNA come across them everybody's gonna stall everybody's gonna have trouble everybody's gonna have to figure things out yeah I guess I've just been proven wrong like so many times like appen is a great example I was the most skeptical of Apple when when this started popping up and I was like don't bother and then you know folks kind of pushed it through in terms of testing and like it actually ended up being like a a pretty meaningful Channel and so there's another example where like we learned at Netflix that you always want to advertise the free trial um and like I had that engrained in my head and then you know I went to a different streaming company and I was like very resistant to testing it because I was like we've learned this like I don't want to relitigate it and we ended up testing it and we found that like we shouldn't have advertised free trial and there's a pretty significant step change in terms of performance there and so like I don't know I've just seen like what challenging my leld beliefs can do in terms of like growing a business and so I'm like I I don't think you so I I actually don't think you tested things there I I what I think you did is you actually ran brand specific things through general principles that you already believe so for example you already believe that there is a way of determining the truth about App 11 which is incrementality testing now you've got a new player on the scene that nobody had access to before and then you're able to go and test that and you say like okay I have a specific prior about this about this thing which is that it's that's not going to work okay that's really different than saying like and it's also new it's like literally never existed before so like that's I think fine to test that right um but then secondly I think on your Netflix example what you're saying is the offer is a place of disproportionate impact on the value of the customer I'm not going to test I'm not going to try to go spend all my spin my wheels figure out like how to optimize landing page is everywhere instead I'm going to say like the offer has disproportionate impact and so that's the thing I need to go figure out for this particular brand like that is the kind of principle that I think is like and so I don't know maybe it's semantics but like I I I think in those cases you actually ran um product or brand specific things through broader principles that you already had in place and the broader principles are the things I care about yeah I I I like the way you put that I think yeah like measuring for these larger Brands like measuring on causation and through incrementality just like lends itself to more testing because that's the only way to measure this you have to like run test so and and in that way it I think we're we we're kind of in agreement the the the last piece on this is like Yeah you mentioned on that pod like listen to The Experts like I know I have come across so many who don't know what they're talking about I don't know like I don't know how to reconcile like how do I actually know who to trust in in this industry it's like well just trust me and trust us like I know you and I know Taylor now well enough to like know that you guys know what you're talking about but like there's also so many agencies who don't know what they're talking about and I don't yeah and I'm sure I'm wrong about some stuff I I I think that's absolutely the best counter advice to what I'm saying and actually the problem of having an earlier stage this is even you know I just wrote like I said like a 10 things I'd tell my younger self piece but the problem with that idea is that like the only way to get that knowledge is the is is experience like it's just this is why experience can matter and some of those things because it just you are able to you're able to uh you know evaluate people better so I I agree I mean I agree with you I think trust the experts is part of the reason I say it again is maybe that is a hotter take is like because it's so like known that there's you know that people are going to disagree and be wrong about things but like I think that's the skill you have to develop is trying to figure out how to Source what good reasoning is and isn't but of course if you aren't good at reasoning yourself then it's going to be really hard to do that and um and so yeah I I I I think it's a fair critique I think what you're saying is a fair critique which is like you're say what you're saying is true in theory but actually in practice extremely hard to do I think that's right it it's part of why I do think people ought to be Taylor's a great example of this like I just don't know anybody who more rigorously supports his opinions with like some kind of approach to data and and who is actually be willing to call to the who's willing to be called to the carpet on on claims he makes um many people of Wiggle out of them maybe that's maybe that's my advice on this is like if the person says it can be different for everybody all the time and just change all the time that's the person you shouldn't listen to the person you should listen to is the person who's willing to make a statement and then have it evaluated because if they're not then they actually can never be held accountable for anything they ever believe it's just all like all over the place it's like a I know this is like a classic thing in political punditry like it's like the reason people use uh what's the betting site um I use it all the time I can't think Market yeah it's the reason people say like poly Market is a way better predictor of of what's going to happen next than like political pundits right and it's because once you actually have to put your dollars on it now you're being willing to held accountable and I think there's something similar going on with advice it's just like yeah like I'm I'm like my chips are in on this idea you know and therefore yeah whether it's your status chips in the community or whether it's your or whether it's something else you know so that's why I like I like a strong opinion I'll take you know Taylor's been anti-a 11 strong opinion great that's fine I can evaluate that claim you know and I can find out if he's right so yeah ultimately I I I do think like a lot of these Brands get get get stuck in like lack of focus um and that like it's a net positive like I I think I would rather them test less and focus more then like I think the the Trap they too often fall into is like lack of focus and testing too too too much that like actually doesn't make an impact so I I've very much come around to like why why you why you all um really like focus on these core set of principles and yeah yeah I would hire you thank you I would hire you olivan I think everybody watching and listening to this I mean I'm actually desperately