Creative Testing Campaigns Are DEAD transcript

you are serious enough right now about the place that AI has and will continue to have in your creative process I can tell you right now if you are thinking too small about that if you are not understanding yet how much that is going to continue to revolutionize it is already revolutionizing that word gets thrown around it's a very AI word actually if you ask AI to write something it will use words like revolutionize this product revolutionize the space or whatever but it's true in this case AI actually is revolutionizing the way that creative production works for meta ads the way that I can tell if you are being Forward Thinking about this enough or not is maybe a little surprising it is this if you are still running creative testing campaigns in your meta ads account then I can just guarantee you you are not thinking seriously enough about the place that AI is going to have in your in your meta ads creative production stack okay and that might sound surprising it might sound like me just sort of um being on a hobby horse of mine about creative testing campaigns and manual bids but I'm telling you right now that you are going to get left behind in the world of meta ads media buying and probably all media buying everywhere if you are still allocating actual extra budget dollars to separate creative testing campaigns and the reason why I know that is because of the way the cost structure of meta advertising works so the the logic of creative testing campaigns for so many people right is that you want to allocate some dollars towards meta's uh towards new meta ads to see how they perform and then decide after you've allocated those dollars whether or not they are going to work and many people will frame that as you have to get statistical significance stat Sig right you have to get statistical significance on every ad that you run and when you get stat Sig on those ads then you can determine whether or not you have a winner or loser and then move it to your scaling campaign and the thing is that requires many purchases per ad now there are there are countless problems with that logic chain that uh that I could go through but I'm gonna leave that for mostly another episode I don't actually want to sort of revisit that probably if you are watching this episode even if you've made it just this far you should first of all subscribe to my content because you're going to like a lot of what I talk about and what I do um so wherever you're watching listening make sure to subscribe but also you should go backwards and look at both of my previous episodes which will be linked in the show notes about creative testing campaigns okay um and the reason why is because I'm I lay the foundation in those episodes for why I think they're a really bad use of your time and money okay and why I think they're actually maybe the biggest cost center in your e-commerce business bar none so so I I I have all kinds of problems with this but I actually want to focus on one really specific thing right now in this episode and that really specific thing is the fact and the implications of the fact that creative production costs are getting so low because right now if you think about it right the idea is you allocate some let's make a distinction between creative production and creative distribution okay creative production would be all the costs associated with making an ad including the humans the software picking up a camera making the edits all those things okay and then distribution would be the cost of uh running Those ads on your meta ads um ads or wherever Google ads App 11 doesn't matter okay anywhere where you distribute the ads to people so production distribution okay let's make that distinction the thing about what AI is doing right now as many people have pointed out right is that the cost of creative production is getting incredibly low incredibly fast so that you could make an incredible volume of ads incredibly cheap okay now the thing is the distribution cost of running ads then becomes the blocker and so if you run if you have to take every ad you make right um and let's just say you make 50 ads in a month if you have to take every ad you make and allocate spend towards each of those ads testing whether or not it's hitting stat s okay or something like that or giving some indication that that ad is going to perform you have to allocate budget towards all that that's already at 50 ads a pretty big budget okay let's say you have to get even just 10 purchases on each of those ads and let's say you want a two to one Rass okay and at a two to one row ass um let's say your aov is $50 so we're going to make this a fairly simple setup okay so that means that you need to get $25 let's say 10 purchases gets you to stat s which it almost certainly doesn't okay but that means that you need 10 purchases per ad okay and so that's $500 right there 10 times 50 okay excuse me uh yeah 10 10 uh 50 ads 10 purchases each okay so there's 500 okay now 500 times 25 because we just made your CAC uh for it to be a performer uh uh $25 okay so $50 aov $25 C gets you to a two to one Rass 10 purchases per ad 50 ads per month that's $12,500 per month on just that little thing now like I said almost certainly 10 purchases is not actually enough to be stat Sig many of these will take less many of those will perform under Target and so you'll perform actually worse than that and 50 ads is really not that many so very quickly if we just double that then you immediately get to 25 Grand a month okay now if you're a really big brand that might not be a big deal but you can see very quickly you go to a 100 ads let's say you need 20 purchases per ad and then let's say you need a 100 ads per month now we've doubled it twice more now we're at 50 Grand a month and the thing is it's going to get crazy because here's the problem okay the problem is and so first of all if you're a smaller brand okay if you're like sub $100,000 a month and spend this already what I have just laid out is already not a viable way to test ads it means if you have to test every ad you're basically stuck at testing 10 ads a month and you're going to be stuck there forever because it's just going to be too expensive and it's going to take forever to learn anything because you need all these purchases per ad like there's all of these problems with that method right away on top of that like I said that's almost certainly not actually stat Sig that's also a $50 aov which is relatively low plenty of brands are at 801 $150 which means the CAC now has to double there's just all of these issues with everything I just said to where the cost snowballs fast and it becomes a totally nonviable way because of the distribution costs of testing your ads if you actually have to do that again on top of that most ads actually don't work and so they're all going to perform so a lot of those ads are going to perform under Target and they're going to drive up cost more as you spend on those ads on top of whatever amount you paid for production so the costs start to add up extremely fast here but but forget even that for a second and instead think about this what happens when the production costs get so cheap that you can with an AI ad Creator okay with an AI ad with just some tools you can actually start producing thousands of ads for month per month for like a couple Grand okay like very clo I very close to free per ad basically where the production cost gets so cheap that you now have this ability to test countless numbers of ads in your account what happens when you can do that well the first thing that happens is it's a huge win for you as an Advertiser as long as you can distribute Those ads because the ability to mass test huge amounts of volume um because the production costs are so low creates a gigantic incentive for you to go and run up those production costs so let me tell you about two meetings I have on the books right now okay one of them uh in about 45 minutes from the time of recording this episode I'm going to meet with somebody who I'm paying $200 splitting the cost with somebody else to show me the Perpetual ad machine that he has made I'm not ready to promote it or anything like that I haven't seen it I don't know if it's good maybe somay I'll have them on if it's good or whatever right but this guy says he has made a tool where in the background of uh running ads he's got um he's got a setup to where for a few hundred a month in uh you know fees for things like zapier and open AI cha PT and some of those things right he is able to make it so that you generate end still image ads for free basically right so for those software costs only like it just does all the still image ad generation now I don't know if that's good enough okay I don't know if that's actually going to work but I also know that there's other software out there that's reporting to do the same thing and it's still pretty early and that all of this AI image and AD generation and all these things are getting better all the time so that's a meeting I'm having today I'm also meeting constantly have a client right now who's spending like lots and lots of money selling products based off of an ad generation machine that is absolutely incredible he's got AI avatars he's got we've got script being written we've got um videos not totally being stitched together but uses um vo from 11 Labs like he's just got all of these tools to make tons and tons of content all the time and so that that has happening right now okay so possibly in my very near future I've got ads being created all the time basically for free or for close to free so I can create endless ads if you want to learn to think the way I am thinking about meta ads if you like this episode I know that you are deep in the weeds of media buying there is one resource that is unparalleled whether you're an operator an agency owner a freelancer a media buyer somewhere that will level up your game more than any other and that is what I consider e-commerce University what I consider met meta ads University like the place to get it except it'll maybe have a much higher Roi than your University experience on a peer dollar for- dooll basis and that is admission from common thread Collective I'm telling you if you are spending less than 50 Grand a month on meta ads right now or even if you're spending more and you just want the very best information the most data backed information from the best Minds in e-commerce to think about how to run a great profitable growing business and not only a business generally but also a specifically um a manual bid based meta ads account so you can leverage cost caps bid caps all those things to the best way possible you need to join ad Mission there is no other resource like it admission is constantly being updated all the time so it's not just a bunch of outdated e-commerce information they're constantly adding new um uh videos to work through new courses to work through there's also uh uh weekly webinars that are exclusive with members of Common Thread collectives teams including a monthly exclusive webinar with Taylor holiday you cannot get access to that information including some Q&A with Taylor that you cannot get if you are not an admission and on top of all that if you go through my link you actually get a free audit of your ad account um a free Cut not an audit a coaching call with uh one of ctc's media buyers who are buying with manual bids all the time so if you're trying to move to the setup that I'm talking about this is like an incredible resource these people are buying huge amounts of media for large Brands and small Brands alike uh on manual bids and they will help you take an hour look through your account make sure your setup is right I get DMS about this all the time I can't help everybody I would like to I always send them into admission it really helps you actually get a free call you actually get four of those calls for free if you sign up for a year uh a year of admission in advance it's an unparalleled resource follow the link in the description of this episode the show notes of this episode to get that uh coaching call for free go join ADM Mission today if you want to level up your game and get serious about getting the best information possible uh if more people join ADM Mission way less people would DM me about all the money they've lost spending spending on ads in terrible ways all right if you're watching this and not listening to it you'll see that my shirt has suddenly and magically changed because in the middle of recording that my three-year-old and 5-year-old came in the house and started making a bunch of noise so um okay so we have picked up the next day where we left off and that's okay because I know exactly where we are and that is the second meeting that I'm going to have I actually just scheduled it in the time between recording the first part of this episode and where I'm recording now and that is um a meeting where I'm going to talk to somebody who has built a tool that can stitch together hooks with bodies of video ads sort of endlessly so he was giving the example of uh of like 20 Hooks and 10 ad bodies that all kind of work together enough and his video editor you know does a bunch of little VAR variations like that his um his team creates those and then he runs them through a machine and now he has 200 ads okay now I believe in concept variations not just hook variations I'm thinking more and more about that but listen if you can do that to where you can create that much variation in your ad that cheaply literally you just create those few hooks create those bodies and then the machine stitches them together and if those can actually work and I believe it's possible they can then um then your going to have all kinds of opportunity that's before you even talk about other tools that do the whole thing from start to finish including analyzing the performance of your past ads and using that information to create basically without you even looking at it new ads now I'm not saying we're necessarily there yet but we probably will be soon on top of that by the way that the tool that I was just talking about that stitches together different parts of video that tool also is um has something built into it that publishes Those ads extremely fast because one of the limitations if you've run meta ads for a while you know is that you actually have to go in the ad account and build the ads while there's people increasing I'm seeing actually multiple versions of this where people are building are um clicking into the to the API and you know like Taylor holiday has said in CTC at some point he's going to get to a point at 2025 where no human ever Builds an ad that will just all be done through an interface that automatically builds via the Facebook ads API and you just be able to have the ads built for you just like that which is to say that would be a another barrier of uh another barrier to creating a whole bunch of ads that is reduced or eliminated by the continued development of Technology around this and so we could just go on and on here and and this is really the point okay the point is that I'm having meetings constantly right now about the ways to use AI whether it can sort of fully take over the accounts or whether it can uh even if it can't fully take over the a generation if it can make it so much faster and so much cheaper that there again you don't have to be like a full AI super Optimist where you believe everything is just going to you know just like sort of you click a button and ads just get created forever you don't have to believe in that world you don't have to believe in the world where it's that simple where it all does for you to to still believe in a world where production costs go way down on top of that if you layer on to that working with teams like the one I work with in the Philippines where I've got great designers and editors who I pay above Market rates in the Philippines um to be uh to to work for me and they bring real expertise great ads into my ad account but at a much lower cost than hiring those people in the US like so now I'm just talking about uh extending your resources via offshoring okay if if we do that like I do that right with with uh behind the scenes Studio they're they they are and their sister company sponsored the show but like you know you've heard me talk about this a lot right BTS studio.com H you can go check them out yourself they're awesome they didn't even pay for that reference somewhere else in this in this uh episode I've got some other actual ad but like BTS studio right they they are um on top of that and they are actually building into their tool set at BTS I have conversations all the time with their CEO uh where I'm looping her into all of these meetings that I'm taking including the ones that I referenced earlier and she is building into their product not only offshore with really high quality designers and editors but she is building into their product lots and lots of use of AI so I basically don't have a meeting about this anymore without her so that it can actually be built into the process that then gets leveraged on my team and then across any of other bts's clients that's great too if they can use it for others because I know other people are using them so so if you look at that excuse me it's BTS studio.

BTS studio. I'll try to link it in the show notes here okay or maybe I'll even have my editor right now put BTS studio. on the screen right here you go check it out okay so uh so like they're doing all of those things all the time and listen the point of all of this that I'm saying forget any individual version of it because I could go on and on and on like it's like the the thing I talk about constantly with my clients and with my team all the time right now is like no matter where this whole world goes again the cost of production is going to keep going down whether AI is just a tool in the tool belt of all this or whether it is like fully a magic ad maker or something like that like whatever version of that you go on the same is the same thing is true which is that the production costs are going down and that means now the limiter for ad the the the cost limiter for ad creation is actually moving away from production to distribution okay and so that's the key point the key point is any cost limitation you have is now not going to be on the production side it's going to be so cheap to make so many ads it already is getting so much cheaper to make so many ads it already is it's been that way for me before I used a lick of AI it was already cheap to make a lot of ads okay now the cost though is on the distribution side and the problem is if you have to run all of those ads through creative testing Machines man you are going to be in big time trouble because now you're going to take this very very low cost on production but then add a ton of cost on distribution and it will simply not be viable and the thing is of course you don't actually have to do that and this is the this is this is the reason why like organizationally it matters so much that you get your media buying right this is the reason why I don't shut up about these things because because the media buying is not an isolated thing with like its own little small optimization thing first of all it's the actual distribution of the probably one of the largest two line items in your business the two line item the largest line items in basically every D Toc business are cogs and AD spend particularly meta ad spend okay and and so you should care about op about the efficient distribution of those dollars BEC just like you would with your cogs right just like you want to get your cogs down and you you should be efforting that all the time okay but like uh and think about your you know your terms and mqs all the things that make um your supply chain more efficient so when you do those things okay at the same time um you know you should manage your media spend well but it's not just about the optimization of that media spend it's actually about the way that everything you do around that in your business before and after and and above and below that media spend right all of those things work into the media spend and so the whole organizational movement moves towards something about the way you distribute the dollars and this is only going to continue to be more true um if you have other channels as well where you're going to distribute the dollars where you you you know you you may be able to extend it to Apple or YouTube shorts or whatever it is okay so um so being able to distribute those things well is so crucial because the whole thing is really really connected and as I've said before there is an alternative to creative testing campaigns okay um you don't have to spend your money and I want you to understand why it's really crucial you don't have to go spend a bunch of money testing the whole idea of running ads to stat Sig um to get a some amount of purchases lined up to figure out whether or not your ads are working or not has so many giant problems with it that uh that uh again I don't want to belabor all of them but there's one really really crucial one okay which is that um you actually are almost certainly not getting stat Sig on those ads your ads are actually um the way you would get stat Sig is by comparing two groups of people holding them out from one another right you you would have a test group this would be called a randomized controll trial you would split two groups an a group and a b group a control and a variant and they would not interact with one another there would be two separate things but that is not how anybody sets up their creative testing campaigns in meta ads they don't actually run them through meta split tests or anything like that what they do is they just run them side by side so it's very possible one group is remarketing the other or whatever it is and you can't see any of that kind of stuff like the thing is meta ads is not set up as an RCT machine that is not how it delivers ads at scale in the auction um and and on top of that even if it was there would be human error at every part of this and I see this all the time people over or under under interpreting uh especially over interpreting small samples of data so people get like three purchases on an ad like oh it's a winner I I know it go put it in a scale campaign and it's like three purchases doesn't tell you anything at all in fact that ad may not work at all and the opposite is also true people will go like oh that ad spent $500 and only got one purchase that means it's not a winner but that just means nothing it means literally nothing like like and so and so people will turn off that out and they may well be turning off winners so it's the scaling of losers and the turning off of winners both are really big problems in the way you do that and it's because the machine's not built for that and that's what I want you to understand what I want you to understand is the machine is built on basian Logic the idea is it meta has access to endless reams of gigantic amounts of data endlessly endlessly endlessly has has access to all of this data and so it can forecast out from what happens at the level of a few hundred a few thousand Impressions something like that very very low amounts of spend okay it can then analyze what users are doing with your creative if you're getting thumb stop if they're engaging with the content for very long and it can take that and probabilistically forecast which is how all good forecasting Works what is going to happen if it spends more on that ad okay and it will either shut down your spend based on that or not and therefore your your ad testing budget can get extremely low because for all these ads that you launch yes it needs to spend a couple dollars ads you know ads maybe need to spend uh the distribution costs are not zero but they're really close to zero they can spend a few dollars and not spend any more money in most cases unless the ad is very very likely to work now probabilistic forecasting is probabilistic if it is is an 80% likelihood of success if you has an 80% likelihood of success and maybe meta puts more spend behind it doesn't show you this information but if it does that then uh at a very small sample then maybe it will spend more well it may be actually the 20% side that uh that is not successful and so your ad may not be a winner so you maybe spend 100 bucks on an ad that doesn't work out very well right um that said uh it's going to spend a lot less than if you did it yourself and it will also read that for you and stop spending because of that information and that is crucial [Music] also all of the ads that I am launching in the way that I just told you I am launching it are being tracked the customer Behavior are being tracked on my websites on my clients websites with Billy b y. a billy. is the best tracking software of which I'm aware is the best pixel setup software of which I'm aware in e-commerce it is really really great software uh because it Ma it gets the best event match quality score that I have ever seen in my meta ads dashboard and so here's the way I've I've tested Billy multiple times with multiple clients and the reason that my clients use Billy is that I have set up tests where I take the native Shopify pixel integration or even sometimes other thirdparty pixels and I take the same ads and I put those in campaign a and then I take campaign B and I I take Billy ads and uh and I do the tracking with Billy and I run the two side by side and Billy has outperformed every other pixel tracking setup that I've seen every time whether that's because it's reporting more accurately or actually generating more purchases I neither know nor care because what I actually care about is getting the most information possible back over to meta from my website from my customers that's what I use Billy for I also validated that objectively by reading the event match quality scores across all of my website events over and over in Facebook ads manager it is simply the most robust tracking software of which I'm aware and magically beautifully it is also incredibly fast and easy to install you can even take it and sort of have it overtake your old pixel so that all the information you have stored in that pixel uh from before doesn't go away it just goes right into Billy and you upgrade your track and going forward um so you don't have to start fresh or anything like that okay Billy can also be a One-Stop shop replacement for Google tag manager so you can use this for like a couple click install without any developers on Google ads Google analytics j4 setup which is a huge pain of course and every other AD tracking um uh every other AD platform that you are tracking ads on Billy can do the install for all of it so uh the point is if you're still tracking your ads with meta ads or with the Shopify native integration with meta it's time to upgrade the bill you get the best information possible sent back to meta Billy b. it will learn that information in real time and it will either continue to spend your money or not now if you're wondering if this even works the thing I can tell you is yes this is how we have been running out Accounts at growth for a long time and I get DMS all the time from other people who do the same thing it is in incredibly frustrating for lots of people to make ads that seem to get very little spend and don't work and I know sometimes they even get only a couple cents of spend and you know I'm not going to sit here and say that's definitely the machine working correctly but but I think on the Aggregate and this is the key point on the aggregate that's the outlier on the aggregate it actually is spending some amount of money on most of your ads even a few dollars is telling it a lot and that is all it needs to do because a is much less than $500 to test an ad or whatever especially when you multiply the ad volume like this and meta will scale winners and it will um suppress losers it will do it and I know that because I'm watching it happen with my clients all the time we launch new ads the good ones spend the bad ones don't if you listen to my um if you listen to my episode recently with Jess Bachmann you know Jess has a has a really different approach than I do I think um I think it's it's from the last couple weeks it's about why his ads are performing better than mine on average now Jess he doesn't do the high volume thing that I do and I think that's great I think Jess has a unique skill set in generating creative that works at scale so he's got a really different approach now it's expensive by the way he charges a lot more money per ad than I do like a lot more way not even close not even the ballpark that's okay it can work like I think I think Jess's approach can work okay but the thing to know is that in the account that Jess and I talked about on that episode where we are both working together and on the next account that we are going to work together on we are not like we are not um tilting the machine in favor of Jess's ass his ads have to perform at the same level as all the other ads do but they get launched in campaigns with bid caps and with tros targets just like all of my ads to and when they perform they spend and when they don't they don't until you know as we looked at it afterwards his ads were getting to 30% of the of the total budget in the ad account he was getting a really meaningful spend total I look at that and I say like that is all the evidence you need that the thing works like it is reading his ads it's spending on them and it's scaling the winners and suppressing the losers and it's doing that dynamic in time of an ad fatigues or whatever it is uh it's doing all of that for us in the background and it allows Jess to go in there make ads and we can do that it allows me at the same time to go make a whole bunch of ads and see what's going to work and see what's not going to work the point in all of this and again I'm not going to re rehash the entire creative testing setup uh the point is the very simple version is launch your ads with your manual bids the real point that I want to make today is that it will simply not be viable you will get left behind very soon if you test your ads any other way it will not work you won't be able to do it that's the point that I'm saying because you will be able to produce so many ads so fast that it will not be financially viable for your brand to test them to actually allocate additional testing dollars to each one that's the point and and so you must switch to this right unless you are going to run unless you are Jess Bachman and you can like make five ads that like hit at this incredible rate every time and you need testing budget towards that but even for him like it works in my system too right so I still think it's an it's an under optimized way to do it right basically almost everybody and and frankly I think Jess should create more iterations faster of the stuff he's doing with more AI um so the point is it is not going to be viable to work that way uh at all anymore to to to work uh by sending uh specific ads to specific creative testing campaigns it's just not going to be viable you're going to get left behind you you have to switch to this um uh otherwise um you simply won't be able to compete in in the rest of the world of ads because it will be too expensive to do it and that should be a good thing you should be excited about that because what I am really telling you is there is all of this wasted money in your ad account based off of uh based on that you are spending on Creative testing that is actually a giant drain on your finances and it's it's actually not not only is it not helping you it's actually hurting you and when you get rid of that you will not only be able to move forward and embrace new ways to produce a whole bunch of ads and level up your ad production and your ad creative output okay but you'll also be able to do that while spending a lot less money on wasted dollars in the process and that is a giant win for you go figure out where these tools fit into your ad creative mix go f out figure out all those things launch your ads with your manual bids just like you launch any other ads every campaign is a scaling campaign and see what happens to the total efficiency of your business when you do all right if you like this episode you're going to like a lot of my content getting into the weeds of media buying and then sometimes pulling out as well into larger e-commerce business principles to help you think about every level of your business from the from the detailed tactics up to the big picture strategy and organizational design and panl design it's going to generate a profitable e-commerce business it's a mix of course of me solo also got some interviews all the time including a bunch of really good ones coming up just go check my last couple interviews they're great my Jeremiah prer conversation was awesome like that took was actionable for me in a bunch of ways my Jess Bachman conversation was awesome there's a lot of good stuff coming so subscribe make sure you don't miss it you should also subscribe to my news letter I'm just sending a very simple weekly like very tactical newsletter at this point it's not very long once a week I'm not going to spam you go to AJF gr. comom drop your email address either in the footer or in the popup that shows up there you'll get on my newsletter I think it's going to really help you and you're going to really like that a lot you also get my four free essential e-commerce resources there including like a cohort forecasting guide a weekly ad account performance um doc that I Ed to track ads every week with my clients really helpful stuff I think that will help you a lot so go sign up for that there thanks so much for watching or listening I will will talk to you next time [Music]