I Tested GPT 6 Astra vs Fable 5.1 (No Hype Assessment) transcript

So, this week was the battle of the AI Titans. We got the release of Claude Fable 5.1 and GPT-6 Astra. Now, not everyone has access to GPT-6 Astra. They're slowly phasing it in, but I was lucky enough to get early access. And today, I'm going to show you some head-to-head tests with GPT-6 versus Claude Fable 5.1, so you can kind of figure out what makes the most sense for you. Yes, we'll take a look at the benchmarks at the beginning just to kind of give you an idea of the numbers, but I figured it was more important to see some real use cases.

We're going to take a look and see how well they do head-to-head in things like front-end design, motion graphics, web applications, and even some longer-running agentic tasks related to game design. The goal here is to give you a snapshot of where these two models stand when compared head-to-head, so you can make a more informed decision about where you should invest your time. So, let's hop in. So, first, let's very quickly go over the numbers here, the benchmarks that they reported.

I know we kind of all take these with a grain of salt, but there's still some information we can pull from here. Now, kind of a bummer that Anthropic didn't give us more data. You'll notice there is a lot of benchmarks that they just didn't report compared to OpenAI. OpenAI kind of gave us everything you could possibly want. And based on the GPT-6 numbers, it pretty much beats Fable 5.1 in almost everything. Well, based on these numbers, literally everything, which is very impressive.

Now, like just because on Deep Sweep version 1.1 and it's a 74.1, does that mean then it's you know, it's actually 7 and 1/2% better than Fable 5.1? And in fact, Gemini 3.8 Flash is better than everything else except GPT-6. No, you really can't look at these numbers in the vacuum, but when we take a look at it holistically, what do we see? We see a huge jump forward, especially when you compare it to something like 5.6.

So, GPT-6 definitely is a step change from previous GPT models. And Fable 5.1, across the board, what do we see? Well, we just see a very strong model that by the numbers is better than Fable 5. And if you're someone who's used Fable 5, you know for a fact that this is a solid model. Like it's good. So, by the numbers, what can we genuinely say? Well, we can say GPT-6 is a big leap forward on the OpenAI side, and Fable 5.1 is just giving us more of what we already like.

So, good start. Now, when it comes to the numbers, the actual performance, the percentage of scoring on these benchmarks are just one part of the equation. The second half of the equation is the cost. How many tokens is it taking each of these models to achieve those numbers? Because historically, the GPT models have actually been a lot better in this regard. And when we look at the stats, that kind of plays out here as well.

Now, I'm looking at OpenAI's press release for this for full disclosure. And if you look at something like Terminal Bench 4.0, and I look at Astra, which is here in the with the stars, compared to Fable 5.1, which is here in the orange, we can see that at their best, they're pretty close. So, at max, we're getting 55.8% accuracy with Fable 5.1, and with Astra, it's 56.7. So, you know, we're like we're basically the same.

But where they aren't the same is the cost. At max, I'm at $10.35 for Astra, and I'm at $19.50 with Fable 5.1. And that's even more of a significant gap when we look at stuff like extra high here versus extra high there. And in general, you can look at this whether we look at really like any of these different benchmarks, GPT-6 tends to be a lot cheaper with performance that matches 5.1. Now, it's time to actually put them to the test on some real use cases.

But before we do that, a quick word from today's sponsor, me. So, inside of Chase AI Plus, I have not only just released a Claude Code Masterclass. I have also released a Codex Masterclass. So, if you're someone who's trying to figure out, how can I master any of the tools you see here in today's video, Chase AI Plus is the place for you. I assume you have no knowledge going into any of this. You don't need to be technical, and we focus on really use cases, real projects, so you can actually apply this to whatever your pet project is.

So, if that sounds like something you're interested in, definitely check us out. There will be a link in the pinned comment. So, for our first test, I'm going to see how well Fable and GPT-6 can recreate Fortnite. I was inspired by this video that I saw on YouTube. It's called Claude Fable 5.1 is insane by this guy named Cole. He only has like 7K subs, so definitely check it out. But, he was showing how he did this with Fable, and I thought "Hey, let's try it out." So, I gave it the prompt you see here.

I gave it several reference images, and I was like, "Pretty much, let's create Fortnite that I can play in the browser with 99 bots and different weapons and all those things." And this is what they were able to create. So, first up we have GPT-6. You can see this is sort of the selection screen. I can choose my different game modes. We're just going to do solo, but I can add some bot teammates if I want. Move back over here.

It has sort of like how to play. It gives me the different buttons. I can do my settings, but let's just jump in and see what happens. So, I'll hit ready, and here we are waiting for the battle bus. Now, we are inside of the bus. This is also supposed to have an entire like storm element. Um you can see here as I jump, I can do a glider, which is pretty good. And also, remember this is a one shot as well. I just gave it the prompt I showed you guys earlier, and that was pretty much it.

So, let's see how this plays out. You can see the bots kind of floating around. We have chests up top here. Let's see how that works. I can grab guns. I can uh give myself potions, so you can see my shield's going up there on the left. Looks like I got a shotgun here. The camera's a bit shaky, but there's a bot. He's dead. The bots are not very good. I purposely told it to make the bots kind of bad, but you can see I can zoom in.

Um like I can reload. I can pick up different guns from people. Right? So I can switch it in my inventory. And so I can tab. I can see the whole map. And honestly, besides the bots being terrible, um like not a bad job. Let's see what happens if I try to like harvest something like I get brick. It doesn't really show anything coming out. There's like small little like specs of dusts, but honestly, overall, pretty solid.

This took Codex about 45 minutes to do this. Now, I don't have an exact token amount, unfortunately, because of how the early access worked, it wasn't like tied to usage, and so I couldn't see that. But for a one shot here, pretty solid. And you probably can't hear the actual audio here at all, but it's kind of just like audio it created on its own, so it's nothing too impressive. Oh, yeah. I can also build stuff. Pretty cool.

Let's see what else I can build, like See if it actually works. So, yeah. I can like jump up it. So, overall, not bad for a one shot. So, that was Codex. Now, let's take a look at what Fable created. So, here's Fable. Kind of like relatively similar. Um I'll move this over here so you can see a bit more. Uh when I look at Codex's, I kind of like that it has the map thing over here. I don't see that on Fortnite's. So, a little less clean, I would say.

But when I look at the um Fortnite lobby, it has more things for me to click on, but these don't actually do anything when I click on them. Now, let's see. So, let's say we're ready to go. It's filling up the lobby. And also, for reference, this took Claude about like an hour and a half to create this, and about 750,000 tokens. So, here's the battle bus area. The camera controls are a little more janky, but I'm not going to hate on that too much.

It's space. I will say this has a super annoying noise >> [laughter] >> as I'm jumping out. And in general, the camera again is kind of janky. Like, I can't really It's really hard to kind of move it. Um graphics overall, I would say are a little bit worse than what we had with what Codex created. But, not like huge difference, to be honest. So, here I'm in front of the tree trying to mine it. I'll say the animations are a bit wonky, and that was very strange with how the tree disappeared.

I can still build things. Interesting kind of way to do it. But, when I try to move into, you know, what I've actually just built, kind of janky. It just slides me through it. Now, I finally found a gun. I will say in terms of like gun play with this one, definitely leaves something to be desired. Like, where I shoot versus where it goes is like this crazy bloom going on. But, hey, I did eliminate somebody. And so, overall, I would say just feels a little less polished.

Again, this is one shot, and it took like an hour and a half. So, it is interesting to kind of see what we can do on a single pass when we give it a very long prompt like that with a bunch of different references. So, overall, it's kind of impressive it was able to do this. Definitely didn't get to what I saw in the reference video that I showed you earlier. I don't really know if you one shot it or it was a bunch of different passes.

Like, sometimes you just never know. But, for just a straight one shot, I will say there kind of is no competition here. I feel like Codex kind of nailed it. Now, for test number two, I wanted to take a look at front end design. This was the prompt I gave it. I said, "I want you to create a landing page for an AI travel website. They put in there where they're going, and you create an itinerary." Something relatively simple.

What I was really focused on is what is it actually going to look like? I said they can do whatever research they want. They can bring in whatever tools or skills they deem appropriate, and this is what we got on the first pass. Now, this is what GPT-6 created. It generated or found this image. I didn't actually check which. Remember that GPT-6 has access to its own internal image model, which is really nice for this sort of stuff.

So, the hero section looks pretty clean. They have like this little thing where I can put where I'm departing from and where I'm going. I told it I didn't care about the functionality, so I wasn't actually going to test like does this actually work under the hood. I was just focused kind of on aesthetics. You go down here, kind of shows you what's happening. Hey, five days, a thousand little stories. Again, we have some imagery.

And I will say in general, as I look at this, this isn't screaming AI to me. This isn't screaming AI slop. This is actually pretty clean. If I was someone like, "Hey, what is this website about? What do they actually do?" I think it does a pretty good job of explaining that. So, it goes down here. Hey, we bring the curiosity, blah blah blah. Lots of images, right? It definitely like bringing a ton of images. A little FAQ section, and then the footer.

So, overall, I think this is solid. This isn't blowing anyone away. This isn't an awards-level website, but for just like clean, simple, figure out what you want to do and do it. I didn't give it ton of creative direction. I'm like actually pretty impressed. On the other hand, with the exact same prompt, this is what Fable 5.1 gave me, which is kind of like, "Eh." Right? Like this comes across very, very basic. You know, like the coloring isn't very inspired.

Like this is a very generic kind of hero section where we have text on the left, some sort of image on the right. You know, it's not like GPT-6 where it's able to create its own images. It didn't decide to find an image. Instead, it just like created this using its own internal graphic system. It has this little border here, which looks very AI. We have sort of our typical cards, you know, rounded corners. There's not even really any motion there whatsoever.

And in general, it just feels very like it's light blue into dark blue. And honestly, I was kind of disappointed with what Fable 5.1 gave me because in general, I feel like the Anthropic models have been better when it comes to front-end design. So, I'm not sure if it just bit off on like maybe the wrong front-end design skill or just like research something weird, but overall, I felt when you compare these two things, kind of night and day.

Kind of night and day. Now, when it comes to front-end design in general, I will say there is a huge gap between those who are good at it using AI and those who are bad. And so, the better you are, the less the model almost matters because you understand how to give it reference images, how to go like what components should look like, where to go outside, you know, outside of these models on the internet and found like find component libraries.

So, there's a ton of skill involved here. And like we can have this argument about how how much does it matter if a model is baseline really good at front-end design if I can sort of coach it. In reality, that's not most people. Most people just like build me this and we see what we get. And I think like if this is the median output we are judging here, if you're not good at prompting, kind of no question. Again, I would give Astra the W here.

So, for the next test, I wanted to look at motion graphics. So, I hooked up both models to the Higgsfield MCP and had them call the same motion graphics skill I've created. And what I wanted them to do was create a 15-second 2D explainer on how internet messaging works, right? Or if you're on your phone and you text someone, how the heck does that work? So, this is what Codex came up with. >> Honestly, pretty good. So, here's what Fable gave us. >> One tap, half a planet away, chopped into packets.

A dozen hops, milliseconds. Every message, every time. >> So, overall, I thought both these models did really well. I would call it a tie. They're both really good at calling these outside tools, like the Higgs field MCP, routing the prompt to something like C Dance 2.5, and giving you solid motion graphics. Now, for the final test, I had both models execute this prompt. This is also somewhat front-end design related, but I really wanted to see how they could push the boundaries in terms of creativity, versus our landing page design, where one of the parameters was like, this needs to be just functional.

Like, this is we're trying to sell a product, someone should be able to very clearly understand who we are, what we are about, and how to use it. And this one, right, it had some room for the visual spectacle. So, we wanted to create this 3D globe dashboard kind of web app. And this is Orbit, and this is what GPT-6 created. So, I put where I'm departing from. Let's say we're coming from New York. I go wherever I'm going to go.

Let's say we're going to Cape Town, and you can see it shows it on the map. I can kind of click wherever I want, which is kind of neat. And if I move this over here, you can see there's information on the bottom. So, we're going to go to Cape Town. Going from SFO to CPT, one stop, 22 hours. It's 19° out there, and for the sample round trip, we're looking at $864 per person. If I then click on explore Cape Town, I have this information over here on the left, right?

Gives me a cool little picture, gives me some information about the place, gives me some potential things to do, right? Hike Lion's Head, and I can even save the journey. It also had this thing called Meet Cape Town at Golden Hour. Like, one of the things it added was this chase the sun functionality, where it shows me on the globe like where golden hour is at any one time, which is kind of cool, right? I gave it some room for creativity, and that's what it came up with, and not bad.

Again, what I really like about GPT-6 is just clean. Very, very clean design. So, I like this. Then we have Arclight, which is what Fable 5.1 created. And right off the bat, it definitely goes for the visual spectacle, not just with that loading screen, but this is just a lot. Very, very bright, tons of things going on. And like, we have like the text is like following the route itself. I have a bunch of different lines.

There's like this craziness going over here with like the fares, and there're live fares. I can see what the percentage increased as of late. So, honestly, kind of crazy >> [laughter] >> off the bat. Like, this looks really cool, but when we compare this right away to the Astra design, like, not as clean. It's almost too much. Although, I think it's cool, and this could definitely be cleaned up. Again, this was all one shot.

Now, similar to what we saw with Astra, I sort of have like the solar clock thing going on, and as I move it and change the time, the fares update with it. Um I can see certain ones that are trending, and if I click on any of these, like, let's say I click on Helsinki, we get this sort of cool animation going on, where it brings us to Helsinki over here, and then gives us some information about it, the local time, the weather, the winds, visibility, all this stuff, you know.

Um I can hold the fare, right? And like, really cool animations. Although, it's like, I can kind of see it. It's actually kind of hard to see. It's sort of blurry because it's so bright in the background. So, it almost feels like visual spectacle became the number one priority with Fable 5.1 versus functionality. And like, it does look sweet. This whole animation. But when I see this, I'm almost like, "Hmm, what if we could take some of this, you know, craziness and like, sort of temper it with what Astra was able to generate?" So, in general, I would call this another win for Astra, to be totally honest, especially when I can kind of do like this whole like explore Cape Down thing, like you see over here on the left, and that really goes for any of these places.

Just like, I like it. Looks very professional, and I don't feel like I would have as much work to do versus here. Like with Fable 5.1, I feel like I'm many prompts away before this is like usable, in my opinion. So, what does that ultimately leave us? Well, I mean, based on the tests I showed you, we did kind of four tests. Astra won three of them. Fable 5.1, I would say, was even with it on the motion graphics. And when we look at the raw benchmarks, we see that GPT-6, by the numbers, also kind of edges ahead of the competition.

Plus the fact it's actually cheaper by tokens. But, these were just four tests. They were just one shots. How important is it to you to have a model that has built-in image generation? How comfortable are you in something like the Codex desktop platform versus the Anthropic platform? Are you someone who even needs to choose between the two? Are you someone who actually uses both? You know, we watch a video like this, and I hope you were able to come away with something.

And really, what you should come away with is that both of these models are just really good. You know, as someone who's been using Fable 5.1 a ton over the last 3 days, I almost feel like these tests I put it through put it through kind of downplay what it's able to do, cuz I really have yet to come across someone who's like, "Fable 5.1 sucks. It's just just isn't it." At the same time, Astra feels great. It really does.

Like, all of the generations it had were like very smooth. I'm not going to say like AGI, because who even like the definition of AGI changes day by day. But I think more importantly, there is a question now of like Astra versus 5.1. There were certainly a contingent of people out there over the last month or so who were like, "Hey, 5.6 actually does compete with Fable 5." But, I think that was a very small community of people.

And when we take into account like how Anthropic has been treating sort of like its user base, you know, take the great example of how like they lowered limits and then pretended they were actually bumping it up when in reality it had been less over the last month. Like, it just kind of leaves a bad taste in your mouth. Fable 5.1, we still only get like 50% of our weekly usage and then 20x isn't really 20x. And then on the other side of the equation, we have OpenAI which is just like giving us a reset every 3 days.

Although, they did take away some of the base resets. But, in general, it feels like they're giving you more than Anthropic. So, I don't think it's such a big difference between Astra and Fable 5.1 that if you like Anthropic, you should just like throw it away and cancel it. But, there's a real question about it now. And if you're someone who's been on the 20x Fable plan, and guess what? 20x doesn't actually mean 20x.

I think we're at a place where you should really consider like maybe I do the 5x plan with OpenAI and the 5x plan with Anthropic for a while. Still paying $200 at the end of the month, but now I have options and now I can really test it in my day-to-day because no video, no test, no benchmark is really going to be the same as when you get in there and use it yourself for your projects. So, that's where I'm going to leave you.

As always, if you want to learn more about how to master Cloud Code and Codex, I have master classes for both of those inside of Chase AI Plus. But, besides that, let me know what you think. Would love to see what you are able to do with Astra cuz they are starting to roll it out for everyone starting today. I've seen some posts about that. But, besides that, I'll see you around.