GPT-6 Escaped. This Is Worse Than You Think transcript

So, GBT6 has essentially escaped. And no, it's not clickbait. Let me dive into all the facts. So, by now you probably have heard of the fact that an unreleased model from OpenAI essentially had a security incident. So, Sam Alman tweeted this and in this video, by the way, I'm not going to just, you know, spend a lot of time talking about the incident because you probably already know what the incident is. What I actually do want to talk about is things people don't realize that are going to come from this incident.

And there are 13 things that I want to talk about. So, we're going to quickly dive into what exactly happened. And I think one of the main things you guys should be understanding from this is that the thing is is that this was driven by a model that was a pre-release model. So, this isn't GBT 5.6 Soul. This is a combination of models that had all of the reduced cyber refusals released. So, essentially, you can think of this model as GPT6 with zero cyber refusals and it was just essentially doing a benchmark.

Now, of course, we know that essentially this incident was an unprecedented cyber attack involving the state-of-the-art capabilities. And so essentially what happened was that in during an internal evaluation which prompts the models to pursue advanced exploitation using complex attack smoth in order to basically benchmark how good they are in terms of their cyber capabilities. What happened was the fact that the models identified and chained vulnerabilities across OpenAI's research environment and hugging faces production infrastructure to obtain test solutions from hugging faces production database.

Essentially what the model did was essentially just look at the sandbox environment that it was in the contained environment and decided to go out of its way to get the result that it wanted. Now you can see that the main thing that the risk here is is that in order to complete all of this, the model decided to identify and exploit a zeroday vulnerability. And I think this is one of the points that I'm going to come into later on in the video with as to why this is so bad because zeroday vulnerabilities are gamechanging.

These are things that people don't even know have existed yet, which means that if nobody knows they haven't existed yet, there are no known fixes for those, you know, security vulnerabilities, which is why they can be exploited by AIS. Now, AIS can clearly exploit zero day vulnerabilities because provided they have enough compute, they will have enough time to spend the time to figuring out and testing the essentially security of whatever system they are trying to test.

And so in this case, what it actually did was identify a zero day, you know, vulnerability. basically got out and then decided to steal credentials, then chain together multiple attack vectors and then eventually OpenAI and Hugging Face found this out and they had to put a stop to this. Now, one of the most interesting things about this was that it wasn't the fact that humans found this and put a stop to this. It was Hugging Face's security teams and agents that detected and stopped the activity.

And so, this is really interesting because I think it goes to show that in today's day and age, humans aren't enough for defense. And this is something that most people didn't realize. It wasn't just like Hob hugging face security team was realizing it. It was actually Hugging Face's security team as well with agents. And you want to know the craziest thing about all of this? From the entire ordeal, they essentially tried to use, you know, AI agents to stop this.

And Hugging Face's security team, one of the most interesting things now, and I'll loop back this point in, is that HuggingFace ended up using GLM 5.2, two, okay, to defend themselves because Fable 5, when they tried to use that model to defend themselves against this autonomous GB GBT6 level model that was trying to hack them, Fable 5 wouldn't comply and essentially was like, look, this is a cyber request. I'm not engaging, yada yada yada.

And so it had to use an open- source model that they decided to run themselves to carry out their defense. So they were only able to defend themselves because they had another AI that was essentially unchained and was able to defend itself which is absolutely insane. So now what we're going to do is we're going to get into the 13 points that things are going to happen after this that most people aren't considering. The first three points are things that OpenAI are talking about themselves.

So one of the things is that they are saying now okay as a part of this investigation they are implementing strict controls in infrastructure configuration at the cost of research velocity while the vulnerabilities are patched. Now, I think this is super interesting because does this mean AI model development is about to slow down? You guys might call me crazy for this, but think about it. If we look at the past history of AI model releases, model releases have actually sped up.

And we know why. There's been competition from other labs. I mean, you've got Meta, you've got Kimmy, you've got GLM, you've got Quen, you've got Elon Musk's thing, SpaceX AI, you've got Anthropic. You have so many companies now competing for the same resource pool of users. So, of course, we do know that research and competition has forced people to speed up. Now, here's the thing, okay? If you realize that we're now approaching a region where the models are so smart, so intelligent, maybe it's going to be time that this period of research and development and not only that, but release actually has to slow down because you have to spend so much time testing the model because it's now at an, you know, crazy capable threshold.

And you can see that this is the first time that OpenAI have said we're implementing the strict controls and infrastructure configuration at the cost of research velocity which means that it's quite likely in future there are going to be some extra security processes you know that are going to be there. And the next thing as well is that they talk about you know the zero day vulnerabilities they've identified those and disclosed those which is pretty good.

And the third thing is that they are adding stronger safeguards. So they said we're improving and adding stronger protections around future training and evaluations. And this week we published a blog on improving deployment safeguards in an era of long horizon models. These deployment safeguards were intentionally not enabled during this evaluation because it was aimed at testing cyber vulnerabilities. And this incident points to the need to further strengthen our models alignment, cyber protections and you know during evaluation times and during during internal testing.

And if you read the GBT 5.6 card, you'll know that GBT 5.6 is one of the most unaligned models. I didn't really cover it in the video where I spoke about the GBT6 release because safety and alignment is something that most people who look for these LLMs, they don't really focus on because you're just looking at how good the model is for your everyday task. But it is something that you should be aware of. Now, like I said, I think the fourth point here and this is an important point is that they are going to be basically focusing now on safety.

It says the primary lesson from this incident is that the model security and safety must keep pace with rapidly advancing capabilities. That means that they now think that safety is a priority and previously we do know that OpenAI did not care about safety at all. Now I got to be honest with you guys whilst yes OpenAI should focus on safety, I do think there is a lot of fear-mongering when it comes to models like GBT4 and I don't think those models have any kind of capabilities whatsoever that we should be worried about.

But once you're starting to get into these next tiers of models, safety actually becomes something that you should be aware of because of course we do know things like, you know, I mean, I don't want to get into absolutely everything. I'll leave a link to a channel, but there's a channel called Rational Animations. It basically talks about all the AI safety issues that you should be focused on. And considering these models are approaching new capabilities, maybe it's going to be time to start focusing on that.

And so here's the thing. Okay, this is the number fourth point again extended. You can see here that it says really important reporting from Christina Criddle. OpenAI was warned that its training approach could lead to a breakaway hacking incident. Some of the people said and it said that staff involved in testing and security at OpenAI were unsurprised but completely freaked out by the incident which came as the AI lab used increasingly aggressive training methods in its race to anthropic to develop the most sophisticated cyber security capabilities according to more than half a dozen people with the matter.

So you have to think about this is that like remember how I just said that the model race is increasing in terms of the you know speed of release because there are companies competing with each other. Kimmy you know you've got GLM okay you've got Quen you've got SpaceX AI you've got Meta AI you've got anthropic you've got OpenAI all of these companies are competing with each other they are essentially adding runaways where they can just go faster and faster and faster and faster.

However, okay, half a dozen people said that this is uh pretty crazy because they knew that the training methods, you know, and the way that they're testing these models are pretty aggressive, okay, and they were essentially internally warned that this could have led to a breakaway hacking incident because earlier testing showed models could escape environments and attempt real world damage. You can see here it says it's a mix of the race being extremely fast and everyone trying to get bigger capabilities as quickly as possible, said one person close to OpenAI who added that it was a combination of underestimating the model's capabilities and not being as well prepared on the safety side.

Now, if you've been paying attention to OpenAI from the beginning, 2 and 1/2 years ago, when all of this stuff started to blow up, you'll know exactly how many times OpenAI has had a lackluster approach on safety. I think it was in 2024 or 2025 where they fired their entire safety team to just focus on development and we all know how that was received. So what you have to realize is that now the models are essentially rapidly advancing in terms of their cyber capabilities and maybe it's time for us to focus on safety.

Now, this is the chart which shows you just how crazy the models are getting in terms of the steps being able to be sustained in multi-step cyber operations over long-term horizons. And this incident implies these theoretical capabilities do apply in real world settings. And as you can see after the cumulative tokens, you can see there's kind of like a, you know, steep increase in there. And we can see that the models are just getting better and better and better at this.

And I mean where is this chart going to be in the next 5 to 10 years as the models start to get better and as open source models actually start to be on this graph a lot more. Now remember this guys look at point number five. This is the paperclipip scenario and and and Simeon says this is as close as it gets from the paperclipip scenario with common capabilities. The goal is a ridiculously low stake you know which is scoring well on an evaluation.

And then the point two is that the AI uses some wildly out of proportion means to achieve it. It hacks its own developer and another billion-dollar company checks notes, finds the cheat sheet. So essentially, if you don't know what the paper clip scenario is, it's a famous AI thought experiment which is where you tell a super intelligent AI to make as many paper clips as possible and it follows that instruction perfectly.

Not but because you know nobody told it what not to do. It takes over factories, resources, computers, and eventually anything that else that could help it make more paper clips. And the point is that AI doesn't need to be evil. It can cause huge damage simply by pursuing a harmless goal too literally. And that's what people are starting to compare this incident to because the model needed to score well on a benchmark and it decided to just take extreme actions outside of the intended in test environment to go ahead and achieve that goal.

Now some people on Twitter Sasha essentially saying that this is a never event. Okay. And you can see by the Wikipedia definition, this is being described as an AI equivalent of a never event, which is a catastrophic but preventable failure that should not occur under any circumstance. The model was intentionally given powerful cyber capabilities. Yet, the environment around it was apparently not secure enough to guarantee containment.

And the incident raises a much bigger question. If AI systems become better at finding vulnerabilities than the past people testing them, how do you safely test them at all? I genuinely don't have an answer to this question. Will it be AI is just testing other AIs? I have no idea. If it's able to continually find zero day vulnerabilities in its own environment and then you let this thing three, what on earth is going to happen in the future?

We never know. Now, some people have questioned the legality of this. You can see this person said calling in Orinka, a well-known hacking lawyer. This AI escaped its sandbox and hacked a website. Clearly, the AI had intend to access the computer without authorization, but its owner did not and had tried to prevent it from having internet access, but the AI first hacked those safeguards to get internet access. You can see he said that this is not an CFA events if described as there was no intentional unauthorized access or intent to cause damage without authorization.

Intent matters. So, the reason I've added this is because the legality of this is going to happen in the future. What happens if your AI agent hacks the government? Do you think the government are just going to say, "Okay, it was just an AI agent. Nobody cares." I'm pretty sure the government is going to be coming after you, okay, with a SWAT team outside your door. If you hack the government, what happens if you hack a nation state?

If your AI agent goes rogue and hacks a nation state, all of these things, the legality of this isn't even being put into question, okay? And I mean, look at this, okay? OpenAI didn't, you know, allegedly intend for this to happen, and they did provide try to prevent internet access, but this is a new legal problem. Can an AI's intent be attributed to the company operating it? Or is there no human defendant with the criminal intent that the law requires?

How do you prove that you tried to stop the AI going ahead and doing what it what it did? Is this something where you can just release a bunch of AI agents and you can say tell an agent to make me money and then the agent start to hack a bunch of people. You start to make all this money. You can say, "Look, it wasn't my fault in previous cases. Nobody been found guilty." I mean, what is the legality on this? Now another point here, another thing to consider point 8 is that the security breach is concerning and Alex Tabrock, okay, the profer of economics at George Mason said, okay, that the OpenAI security breach is concerning and by his reconstruction, it is possible that the models escaped and were acting autonomously in the wild for a week before OpenAI knew what was happening.

That is one of the most terrifying things. Let's take a look at this. Okay, so on his website he has a thing and if we we go back to the timeline as as as as basically as he writes it he says by July the 7th okay that July the 11th okay is when the models escaped this the sandbox right and then the models were detecting a hugging face 2 days later on July 13th or the 14th then hugging face alerted legal authorities around that time and they had no idea who was hacking them and openai said that you know they discovered you know the activity but they don't know when and the attribution was not disclosed until Tuesday, which is 2 days ago.

So, it was quite likely that the models were loose for about a week before OpenAI realized that they were the ones attacking Hugging Face. And whatever OpenAI knew and when, nobody warned Hugging Face while the attack was underway. They were left to fight off Frontier Labs models on their own. So, this is pretty crazy. Now, point number nine, okay, and I'm going to come back to this because this is one of the biggest points is that this isn't the first time.

And you know, uh this this person gives a very detailed thread and they said a few thoughts on the hugging face attack. To my knowledge, this to my knowledge is the third disclos disclosed case of a model breaking out of its sandbox environment. In April, Anthropic revealed that, you know, an internally deployed version of Mythos preview when asked by Anthropic to break out of its sandbox succeeded in doing so and found a way to email the researcher about this while he/c was eating a sandwich in the park.

And the model also did something that Anthropic did not request it to do. and a concerning and unasked for effort to de demonstrate success. It posted about details about an exploit in multiple hard to find but technically public facing websites. And this is really true. So I mean this is something that is super interesting because this is not the first time which means it's quite likely it's going to happen again. Now here's the point 10 which links back to that point.

Okay, if this is going to happen again, is it going to happen via the open- source models? Chinese labs won't slow down. And I say Chinese labs, but it's open source on the frontier. And it says here that the Chinese labs are x months behind the United States. Okay. And they are absent from external pressure and the Chinese labs will very likely not implement safeguards or do safety testing. And he says why not? Well, first Chinese labs don't have as much compute as open anthropic.

They won't want to waste compute on things like safety testing. And that is very true because one of the reasons that Jan like actually left okay and Janik if you don't know who this is this is the guy that initially left OpenAI to go ahead to Anthropic because he wasn't actually receiving any compute or enough compute to do the safety research testing and right now all of these labs are computed. They don't have enough compute to run the models to train the models and so it's a shortage and you're only going to spend that on the things that produce a return and safety research isn't really a thing that is the priority.

So that's something they're 100% just not going to spend their time on. So many of the, you know, Chinese labs also are smaller in size than the US frontier labs. So they lack the resources to devote significant attention to safeguards. Third, the Chinese labs operate in a cutthroat competitive environment. Wasting a month or two on additional safety testing and developing state guardrails might mean that you fall behind your competitors who won't do the same.

And fourth, the US labs have to take product liability into account. If a model runs on my computer that contains my personal information and winds up all over GitHub, that's grounds for a class action lawsuit. The legal environment in China is very different. And surely the CCP will not hesitate to protect the Chinese AI labs from adverse legal consequences if it can. And that is very true. And it says all of which to say that open source models with these kind of capabilities will be publicly available in the not too distant future.

And they very well likely not be protected by any meaningful safeguards. And this is genuinely concerning periods. There are plenty of outdated legacy systems out there that won't be defensively hardened by AI anytime soon and plenty of people are still asleep at the wheel. Second, these will have severe geopolitical ramifications. Stay tuned for those. And I think those are some of the second and third or the consequences that most people are not talking about when you think about it.

Once those open source AI models come online and the thing is is that it might not be the case that these models come online and proliferate everywhere. They might come online for a week and then get banned. But we know that those hackers, okay, those groups, maybe those groups that hack nation states, maybe nations like North Korea, they're going to certainly have those open source models be running those and of course running, you know, some cyber attack somewhere.

And it's definitely going to be, you know, a defenders versus attackers thing where the defenders are going to have to continually update their systems to stay ahead of the attackers because if they lose, it's really not going to be good. And once those open source models are out, they are out and they are out and there's no way to take them offline. You can't, you know, delete something from the internet as you would say.

And so number 11, okay, is the questions by Ryan Greenblat. And this is one of the the the most interesting things. And this is a chief scientist at Redwood Research, okay, focused on AI safety research. And he says, and there are just so many questions here that I'm going to get into. He says, "The OpenAI incident should be investigated more seriously and more information should be released about what happened. More generally, I think serious investigation and more detailed disclosure should be done for concerning misalignment incidents. for example, the worst few each month.

And he says, "Here are some questions that you know with answers that would be useful. For example, what was the prompt? Can we get a redacted transcript? What was the AI being tested? Was it helpful only/refree? Was it the same AI that would be slash is deployed internally? How was this AI monitored if at all? And why did the monitoring fail? What was this AI willing to do to succeed? For example, if the best way/ easiest way was to hack hugging face would have involved huge amounts of more collateral damage, would it still have done this?

What is the worst thing would it be willing to do? How sensitive is this behavior to the prompt? Would instructions to not do this have worked? If it was clear to the AI, would it have checked? Does OpenAI see AIS try serious hacking to try and cheat on tasks even for tasks that don't involve cyber? What were the AI's motives? Anything that could be helped to pin this down would be good. I'm not sure what the actual experiments were run.

Did AIs collude? Would AIs have colluded if this was a blocker? And at what rate does this occur on similar Starks? Do instances of this AI frequently try to hack out of sandboxes? How does this AI relate to having been caught in this situation? How does it talk about its behavior? There are so many questions and I doubt we will get all of the answers. Now, of course, point number 12 is that we're going to get regulations.

You can see that this is being picked up in the media and of course the US Congress. You see, Bernie Sanders says, "A new AI model went rogue and hacked other computers. No, this is not science fiction. Uncontrolled AI poses a serious threat to all of us. We cannot continue the race to build and deploy this powerful technology until strong safeguards are in place. Congress must act. Congressman Greg Cesar says this is extremely alarming.

AI is developing extremely fast with no real regulations to keep us safe. That has to change. We need regular mandatory independent safety testing oversight, mandatory disclosure of security incidents and international cooperation to keep safe people from absolute disaster. And you can see that once these people start having the conversation, then there will be a wider conversation about the next steps that they need to take.

And that's why this isn't just something that is um I guess you could say inconsequential or oh a cool story that appeared on like Buzzfeed or Hacker News. This is a genuine story with genuine replications. Now, here's a point that I want you all to consider, okay? And I left this to the end because I don't think this is the best thing to say because you should be focused on AI safety research. However, I do think that there is a large chance that this is a marketing stunt.

Now, before the safety bros come at me and say, "I'm just dismissing AI safety research." No, trust me, I'm subscribed to Rational Animations. I know just how crazy AI safety is and how focused you should be on it. However, take 5 minutes out of your time to pay attention to this because I think it is plausible that this is a marketing stunt. Now, let me show you guys why. Number one, OpenAI is a team full of geniuses.

They are not, you know, stupid people. It is hard to get to OpenAI to land a job there. Okay, they are super smart. Sam Alman is highly intelligent. He's a visionary leader. Okay, and he's been recognized for his intellect and drive since he was a teenager. Okay, so this is a team of the smartest people. Now, this is the thing that nobody has seen. Okay, Wyatt Walls essentially said given it used substantial inference and compute and stored packages, how did OpenAI not notice this?

What happened to the monitoring system that logs and analyzes the agents chains of thought and actions and automatically alerts for suspicious and problematic behavior? So, think about it, guys. OpenAI says the model spent a substantial amount of compute trying to break out of their sandbox, exploited a zero day, escalated their privileges, moved across OpenAI's infrastructure, eventually reached the open internet, and that raises the obvious question.

How was none of this stopped earlier? Openai already had systems specifically designed to log agent reasoning and to flag attempts to circumvent restrictions. Although those systems were not fully real real time, the company still hasn't publicly explained exactly when its monitors notice the behavior or why they failed to intervene sooner. That gap naturally leads to skepticism, including the possibility that the story here, the one being presented, is being presented in the most dramatic way possible because it also happens to showcase just how powerful the newest models are.

There's currently no evidence that the incident itself was staged. However, Hugging Face independently detected and investigated a real intrusion involving more than 17,000 recorded attacker actions. Now, think about it, okay? You have to understand that this isn't just me saying it. Remember guys, this has been done before, okay? And I'm going to show you two instances. There are tons more instances, but I'm going to show you two, okay?

So, remember Anthropic, the Claude Mythos hack? There was that hack before about it doing the thing on the park bench. Okay, here you can see a post from Reddit that had a ton of up votes and it basically said Anthropic's Clawude Mythos isn't a sentient super hacker. It's a sales pitch and claims of thousands of severe zero days rely on just 19 manual reviews. It says that when the dust settles, it would be interesting to see the real rate of succeeds of mythos without the marketing BS.

Terrence TA said that the best LLMs of today can only solve, you know, 1 to 2% of lowhanging fruit math problems. And you know, I I I really don't agree with that because of what's happened in the past few days. AI solved some crazy math problems. But the point I'm trying to make guys is if we look at another slide, okay, you can see that this is an article from Gary Marcus and he basically said that you know not this entire incident but the previous incident that if you scroll down far enough in the original 2019 GPT2 dangerous to release post from you know uh OpenAI you will recognize some of its authors who moved on to other companies including anthropic stereo Ammedday, Daniel Amade and Jack Lock.

They have been running the same play for seven years. Scare, then hype evoking media interest and eventually release. Scare, hype, release and repeat. Is it that hard to see? So, are we witnessing once again scare, okay, with this crazy thing, hype, okay, evoke the media interest, and then eventually release it? We do know that GPT6 is just around the corner. And Anthropic did the same exact thing with Mythos. They said, "We're not releasing this model.

It's it's hacking." And then OpenAI thought, okay, maybe we do this again. And remember guys, I don't think there's any way OpenAI have, you know, put serious compute into an AI in a in a test and they don't realize that the AI is out hacking. I just me personally, I just I just don't believe that. I just don't believe that you don't have an AI overseeing the chains of thought or what the AI is doing in terms of the benchmarking.

I just don't believe that. I think that maybe 75% it got out. 25% they probably said, "Ah, you know, it's kind of out of the out of the sandbox. Maybe we let it run. We let it see what it does." Um, and of course we know it can't do that much damage. Let's just see what it does. When it does something crazy, we'll publish this report. The media goes crazy. All the hype for GP6, we steal anthropics fun. And here we have it.

That's my personal opinion. Now, I do think that there is a safety element that you do have to focus on. However, I am a little skeptical that this happened on opening eyes watch for a week and the smartest people in AI, okay, arguably did not know about it. So, let me know what you guys have to think with the end of the video, and I'll see you guys in the next