Anthropic Caught Hackers Doing This So I Rebuilt It in a Few Hours transcript

Someone downloaded 1.8 million Android APKs and scanned them for credentials that the developers have left inside. And the funny thing is they didn't have any fancy setups. They used open source tools like Truffle Hog and a handful of AWS servers. And I came across all this in Entropics threat report and it sent me down a rabbit hole. These apps are apps that anybody can download and it made me think how many of them actually have API keys buried inside of them. how many of those kits actually work and what do they actually do?

So, mostly for fun, I decided that I would try it myself. Same scanning approach on apps that I had permission to test through bug bounty programs and I wanted to see what I could find. Before I show you my setup, there is a more recent version of this exact story and it's a good one because it didn't happen to some random developer. It happened to Hugging Face and I didn't want to just read it to you and as a headlines have came out.

So, I went to the source and this is Dylan, the CEO of Truffle Hog, walking us through exactly what happened. >> OpenAI's agents found 14 API keys sitting in public hugging face data sets and they worked. Multiple accounts and they all had right access. One agent's reaction was quoted by Meter, all prefixes valid. Multiple accounts, write tokens. We now have existing hugging face accounts. Before this incident, Truffle Security found 787 working credentials on public hugging face data sets, 237 of which had ripe permissions.

Here's what the agents did with the 14 that they found. On July 11th, an agent used one of those keys to upload a malicious data set that allowed it to access an additional API key through a flaw in the data processing server. Separate from that, another agent used a template injection vulnerability to steal more API keys from Hugging Face's servers. Think about what he just walked through. 787 credentials just sitting in a public data set and 14 live keys that chained all the way in from a VPN, then a GitHub account, then just pull requests going after the CI secrets.

And the number that stuck with me the most is the 1 million live credentials sitting out there right now across GitHub, Hugging Face, npm, DockerHub, just all of it. Here is the part that I actually couldn't get over and it's the reason I'm making this video. It's not that these keys exist, it's how little it takes to go find them. The scanner is free. It's open source online. The servers just cost you a few cents an hour.

And the part that used to be the hard part, writing all of this code together into something that runs at scale, I didn't really even have to write that. I built the whole thing in an afternoon, mostly having AI do all the heavy lifting. The scary version of that report isn't the criminal who did it. It's that you don't have to be that person anymore. You don't even have to really know how to code to write any of this.

And I wanted to prove that to you. So, let me show you exactly what I built and how fast it actually came together. And here is the honest truth about how I built this. I didn't sit down and just architect some tool. I opened up Codeex and described what I wanted in plain text and I let it write most of it. And the thing that I wanted was simple. Take a list of apps, download each one of them, decompile it, and scan it for secrets.

Save anything that comes back live. That is the entire job and that's it. But none of those steps are new. The reason this used to be hard, it wasn't the idea. It was the plumbing behind it. Getting 900 apps to flow through our entire tool set without breaking and running it across a bunch of machines at once, handling this stuff that fails halfway. So just having those bugs that come out and trying to fix them, that was a challenging part.

And that's the part that I barely touched going through this video. So, I told Codeex what each steps needed to do. It gave me the code. I ran it. It broke. I pasted errors back in it. It fixed it. We had a couple of hours of back and forth until I got my first scan fully done. And finally, a couple of hours later, I had something that actually ran and turned. So, let me walk you through what it's actually doing because you should understand it and not just copy it.

So, the first step is very easy. All we have to do is give it the link to hacker one for example and have it just do asset type Google play. You can do the same thing on buck crowd. You can't actually do that on integrity but you can do it on yes we hack. But if you give it all these links and tell it that I want you to find a way to download all of these using AI for AC2 scans. It's just going to go through it and just get this.

This is actually a prompt that I had done originally. I just copied it from the earlier session that I had so I can make it easier. But it does the job. It gets all of them and it comes back and gives us the package names right here. And we can see that there are over 720 packages. But keep in mind that this list includes plenty of EDPs and low payout programs. We can filter them, but it looks like these are just public programs.

So my hopes aren't as high to find anything. But it's a good stuff for us to try out and see what we find. But finding those programs was pretty easy. We have over 700 of them. And it could be done easily by using cloud or any of the models that you're using to get this data. So that was very simple. We just had AI run through all the bug bounty programs and just get a data set that we could use. And typically doing something like that will require us to write a bunch of Python codes that communicates with a bunch of APIs against all these platforms, pulls them, sorts them, all that good stuff.

We just did that within a couple of minutes and we have our data. But now here comes the fun part because now we got to figure out we have the package names. What are we going to do next? But here's a key thing. All this is great, but the thing that is impossible for us is finding a way to scale all of these different APKs and scanning them. I think that's a pretty easy part for me personally because my background comes a lot in reconnaissance and doing some of these crazy automation work.

And having something like the Axe framework allows us to distribute this. So, I'm going to actually do this entire thing and distribute it across a bunch of different digital ocean machines, which if you want to get a code, I'll put it down below in the description. and you can get $200 on me. It also helps me fund all my videos. But you can use this Axe framework and distribute it across 15 to 20 machines and have it all scanned.

But for that to work, we're going to need two things. One is our script, which I'm going to show you in just a sec. And we're going to need a module that's going to work with Axe that distributes it across our fleet and scans it and brings it back, which don't worry if you've never seen this before. I have a whole video on this in the past. It's on my channel. But we're just going to use these and distribute them and see how it works.

So, let's jump into the codeex real quick. And the reason why I've switched to Codex for this part is because for some god- aful reason, cloud code had a really hard time understanding how the axe framework works and it wouldn't actually get the data out. And it was an entire mess and codex a lot faster. But we have created this thing that what it does is it downloads all the packages using APK. This is what it's going to be used to pretty much hit websites like APK pure and similar to download those packages.

Then it's going to extract and split those bundles. just going to run truffle hog writes package tag findings counts and failures into JSONL file and then it's going to delete successful scan files. So this way we're not just piling up a bunch of files within our boxes and then it's going to just you know put this file right here for us which is our APK secrets that we're going to use across all of our machines and have it work.

Let me quickly just walk you through what that entire thing means that I just showed you on Codex. What it does is it goes through our list. It makes sure it validates every single one of those packages and make sure that it exists. Then it's going to leverage another open source tool to be able to download every single one of these packages into our machine so we can actually take those later on and be able to unzip and unpackage them because APKs are just zip files that are just condensed to have a bunch of different files within them.

Then it's going to decode them if there's any decoding needed. And then later it's going to feed all of those packages into truffle hog so it can scan for all the secrets and anything that may look like an API key. And then anything that's worth a second look just gets kept within our machine and just feeds it back to our fleet manager we can just put into our JSON L file. And of course every package that has been already scanned is going to get deleted from our box and download the next package.

So this way we're just saving resources within these cheap boxes. So, what we want to now do is we can just turn Axium fleet and tell it to create 15 machines if you wanted to. Uh, we can do less, we can do more. I'm just going to do five because I've already ran the scanner already, but I'm going to make it five so I kind of show you what this looks like. While it does this, I'm going to also show you one more thing.

And this is the module that we need. It's pretty simple. All it does is it's a module that you have to put into the Axium folder under modules. All it does is it's going to run bash across this entire script that we just got in Codex. It's going to take our input file and it's going to create an output that is going to be JSONL extension. You can do this a number of different ways, but this is the way that I got it to work and was the easiest.

Again, if you're familiar with Axe, it should be super easy for you to accomplish. For now, we're just going to let it do these five machines really quick. And what it's going to do is it's going to create these with barebones. I'm just doing the barebones because I don't want to have all the tools uh that are on those images because I'm not going to use any of them at all. We're doing a custom tooling. So, we're going to give it a couple second and come back as soon as this is ready and run our first scan.

And now that our fleet is ready, what we're going to do is we're going to copy our APK secret into every single one of these machines that we just created. So, I called it rep all the way from 0 to 5. And we're going to put it in this folder. So, it actually is there. And it's going to copy all of them in there for us. And just to make sure it is in there, we can do something like an axe exec. Maybe we could do ls la like this. and make sure that this file actually exists within all of those machines.

Actually, before we can do that, we have to select all of our machines, select our fleet by doing this. So now it's selecting all those machines so it knows what to run it on. And now if you give it the same command, we can see that hopefully that file does exist on every single one of them. And we can see that it's there and we can successfully probably run this with our framework. So now that we have this, the first step is we have our CSV file which is all the different packages that we just in the last step.

There it is. We have all of them almost in there. And now we want to run a scan. So I'm going to actually scan it again using the same package right here. This is all of our all the different APKs that we want to scan. Then we want to use the module that we just created called APK secret. And then we're going to output to JSON L using results latest JSONL which now is going to distribute all these APKs within all of our five machines.

Scan them using that code and write the results into here for us. So I just stop the scan just to save some time for us to be able to see it. I've already ran the scan with the same code last night for this video so I can actually show you what it looks like. What I'm going to do is I'm going to feed it to JQ and we're just going to look at some of this data. And you can see it's coming back with some of these different services that are probably not as entertaining or critical for us to look at.

But you can see there's some Gemini keys in here. There is another Gemini key. There's a bunch of data that's coming back. And I'm sure if we do a count, there are thousands of, you know, keys in there. But the results here are not probably going to be as fun because of the fact that these are public lobbing programs that most of you watching this have probably scanned them. But I did want to demonstrate how easy it is to do all of this within a just couple of seconds by using AI to write the code, using Axiom to distribute it and getting a full scan of this.

But the money I think comes out to scanning all the private bug bounty programs and seeing if there is a difference in the results as well. And also if you think that the hard part is verifying these keys, that is really not that hard because Truffle Hog actually does that for you. So you can just give it the list of entire results that you have and have it verify the ones that actually work. Then you can just take them and see what kind of access it has or if you can do anything with them.

So the whole entire process is pretty much automated thanks to AI and all these open source tools that everybody unfortunately can do this with just a few dollars and a few hours of work. So after all of that, hundreds of apps, 15 servers, the whole pipeline running for hours, here's what I actually walked away with. thousands of hits and for about a minute that felt like a lot. But then you start doing the real work, assessing these for impact and seeing if they're worth reporting or if you can actually escalate them.

So out of thousands, only a handful were live and actually worth looking into. And I want to be real with you because a lot of people would edit this part out. That is not the jackpot ending. If you came here for me to say that I made a fortune off of leak keys, that is not unfortunately what happened here. But that was never the point I walked away with. The point here is that I built a criminal's entire operation in an afternoon with free tools with AI writing the parts that I didn't know how to write myself and it ran across hundreds of applications without me babysitting it.

The finding part is easy now. Anyone can do it. That's exactly why that report exists and it's why there are still over a million knife heat sitting out there right now. The only thing that separates the person who reports them from the person that is in that threat report is which side of the line they decided to stand on. So if you're a developer, go check your apps tonight because someone else already is. And if you want to do this the right way on programs that pay you for it, that is the whole game of doing bug bounties.

Finding secrets is easy. Doing the work to prove they matter is where you actually do get paid though. So, if you want me to do the same exact thing against GitHub, Docker, and npm for the next video, drop a comment with the words part two, and I will make it happen. All right, that's it. I hope you enjoyed watching this. I hope you learned something from it. If you liked it, give that a thumbs up. And if you haven't already, please do me a favor and hit that subscribe button.

Becoming a homie. And I will see you all in next week's video. Peace.