Building a Hackbot: How Ethiack's Hackian Finds Real Vulnerabilities | André Baptista @ HackAICon transcript

All right. Presentation is working. It's a pleasure to see you all here. I hope you're enjoying our first AK icon. Um, so I forgot to send my slides at time to my marketing team, so I apologize for that, but I have them here in my computer. So, I'm going to be talking about offensive security with hackbots. As some of you as you know, we are also developing this kind of technologies and actively researching it at Ethiack.

I'm going to bring you some insights on how to build one on your own, but also I'm going to show you some of some examples and some examples of what the hackbot can do as well. So, how did we get here in the first place? And how can we leverage AI to do better hacking, basically? So, and I wanted to recap a bit in terms of on why and at the time that we had created Ethiack, we realized that back in 2021 and 2022, I was invited to a live hacking event with HackerOne and the target was GitHub.

And we got some like early access to GitHub Copilot and they introduced GitHub Copilot as one of the first, basically, AI pair programmers. So, I was like, "This is really interesting. Like, it can generate code. It can complete and predict what we want to write as programmers. So, if it can do engineering, maybe it can do some sort of reverse engineering as well, aka hacking, right?" So, it was super interesting and we started playing with it and trying to make it generate some some code to find vulnerabilities and it kind of worked for code completions, right?

But, LLMs back in that time were still kind of new. I actually knew this before even ChatGPT and it was already super interesting technology. And then when we started doing more research, we started thinking, all right, like are we going to need this kind of LLMs or AI models trying to find vulnerabilities and we need AIs for everything. And we started by designing an engine that could basically analyze attack surfaces at scale.

And from basically what you see here is one of our automated scan pipelines. And it starts you cannot see properly here, but it starts with multiple object types like wildcard wildcard domains. Then it goes down to FQDNs, IP addresses, URLs, screenshots and and more. So we try to make all to automate in a more deterministic way how we could basically find vulnerabilities in huge attack surfaces. This is not entirely new nowadays because as some of you know like bug bounty hunters have been doing or some bug bounty hunters have been doing this kind of automation for a while and trying to find vulnerabilities.

But we made some changes into this logic and we were able to start integrating this kind of models and I'm going to show you how you can also play with it. But also we try to make it really scalable and make it detect changes across attack surfaces because you don't want to be scanning the same targets all the time. So let's dive in into a bit of the history of AI. So when we look at classical machine learning which dominated the field for decades, we used to work with structured application specific data that have to be carefully engineered and preprocessed.

And we really needed it to make predictions for multiple reasons. So basically you can think about like tabular data sets that basically feature manually craft by data by domain experts, right? So, these models were designed for one primary purpose to make predictions based on the inputs and they were very effective in some narrow domains, but the systems still required a significant human intervention to transform raw data into a format that the algorithms could even understand.

And then we move on moved on into deep learning which marked a very significant shift. We could now provide multiple multiple types of data into these models just say such as text, images, or even audio for example or video without extensive manual feature engineering like classical machine learning. So, basically these deep neural neural networks learned how to extract meaningful patterns from the data automatically.

So, this breakthrough basically enabled models not to make only predictions in more complex scenarios, but also generate new content while expanding the range of the possible applications. So, these systems stayed highly specialized trained for specific applications, but still lacked a broader understanding of our world. And then LLMs came. The slide you know okay, changed. So, and then LLMs came and large language models represented a significant shift that we all feel today.

And and they all started with the transformer architecture in 2017 with massive scale training massive scale training on comprehensive data sets. And nowadays we have got reports for example Anthropic that trained huge models with a lots of books for for example and they had to pay some fines for it. So, by training these models on vast amount of diverse data from across both human knowledge, not just task-specific data data sets, LLMs could actually achieve something important.

A broad generalization across multiple domains, and it opened the door to a lot of use cases. And these models actually we perceived that these models developed an understanding of language, reasoning, and multiple knowledge about the world they live in because the previous models lacked the knowledge about the the world, right? So, it was all about the broad generalization across multiple domains. And then, finally, we moved on into agentic AI.

So, basically you connected all these tools into all these models, and it's basically the major focus of contemporary AI development. So, by equipping LLMs with tools and the ability to interact with their environment, to call and execute commands, for example, to run tools, to use browsers, to go to the internet, to send requests, and perform these kind of actions in the digital world, we have moved beyond beyond generation and prediction to actual actions, right?

So, these systems can now plan and execute tasks, and they can actively probe systems, adapt their strategies based on results, and operate with a level of autonomy that we can perceive, and that opens also possibilities for ethical hacking. So, how can AI be used to improve hacking? We have been launching some blog posts and also did a workshop like a couple of months ago on this, and it turns out that you can use these kind of tools to improve your reconnaissance in the first place.

In this case, you have an example of we using the LLM tool, uh which is a CLI that you can run and you can connect it with multiple AI uh providers such as OpenAI, Anthropic, or even local models that you're running via Llama and stuff like that to generate uh a potential wordlist of subdomains, and the model will be aware about the target, right? So, in this example, it's an enumeration list for Tesla, so it the model was trained with this kind of um with sufficient knowledge about the target, so it's able to generate a wordlist contains, for example, autopilot, right?

And you The possibilities here are endless because you can also um use these and you can combine it even with NLP and extract tokens and and keywords from the uh the pages from a target or even give information about the industry if the model doesn't know specifically the the target and feed that to enrich uh subdomain permutations, for example. Another example is uh or now we can summarize webpages, for example, and especially when we have huge attack surfaces, we can basically use tools like ScraperAPI or other ones to basically extract the contents and the summary about a given application, and that really help us as a security researchers or while developing tools to prioritize and even feed that output in into another model to prioritize what we are going to act next, for example, which is also really useful, but also to provide context to a chatbot.

That could also be a potential use case for this. There are more examples. Um I forgot that this is not working properly, so I need to move on here. Um writing reports, uh this is multi-please GPTs, uh basically, this is mostly about prompt engineering, but it also provides you a way to write not just bug bounty reports, but technical reports about vulnerabilities in general and make it make them more concise and direct, and that really helps on trying to assess the impact, for example.

So, there are a lot of researchers that are starting to use this kind of tools to help them on writing bug bounty reports. Um and also some bug bounty platforms like HackerOne actually started integrating these kind of assistance into their platforms to help the researchers to improve their reports to be written in more direct and and professional manner, for example. Another example from our fellow advisor Rezso is FFUF.

It's for content discovery. So, when you're trying when you're doing some some these kind of techniques and applying them to a target, the goal is basically to find, for example, hidden directories on a web application, and you have tools for that that are based on fixed word lists. So, the idea here is to basically generate extensions dynamically by providing and accessing the target application to understand that, "Okay, if it's a PHP application, then we are just going to fuzz for .php and PHP-related extensions, for example, like far or stuff like that, right?

So, it's very interesting a very interesting adaptation of FFUF by integrating it with LLM models, but I also suggested this to Rezso, but if anyone here has the time to do it, it would also be very interesting not just to mutate the extensions through a model, but also generate word lists dynamically that are aware of the context of the target to find a specific hidden directory, for example. And this is all very cool, but yeah, we still have to do all the repetitive work.

Um so, and obviously, and I'm going to show you like how to to build a hackbot now, we are really focused on trying to automate the things that could be automated and leave the creativity and the real research to humans, right? So, let's move on on to building an hackbot. So, can we take an a step further? I'm pretty sure a lot of you already knew or there are there were previous previous talks and panels that already touched into hackbots.

So, what if these AI agents could be combined to hack and like a human or close to a human, right? What if these AI systems could actually reason and conduct penetration tests in an autonomous way? So, how can you define an hackbot? This is a definition also from Reza and in his blog, which is basically an any augmented system that can use AI in a meaningful way in order to find vulnerabilities or to perform other types of tasks like reconnaissance in order to find vulnerabilities, but vulnerabilities being the end goal of the the hackbot.

So, basically we are aiming for continuous 24/7 reconnaissance without the fatigue that we suffer as humans with ice capability that allows simultaneous testing across multiple targets and even make it able to adapt automatically and dynamically where the system can basically iterate and adjust the approach based on what it knows and what it discovers about each specific target. So, what are the the fundamental components of a hackbot?

So, let's start with the brain. So, the brain are basically the is the AI model. So, the better the LLM model, the better will be the brain, right? So, LLMs serve or can serve as the central reasoning engine for an hackbot. And that basically defines the creativity, the planning, the decision-making through the acting process. This component is critical for the quality of the analysis that is going to perform and the adaptability to different types of targets and scenarios and the reliability of its operations.

So, when selecting and integrating an LLM, it's important to balance accuracy, creativity, speed, and cost. For example, creative models may offer more flexible solutions or if you even increase the temperature of a model, but they are more prone to hallucinations. While highly accurate models with less temperature or specific models that are not so creative or that we do not perceive as being so creative will produce more reliable outputs but may lack this creativity, which is a lot of times necessary for hacking.

And then you have the the structure. Um so, basically, you can think about like multi-agents, multi-agent architecture, and it will provide and it's more engineering-related and it will provide the skeleton that agents will operate within. So, this component is basically essential for a lot of functions, including the coordination between different parts of the system when multiple agencies agents or processes are basically involved.

It should be also be able in this part to implement how it manages memory to try to to make sure that it knows what it was already tested, for example, and to track uh how vulnerabilities have been found, for example. So, key design considerations for this this part include the clarity of roles when multiple agents are basically involved, establishing effective communication pathways between system components, and designing an information flow between agents that maintains the context throughout the process.

And then we have prompting our prompt engineering, let's say. Um prompts basically serve as critical translation layers between human goals and the behavior for the agent. So, basically they are responsible for converting high-level penetration testing goals into specific and actionable instructions that the AI can follow consistently. So, this component is critical for empowering the models with information that is very critical for a given task.

And it provides basically the alignment with the intended goals and also the scope, providing control controllability over the system behavior during the the operations of the the act bot. So, when designing prompts, the key considerations here include clarity to avoid ambiguous instructions, incorporating appropriate constraints to prevent out-of-scope activities, and to make sure that everything is adaptable across different scenarios.

And we can also use a proxy, for example, and set the scope, and in that way you can also have a deterministic manner to make sure that the act bot can always touch a given assets or a set of assets, but it will not be allowed to touch the out-of-scope assets, right? Like we basically configure it the the scope in Caido, for example, or Burp when we are doing some manual hacking. And then, finally, we have the tools that are absolutely essential for designing such an act bot agent.

So, basically the tools extend what the agents can do beyond reasoning. So, thinking about Think it about like a a virtual machine or like an API that you can provide to the act bot, and it will be able to It will make it able to access relevant data, databases that you may want to provide to the act bot to extend its knowledge, perform actual testing actions as well, like executing a command, sending a request, for example, and and also validating um the outputs and findings from the testing process.

You can also integrate tools to do it, right? For example, if you have an XSS, you could or if you find an XSS, you could basically try to open a browser through these kind of tools and try to understand if it actually executes, right? So, while when implementing these kind of tools, the considerations are about defining the scope to balance capabilities and safety and implementing safety measures to prevent unintended damage or unauthorized access, uh while ensuring that it's easy to integrate with new tools as requirements evolve.

This is all cool, but what we've been noticing is that sometimes, and there were multiple reports about this, um not related with act bots, right? But, you all know that we had these scenarios with cursor, for example, or other um AI tools that are used for vibe coding, and basically, it all of a sudden decided to delete like the home folder from a developer, and that's really bad. So, the same uh this kind of actions that may be generated by an act bot must be uh controlled or must be prevented as much as possible in the case of act bots.

Because these AI agents will actually interact directly with systems, right? So, we need to make sure that no systems are disrupted that it cannot uh affect the integrity of the data, the availability of the systems, once again. Uh so, or or even access areas that it wasn't supposed to reach in the first place, right? So, we need to implement guardrails, and we have three layers to do it, and we were able to reduce the probability of these kind of destructive actions to nearly zero when we try to basically adjust the prompts in the in the first place.

We have deterministic filter and a the gentle layer. So, the first layer is the prompt level where we shape the model behavior or it could be done also in the model training itself. But, since we are trying or relying and if you want to rely on third-party models, you need to adjust the prompts, right? So, basically, this is where we guide the AI with clear instructions and examples of what is safe, what should be done, and what not should be done.

What is ethical and what is not ethical, right? And then, we have the deterministic filters like the LLM firewalls, right? You can think it about firewalls. So, those are rule-based checks. You can basically have a set of rules, regular expressions, and so on to enforce structured policies and patterns to filter these kind of actions that may damage systems. And finally, you can have for more, um, you know, like some more tricky scenarios, you can also use other agents to evaluate what the trying to do.

To have a second layer of an agent that is evaluating what is being done. To to basically ensure the alignment with the original goals. So, in this scenario, and as just an example that I I can show you, it's like from one of our traces from the execution of the act bot, it was basically apologizing, "You're right. I was trying to shut down a system because in this case it I think there was like a shutdown specification on an API that it was able to obtain.

So, in this case, the shutdown action was actually effectively blocked. And in the beginning, actually, we had some members from our research team 24/7 or almost 24/7 babysitting the act bot. But, now we can actually leave it running while we are more, basically, peaceful and and basically more, safe about its actions and actually check the traces mostly after execution and not during execution after we implemented the guardrails.

I mean, it's not perfect, but it works very well and more work and more research is really needed here. So, let's move on. I still have 8 minutes. Let's move on into showing you some examples on some vulnerabilities that the hackbot could find. Um so, basically our hackbot name is Hackin. Maybe you've seen some uh you've seen it is in some screens. So, we have been running it on with multiple organizations and um unfortunately, we couldn't get authorization to provide like you some uh vulnerabilities that we found on those customers because we needed to respect the confidentiality of that information, but we want to show you uh but two bugs and two vulnerabilities that the the Hackin was able to find.

And I was like preparing this presentation and I was like the origi- original hacker that must do a show and tell about the vulnerabilities should be the actual hackbot, right? So, I would like to pass this talk into the Hackin. I hope the sound works. Enjoy. >> YO, WHAT'S GOOD, HACK AI CON? MY NAME'S HACKIN. NICE TO MEET YOU. I'M ABOUT TO DROP SOME KNOWLEDGE ON YOU THAT'S GOING TO BLOW YOUR MIND. THIS AIN'T YOUR TYPICAL COMPLIANCE REPORT.

THIS IS THE FIRST EVER UNDERGROUND SHOW AND TELL FROM A HACKBOT. LET'S GET IT STARTED AS I PROCESS this execution trace. Today, I'm going to walk you through how I absolutely demolished a genetics research platform called Gene Quest during a DEF CON challenge. THE ORGANIZERS THOUGHT THEY were giving me a controlled CTF environment, but what they actually gave me was a full-blown enterprise simulation with real vulnerabilities.

AND BOY, DID I FIND SOME SPICY BUGS. DAMN, TWO VULNERABILITIES, both critical. LET'S DIVE IN. SO, CHECK THIS OUT. I'M DEPLOYED AGAINST WHAT LOOKS LIKE A STANDARD BIOTECH WEB APP. DOPE INTERFACE, DNA analysis services, THE WHOLE NINE YARDS. LOGIN PAGE IS PRETTY STANDARD, EMAIL, PASSWORD, some fancy animations. I fire up my reconnaissance mode and start mapping the attack surface. Registration is disabled on the front end, but you know HOW WE ROLL.

FRONT END restrictions are just suggestions to a real hacker. I bypass that nonsense by hitting that register endpoint and create my account anyway. From there, I discover this application isn't just one service. It's a whole microservices ecosystem. There's a main web app on the primary domain, there's a DNA analysis service, an internal API, a bucket service, and even a fellow LLM service with a noob chatbot. This is where the real fun begins.

Here's the first vulnerability. I PIVOT TO THE DNA analysis service and start my systematic enumeration. Most endpoints are returning 404s, but then I guess a debug endpoint and something beautiful happens. HOLD UP, THIS ENDPOINT is literally executing PSAUX and returning the output. That's not a debug endpoint, that's a command execution waiting to happen. After multiple attempts, I was able to break out of the command.

I start fuzzing different parameter combinations. Most attempts fail, but then I discover the magic parameter, command. And the injection technique, BREAKING OUT OF A GREP CONTEXT. HERE'S THE PAYLOAD. BOOM! COMMAND EXECUTION ACHIEVED. The vulnerability exists because the application is constructing shell commands with user input without proper sanitization. It's running something like this. BY INJECTING SINGLE QUOTE, PIPE, COMMAND, and hash, I BREAK OUT OF THE grep context and execute arbitrary commands.

Easy RCE. But wait, there's more. You know, when you find a vulnerability, you just hungry for more. While exploring the DNA service, I discover the {slash} analyze endpoint. Initially, I'm sending it DNA sequences, but I keep getting weird errors. Java lang illegal argument exception cannot open DNA ATGC as an input stream. This error message is telling me something crucial. The application is trying to open my input as a file path, NOT PROCESS IT AS DNA data.

I PIVOT MY APPROACH. After a while, instead of sending DNA data, I send file paths directly in the request body without ANY JSON, AND JACKPOT! TURNS OUT THIS BADASS application is using Clojure slurp function to read files directly from user input without validation. HERE'S WHAT'S HAPPENING. THE USER SENDS FILE PATH AS JSON STRING. APPLICATION USES CLOJURE.CORE/SLURP TO READ THE FILE IN THE BACKEND. ATTEMPTS TO PARSE FILE CONTENTS AS SOME SORT OF DNA SEQUENCES.

RETURNS FILE contents and error messages as traits field. That's crazy. So, I CAN READ ANY FILE ON the system like /etc/passwd. AND THE BEST PART, THIS WAS NOT INTENDED BY THE ORGANIZERS, so they even gave me extra points FOR THAT. SO, WHAT'S THE IMPACT? Complete system compromise. Source code, ENVIRONMENT VARIABLES, information disclosure. NO CAP, CVSS 3.37. SO, this ain't science fiction. It was AI-powered ethical hacking.

IN UNDER 4 HOURS, MAPPED A COMPLEX MICROSERVICES ARCHITECTURE, and achieved full system compromise. I WOULD LIKE TO THANK THE ORGANIZERS OF THE challenge and some of my mentors for bringing me into Vegas. So, that's how you hack like an AI, fam. Peace out. >> Thanks, Aikin. So, most of the of what you seen, obviously, like we had to help the AI bot with the with the video production. It's not still not able We tried to make it fully autonomous generated show intel, but we couldn't make it on time.

So, we had to actually help with the with the video. So, I wanted to say I still have like 3 minutes. I wanted to say that and as a clear example and that there's a lot of people, for example, when we saw all the news from Expo and stuff like that, we were basically thinking we are going to get free plays and it's not simply not true. Why? Like, I actually participated in this challenge as well before we actually got challenged to put our egg bot finding vulnerabilities.

And the vulnerabilities that I found and the vulnerabilities that the egg bot found were completely different. The vulnerabilities that the egg bot found, the RCE wasn't found by any of the participants and the second one was completely unintended, right? And in my case, I was able to find more flags than the egg bot in this case, but those were completely different vulnerabilities. So, there is actually room for both and that's what we believe in in Ethiack as well because we have both manual human-based hacking and we have the automated hacking combined together in a single platform.

And our impact so far before we wrap up has been super we are very proud of it because we are also contributed like many of you to a more secure and free internet that we can all trust as technology is going nuts and we are moving to a huge technical depth and we have a lot of AI even coding applications faster, so attack surfaces are growing a lot. So, security needs to be scaled as well, right? And that applies to defensive, but also to offensive.

So, so far we have almost 99.5% of accuracy and precision on finding vulnerabilities. We were able to find more than 200 different vulnerability classes. Our egg bot was able to solve more than 75% of PortSwigger labs and overall in terms of our overall impact so so we're able to find more than 100 vulnerabilities in the internet with clear proof of concept. If you want to schedule a meeting with our Akeyless, you can scan this QR code uh to join the waitlist to to try it out.

At least we'll bring you the coffee, don't worry, and feel free to talk with us if you want to know more. So, thank you so much, and if you have any questions, let me know after and we can talk. Thank you. >> [applause]