This LPU is 2000% Faster Than a GPU! transcript

When a startup built a chip that can run AI 20 times faster than Nvidia's best GPUs, that too while using 10 times less power doing it, Nvidia didn't try to beat it. Instead, they paid $20 to make it disappear. And I'm not even joking. Nvidia controls roughly 90% of the AI chip market. Governments are stockpiling their hardware, and every major tech company is spending billions just to get enough of them. But in December 2025, Nvidia acquired Groq in a deal specifically structured to avoid antitrust regulators.

They silently absorbed their engineers, the patents, and the entire architecture, then folded it into their next-generation hardware. That tells you exactly how dangerous this thing was for Nvidia. So, what did Groq actually build? And if this architecture is truly 20 times faster, why hasn't it already replaced the GPU? This is an LPU, and the reason it demolishes a GPU at inference comes down to a fundamental physical flaw inside every GPU that ever existed.

You see, a GPU was never built for AI. It was a graphics accelerator. It was built for gaming. And when you use it to run a large language model, you immediately run into a problem. Every single time the GPU generates one token, it has to go fetch the entire model's weights from VRAM, drag them to the cores, do the math, and then send the result back. It does this for every single token. At 40 tokens per second, that's 40 round trips to memory every second.

The actual math is almost instant. The chip is literally just waiting most of the time. But LPU's entire architecture is built around one idea: what if the model never had to leave the chip at all? And to make that happen, they had to abandon traditional GPU memory completely. Modern GPUs store massive amounts of data using DRAM. To hold a single bit of information, DRAM uses just one tiny transistor and a microscopic capacitor.

Because it's so small, Nvidia can cram 80 GB of it onto a single GPU. But there are two massive catches. First, capacitors leak electricity and have to be constantly refreshed. Second, because this memory bank is so incredibly huge, it cannot fit inside the compute core. It sits outside, centimeters away from the core. A few centimeters sounds tiny, but when a chip is doing billions of calculations per second, those centimeters turn into hundreds of miles.

Groq bypassed this entirely by using SRAM. SRAM doesn't use leaky capacitors. It uses a locked circuit built out of six transistors. And more importantly, the memory isn't sitting on a separate chip across the board. The SRAM is baked directly into the silicon die itself. The memory and the math cores are literally millimeters apart. The data has no commute. It's instantly available the exact millisecond the core asks for it.

That is how the LPU is able to hit 80 terabytes per second and obliterates the GPU in raw speed. But here's where it breaks. When every single bit of memory requires six transistors instead of one, your memory explodes in size. SRAM is incredibly fast, but it is physically massive on the silicon die. Now, you might be thinking, "Hmm, isn't that an over-glorified L1 cache? What a big deal?" But it is indeed a massive deal.

There is a reason why you don't get 200 megabytes worth of L1 cache on your GPU. You don't even get 200 kilobytes of it, actually. Printing this many six-transistor circuits onto one continuous die is not a joke. It pushes the absolute radical limit of semiconductor manufacturing. But even an engineering miracle can't break the laws of physics. Because those SRAM blocks are so physically large, you simply cannot fit very much of it on a single chip.

While an Nvidia GPU holds an easy 80 GB, a single LPU can still only hold 230 MB, at least for now. And unlike a GPU, it cannot train a model. It can only run it. That sounds like a death sentence until you see what happens when you start stacking them. But first, how did Groq manage to print 230 MB of it when nobody has dedicated this much silicon to pure on-die SRAM as a memory replacement? Simple. They deleted the rest of the chip.

Nvidia uses up their silicon space by building complex physical hardware to direct traffic and schedule data. Groq realized AI math is entirely predictable, so they ripped out the hardware schedulers, leaving massive amounts of empty space on the silicon. They filled all that empty space with SRAM, but by deleting the physical traffic routers, Groq had to move all that traffic control into the software, and that software is the actual reason Groq is so incredibly terrifying to Nvidia.

But even though it's a big deal to have this much SRAM on it, it's still nowhere close to running an actual state-of-the-art model. Real models are hundreds of gigabytes. So, how is that even competing against GPUs if it can't even run some of the smallest AI models? And that's why the chips are built from ground up to be clustered with hundreds of them together. And here's where every other chip manufacturer fails. The moment you chain that many chips together, they spend more time waiting on each other than actually doing what they're supposed to.

But Groq's compiler is perfectly deterministic. It knows everything before it even happens. Before the code ever physically touches the silicon, Groq's software maps out the exact path of every single electron. It pre-calculates the entire journey down to the exact nanosecond. The chips never have to ask each other for data, and they never wait. They just blindly execute their math, and the data arrives perfectly on time.

The compiler forces 256 independent chips to act as one flawlessly synchronized brain. And the real reason why Nvidia paid $20 isn't actually about the raw chips. Their GPUs are already the undisputed kings of raw compute. Nvidia's actual nightmare is the network. When a company like OpenAI builds a data center, they are never using just one GPU. They are wiring together tens of thousands of them. And right now, getting 10,000 GPUs to talk to each other without causing massive network bottlenecks is one of the hardest engineering problem on Earth.

Groq solved multi-chip scaling gracefully. The more chips you stack together, the better it gets. They bought Groq to rip out their deterministic networking technology, fold that software into their next generation of GPUs, and permanently neutralize the only company that figured out how to beat them at scale. If you liked this video, you will definitely like this one where the chip literally just stops using electricity to compute.

See you there.