DeepSeek’s Insane New Architecture transcript

The new DeepSeek is here. 4.1 flash and it is wow. Look at that. It is incredibly fast. And I was also stunned by this. Look, it can outperform Claude Opus 5 Kimi K3 on some tests. Not everything. We'll see about that. But there is a catch. I'll show you at the end. It also reliably outperforms DeepSeek 4.0 Pro, which is a huge surprise. I mean, that system costs maybe 300k dollars to run locally and this one can be run for a quarter of that.

Well, that's still a lot of money, but the tendency is undeniable. This might run in our pocket in a year's time. It also has native visual understanding. So, finally, you can give it images of an iconic game menu and have it write a game that reproduces it. I'll tell you more about my other experiments at the end. And this is huge and small at the same time. How? Well, now hold on to your papers, fellow scholars. It's small because of the KV cache.

This contains the context and is 437 times smaller than V1 was 3 years ago. But also four times smaller than the previous 4.0 flash. But that was just a few months ago. And you compressed it 4x. That is what? How did they do that? Do we know? Yes, we do. Dear fellow scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér. Why? Because this is an open model and there is a free research paper explaining [music] it.

I am not an expert, just a student, but I'll try to break it down. They call this technique CSA2 and they are finally onto something here. You see, a neural network is given by a lot of layers of neurons, and as information is propagated through these layers, each of them has its own KV memory. But, not here. DeepSeek 4.1 Flash finally gives us shared memory between the layers. Not everyone has to remember everything.

No. They share the memories. [clears throat] It's a nightmare to pull off well, because different layers need different views of the same history. And if we pop the hood and look closer, hohoho, look, an encoder-decoder structure. The encoder creates a shared global memory, and the decoder reads from it. And all that is what finally gives us a much smaller KV cache, which is the bane of my existence, because it needs too much video RAM.

Incredible leap forward. Now, I also said this is huge, more than 500 billion parameters. So, I have no chance to run this at home whatsoever. And when I tried to reproduce this beautiful honeycombing paper, GPT-6 Astra nailed it. Absolutely stunning. Ooh. Opus 5.1, not as stunning, but the physics is quite formidable. Rendering a bit less so. Now, this one. Well, we still need to get a couple more papers down the line to match that, but it will happen.

Of that, I am certain. And you see, I am here to show you the truth, not just believe the headlines. Now, also, since YouTube stopped recommending my physics simulation papers, which breaks my heart, at least I get to show you this, which fills me with joy. Subscribe and hit the bell to see more. Also, the catch, 4.1 flash likes to think a lot and burns a lot of tokens. Fortunately, it is not that expensive, but man, that's a lot of tokens.

But, if you have the hardware, you get to run all this for free. I don't, so I run it through the API or lambda, but given that every time Deep Seek publishes a new paper, AI gets cheaper for all of us. So, thank you so much for that. Huge contribution to humanity. This is going to help a lot of doctors and scientists do their work. Open science for the win. What a time to be alive. I use lambda to reproduce AI research papers often in minutes.

It's also great to train your own models or fine-tune an existing one. Run inference or text to image or video, easy peasy. Running a Deep Seek chatbot or agent, super fast, super reliable. Lambda gives you powerful Nvidia GPUs to run your own experiments. I test ideas from the papers I cover, and moments later, results. Love it. Seriously, try it out now at lambda.ai/papers.