Xiaomi and DeepSeek Just Solved the Biggest Problem in AI transcript

Your agent runs NPM test. Eight characters, back comes 20,000 tokens of logs and stack traces. It typed eight characters and got back a novella. And before it can type the next command, the model has to read every one of those tokens. Then it runs the next test, and the whole thing happens again. Xiaomi's Fully Lu describes exactly this loop in her announcement. Each round, a short action returns a long observation that has to be pre-filled, and the context keeps growing.

So, three things squeeze you at once: the compute to read the new chunk, the memory everything, and whether the model can still find the one line that matters. Her team's answer is Hypar sparse 2, and she says it's the core of Mimo V3's new architecture. At a million tokens, it shrinks the model's memory of the context from 12.09 GB to 2.69, and cuts pre-fill compute about five times. And retrieval, which you think would get worse, actually improved on their 256K test.

That's measured against Xiaomi's own previous design, and it's still a big jump. It also has a strange twin. 12 days before this paper landed on archive, DeepSeek shipped V4, OneFlash with the same two tricks inside. By the end, you'll know why that happened, and why copied is the wrong word for it. First though, you need to see what's actually being skipped. Picture the model as a tall building. Each floor is one layer, and Xiaomi's test model has 49 of them.

Every token, every word fragment of those logs, walks in the front door and gets read on every floor on its way up. Each floor keeps a filing cabinet and files a card for every token it reads. The card's label is the key, roughly what this token's about. Its content are the value, what the token contributes when it's relevant. When the model writes its next token, each floor checks its question against the labels and pulls what matches.

That cabinet's the KV cache. It saves the model from re-reading the whole conversation every turn, but it grows by one card per token on every floor that keeps one. In Xiaomi's previous design, a million tokens came to 12.09 GB. Prefill is the filing itself. When those 20,000 tokens of logs arrive, every floor has to read them and file its cards before the model can say anything. In an agent loop, you pay that bill every round.

That previous design is hybrid SWA, short for sliding window attention from the Mimo B2 series. Nine of its floors do full attention and keep a card for everything. The rest mostly watch a sliding window of recent tokens. It's the yardstick for Xiaomi's numbers along with the original HiSparse. Trick one is KV bridging. Split the building partway up, roughly in the middle. The lower floors read every token the normal way.

The top floor of that lower section becomes a handoff floor and it leaves behind a set of notes, its understanding of every token so far. Up top, the full attention floors still keep cabinets, but they don't read the tokens themselves. The handoff notes go up a chute and each of those floors fills its cabinets straight from them. So, once the lower section finishes, prefill's done. It gets to go home at the handoff floor.

The paper calls the lower section the self-decoder and the upper one the cross-decoder and it puts the idea plainly. Prefill can exit after the self-decoder. That layout comes from a 2024 paper called Yoko, short for you only cache once. You do give something up. The upper floors never form their own view of each token during prefill. Their picture of the past is whatever the handoff floor wrote down. Xiaomi's betting that's good enough and the retrieval test will show how that bet went.

Trick two, KV reuse, works inside each block of floors. A block starts with one full attention floor that reads everything and keeps a proper cabinet. Above it sit a few sparse floors, which only look at a small selection of tokens. While the full floor reads, it works out which tokens matter most and writes them on a short list. The sparse floors above don't build cabinets at all. They walk downstairs, borrow the full floor's cabinet, and read only the cards on that list.

The sparse floors are the flatmate who never buys groceries, but only eats what's on the list. The trick works because the full floor already did the hard part, deciding what's important. Borrowing the list saves the choosing, and borrowing the cabinet saves the memory. Reuse itself isn't new. Xiaomi published it in February as the original HighSparse. Sparse floors borrow the list and the cabinet from the full floor below.

HighSparse 2 keeps that and makes two smaller changes. Change one is how cards get picked. Picture the cabinet holding binders. The earlier approach pulled out whole binders. HighSparse 2 picks individual cards, which the paper calls token-level selection. If the line you need is buried in a binder of noise, you get the line without hauling the noise. Change two is recent context. A sparse floor always needs the last few things that happened, and that used to come from a separate sliding window branch with its own storage.

Now it's a fixed shelf in the same borrowed cabinet. The most recent one, 28 tokens, always kept next to 1,024 picked ones. So, a sparse floor facing a million token context reads 1,152 cards, about a tenth of a percent of the cabinet. It gets away with that because the floor below already read everything and told it where to look. Now, count cabinets. Only five of the 49 floors do full attention, down from nine in Hybrid SWA.

Sparse floors borrow instead of building, and the whole upper section skips reading during prefill. Fewer cabinets, less filing. At a million tokens, that buys you this. The KV cache is 2.69 GB. The original HighSparse needed 6.72. Hybrid SWA needed 12.09. That's roughly four and a half times smaller than Xiaomi's previous design. Prefill compute drops 5.02 times against Hybrid SWA and 2.92 times against the original HighSparse, which already had reuse.

These are computed flops, a count of arithmetic operations on an 80 billion parameter research model with 3 billion active. They aren't stopwatch timings, and they aren't a comparison with anyone else's model. Does all that borrowing make the model worse at finding things? You'd expect it to. On Ruler Daji 2, a long context retrieval test at 256k tokens and after a light post training stage, high spars 2 scored 58.45.

The original high spars scored 32.61 and hybrid SWA 35.74. So, on this test, the borrowing didn't cost it the one line that matters. One catch. The cash and compute numbers are at a million tokens. The retrieval table stops at 256k, a quarter of that. It's the long context version of the test passing on a smaller fixture. Hold on to that gap. Now, leave Xiaomi's building and rewind to the 10th of September. DeepSeek released V4, one flash, a 552 billion parameter mixture of experts model.

So, only a slice of it runs for any given token. It has a million token context and it activates 8 billion parameters while processing the prompt and 16 billion while generating. Half the active parameters for reading the prompt. It's what you'd expect if prefill stops halfway up the building. DeepSeek calls this the causal encoder-decoder. Its upper layers don't build their keys and values themselves. They're projected straight from the middle layers notes, what the paper calls its hidden state.

Move for move, it's KV bridging. A handoff floor in the middle and upper cabinets filled from the notes it sends up the chute. The second trick is in CSA2, DeepSeek's sparse attention. Its reuse mode lets layers borrow the main cabinet and the latest top K picks, the short list of most relevant tokens from the layers below. Same staircase, same borrowed list. Xiaomi calls the first trick KV bridging. DeepSeek calls it a causal encoder-decoder.

And Yoko called it a decoder-decoder. That's three names for one shoot. Turns out the caching was the easy part and the dates are close. DeepSeek released on the 10th of September and Xiaomi's paper went up on archive on the 22nd of September. Did someone copy? No. And the citation show why. DeepSeek's report says its encoder decoder is inspired by Yolo, a Microsoft and Tsinghua paper from May 2024. Yolo's pitch was this exact move.

A self decoder builds the cache once. A cross decoder reuses it. And prefill ends early. The reuse half has an even clearer trail. DeepSeek's report credits HighSparse by name for letting sparse layers reuse the KV cache of dense layers. It's Xiaomi's own February paper published 7 months before either September release. The fair version then. Same idea, shared ancestors. Both labs built on Yolo's bridge. For the reuse half, DeepSeek credited Xiaomi's February paper in writing.

Then each put out its own version of the pair 12 days apart. And the trail runs both ways. Xiaomi's new paper sites DeepSeek's report too for a different trick. Could you say whose version is better? Not from these papers. DeepSeek reports 890 bytes of cache per token. A figure that already counts squeezing numbers down to 4 bits called FP4. Xiaomi reports total gigabytes at a million tokens on a different model. Two rulers, two buildings, no fair side by side.

Did Xiaomi kill it? Against its own last design, yes. About 4 and 1/2 times less cache. Five times less prefill compute and better retrieval at 256K. If you run or fine-tune models for agents that pour long tool output back in, this is the family to watch. When two labs land on the same pair of tricks and cite each other's work along the way, that's a decent signal. If you just call an API, there's nothing to change yet.

Mimo V3 weights haven't shipped and nothing confirms the production model matches this ADB test setup. When the weights do land, the question I'll be checking is simple. Does retrieval hold up at a full million tokens?