Advanced Thermal Management for High-Density AI Servers: Cold Plate Optimization and CDU Specificati transcript

Hello. Okay. Uh hello everyone. Uh my name is Folin. I'm from ABC. So uh it's uh been great honor to be here to share our knowledge in uh AI liquid cooling. So today I going uh I will bring two topics about uh first one is the uh co-play optimization because we know uh since 20 20 uh 24 uh 24 is lot of change uh since that yeah and I also want to talk about the Titan uh CDU specification requirement. Okay. So here's uh our my my outline.

So there's a five topics. So uh first first of all I will talk about the surging uh thermal power from the XPU or you can call SI. So basically the package size going to be bigger to big and bigger and the TDP will be over uh 4 kilowatt even 6 kilowatt in the near future. But the the most important one is the hotspot. The hotspot inside the the asset uh will be uh over six or even to seven watt per minimum square in the very near future.

So that is a big challenge for for liquid cooling especially for the co-play and then I'll talk about the uh the co-play evolution because uh currently we we need use the 45 uh TCS uh secondary uh side water. So that would be the uh there's no lot of merging between the junction to the 45° C but the the power is keep on going up and up. So we need to use very high density microch plate to to solve the the junction temperature.

And then uh I'll talk about the the cooling cleanness. uh currently uh we we need use the 25 micron filtration and some uh leading CSP layer using 02 micron as a side train side stream filtration system. Yeah. Yeah. And for the CDU wise uh currently we we have the four approach temperature difference uh CDU that's there's a like a new standard and finally I would talk about uh if you want to have the a very good uh AI coding system you need to consider about the whole system you need to do the vertical integration from your core play to your CDU to your uh L11 to L12 the all all thing you Think about it.

Yeah. Yeah. So the first one is uh the left side is the traditional uh chip. So it's like uh the heat loss inside this chip. The pardons is quite low. It's around two watt per minimum square. So in that case we can use a traditional coplay like a a side in side out is a a uniform cooling. That's good. That's that's a doable. Yeah, but for the next generation or even for a future generation uh package uh the package size will up to nine times or 14 time even other uh package company can do 24 retical size.

So you can think about the the package size will be huge around uh more more than 100 mm by 100 even even bigger. So the power will be pretty high uh would be uh 4 kilowatt or even 6 kilowatt. So uh but the the most critical thing is the uh hotspot inside the the XPU or you can call uh S or yeah GPU or XPU. Yeah, there will there will be uh multiple hotspot in the the retical die. So uh it is very hard to solve this husba and when you consider about the HBM the highstake HBM the the traditional cop play cannot cannot handle that the power uh especially when you think about the the inet temperature is is locked at 45° C so there's not no another margin so uh the strategies, you need to consider the the precision cooling at the lo localized hotspot.

But when you want to first first uh for the first thing is you want to solve the hotspot and the the high power maybe you don't want to increase the fin area or narrow the fin gap because when you think about that you you will get a very high heat transfer area but uh when you reduce the fin gap you'll face another hydraulic dilemma which is a very high pressure drop inside your loop. or inside your core play. Uh the left uh the left one is a uh is a rough estimation for the uh thermal performance and pressure drop. when you reduce the uh fin pitch down to uh one 100 or even uh se 7 75 micron uh you can do you could get the pretty high uh I would say pretty uh a little bit higher uh thermopos but you get lot of pressure drop so uh for microch the middle one middle uh picture.

We uh we developed some uh calculation tool. We can optimize the the fin gap fin high because this uh fin aspiratial we were think about the hydraulic diameter of the fin. Uh and when you when we uh design we were first first one we were consider about the the flow rate and uh the hydraulic diameter. So we will choose the the right uh fin gap which is micro channel gap and uh let me use here sorry excuse me press long Yes.

Oh, it's okay. Okay. Uh so uh this one we just showed uh we were design for example we were narrow down from the the uh channel width from uh 150 to 80. So we can see the the the trend of the pressure drop and then we uh got a a maximum point to like a B point and then we'll think about to increase the fin height and this calculation also we were uh compare with the the right right hand side uh chart. So we will compare with them so we can get the best uh optimized heat transfer coefficient during our uh calculation.

Yeah. And this just the first step to optimize your uh fin structure and fin geometry. It's a single channel but but this one cannot uh 100% solve the the current situation. So we need to introduce another concept and which is the multiple uh I will call multiple manifold or the uh multiszone core plate design. So this concept is to bring the uh the core coolant direct to the hotspot or near to the hotspot. So we can separate the several uh coolant to the different uh uh like ESIC as different area of ASIC on HPN and that one we can uh dramatically uh uh reduce the pressure drop.

Yeah. And also we can uh solve the hotspot issue. But uh the I I want to mention one thing uh if you want to use this this design you need to think about uh what is the developing uh length of your channel because you have multiple uh multiple inlet multiple outlets. So when you design the the the inlet and outlet you need to you need to calculate the what is the uh the flow region from your end in and out. So we need to calculate uh the the flow region is during the developing or or going to the fully development that is very important part.

Yeah. And beyond that uh we also need to introduce some high performance team 1.5 one team uh team one and that is pretty important part for so for the the co-play. So when the uh fin pitch from down from 150 to 100 micron even some some design we need to use 80 micron and then we need to think about the uh the cooling cleanness and the filtration issue. uh the traditional one is uh like CDU we were we suggest use 50 micron uh filter inside the uh TCS uh TCS size that's okay but uh currently it's not enough because uh our the co plate will be the fin gap will be only 100 micron so it to prevent any uh blockage or any uh contaminant we need to think about the 25 micron filter.

So uh and some uh leading CSP like Google they are using uh 2 micron size stream loop for their uh their latest uh dish CDU. Yeah. So clean is not the longer the facility preference. So it's a pretty strict u uh mechanical requirement for current AI server and uh so as uh for the 45° C tcs water temperature. So this is quite a tight. So when when think about uh if the the CDU cannot have the uh good uh approach temperature they they take one degree.

So uh our co-play your co-play or cooling full cooling loop design they have to take another uh one degree from from your co-play. So if your CDU is not good is not good enough. So your your your co-play design will be pretty tough. So currently we have designed our uh liquid to air CDU and liquid to uh liquid CDU. They are uh follow the uh four approach temperature rule. So we can align the the warm water 45° C trend and they are currently verified and as as a reference vendor.

So we can provide uh from uh seven 70 kilowatt to 1.2 megawatt CDU. Okay. Okay. We can also uh vertically integrate our CDU because we have our uh in-house uh uh parts like a fan and pump and also we can inhouse manufacturing our uh stainless piping because we uh we know the the cleanness issue is not just happening in a clay uh they're happening in the whole system. So uh the piping is very critical. We need to consider about the the surface roughness inside this uh stainless piping.

Uh not just the uh rack manifold. We we need to consider about uh the CDU. Yeah. Okay. So here's two two uh call for action items. So currently we don't have the rigorous or comprehensive for the cleanest. So it's very uh quite tough for for uh like ABC as a co-player or cooling uh component provide uh provider. So I think we should have the more good definition for the whole uh cleanness guideline in in the the TCS side.

That would be pretty important I think. And another one is the even we uh we know how to optimize our uh co-play but when the uh power low is higher and higher uh but I think that the co-play size should be uh they were not not not bigger they are limited in some some range. So uh the fro would be pretty huge. So I think the the another one is to we should have more corrosion uh studied especially for the high flow velocity in the like a manifold side or or adapter side.

I think that's quite urgent. Okay. Thank you. Yes. Yes. You mentioned I want to put a high flow rate now. Yeah. about the solution for the co uh for the coplay I I would not worry about the fin the fin parts but I worry about the like adapter because there will be like l uh n 90 degree angle or something or bend yeah so currently maybe uh first one is like use like a stainless part or at least some uh with some surface treatment like a nickel plating or something.

Yeah. Uh because the cor I would say is the corrosion with the erosion issue that they are combined. Yeah. So erosion is talk about the the velocity and the the cleanness of the the coolant. So I think it's the combined issue. Yeah. So we also we need to think about how to control the the cleanness needs in the whole TCS uh coolant. Yeah, that's a great question. Thank you. Okay, so which study? Sorry. a channel. Uh we we can uh both both uh they can use both uh P25 and water.

Currently we don't use any like a surve to to change the the the weightability. Yeah, we we're not Yeah, we just talk about the the thing structure. Yeah. Okay, times up. Thank you. Thank you, Vanjong. And