China, Open Source & AI Competitiveness I Andrew Ng transcript

If an adversary of the United States wanted to slow us down, I think they couldn't wish for almost anything better than these silly moratoriums on building our data centers. I've been alarmed at the amount of lobbying that a handful of businesses have been doing, saying that open models are dangerous. I think that's false. I think the concept that distillation is a [music] major factor has been overstated. Sometimes the most powerful forces in the world go unseen.

A critical layer of intelligence driving [music] possibilities. This is Micron Memory and Storage. Intelligence that uncovers new cures, models changing climates, and build smarter, [music] safer cities. Every breakthrough in AI is driven by data. And data lives in Micron Memory and Storage. Hello everyone. I'm James H. Homeman, deputy opinion editor at the Washington Post. Welcome to the Post's Building America podcast about how America builds, manufactures, and innovates better for the future.

Today, we're speaking with Andrew about how investing in AI infrastructure will shape America's long-term competitiveness. Andrew is a true pioneer in AI. [music] He led the Google Brain Team, which launched much of the tech giants AI work. He directed the Stanford AI lab and has authored over 200 papers on AI and related issues. He has since co-founded deep learning.ai, which provides technical training. And he's managing general partner of a venture fund.

He is also on the board of directors of Amazon whose executive chairman Jeff Bezos owns the post. Andrew, welcome. >> Thank you, James. It's always good to be with the post. >> Well, I for those who don't know your incredible story. Take us back to the start. Uh you conducted research on neural networks that led to the formation of Google Brain. What was it that you pitched to Google founder Larry Page back in 2010? At that time I had this controversial idea that if we build really big neuronet networks and feed a lot of data to them that they will learn incredible things.

Um and so when I started the Google brain team the number one direction I gave the team was let's build really really big AI systems and feed a lot of data and with that oversimplified direction we wound up um having some incredible accomplishments. I think that culture of the Google brain team led it to later indent the transformer neuronet network which is really the one key technology that the Google brain team openly published and that open eye adopted many others adopted and sparked off the modern AI revolution.

It was really driven by this one idea that I saw in my very early data at Stanford that the bigger you build AI systems the smarter they get and that idea is still carrying us forward today. I mean, fast forwarding to 2026, are you surprised by the speed in which we've gotten to this place, or did you kind of expect that this would be where we'd end up? Obviously, a lot of this was theoretical uh and now it's the biggest thing in the world by far. >> Even back then, I was very optimistic about where this would go and I remain very optimistic about where we can get in the future.

But, uh, exactly how far we are and how far we could go. Some days I'm excited, some days I wonder why we aren't even further along. And I think what we're seeing is that um as amazing as AI technology is, and it is amazing, businesses do not feel like it's helping them yet. And one of the challenges is no company ever gain competitive advantage just by buying a CH GP or Microsoft co-pilot license. The problem is finding the right use cases and um that's a people change management having the right people understand the technology marry it to the actual use cases and of course supported also by the infrastructure that you talked about. >> Yeah, I saw that you've just uh unveiled open worker an open-source desktop agent that produces finished work rather than just a conversation.

Is that part of the the answer to what you're talking about? >> Uh yeah. So uh my my my friend um Rahab Prasad and I have been working on a open version of uh AI that you don't just chat with um but that actually does work for you. So a lot of people still will use chat or claw or Gemini or tools like that to chat and get back an answer. But today AI could go further instead of just chatting with me. it can produce a finished polished document rather than just give me things to copy paste or it can actually send an email for you or send a you know short message uh in in in Slack or messaging system for you um or build a dashboard for you and so open worker is a a fully open source you know free uh uh piece of software to let people do that on their computer and I'm actually excited about uh uh tools like um uh clause co-work and uh chat GPT work uh and uh Gemini anti-gravity which are all good tools and open worker is a free open source version that anyone can use and there's a lot of talk right now in DC certainly about open versus closed open weight models closed uh and you've done a lot on the the open side you see a lot of the the frontier models developing these closed models where you know people have to pay for tokens that's become sort of you're you're seeing companies push back on that as the the costs and the bills start to run up.

How do you think about the open versus closed debate, if you can call it that? >> Yeah. So, to sustain competitive advantage in America, one of the most important things we have to do is um support and sustain open models. So just to drown to terminology um uh by the way I think I'm the only person that both Sam and Dario has worked for and so actually you know support uh open anthropic I I really hope they do well and have fantastic IPOs and so on.

So I really hope these great American businesses will succeed. At the same time the tension the success of these businesses cannot be at the cost of shutting down everyone else's access to open models. So, OpenAI, Anthropic, uh, Google, um, have done a great job training closed proprietary models that they sell access to, uh, for, you know, decently high price. Uh, [sighs and gasps] open models are models that people have published on the internet, free for anyone to use.

And those are extremely performant, very intelligent, and often much cheaper. And so, I've been alarmed at the amount of lobbying that a handful of businesses have been doing. um saying that open models are dangerous. I think that's false. Uh or saying that open models are uh uh you know kind of somehow not as good as the closed models which they just don't get. Uh open models are a form of um gives businesses more choice.

It means you're not locked in to one closed proprietary provider. In fact, one piece of advice I often give to business leaders is I can't forecast in a year or even six months what is going to be the top model. So as we build our businesses, one of the most important things to do is to preserve optionality. So I'm not locked in. I I use openAI, I use cloud, I use Gemini, I use uh you know a lot of different models but I don't let myself be locked into any one of them because a year from now we want to use the best one whether it's the open model that we cheaper or closed proprietary model so preserving optionality is very important and the world supply chain is using open models so making sure that America continues have access to that will be important for us as well.

Yeah, I mean part of the lobbying campaign that you are referring to, they're trying to sort of they're basically trying to portray open models as as one and the same as Chinese models and then they're making kind of this national security argument, but it it doesn't necessarily need to be or stay that way. Uh and I mean I also wonder about the tension in the Chinese system of having open models and whether the government there will continue to allow that over the long term. >> When Chachi P was first released a few years ago, America was decisively ahead in of China in uh generative AI technology.

[sighs] Since then, China has played its hand really well. One of the things that China did really well was embrace open models because it turns out that when you release models freely for anyone to use, it helps the whole world. Yes, but it helps you even more than it helps the whole world. So the openness, freedom of communication, fast diffusion of knowledge has meant China has rapidly accelerated and is approaching maybe not quite at par but approaching par with um the leading Chinese models.

In contrast, because uh a lot of American work has gone into closed proprietary models, it's just much harder to know if you have a question, who should I call up to understand how to do this modeling thing. >> Um and maybe one company recruit another engineer for, you know, a very high price tag and then there's a little bit of diffusion of knowledge. But I think this has um uh the the the lobbying against open models has hampered American AI development.

We do have some great open American models. I think um Nvidia's Neotron models are strong models. Uh the uh thinking machines Inkling lab feels like a very strong showing. I I'd love to see more of this. I I think that there's room for proprietary models and open models to succeed. >> And the important thing is to maintain a level playing field so that any model, you know, so that so that people have choice to use the best model.

How much do you think that the gains that the open Chinese models have made are the result of distillation versus their own progression and the kind of the democratized nature of an open model. >> So it is clear that we have great technical innovations in America and China has great technical innovations. In fact um many of the US frontier labs are actively reading a lot of the open research that the Chinese lab published.

I mean you have to be dumb not to. So America labs are definitely benefiting from tons of Chinese research lab innovations. I think the concept that distillation is a major factor has been overstated, vastly overstated. Um you know there's a question of what the world should consider fair. AI labs all around the world took data off the open internet and use it to distill knowledge from the internet into their AI models.

Is it fair for them to turn around and say, "I've distilled the internet into my model. If anyone distills my model from here on out, that's not fair." Um, I think that's an open question of what society should consider fair or not. Um, the the the some of the claims that uh uh some I think that uh you know, Kim K was released uh uh recently uh I guess with ways coming soon. Um given that the Fable model was available only for a short period of time, the idea that Kim K was trained primarily for by distilling Fable, I just find that very hard to believe.

Uh uh given, you know, there just couldn't have been that much fable data and how could Kim Casey have been trained in such a short time. >> That's really that's a really good perspective because I think that's definitely not part of the conversation in DC. But that that's a great point. Yeah, there's certain parties that eager to throw whatever they can at the wall to uh attack open source models including Chinese but also uh when you take an open model whether it's released from a Chinese lab or American lab and it run them on American infrastructure you know it basically becomes you really American infrastructure can control that model for example when um I you know in the US publish code online that is used by some other country well that other country controls it from them or no.

And so I think the amount of FUD to slow down uh uh American adoption. Oh, one unfortunate thing, the whole world's going to use open models. A lot of the world is going to use open models. And so American underinvestment in open models means that there are many nations, for example, in Africa, deepseat adoption is through the roof. We don't see this that much in the US, but in places where they're a little bit more price sensitive, Chinese models have really gained tremendous market share.

And this is regrettable because I would like American open models or American models to be more competitive all around the world. And it is a competitive massive competitive advantage disadvantage. If um an open model allows you to get intelligence at I don't know 1/5if or one third or one the cost then if um for everyone wanting to build AI applications if your supply of intelligence costs three times more that's a very fundamental uh business disadvantage.

I guess I'm sort of curious how do you sustain the like a business model for an open model? Uh you know if you're charging for tokens, if you're anthropic or open AI that's helping obviously there's some profit but it's also eventually hopefully uh but you're it's also helping do this big buildout. Where does the kind of the money come from for the capital expenditures when it when you're talking about open AI models?

You know, Bill Gurley uh wrote an article, I think it was in the Washington Post recently, um talking about how open source is a well-established business model. Uh Red Hat built a great business by open sourcing, you know, Linux operating system and many businesses have been built with open source. The details of how to do this with open models, I think, are still being worked out. Um but there are um uh publicly traded Chinese companies there that seem to be doing just fine at least in the stock market with smart investors with their open source strategy.

So that's true that the capex of training open models is higher than the capex of of you know writing traditional software but um uh I I I feel like uh their business models to be worked out and to the extent that open models gives you a fundamental cost advantage >> that's something to to pay attention to. Totally. One of the focuses of this series of interviews that we're doing is that AI is not just a digital story but also a physical one.

Uh and that you know there is this huge buildout. Obviously it's really kept the economy growing. Uh how much do you think all the breakthroughs that you're seeing and working on depend still on the built world? I mean, we're talking about training, running, using AI models, uh, data centers, grids, semiconductors, cooling equipment, other infrastructure. How do you see the the physical and the digital space in conversation with one another? >> The fact that the world is investing in building more bigger data centers, I think that's a good thing.

Um I feel like there's one part of the story that's underappreciated which is there's so much investment in data center buildout buying GPUs the capex for the infrastructure layer and I think we definitely need more um AI inference capacity which is delivering the intelligence after the model train so we definitely can't seem to get enough of that but the part of the story that is going to be even more valuable and that is vastly underappreciated is the value of all the applications that'll be built on top of this infrastructure.

So for example, when the internet came up, you know, Cisco did well. I mean, good for Cisco. Happy a lot of friends at Cisco. But it was the applications built on top of the internet that became even more valuable. And it had to be that way because um you need the people building applications to generate enough revenue to pay the infrastructure providers. And so, um, a lot of the time that I spend at at AI Aspire with my, um, business partner Kirsty Tan is helping large businesses think through what are those applications that are not just, you know, let's let's just chat with a chatbot and copy paste, fix my email grammar, but to think through what are the automation scenarios that let you change the business model and not just get cost savings, but more excitingly get growth and people still underappreciate that's going to be even more valuable than this infrastructure story that we're also focused on. >> I think you know when you talk to some investors you're obviously like in this how do you avoid investing in a company that might be making a product or an application that could just be basically like oneshotted by the next anthropic roll out or the next open AI rollout you know where they they can perform the same function.

How do you how do you think about that when you're making investment decisions? >> Yeah. So, I think um uh in the for both small startups as well as large incumbent companies uh we sometimes think about what is the moat that makes the business defensible and sustained for a long time. um for large businesses uh which is I I find that a lot of his existing assets have been built up over decades or sometimes even more than decades and are actually really difficult to displace but the lot of work that needs to be done is um to have the right top down leadership to think through given all the valuable things we built all the customer relationships all the technology all the data all the supply chain all of these things where do we insert AI to get efficiencies or maybe even more excitingly drive business growth.

So maybe one one example earlier this week um I was uh visiting Morgan Stanley uh where you know I was chatting with um uh the CEO Tedpic uh and then Dan Sinowitz Andy Sapposine and I find that that's one example of a great financial institution one of the most important institutions in the world that has set a tone from the top of using AI offering to train everyone in the company globally and then executing a set of exercises that they call challenge labs where um they assemble business leaders and technology leaders to collaborate to find use cases.

And I find that those partnerships uh between business and tech leaders can often identify um really valuable use cases that are not just incremental efficiency shift >> where you think about how does the business create value and then you don't just make a little bit cheaper to do but maybe think about how to do this service 100 times more and drive growth or do the service 10 times faster so you can get back to the customer faster and offer more valuable products.

I find those things really exciting and will be even more valuable than the you know kind of business efficiency cost savings that that people also talk about. >> That's that's heartening. Uh and I think it helps explain some of these valuations too. Uh when you think about with the scalability uh I mean we are continuing on the infrastructure side to just see this >> really amazing uh capital expenditures. We just heard Google basically say they're going to spend almost a quarter of the US military budget on capex this year.

You know, that's four times the budget of the whole Marine Corps. I mean, it's it's just incredible sums. What do you say to people who worry that there's some kind of bubble uh or that we're going to have you too much investment uh too fast? >> So, I'm not giving anyone investment advice. No, I know. And I think [laughter] I know um I feel like one thing I'm confident about is we need a lot more inference capacity. Yeah.

Uh meaning AI to generate tokens, generate outputs for us. And >> take software engineering which is the sector that AI has uh accelerated the most. It's covered the most because AI is writing tons of software. Uh two interesting observations. Penetration is still low. A lot of software engineers have not yet fully embraced AI tools, but the ones that have are just so much faster, more productive, and frankly, I think have more fun as well and already we just can't get enough inference capacity.

So, as penetration goes deeper in software, we just need more AI. Doesn't mean people won't lose money building capex, but I think it will get used whatever we can build out. The other piece of good news for that is one of the fierce narratives for AI is there'll be some sort of job apocalypse or AI job apocalypse where some people have said 50% of the people we other job may be rioting in the streets that's not going to happen right >> I think the opposite is see already very visible in software engineering where AI is automating so much work >> frankly we can't get enough skilled AI engineers yeah I mean AI server engineers that's made software engineers even more valuable and so software engineering job postings are up uh uh and the the industry is healthy and growing.

So I think to the extent that AI um in is infiltrating software first and then other sectors later be it marketing recruiting and so on I think on average there'll be small exceptions but on average for most job categories because AI can't do everything people are needed to steer and to complement AI and the challenge will be that people will need new skills so I can't write we can't write software the way we did two years ago and expect to you know frankly have have a job >> but by shifting the skill mix and learning new skills we can't find enough software engineers and AI engineers and so as AI starts to enter more fields I think software will prove to be a harbinger or a forerunner of a trend we'll see in other sectors as well where you do need to grow the skill shift in skills but people will be able to do more hopefully get paid more uh but but that'll be actually the challenge is not dealing with the job apocalypse right the challenge is how to help everyone gain the new skills they will need. >> Yeah.

I mean that when you talk about developing inference capacity I mean is that that in practice what does that mean? Is that building more data centers? What is the how do we physically do that >> to get more inference capacity? One way would be build more data centers and I think uh frankly I think build more data centers they will I think they will get used. you know the exact amounts of profit or loss I don't know >> right >> harder to pine on but I think there is there is very very high demand for inference data centers um >> and then will there be more creative ways new tech breakthrough technologies to bring down the cost even further I'm actually pretty confident there will just from the things I see researchers working on uh but I think uh maybe with AI computers and data centers have become even more valuable than before so building a lot of them just makes sense uh exactly how my shield we need to calibrate but there should clearly be a lot more of them and and then maybe if I think about foundational pieces as well one of the big buildout and I was just chatting with my collaborators at the centure AI is by centure chat about this is data architecture so it turns out AI is fueled by data and I see a lot of large businesses um rethink how do we organize our data so that data is not ready only for humans to use but for agents to use So actually in in in my team and I we actually spend a lot of time thinking through in the future data will be used by humans absolutely but it'll also be used by our AI agents and one of the key ways to unlock a lot of value in businesses is to make your data fabric agent ready but agents use data very differently than humans.

You access it a lot more. The patterns are sometimes more chaotic but you need automated interfaces. Um, for example, if my agent wants to access the data, I can't have it stop me every 60 seconds have me type in the password, right? So, just just little plumbing things like that. Uh, but I think that's another thing that unlock a lot of value in businesses. >> That's so interesting. The uh on data centers, what do you make of the backlash that we're starting to see, the push back, the fear?

How do we get past that? How much of a threat is something like the moratorum for a year in New York to the buildout to getting the inference capacity that we need to be able to achieve all the the great stuff that you're talking about? >> You know, if an adversary of the United States wanted to slow us down, I think they couldn't wish for almost anything better than these silly moratoriums on building our data centers.

I don't want to dismiss the concerns that people have. Uh I feel like yes, data centers are kind of an eyes saw in some places. You know, I don't know, maybe we could make maybe we should figure out a way to make them prettier. Um I think a lot of other things have been overhyped though. It turns out that one of the best things we could do for the environment is to concentrate our compute in the data center. >> So you know the I I have a rack of service in my office.

Frankly, my rack of service is so much less efficient from energy use or water consumption or any point of view than data centers which have been engineered to be hyperefficient. So, one of the best things we can do for the environment is move all the on premises stuff into data centers where the concentration and and I think you because we put a lot of compute together a data center does you know use energy and water but it's better than the alternative of having be distributed.

Um, I think the real question is we're using so much compute, so that starts to have an environmental impact. But if we're going to use this much compute, it's clearly better put in a data center. I suspect that backlash against data centers is more backlash or discomfort against AI as much as or even more than data centers per se. >> Um, and I am worried that um, >> you know, AI is going to make so many people's lives better.

Uh I think it will help people learn, improve health care. It will drive business outcomes. Uh I feel like my life personally is more fun that AI do some of the less fun things for me. So I see AI as a very positive force, but somehow the amount of weird anti- AI lobbying has been excessive and this will really damage America if we let this continue. >> Speaking of talent and skills, uh the you have deep learning.ai AI, which is an education technology company that's helping people build careers and skills in machine learning and artificial intelligence.

Where do you see that being most helpful for preparing young people to fill all these roles that you're talking about will need filling? The challenge that uh many countries have is uh traditional educational system is slow to adapt to new technologies. And the reality is um we don't want to train up software engineers for the jobs of 2022 or frankly we don't even want to train them up for the jobs of 2026. We should be training them for the jobs of 2028 and beyond.

But how do we get the educational system to adapt where the future of software and AI engine is going is becoming clearer to me. So I think I'm able to have a view on what are the new skills people need. Um and I think I'm actually working hard to figure out for all of the other job roles as well. You know, we don't want everyone to be a software engineer. It'd be fun but be kind of weird. But for the journalists and the marketers, the recruiters, the HR professionals, the operational specialists, their jobs will change.

But and I don't think their jobs will go away. We still absolutely need lots of people to do this. But what are the new skills that they will need to be the next gen marketer, the nextG recruiter, the nextg journalists? Um I have a view on that. But I think firming that up to give people a clear path to learn those skills. Um I that's what uh you know Corsera Udemy Danti uh some of the things um my collaborators are often are working hard on. >> Yeah.

And they're worth they're worth checking out and I'm sure they'll just keep getting better. To close out something we're asking everyone who we're talking to as part of this uh building America project is who is an American builder living or dead or innovator an American builder or innovator who you most admire and why? You know, I'll tell you the name that popped in my head as you were talking, but it'll sound so conflicted, but honestly, Jeff Bezos is one of the leaders I most admire.

Um, [sighs] follow him for years. Uh, uh, and just from afar, you know, before I had the privilege to to pass in Amazon, was trying to listen to a lot of talks. Uh, uh, learned so much from him. Uh, uh, and then and then also more recently anti-Jassie. But I think that, uh, I don't know. I I I think Jeff Bezos has been uh one of the most amazing VAS America has seen. >> Yeah. >> And then clearly what he built at Amazon, you know, is is is something that is uh benefiting a lot of people >> objectively.

You can't argue with that just >> in terms of >> impact. >> Uh well uh Andrew, thank you so much for coming on today. >> Thank you. This is really fun to be here and this is I think this is important topic for America. So ensuring American competitiveness where I want our great American businesses to win but one of the best ways for America broadly not just the AI sector to win is preserving open models so that all American companies can do Well, [music]