Chip War (with Chris Miller)

Welcome back to Cerebral Valley. I'm here with Max Child and James Willstrman, my friends and co-founders of Volly. This episode we are talking about chips and big tech. My interview after a conversation with Max and James is with Chris Miller, the author of Chip War, the fight for the world's most critical technology. He is an expert. We talk a lot about Nvidia. Yeah, so stick around for that. Max, James, welcome back. Nice to be here. Eric, glad to be here. The starting point question that I wanted to frame things up with is, do you think GPU capacity, the quality of these graphics processing units, and what Nvidia is putting out is the main driver for the sort of generative AI revolution we're seeing right now, or do you mostly attribute it to the research papers, the new approaches? Who do you give credit? I guess I give more credit to the research papers, not that I don't discount what Nvidia has done here. I think they were in the right place at the right time with the right technology to accelerate what was possible due to the research, but yeah, the research was the breakthrough. I do think there was some intentionality. I mean, it's always hard to tell with AI when there's so much, it's a hypey thing to talk about, and this is a CEO that also made a bunch of money off crypto mining, so happy to dive into the wild speculation clearly. I don't know, Max, do you have a take here? I mean, I think I agree with James. I just think that even four or five years ago, we were already starting to see the beginnings of this with GPT-2 and earlier versions of the foundational models. Also in areas outside of large language models, you saw big progress in our hobby horses of speech recognition and other areas, so I think that even with five-year-old GPUs, you still could do pretty cool stuff with the new research that was out, whereas if the research never comes out, I don't think that you're making this kind of progress in any of these large language model areas that we have seen so far, so. Right. I mean, chips, even though there's potentially exponential improvement, it's still on a trajectory, whereas a paper comes out of nowhere, gives people a new approach, and then revolutionizes what's possible. I mean, I think a different way to think about it would be like, GPT has gotten 1,000 times better in the last three years, and the chips haven't gotten 1,000 times better in the last three years, so what's driving that? It's probably the software, and then the people figuring out how to make GPT, right? So yeah, I just think it's clearly based on the ideas in the software. The chips are great, no doubt, but it's not the key driver. I do think... So Nvidia has clearly gotten rich off this stuff. It is now a $1.1 trillion company. In one year, the stock is up 258 percent, and in five years, it's up 564 percent. So this has been a great run for the company. I mean, what do you guys make of Nvidia investing in all these companies? I mean, similarly, it fits into Amazon, Google, all these companies are making investments. The money that they're spending obviously comes back to their business, right? Nvidia wants AI to be super active, so it puts out money that gets spent on Nvidia chips. Yeah, what do you make of... I mean, it's round tripping, or it's spending money to make money, but it doesn't seem as sort of negative as in some cases. I mean, isn't the investment Nvidia is putting out an ecosystem like a rounding error compared to their revenue or compared to their profits? I mean, I think that they're making such ludicrous quantities of money that kicking back like one to two percent into the startup community that might invest in Nvidia chips in the future is obviously just a very good business practice. Right. Well, it seems like they're literally playing Kingmaker, right? I mean, like companies like CoreWeave and who are sort of doing chip adjacent stuff depend on their access to H100s, and a lot of these foundation models seem like they're fundraising on the premise that, hey, we have access. This is differentiated partially because we have the deal to get GPU access when other people don't, and so there's this hope that you have a mode in that you have access to chips that other people don't. The scarcity today of the best in class chip, the H100 is has to be a temporary situation. I just like the economics of like making more of these chips is very good idea for Nvidia, so they're going to do it. But they clearly want to build like Amazon Web Services type competitor, and so having eyes and ears in a bunch of startups and seeing maybe who they could acquire to do that is smart. I do think the interesting question again is like Intel basically had like two decades of like total monopolistic dominance of PC chips, right? And the reason basically was that it's like really, really, really hard to build fabrication plants to build CPUs, and it took a long time for like TSMC and other people to sort of catch up on that front and then eventually kind of surpass them in many ways. But the question I guess for GPUs I think is like, is it that technically hard to make GPUs that like whatever the next version of the H100 is, you know, that they're going to be onto that before anyone's even caught up to the H100 and so they'll have this instrumentable two to three-year advantage where if you want the best AI GPUs, it's always going to be in video for the next 20 years because that seems like a plausible case to me. Do you think Microsoft was smart to invest $10 billion in OpenAI given it's clearly going to distract the company as has been reported? I think my friend in the Wall Street Journal reported as much that they directed sort of their attention to OpenAI away from Microsoft. And part of what you're saying is they are giving up access to their own GPUs to OpenAI. Is that accurate? Just to say. Yeah. So yeah, I mean I think it was, I think it was a good decision and I agree. Definitely a good decision. I can't imagine they would be as critical to where we are in the AI era right now had they not made that decision. They have, you know, early access to all of the technology that OpenAI is creating. They got GPT-4 into Bing before it was in public in, you know, in front of customers for in front of OpenAI's customers and seems to be continuing with what they're trying to do with Office and Windows, you know, the Office suite. So I think just that early access to OpenAI products seems worth the investment. I think the sort of meta question you're asking that I think is really interesting is like is Big Tech going to win the AI race or whatever, right? It is like, I mean, we always used to talk about fang, right? Like Facebook Amazon Apple used to be Netflix and Google, right? You could replace the end in fang with the end video, it would be perfect. So fang. Okay. And like, goodbye streaming, hello chips. Yeah, goodbye streaming, hello chips, right? And I love Netflix. So they never got over like a couple hundred billion, which is where you said, you know, Nvidia is a trillion now. So pathetic, 200, 300 billion dollar company. Yeah, is fang going to win AI? Like, I think that's a really interesting question. And I think like, I'm kind of leaning, yes, I guess to your point, like it seems like Microsoft's been able to grab, you know, OpenAI, which is per our last podcast still the most valuable asset in the entire industry, right? It seems like Amazon's getting very cozy with Anthropic in case that's actually the other big asset in the foundation model category. Facebook is absolutely like ripping on open source models, you know, Lama, they're going to put like LLMs inside their ad products. So you can have automated ads on Facebook and Instagram and all that stuff. You know, Apple also putting, they're also putting LLMs into consumer chat, you know, products, right? You can chat with celebrities and I'm sure they're doing a lot with image generation. Yeah, exactly. And then Google seems to be like fully ripping on just like dumping large language models into every single Google product, like a Google, you know, Google docs, Google, RIP photos. I don't know what that is. The conventional wisdom and so on. Okay, Google ripping for Google for the record. You have to like grade them on the Google curve, which isn't there. Yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, new ideas. Given that Google has not come out with a new idea in 15 years, they are doing a tremendous job putting AI products into their existing, into their existing products, right? And so, yeah, for them, the number of launches this year has been tremendous, right? Graded on the Google curve, right? And then Apple has basically done nothing, but Apple always does things like two years late and then hopefully they get it right. So the great dream of Apple as I understand it is just that, you know, we get to a point where a lot of these small models can be run locally, local, and they have these great M1, M2, whatever generation we're on now, chips, and that even though they're not GP, there's sort of combined chips, right? So there is some graphical component, right? And so that those chips would be sort of pretty capable at handling sort of small local models. Totally. And I mean, Apple's argument would be we're using transformer based models in photo recognition stuff, we're using it in speech recognition stuff, we're using it in auto correct, they just put it in the iPhone, right? They're like, their take is always like, well, we will do it when there's a real customer need, and this tool can really provide value. So I think like Jerry's still out there, I also think with Apple Vision Pro, it'll be really interesting to see what kind of generative AI and also just, you know, the AI driven toolset that launches with Apple Vision Pro. So I think like it's a really interesting question as to whether or not like this is a whole AI revolution is just going to entrench the power of the big five tech companies, because I think they look pretty good right now. You didn't mention Amazon, right? I said Amazon was like co-seeing up to Anthropic, and also, I mean, you can elaborate on the other stuff they're doing, which is quite copious on Alexa specifically. Well, and also AWS, right? Bedrock and everything, right? Yeah, right. Well, it feels like Amazon, I mean, Microsoft is clearly ahead. Like they, Azure and AWS for those two companies, I think, are the most important. I get you guys really care about Alexa and stuff can happen, but for the businesses today, like Amazon Web Services is so much more. And like Microsoft can go to people and say, like, Azure has a direct relationship with OpenAI, whereas Amazon, it feels like they've been pretty slow on this. But like you said, they have Bedrock and they're trying to come up with these sort of partnerships, but yeah. Well, their strategy is they're going to put everyone else's model on AWS, right? So they're going to be the middleman, like they're going to sell you access to other people's models. So they Anthropics models and I think to the Lama models and do whoever other models they can get on there, they're going to get everything on there. And the only, like, maybe they will, maybe they won't end up with OpenAI as models on there. I think that's an interesting ongoing business discussion, but like in the end, they're going to try to be, you know, the supermarket for models in the cloud, right? As they have been in the last generation of all the different web tools they sell. But maybe Azure is not allowing OpenAI to put their models on AWS essentially, right? Yeah. So is OpenAI so good that that will cripple Amazon? But it is interesting that I imagine most developers who are using GPT models are doing that directly through the OpenAI APIs. Yeah. Probably not Azure. And there is, you know, there's this world now that was already trending this way of big companies wanting to have like a foot in Azure and AWS so that you can sort of get the best of both worlds. But that would seem to be good for Microsoft, given that they were not the first place player. So even saying, okay, this further creates this world where you want hybrid cloud or you want to dual app clouds is sort of good for Microsoft. I can't believe I'm talking cloud computing on the podcast. I feel like even even as long as we're willing to go, it's funny to be in that room. Okay, Facebook, I think, you know, Facebook is releasing Lama, which is sort of they're sort of the king public company and open source models. Their top AI person is Jan LeCoon who's sort of been at this forever and has had a lot of spicy and open takes. I mean, I guess the core question is just like, what do you make of Facebook, which is sort of a closed social network, being so pro like, let's just give foundation models away, make it easy. Let's commoditize them. Let's just like give away. What do you make of the Lama strategy? Like, what's the point of giving away foundation models and making that such a core piece of your approach? I mean, I love it. Yeah, I think it's great. I think it's really it's really probably primarily about developer relations and creating good will in the open source and the developer communities that for them is more important overall to the like longevity of their business than owning, you know, or monetizing these APIs, these models through APIs, which would be just an entirely new business for them. So I don't know, that's that's my take. I don't know. Do you guys have a different perspective? I just think that I think that there's basically two strategies here with like open and closed source software, right? One is like closed source where you keep it private. You're like, we have the best stuff and people will have to come to us even though they won't know the sort of how the internal working to this software are because it's the best, right? And that's pretty much the open AI strategy, right? Like and some of the other foundation model companies like Anthropic, whatever. And then Facebook is like, okay, somebody already took the closed source. We're the best strategy. We're going to take the the, you know, opposite strategy, which is we're going to open source all these models, right? And we're going to hopefully build developer communities around these models so that people can make them better. They can, you know, fine tune them. They can like, you know, help optimize them over time. And then we will have like the, you know, the thousands or, you know, tens of thousands of developers who are working on top of these models, like build and improve them and build a network of plugins and, you know, optimize them and all this good stuff. And so we're going to get value out of the developer community and therefore the models that our business Facebook is built on top of will get better, better that way, right? And so I think it's like, it may not work, but I think it's better than just trying to be the third foundation. It's different. Yeah, it's differentiated, right? Yeah. And, you know, there are lots of great startups that have been built, you know, where it's like, we're going to give away a foundation. We're going to give away, sorry, an open source project. And then you could build a business around if it's super successful. Also for Facebook, they probably benefit a lot from hiring great AI talent in the long run. Right. And if that alone just gets them, you know, marginally, you know, a percentage. I know reals is getting better, you know, I feel like the, the reals is getting reals is a good algorithm. When a Facebook and Google nail AI generated ad advertisements, right? Or ad just even the words in the advertisements, the copy, right? Like that, you have to imagine that's a huge tailwind for the business, right? Like, like, you know, personalized ads that are created by AI in real time, as you scroll to Instagram feed or the Facebook feed or whatever, like, you have to imagine that. Yeah, yeah. You have to imagine that's a crazy good business for them, right? Or improvement to their business and lowers the friction to actually buying the ads. It's kind of interesting if you think of it that way because what you're saying, Max, is AI just improving and getting better faster is what's best for Facebook overall, almost like if we had a breakthrough in, you know, energy production of energy or clean energy or something, that would be great for Facebook too, probably because they would, you know, have a lot, we could run their data centers a lot cheaper and you know, all these things, right? So it's almost just like better for them to encourage the advancement of this industry faster because a lot of the gains will accrue to them. We jumped into this with business strategy, which I am enjoying. But to me, what's interesting about Facebook's positioning is that, you know, Google, Microsoft with OpenAI, to some degree, they're playing the responsible steward, right? It's we are going to make sure our language model behaves appropriately, doesn't get off the rails. We're, our brands are at stake and we need to really guard them. You know, Mustafa, Suleiman Ed, an inflection who's going to be speaking at the conference, has sort of staked out and, you know, we're going to integrate this more. And so I don't want to caricature it before he speaks, but like a somewhat open source skeptical position, write it, it undermines the idea that we need to really be thoughtful and protective about who gets access to what and what these systems can do. And so Facebook is playing sort of a, I don't know, Flame Thrower, Fire Thrower type role here where they're throwing fuel to the fire on open source in an area where a lot of the other big tech companies would say, the thing we can do is like, do this really well and then keep it really closed off so that, you know, we have basically, we're almost as functioning as a state ourselves where we're really being thoughtful. What do you take of Facebook, the big company playing this role of democratizing? I do worry, I guess, for them if someone uses Lama for nefarious purposes, like essentially it hasn't happened. It's a great problem. Yeah, I think that could happen. I think that this could be like a long-running issue now for them going forward the next few years, if for, if, as we've talked about in previous episodes, these models can be used for nefarious purposes. Really laying the groundwork for reporters here, you're like, okay, people should be held accountable for what their foundation models. Not even like their open source project does. I don't, I don't know if I, I don't know if I personally agree that I'm saying that could be an narrative for a share. And it just depends on how, how much harm can be done with open source models. I think we don't know yet. I mean, Facebook's like, look, our brand for trusted safety is already so bad that there's nothing we can do that could possibly make it worse. So why don't we just go for it on the business front? They're like, Cambridge Analytica, wait till they see what people will do with this model. Well, Cambridge Analytica is a great similar analogy because they thought they were, at the time, they were opening up the Ferngren graph and they were giving data to other developers and companies. And that was encouraged almost at the time, right, by developers. And it blew back on them entirely. And it wasn't even good for their business to be doing that, to be leaking that data. So it is, there is kind of a similar thing occurring here potentially. Facebook's willing to be wild. Well, the narrative kind of can change on you, right, that with at the beginning of social networking and technology, this was encouraged that we would want, you know, more data sharing or something with developers and then fast forward fact and years. Have we all staked out points of view? I think we're all on the same page on, do we believe in open source? Like Databricks, Super Pro open source, like a lot of the, you know, obviously hugging face, Clem at CV1, Super Pro open source, like I feel like generally we've heard from a lot of pro open source people, do either of you have reservations or you fully in the camp of like, it's an arms race and just like get the best stuff out and like good people sort of take advantage of. My take is that open source right now is great for the community, for developers. It's awesome to be able to, you know, edit weights or use smaller models or run your own inference, all these things fine tune, right. And that just might not be the case going forward. I don't know, at some point, maybe I will change my mind and be like, no, the models are too powerful. Like I don't want these open source models in the hands of the various actors or, you know, China or something. So I guess my take is it's great right now. But it's hard to pull, you know, that's sort of the, once they have the models and it's gone too far, it seems hard, hard unwind the clock or whatever. Yeah, I guess I, I just believe the nefarious actors in this scenario are unstoppable. And as we discussed in the AI kills us all episode that somebody's going to figure out how to, you know, some bad, bad actor is going to get control of a really powerful model regardless of whether or not it was open source or not. So, yeah, I mean, give the, you're, you're very, give the, yeah, I'm pro open source, give the, give the fruits of the labor away to the community. I mean, why do open source, why do we think, why do we think open AI changed its tune so dramatically on this question? Awesome business strategy. Just pretend business is good for the front runner, right? Like, like, like, regulate it, like, let's tell, make sure you have to spend a lot of money, like, following rules, like, that's good. Cause for big companies, because once they realized how good the stuff they were cooking was, why give it away? I mean, open source is the strategy when you're not the leader, like, not, you know, you never give away the high quality stuff. Yeah. Or, or the strategy that makes sense for meta, because they have this, these huge other businesses, right? Like, you would, it doesn't really make sense to give away models as a startup, I guess. I mean, maybe, maybe that's what stability is kind of doing or mid journey, but you said they're kind of like, becoming more close source. Imagine a world where, like, these algorithms are near AGI, like, and then they are locked down, you know, that's the ultimate elite. So, like, a small group of companies gets to control, like, this sort of infinity resource, like, it seems, that seems insane, you know what I mean? I feel like that would become sort of one of the great freedom questions of our time, that just in any way, restricting the set of people who have limited access to, like, a god-like reasoning and information system. I feel like we're backtracking on the AGI kills us all episode, but I mean, I did think open source was key to Facebook, and that's how we got down this path, but reasonable that we've gone far from that. Though you've been the one that makes the point that, like, often these discussions undercount the AGI potential and how disruptive that would be to any of the other arguments that we're having. I totally agree, and that's where I would question your assumption that everyone should have AGI, like, without knowing how dangerous that could be for the world, that seems, like, pretty presumptuous to just assume that it should not be regulated by a handful of companies. Yeah, that means it's a lot of just give away the god-like power. Everyone deserves god-like powers. Everyone deserves god-like powers. I mean, presumably they will have some counter, you know, countervailing force. Presumably, sure. Yeah, I guess if that's the case, you know, but... I guess here's an easy way to make sure we haven't missed anybody. Let's do a collective ranking of who we think benefits the most from AGI in terms of market-cat movements experienced and to be experienced. I'm putting in... No, we're doing it together. I think NVIDIA should be our number one. Do you think NVIDIA would be the close one there, I think? I mean... Microsoft's stock isn't actually up anywhere near... So NVIDIA then probably, I mean, to your point, I guess. But I guess the question is... What about Google? I would put Google pretty high here. Microsoft has a lot of time for it. 34% over the year and 187% over five years. Google, you're putting Google? I know what... To me, Google is in the like... Tech becoming more important and Google being all over it is good for tech. It's good for Google, but like Google's relative dominance, which was so powerful in search, feels threatened by this. So I just... No, I think that's worth talking about because I just feel pretty confident that that whole search impact of AGI is like very, very early and Google will have plenty of time to potentially continue dominance with using LLMs and AI and multimodal models in their search products. So they have the front page that everyone goes to for search and they have really powerful AI internally. Right. It just hasn't played out yet, but I believe that they will get there. All right. So, I mean... You want Google first? It just depends what we're talking about. Are we talking about which of the biggest companies will have the highest market caps from AGI? I mean, or just like relatively, like, you know, is their position improved by AGI? How much the percentage their position is improved by AGI? That's what you're asking. Yeah. I mean, Nvidia... Yeah, Nvidia for sure. Microsoft, second. Agree. Facebook, third, I would go. What? Yeah, I don't agree. Because algorithms are core to what they do in their discovery mechanism. I think if you actually... Oh, because of algorithms are core to what they do. Okay. I just think if you believe there can be a 50% improvement in advertising at scale through AI generation, then Facebook benefits more from that than anyone except Google. And they might think Google... I have just a Google before. Okay. I mean, that's fine. It's one of those. But I would agree with Eric. But to max the point, it's search versus the Instagram feed, which one gets more benefits out of AI. Like, I think that's the real question here. Like, or search versus the Instagram feed plus the Facebook feed, right? And I guess I like YouTube, which is competing in like shorts. Yeah, but it's going to take a while to do video generation, I guess. I don't know. Like, it's sort of about the ladder of how quickly generated personless ads kind of get scaled up. So, it's, yeah. Facebook, you're mostly talking about their ad products and ranking algorithms, but there's also an argument around on the content generation piece itself that, as we've talked about before, there's, you know, ability to create more compelling content and social media posts. I really think the only way you could make the case for Google is that you would say the ads on Google are going to get much better, right? I mean, like, well, what about search itself? You think search is going to get so much better? I think search itself is going to get just so dramatically better. Yes. Like, I don't know. I mean, time spent in search, you would think to go up if they're chat type products. Well, there is an argument that time spent goes down, right? Because you're just getting your answer faster at an end. But it's, you know, it's no longer a link. It's like less of a link system and more like in the search, which is where Google ads are. Yeah. Well, it just depends if it's more, it becomes more like a Chatchee BT model where you're chatting with Bard or instead of searching. I guess that's where I'm coming from. I see myself doing that. Like, almost more than I search Google today in the future. And I think they're the best positioned to be that like true assistant that I use every day to talk to you and instead of searching by typing into a, I mean, I was down on Google before, but like Google versus Facebook, I'm getting heated up just in the sense that like if you think of every product that Google offers, like if Gmail was amazing from AI and like the prompts were great and those writing like could become an even more insane product. Obviously, like Google cloud services, wait, sorry, it's Google cloud platform. Google cloud platform is key if they can improve their like search, YouTube, like there are just so many pieces of their business where they have audience and they could put this to work. The only question we all have probably is execution on all of these areas, but the opportunity to me seems massive. I mean, I guess I'm just going based on the fact that Google makes no money on anything except search. And the other stuff is great, but it's really just a data vacuum for search ads. Like they really don't make any money. I mean, they make one on YouTube now. So that would be the other big thing. But like search ads is the whole ball game. If you're just purely talking about the business of Google pretty much. And so you have to believe that search ads are going to get way better and or the people are going to spend way more time in search, which again is an execution question about like bar versus all the other, you know, large language models and chat models. So like I buy the case. I just don't think it's as clear cut as you guys do because I just think Facebook like Instagram ads and Facebook ads getting better would like just immediately dump right into the business. And what you're saying is that there's more of an obvious trajectory for the Instagram ads and feed to get better from AI than there is for search ads to get better from AI. Yeah. I mean, it's close. They're, you know, they both are going to get a lot better. So yeah, but it's about which one's bigger and what's the bigger impacting us? Yeah. So notably, notably like. So yeah, we, we, it's Facebook or, I don't know, I don't know. I'm not necessarily being my next either. Oh, really? Interesting. I'm willing to, I mean, I just I would put it for at least, but what what's next? But we have Apple and Amazon, you know, like are now towards the bottom of our at least tech list. Obviously, tech being better in some ways lifts all boats, but I don't know. What do you make of those two and like what it means for them? I would put Amazon next. I think they have an obvious role to play with AWS and they have made a lot of great partnerships there with Anthropic already. They are probably working internally on models themselves. They are the number one cloud platform. They will benefit in their other core businesses through AI, potentially, you know, Amazon.com and ads there, which is a increasing part of their business. And not to ignore. You're allowed to talk about it. Yeah, Alexa, which is critical to our business, they have an endpoint in 30% of US homes that is an AI assistant. It historically hasn't been amazing at talking to you beyond, you know, controlling smart home features and telling you the weather and playing bally games. But it's it's completely possible that they will be able to add LLMs and AI experiences as they've already demoed that would transform that device in a way that would be, you know, essential, would would be a game changer for the Alexa business. So I think they're the furthest ahead in that area on like potentially integrating AI with, you know, your average day through a voice assistant. And Apple, I mean, Apple, I mean, the argument Apple gets away with everything and that we can put them in like basically last place. And it's still like, well, they move slowly. And maybe they'll figure something out. And like, I don't know. I mean, it's not only their wheelhouse. It's like, right. It's not it's a lot of data. It's not about privacy. It's sort of not a device. I mean, I guess, well, I mean, if we really want to criticize Apple, we saw this report that, you know, maybe Sam Altman and Johnny Ive and Mossas son, Mossa Yoshi son of Softbank are getting the team together. We're the weirdest team headline you can imagine to build a phone or some device that's sort of got AI at its core. That's a threat. I mean, I could buy, you know, a next generation sort of AI first device. I just think Apple's moat has always been how unbelievably difficult it is to build hardware at scale, right? Especially like high margin hardware. I mean, there's almost no other large hardware businesses in the industry. And even if Johnny is on board and Mossas on board, you still need like, Mossas got a great history. You still need, yeah, you still need about three to five years and three to five billion dollars to build even like a small scale high quality hardware device. And just like, I think that's just a long way away. And anywhere, we're, you know, Johnny and and Sam are playing like Tim Cook will pay some attention to that and make sure that they're not completely caught off guard, right? You know, they all talked to the same people manufacturing the stuff in China. So, but would you agree though that like Apple, while they're not threatened that much like relatively, they are probably lowly ranked here. Yeah, I think they're the most most protected on the downside in some case, in some cases, you could argue. And then also, they don't have a huge outside. Yeah, there's all bullshit. They're fine. And if it's good, you know, it doesn't feel like fun to interrupt it. Yeah, right. I would like to throw another company in the mix here. What do you guys think about Tesla? Oh, yeah, the old self-driving and the general X AI suite now, you know, it's like, he's somehow allowed to like merge AI research. Like, it seems like across all his companies. I mean, I just think that I think I'd be much longer wamo and cruise than Tesla or whatever. I mean, like, I just think that Tesla is fundamentally built around a hardware business model where you sell cars. And like, if you believe we're entering a world where all every car is self-driving and it can take you anywhere, like, why wouldn't you be long like, yeah, Uber or Lyft or Waymo or cruise or somebody who's built around this model of like providing cars as a service rather than cars as like a jewelry object that sits in your garage 99% of the time. Like, I just don't, like, I think Tesla's a great product, but is it an AI for self-driving first product? I always thought the idea that it was going to turn into a self-driving taxi fleet was like pretty absurd. But, you know, I think that's a good point. I think Elon's, his hype or marketing spin on this is that the AI that they've built to successfully get self-driving working at Tesla is so valuable in future endeavors, right? Like, robots, robotics. But didn't they literally admit they gave up on all the AI they've been building for like until a year or two ago, they had been using the last generation of, you know, deep learning tools or whatever. And then, two years ago, there was like a, like, recently there was a story that came out that basically said, yeah, we had to redo the whole thing once we realized like transformer models, large language models are better, right? So, like, they've been saying that for like seven years that like, we're way ahead and everyone on building a self-driving model as we have a bajillion trillion miles of data blah, blah, blah, which is fine is true. But like, they had to throw it on the trash can like 18 months ago. So like, I don't find that like a super compelling like case that they have some moat around that sort of stuff. But yeah, who knows? I mean, they do have a lot of trust with like regulators. Like, I feel like, Waymo and Cruz have just tried to be as responsible as possible. You know, I mean, Waymo has been very conservative in deployment. And, you know, Cruz is like general motorers like a company that I feel like every American politician is rooting for, right? I mean, Tesla, it seems like now has been able to position itself is very close to the Republican party. But it feels like the administrative state and Democrats are going to be like extremely skeptical of them just throwing cars on the road. I mean, that's sort of cynical even about, I guess, the politicians I like. But I, yeah, I think I agree with Max's point of technology. I also think having a reputation for being a responsible sane actor is important when you're doing something as dangerous as deploying the first self-driving cars. I think it's a fair assessment. I mean, we'll see. But I'm sure they're going to stop premium for it. I'm sure. I think they pivoted and you can't, I don't know how you can ever measure this because it's like, well, the stock can stay inflated forever. If they pivot the entire company to robotaxies and they start selling $5 Tesla rides all around America, I will totally change my tune on this. But you pretty much have to embrace a disruptive business model in the next three to five years, which has historically been very, very difficult for any type of company because you're kind of saying, all this money we're making today, that's going to be zero in five years. And we're just going to have to jump on the new thing and ride it out, which is really tough. I, to be clear, I don't think the self-driving deployment cycle will be anywhere near as fast as that would suggest. So I don't think it's going to destroy their like existing car business, but I agree with the idea that it's hard. The classic startup situation, which are articulating is that like people don't disrupt their own good businesses. And so Tesla isn't sort of the worst situation for that. Well, just a counterpoint to that. Do we believe that other companies are going to be able to sell fully self-driving vehicles anytime soon? It doesn't, it doesn't seem like it at least. Besides the three we've talked about? No, I mean, those other companies seem to be only focused on the robotaxie market as opposed to getting consumer vehicles that can fully self-drive themselves into the world. Like if Tesla is the only game in town there, and I'm not sure if that's the case, but right now it seems to be the only one that is going in that direction. That could end up being a very, very compelling offering if the only car you can buy that will drive itself anywhere is a Tesla. Big if. I'll see. I feel like Waymo and Cruiser are going to deploy city by city geo-fence where they know they can do it with no deaths, you know. Yeah. Anyway. Yeah, are there any other? I mean, this is great. I'm glad you brought up Tesla. Like, what are any more edge case companies we haven't considered? I mean, this is part of the problem I think right now. I think a Databricks IPO and the Mosaic deal will be welcomed. It's like, there's a public market desperation for a way to bet on AI, which is why I think Nvidia has done so well because, you know, these big tech companies are already very highly valued, you know, the startups are private. Cool. Great. Thanks for coming on. And now we're going to Chris Miller with Chip War, who is a real genuine expert in the wonky world of chips. And so we go deep in Nvidia sort of their history and, you know, the big geopolitical question around the race with China. So give it a listen. Chris Miller, author of Chip War. Thank you for coming on the show. Thank you for having me. I feel like so much of what sort of the regular person or even, you know, I don't know, the average startup founder, venture capitalist and Silicon Valley thinks of this artificial intelligence phenomenon starts and sometimes ends with chat GPD three. And it's certainly, it's very much like what bots can we play around with? Like, how am I talking to some sort of AI? But, you know, in the background doing reporting on some of these conversations, Nvidia and like H 100s and like weird chips come up over and over again and sort of people's ability to get access to those chips. So I really want to have you on just to sort of like interrogate that and understand sort of the technology behind it. Can we just start off like, when does Nvidia start to matter as a company or just give us like the brief history of this company? Because it was like a weird, weird sort of gaming chip making company right for a long time. Yeah, that's right. Nvidia was founded over three decades ago, but its earliest couple of years was all in graphics. And graphics was most important for games, computer games, video games, which, you know, think back two decades, they had the most complex graphic demands. Figures moving across the scene, pictures changing very rapidly. And there were special types of chips that were produced by companies like Nvidia for processing graphics. And if you think back not so long ago, people would buy specific computers to have the best graphics capability. Now we sort of take it for granted, but for a long time that was a real differentiating factor. Can you explain sort of the difference between a GPU, a graphics processing unit and a CPU? So a CPU, which is the workhorse of traditional computing and computers and data centers, they're very good at doing many different types of things, but they do every computation serially, one after the other. And for most use cases, that's what you want, because you're undertaking different types of calculations. But for training AI systems, you want to undertake the same type of calculation repeatedly. And so parallel processing, which is what a GPU does, is capable of doing multiple things in parallel at once, which is why they're vastly faster for this type of calculation than a CPU is. But why does AI, why do AI tasks need to be parallel while like a normal computer task is sequenced? Well, they don't have to be. It's just really inefficient. Right. You know, I think he trend, there's been great research looking at the amount of data on which cutting edge AI systems are trained. And what you find, if you look at this 10 years or so in chart it, the amount of data used in cutting edge systems is doubling every six to nine months, which extraordinary demands for putting as much data as possible into training. And in order to make sense of all this data, to train your model to learn from the data, it has to undertake lots of calculations. And so the computing demands of training have shot upwards, not just exponentially, but exponentially at a rate that's faster than Moore's law. And that's why having the right chip for training has become very important. Did Nvidia see this coming? Like, you know, their stock is up year to date. I think it's some like 200 percent. I feel like what Jensen Huang, the CEO, you hear about him all the time. Like, was this a happy accident? It's like we're we're making great video games and all of a sudden people want to buy our chips for the hottest thing in the world. Or did they have some foresight here? How much was this a plan on the part of Nvidia? You know, it certainly was a bit of an accident. And it's talked to the company. They'll they'll, they'll readily admit that when they founded Nvidia, they had no idea that AI would be an application. But over a decade ago, they began to realize that there were a bunch of PhD students and researchers at places like Stanford and Berkeley who were using their chips for things other than gaming for complex calculations. And Nvidia realized that if they tried to build out chips around this application and no less importantly, build out a software ecosystem around it that they could provide the chips that AI would require. And so they've been investing very heavily for now over a decade. And for a long time, that investment seemed actually quite foolish because most of their money was still in in graphics and in gaming. It's only quite recently that data center in AI has become their their primary. Well, you just people talk about artificial intelligence as a way to boost their stock price or to, you know, seem like a more futuristic company than you are for Nvidia. It's like, okay, you're a gaming company guy like chill out. And now, you know, it's been totally validated that sort of all this talk of artificial intelligence wasn't total wild hype, hypemanship or whatever. Well, in between they were also a Bitcoin mining. Yeah, I know. Yeah, exactly. They did have the crypto. They're definitely good at seizing the moment. But they were they were being used for crypto mining, right? Is that what it was? That's right. And like, yeah, how do you see Jensen, the CEO? Like he's one of the co-founders, right? Or what sort of his role at the founding and how do you see him as sort of a CEO today? He was one of the three co-founders of Nvidia. They met in a Denny's in San Jose to devise the business plan in the the 1990s. And Jensen, I think deserves a lot of credit for realizing a decade ago that AI could be a real growth driver. Yeah, at the time people thought it was a really irresponsible thing to do, plow all this money into building a software ecosystem around chips. No one else did that. Most companies just made chips. And if you go back to the debate in 2015 around that time period, you can find lots of equity analysts saying, what's this company doing, spending all this money on software rather than selling chips? It's a waste of money. And worse, they were giving the software away for free. But it turned out to be a brilliant move because everyone started using the CUDA ecosystem, which is the software layer on top of Nvidia chips. And that's put it at the center of the AI world. A key piece of the chip world is this sort of like, are you designing chips? Are you the fab? Can you just sort of explain that and where Nvidia sort of fits in? Are they, are they actually, we say build the chips? Are they like building the chips? Yeah, that's a key distinction. They're not actually building the chips. They only do chip design. From day one, they were what's called a fabulous chip company. They didn't have a fab. They only did the design. And chip design in many ways, like a software design type of business, you basically write the code of the design and then you email it off to the factory or a fab in semiconductor parlance and the fab actually does the manufacturing. And so in the case of Nvidia, their most important manufacturing partner has been TSMC, the Taiwan semiconductor manufacturing company, which manufactures most of their chips. And so today for all of Nvidia's key AI training chips, they're all manufactured by one company in Taiwan. And do they, they make all basically all Nvidia chips or they make, they currently make all of the key AI chips that Nvidia produces and been the most important producer of video chips from day one. And is Nvidia, like how far ahead is Nvidia right now in this sort of AI chip? Are they singular? Like, yeah, where, where are they today? Well, in terms of market share Nvidia's far far ahead. So it's estimated that up to 90% of cutting edge AI systems are trained on Nvidia chips and you can debate what's the definition of cutting edge, but Nvidia's market position is extraordinary right now. And that's reflected in their stock price. I think if you ask, are they technologically ahead of competitors, whether AMD, which is another chip maker or Google, which has a chip called the TPU, which is also used for AI training, that's a harder question to answer. Exactly, exactly, exactly. And that's a hard question to answer because there's both, you know, technical specifications you could look at, but there's also ease of use. And so if you've built a system, trained on Nvidia, if you yourself as a engineer, have gotten comfortable with Nvidia's products and with CUDA, there's also a switching cost of moving to somebody else. And so Nvidia's both got their own capability. They've also got the first mover advantage ever was used to their products. But, but they're really scarce right now, right? I mean, my sense talking to startups and investors, I mean, you hear Nvidia's like in every venture round now because as part of the funding, they can sort of promise people their chips, which then they make money back from the startups. It's like a great world. They're playing Kingmaker like, what's your sense of how scarce their artificial intelligence chips are? Right now they're scarce. I hear the same anecdotes that you're hearing an Nvidia and TSMC, the manufacturer partner, have both said it's going to be a year or so until supply normalizes. And the problem is actually not that TSMC can't produce enough chips, but they can't package them because they use a very complex, a new packaging technology to put the chips right next to the specialized. I assume you don't mean in a box. Yeah, you're like, you place the chip on something, right? Yeah. That's right. Yeah. Traditionally chips were packaged. That's only funny, though, if it's like the Apple side boxes, like holding them up at the end. Anyways, go ahead. Yes, so packaging traditionally was a really boring part of a chip business. You take a chip, you put it in a plastic or a ceramic package and that was done. But now it's increasingly important because you need to get your processor and your memory as close as possible with as fast and interconnect as possible. And so companies like Nvidia are leaning more heavily on having the right packaging capabilities to provide the most advanced outcomes. And so it's actually TSMC's packaging capabilities out of the limiting factor right now for producing more Nvidia GPUs. And what is an H100 or like what are how different are these chips from the the GPUs that everybody got excited about in the first place? Like are they server chips or explain what is an H100? It's just the newest version of a GPU that Nvidia offers specifically for AI training. And the reason people get excited about new versions of chips is because the rate of change is so fast that a new version is always not just incrementally better but a lot better than the old version. You know, it's it's not like the iPhone 15, which is a bit better than the iPhone 14, but I wanted to update and I'm like, this is not enough necessarily to get me from 13 pro or whatever. Yeah. Yeah. The chips are different because of the the Moore's Law dynamic, which is, you know, says you roughly double your processing power every two years and that's not a perfect a perfect metric, but ballpark that tells you just how rapidly the technology improves and the H100 is the most advanced chip that Nvidia offers. So yeah, to put Moore's law is still alive today. So Moore's law is actually not a law. It's a prediction that was set out by Gordon Moore, who founded Intel in 1965. He predicted that the number of transistors per chip would double every year or two, which means that the computing power of chips would double every year or two. And that's been holding. And he said it would be a decade, but then it's been true for a long time, right? That's that's right. I'm just stealing from your book, of course. I mix me sound smarter when you get to, you know, read the book in advance so you know all the answers. Anyway, go ahead. But yeah, that that's right. It's a Gordon Moore father last for a decade and here we are. You have a century later. And Moore's law is still basically holding up. The rate of change increases and decreases at certain times, but it's basically true that if you wait two years, a new chip will have twice as many transistors and therefore twice as much computing power. I mean, this is sort of a hard question. So I don't know if there's an easy answer, but yeah, how much credit would you give then the improvement of GPUs for what's going on in artificial intelligence right now relative to, you know, attention is all you need or, you know, the papers in terms of the approaches to actually use these chips to run software. Well, you know, I think it's certainly it's the case that there's multiple factors driving innovation. But I just the way I like to think about it is, you know, what's improved most over the last decade is it the case that that software engineers are 16 times smarter or is the case that algorithms are 16 times better or is it the case that because of Moore's law, two to the fifth, we've got, you know, a vast increase in the number of of transistors. You know, I think it's it's primarily computing power that's that's driving us. That's what you want. The chips guy comes on and he says it's the chips. Yeah, a key theme, you know, obviously in the book and sort of for anyone thinking about the situation is, you know, the rivalry with China and sort of the delicacy of the global supply chains. And, you know, that there's the chip rivalry with China, but also in artificial intelligence, you know, if we're worried that this is the potential sort of path to some sort of, you know, generalize intelligence, there's also the real question of whether the US gets their first. Yeah, so I mean, there's so much to hit out. I mean, I guess the first question for me is, is the United States getting most of NVIDIA's chips or like how much, how many of those chips are going to China is China getting access to those right now while they're scarce? Well, the US last year imposed new rules that said NVIDIA can't transfer its most advanced GPUs to China and not just in NVIDIA. Any company producing chips above a certain threshold can't transfer them to China. So each 100s are illegal to transfer to China. Wow, okay. China's not getting any of those, but NVIDIA has designed chips that go basically right up to the threshold called A800s and Chinese firms are reportedly buying very large volumes of those. So China's getting a lot of GPUs, but they're less advanced than what a US firm can buy. What would their next best option be? Well, their next best option is NVIDIA's second best chip. Oh, man, okay. Right. And China tried to build its own fab, right? A competitor of TSMC. What's the status of that? And how much, you know, it's fun, especially in the US and given the market returns like to talk about NVIDIA, but like, how much is this all hang on TSMC? So China has been investing very heavily in its own chip industry. It's got both companies doing the design side and the manufacturing side. There are a number of Chinese GPU designers that seem to have pretty competitive products, although even in China, Chinese firms prefer to buy NVIDIA's chips, so they're not as good, it seems. But the big challenge in China is the manufacturing side, because TSMC, the Taiwanese firm, has been around five years ahead of the leading Chinese firm, SMIC, for a very long time. For at least a decade, there's been a five-year gap. So both companies regularly improve their manufacturing processes, but the rate of change in both is roughly the same, so TSMC's always half a decade ahead. And that holds true up to today. And what about the United States? Like, hadn't Intel been trying to become a fab? Like, what's the state of our ability to actually manufacture these chips if there's a warrant Taiwan got for bed? Well, yeah, that's the dilemma. So Intel does make its own chips, but around five years ago, it faced some severe problems with its manufacturing operations. And so for its most cutting-edge chips today, Intel now turns to TSMC for the manufacturing. Now, until right now is trying to reformulate its manufacturing processes, and it hopes that by, they're saying by 2025, they're going to be producing their most advanced chips back in house again, and that those chips will be as capable as a TSMC-made chip. But right now, Intel's most advanced chips are actually manufactured by TSMC. And then there's like a key, the people who make the machines that helped that allowed TSMC to do what they do. There's like one company of those, or what's that company? Yeah, that's right. So TSMC, they know how to use the machines to make chips, but the machines themselves are produced by a handful of other companies, a couple in California, a couple in Japan, and then one of the Netherlands called ASML, which produces the most complex of these chip-making tools. So, it is quantity, everything here with the H100s, like, I mean, Microsoft has this cloud computing offering, Google has it, Amazon obviously has Amazon Web Services. How much are those services trying to stockpile AI chips, and then these startups, whether it's Anthropic or OpenAI, how much is their game just like, we're going to win if we assemble the most H100s, is quantity sort of the game right now in terms of getting access to these chips? So long as there are shortages, I think, quantities that keep part of the game, and what you see is that Nvidia now realizes it's got a lot of influence over the future of the cloud computing market. And so, Nvidia has been, I think, pretty actively trying to build up other cloud computing firms, by giving them- Right. Exactly, giving them access to GPUs, and Nvidia, I think quite rationally wants a more fragmented cloud computing market, because that's a market in which it has more market power relative to Microsoft, to Amazon, and to Google. So, we alluded to this, but sort of- Can you walk me through the tech giants in terms of credible competitors to Nvidia? I mean, we talk- I guess start with Google, it sounds like maybe they're in the lead, I want to know about Intel, and then is there anybody else who's relevant or even close? So, Google has a- a in-house design chip, they've had a big design team that has designed a chip called a TPU, which stands for a tensor processing unit, but fairly similar to a GPU that is used for AI applications. And, Google has a vast cloud computing business, as well as its own- its own vast data center, so lots of experience running very complex computing operations. But empirically, we know that companies prefer to use Nvidia chips, at least they do right now. You know, that could change as Google chips get better, as scarcity drives up the price of GPUs, but that hasn't been the trend. The preference has been for Nvidia. Then AMD is the other company that is a competitor in producing GPUs. It's a chip design firm, they also manufacture with TSMC that in- they have more- they're an American firm, that's right, based in Texas. And they manufacture fairly competitive GPUs, but the difference with- between their chips and Nvidia is that Nvidia has the the ecosystem around it, the CUDA ecosystem. And so once you're bought into that ecosystem, right now, the switching costs are substantial enough where you just prefer to stick with Nvidia if you can get access to enough chips. Is Intel relevant at all? Intel is- is trying hard to produce competitive chips. And I think over the next couple of years we'll see it roll out a series of new products that are designed to compete, but right now it's a small player. In my understanding, like in cloud computing, like part of the beauty of like an AWS is that you can sort of, you know, it's flexible, it's elastic, we can sort of have some people do the computing, then you sell to another customer at a different time. Why hasn't that been the case with this artificial intelligence computing? I mean, I hear people talk about it like you're you're running full-bore like these data centers are super hot like what's so different about AI processing? I think the key difference is that the demand for compute power in AI training is so large that there's just a deficit of it. There's just a deficit. And so the systems that firms like open AI are trying to train are so vast that they they can't share their infrastructure because they need all of their infrastructure and they need even more of it than they actually can get access to. And so the cloud computing business has been a business as you say about engineering more efficient systems via sharing. But in AI training, no one wants to share because everyone needs more compute and they can actually get access to it. Right. And then there is also this sort of perceived almost like zero sub like you wanted to deny your competitor access. One company we haven't talked about you know that we talk about in the laptop world for chips is Apple. Like what's what's the status of Apple on this? Sometimes people talk about Apple and artificial intelligence on like local processing. Is that a possibility or how do you see Apple as being relevant or not in this race? Yeah you know thus far we've been talking about AI training and AI training happens almost exclusively in big data centers. But there's a big question about what inference will look like in the future. Some people think that inference will mostly happen in data centers that you're going to ask a question of chat GPT you'll send it back the internet to the data center tips the data center will think about it and give you an answer back. And that is efficient in some ways you have all the compute all the chips in one big data center and so you can start to set up some of the sharing and efficiencies that we just discussed. But the downside is you have latency issues if you're sending all the data back and forth and there's costs associated with moving data. And so an alternative paradigm is doing more of your inference on the edge of networks in your phone in your car for autonomous driving systems in your PC. And that's where companies like Apple could start to play a much bigger role. And right now I think the market for inference on the edge is still very much in flux. We're in the process of seeing many new types of chips rolled out precisely for that purpose. So it just sort of depends on where it's one of these like sort of personal computer versus server level questions back in the 90s where it's like okay is computing going to continue to happen. And the cloud or is there some model where local computers are going to do and you're saying that if it is local then maybe apples are a better position because they have these great chips for for individual computers. Is that right? Yeah that that's right. And I think it's a pretty straightforward cost equation of you know if the cost of data transfer is high you'll try to do as much as your processing can on the edge. If the cost of the transfer is low and the benefits of having being centralized facilities are substantial in terms of the efficiencies you can reap then you'll have a much more centralized inference process. And it might vary even between different use cases. In a car for example if you've got a pretty autonomous car your willingness to depend on a data center to tell you to turn left or to turn right is going to be pretty limited. I would hypothesize for your film. You want to make sure no matter what you can get the answer but you know if it's for mid-journey I can you know I mean we talked about in media earlier on in terms of oh there was a brief period where they were essential to crypto mining and I feel and one of the key story lines I think for the non crypto world that was just watching it was all the energy consumption around those crypto mines. Is that true for AI? I haven't heard that same environmental story get picked up in this case but are we destroying the environment in our pursuit of artificial intelligence? Well I think we haven't heard much about it just because there's such an extraordinary demand for computing that companies haven't gotten around to thinking about the consequences but the short answer is yes it's extraordinarily energy intensive and we're going to get more efficient at it as time passes but right now there are such demands for energy in data centers that there are some places in the US where it's difficult to build a data center because there's no spare electricity capacity for the days it's it's already a dating factor. Have you seen any estimates about energy use or anybody trying? Well it's it's it's tricky because everything depends on how much more efficient you think things will get over time so you've got to assume we're going to be a lot more efficient making the same compute available at lower energy demand in 10 years time but at what curve it's it's very difficult to say. Yeah and to be clear I think what what is the point of energy if not to sort of push the cutting edge of like computing it seems like a very valuable use to me but yeah it would be interesting I mean you obviously want to make sure companies aren't being sort of cavalier about it. Yeah initially not cavalier because if you look at the cost of running a data center. Yeah critical cost. Well I mean I don't know if you're open AI raised 10 billion dollars it's you can become sort of irresponsible you know there isn't the same market pressure if these companies are able to raise what are basically like research grants to to figure this all out. Can you I mean we sort of jumped into artificial intelligence but I mean the broad theme of the book the idea that sort of that we often think of global infrastructure and the sort of weaknesses and vulnerabilities in terms of like oil right I mean it's like oh did we go to war in Iraq over oil. What what is sort of the real sort of what are the weaknesses with chips and what what why do you think sort of this chips arms race is so important when we're thinking about sort of geopolitics. Well the reason I I wrote chip war was when I realized first that we're surrounded by thousands of thousands of chips over the course of our daily lives not just our phones and our PCs but it's cars and dishwasher and coffee makers too. Second when I began to understand how critical they are for AI and then third when I realized how concentrated the production is and just a tiny number of countries and really just a handful of companies Taiwan being the the most surprising example of that and so when you look at the the US China race to develop more advanced AI systems and intact in general then realize that both China and the US are dependent to a shocking degree on chips made in Taiwan to power their most advanced AI systems it's an extraordinary situation that we've all found ourselves in and it's a very dangerous one given the concentration and given the dependency. What would be the consequences of China having a chip edge broadly in artificial intelligence? Well just like we're preventing China from accessing our most advanced chips I think one has to assume that any country with that position would do the same to us and right now Chinese firms that are trying to develop AI systems are doing so at least severe disadvantage because they've got much less access to high quality hardware than does open AI or Google and that's I think only going to accelerate into the future as countries including the US become more restrictive in terms of controlling access to different types of AI hardware and so it's a good thing that we've got most of it designed here and most of it manufactured in friendly countries because what we're going to see what we're already seeing what we'll keep seeing is more politicization I think of the hardware that makes AI possible. Is the United I mean the United States still seems great I guess that chip design given we have so many of the leading firms could do you think we could ever produce a fab or like what what makes TSMC so uniquely capable at building a fab like we're talking culture money like what's what's driving that in your opinion yeah you know it's it's not it can't be a cultural thing because Taiwan hasn't always been the center of the chip industry if you go back four decades ago it was Japan that was the world's biggest producer of advanced semiconductor so it's primarily I think the result of TSMC's business model which has let it scale and the scale has let it drive efficiencies and so when TSMC was founded in 1987 at that time almost all companies both designed and manufactured chips in house it had both sides of the of the equation and the founder of TSMC Morris Chang realized that as it was getting more complex to manufacture companies would prefer to outsource it and so he decided that he wanted to be sort of like Gutenberg was for books Gutenberg didn't write any books he only printed them Morris Chang and TSMC they didn't design any chips they only manufactured that what that meant is that he could produce chips not just for one company but for lots of companies for AMD for for Qualcomm for Nvidia and so today TSMC is the world's largest chip maker and precisely because it's the largest it's also the most advanced because it can hone its processes over every single silicon way for that it manufactures how can you can you make any predictions about how much do you think AI chips are going to improve or like what do you see sort of the next couple years looking like in terms of like do availability problems resolve and how much more do you think these chips will improve over the next couple years yeah I'm the about the the availability front Nvidia and TSMC have said they expected to be a year or so until the shortages get resolved but I think they're going to get resolved prices are such that there's a very strong incentive to solve the availability problem and sell more so sometimes people think they want to keep a shortage to keep sort of demand high you don't you think they want to produce as many chips as they can get out there to make the money while it's there to be made or yeah I think that's right because they're they're they're it looks like they're a monopolistic producer they got 90% of the market but if they can't supply chips there are competitors there is Google with TPUs there is AMD and so if there are long run shortages that are in the supply chain people will turn to competitors and so they got a strong incentive to get get supply increased and actually part of their strategy is again to put their chips at the center of the AI world and make it the the gold standard and the more people that design the more people that train more people that study using their system using the kudi ecosystem the more likely and video is to be at the center going forward and then I guess it's sort of the next chip they have is hard to know or have there been any leaks about what what the future looks like in terms of these artificial intelligence chips I mean I think the best guide is is to look at the past and what you find is that that for over the past half decade they've released a series of better and better chips that you know ballpark Moore's log gives you a pretty good guide is to put things should be in a couple years time bigger more powerful hopefully somewhat more power efficient and and the other area focuses the the interconnect speed between semiconductor so there's the GPUs themselves there's all the networking equipment that moves data between chips that's also an area of really intense focus right now and we're I think we're going to see a lot of progress really coming years jensen I mean did you interact with him at all for this book or no I didn't I say I interviewed his his co-founder but not just yeah I mean just like observe like it must be crazy to go from sort of being seen as this like I don't know wild sort of futurist all of it seeming to come true or the world sort of a green with your futurism I mean sort of project where he wears like leather jackets and dresses all black or like what do you make of him and yeah how much has this been sort of indication for him well I think he sort of sees himself like like Steve Jobs was for the iPhone moment he is for the AI moment and I think that's right I think he has played that role because he does with a lot of credit for investing in the types of products that that have made AI possible at a time when most people thought it was either crazy or possible but only relevant for niche academic uses or something that would never be financially viable and look he's built one of the world's largest tech companies around it and already there are products being produced on a regular basis using chips that his company is pioneered I believe I agree with you you can make a fortune Silicon Valley can love you but I feel like the world will only love you if it's your device in their hands or it's your brand you know it I just doesn't seem like they have a path for like consumer love right I mean you see them trying to take advantage of this and go you know offer other products or you know they're so good at chips they're they're gonna the win the technology's heart and be happy with their capital yeah yeah they're so far from the computer in terms of where they sit in the tech stack that the most consumers don't even know they exist or gamers do is sort of the funny thing right yeah how did how did you get into this like what sorry I don't know as much of your background yeah before you wrote this book were you chip nerd or what no another oh really okay well is your story to the bug so I I'm an economic historian by training written a couple books on different aspects of economic history and and decided to write the book when I came to realize how important chips were and was shocked myself that I'd never paid any attention to this industry and so it was the combination of learning that almost all advanced chips are made in Taiwan which was shocking learning that chips are so extraordinarily difficult to manufacture and transistors in your iPhone for example are small in the size of a coronavirus manufactured by the billions that was an extraordinary fact they got me into it and then finally looking at the ramifications for AI I like most people have thought about AI as a as a question of of of either software or of data data is the new oil I'd read a million times but it turns out that's not exactly right did you go dump all your money into the video or I should have did you see the stock mark I mean I know we are if you're an economic historian like do you have a view on whether okay this is a great company but it's overvalued I mean you know yeah I mean you should turn to me for for stock market advice but when I think the key question about video is is will it I think it's clear it will continue to play a dominant role in AI training maybe to lose market share made a win market share with Google but it'll play a central role but in the inference market will Nvidia be as central there or not that's I think the key question that we're gonna find the answer to in a couple of years and do you think China is close to catching up you know I think China is is not that far behind but China has been not that far behind for some time and because the rate of advance is so rapid in terms of design in the US and manufacturing in Taiwan it's very very hard to catch up and so long as that companies like Nvidia and TSMC keep racing forward the catching up is just going to be extraordinarily challenging for China no matter how much money they pour into it and are Japan and South Korea relevant to this conversation well that's an interesting question Japan is relevant because if you look inside of a chip-making facility there is a ton of Japanese machine tools and there's also a wide variety of ultra specialized chemicals made in Japan so a lot of the chemicals used in chip-making because of the precision required have to be purified to the 99.99999 percent level a lot of those are only made in Japan I wanted to go back and I know this is like a complicated question but like what are the tasks that are being thrown at these chips right or you talked about sort of matrix computing or are is there like one task over and over again that these chips are being asked to do or it's a lot of different things or how how are their foundation models sort of translating to the chips as best you can explain it I guess the way I would explain it is you know if you're trying to train a chatbot like chat GPT you train it by taking all of the language data in a vast data set like Wikipedia essentially read Wikipedia and identify patterns and so and that's like inference that you've been taught that's training that's training okay training is identifying the pattern so what's the what what is the next word I'm going to say you know you know it say and chat GPT knows that because it's identified thousands of other instances where sentence like that ended with the word say and did that because it read lots and lots of books and articles and reddit posts online and then when you go back and ask a question which is inference it gives you a pretty good answer because it has a pretty good sense of what the most likely word is going to be so you're saying they're training the chips based on a lot of data and then it makes an inference about what it thinks the sort of fill in the blank of the prompt basically yeah inference is just an educated guess a probabilistic guess and what's cool about chat GPT is that it's not just fill in one word it's fill in doesn't I know it's insane one thing I'm still trying to get my head around is you know like deep mind right like when they trained you know computer systems I guess to to compete on chess or go or Dota like my sense is that you know they're using sort of this sort of scale data approach but they also have like specific ways they're teaching it to think about specific problems or like do you are those types of approaches to AI also using these same chips or are they still sort of invested in the same arms race yeah well the chips just do the math for you and so they'll do it other math whatever math you're asking to do and they're not and then the interaction between like Nvidia's like programming languages super interesting because they're not like the foundation models are not being written in that language right but they're they're sort of bridging that's right yeah and it's it's just about it's fundamentally allows them to optimize what's happening on these chips much more than maybe they would on like an AMD chip even if we're it's easier it's more familiar and therefore there's a big moat around Nvidia's position yeah so I have you like become like were you bullish on like AI before writing this book or I'm curious how much your view has really changed on yeah like I don't know like generalized intelligence or like the imminence of self-driving cars or any of that you know I think that actually we've I'm in a long run I'm very bullish but I think in in the the short run the last 12 months there's been a whole lot of excitement of the imminent arrival of AGI right which I'm a I'm not so sure I see it happening in 2023 I think if you look at the trends the trends are all very positive AI systems getting better more capable but I think there there was due to chat GPT a popular sense that we're going to get some extraordinary products in our hands right away and actually it's going to be a couple of years before we see AI the current wave of AI products deployed in a big way that meaningfully impacts economy or society yeah I think we're all watching to see if this hype wave is about to end or not I mean there's there's a world where this is sort of peak Nvidia right like we have this sort of mania around AI everybody's running towards it and then it turns out you know chat GPT doesn't improve that much and self-driving cars solve a lot of edge cases and you know the chips don't solve everything even if they do get better like you're saying like I don't know how likely do you think that is or do you have a sense of yeah whether better years or ahead or this is sort of a peak moment for the company well it's it's been a very very good year for the company so I wouldn't be surprised if it's a it's at least a localized peak but I I think you know for for AI applications most people think of what's the consumer application for AI you know what's what's the iPhone of AI but it seems to me that for a couple of years actually most of the application and most of the money and AI will be in enterprises as enterprises that have vast data they've got a desire to use it efficiently to monetize it and so that's where I think we're actually already seeing a lot of the investment I had a lot of the early products are happening in pretty boring places inside of enterprises but that that's probably okay that's that's what's going to drive I think the the building of effective products in the long run uh enterprises you know businesses just translated for like the regular person you know I used to work at Bloomberg they've actually come out I think with their own foundation model to show how they can train financial data you you have a good example of where you think corporations will put this approach and data to use but if you imagine a a company like Walmart they have to every day decide what price to put all of their products and they decide pricing based on a whole variety of different factors what their competitors are pricing the availability of products very complex uh process and that seems like a perfect use case for trying to use more advanced algorithms to set better prices more rapidly who gets is it came are this blue light discounts with Walmart I forget but who gets the discount you know where right uh a lot of products to keep track of have you like it feels absurd to ask this but like what the chip business looks like in a world of AGI generalized intelligence or like have you game that out at all I mean there are people who think it could happen you know with five years from now or less like do you have a view on it first of all and then second you know what what would it mean for the chip world well I guess my sense is that we're we're we're a long way away I'm still waiting for Chatshey PT to accurately complete all my senses it's still at a 90% rate and the other 10% are pretty bad um I think it's going to require uh tremendous advances in semiconductors to make possible the tremendous increasing computing that more advanced AI systems will need I mean that that's been the trend the trend has been AI systems only advanced when we apply more computing to them and so if we want systems to be twice as good as they are today we need something not too far off from twice as much compute to make it possible are there you know there was this I covered regetti and sort of the quantum computing world do you have much optimism there do you spend time on them in in the book you know not not in the book I my sense uh in in in in speaking to people in the quantum world and then looking also at how in history new computing technologies have disseminated is that actually even revolution technologies they're implemented slowly and so suppose we get to the next couple of years the first practical use case of quantum computing it's going to be a years-long process of beginning to implement that in all sorts of different computing use cases and so I think we shouldn't expect to have you know a quantum powered iPhone anytime soon it's it's insane to think like on the one hand we have we have public companies that people are sort of speculating and betting on that yeah produce quantum computers but they really do nothing practical I you can go with you know I swing back and forth on the one hand it's like great it's amazing that our system will invest in such bleeding edge technology and give it a chance on the other hand for the shareholders you know like everybody could decide to give up on the effort with like higher interest rates it's I don't know it's a crazy function of uh the global economy well I think it's interesting that a lot of the companies that are investing the heaviest are also big cloud computing operators and that's I think because they believe that quantum will be actually most useful in a context where it's closely interlinked with huge volumes of classical computing and so actually we're going to need more advanced silicon ships to make quantum practically applicable and do you know is there any sense whether quantum and the GPUs sync up or where they fit and that story there are many different paradigms for how you specifically sync classical quantum computing but right now nobody knows which of many paradigms will win if any of them so wrap it up I mean what do you think the regular person should take from all this like it feels very far from their lives like they they maybe don't even know what it is like they try Chatchy BT their kids using it for homework it's not in their life yet like you know silicon valley loves to get itself worked up about things that don't sometimes translate don't like what do you think the lesson or the key thing for like the regular person right now is in terms of what's happening with these chips I think if you went back to 1965 which has been Gordon Moore first set out the the phrase Moore's Law and you asked what's the impact of of Moore's Law of the average person the the answer in the short one was approximately nothing but the answer in the long run is that it totally transforms society economy technology everything because we put computing and therefore some I conductors into basically every product that we rely on and I think we should expect the same to be true for AI you know what does it mean for me tomorrow probably not much but what is it going to mean in in 10 years and in 20 years when every aspect of human life is being impacted by it well it'll be transformative and all sorts of ways most of which we probably can't even imagine today for the startup entrepreneur where do you think there's opportunity to build a business you know they I mean maybe you think they should go out and start the next in video like it's a heavy lift but like where where do you think really with the progress that we've made in these chips that there's real business opportunities still why I think you've got the idea of the next in video you should absolutely go do it or America but I think you're right that you know companies like Nvidia they they create the infrastructure on which many different types of systems can be built and and if you go back to you know the smartphone for example smartphones were themselves a platform once you build lots of different things and I think we're still in the super early stages we're basically in stage zero in terms of figuring out what are the ways you can create products out of generative AI systems right yeah there are hardly any companies that make money selling the great systems and of course the challenge for entrepreneurs is you know people who built early iPhone apps and Facebook apps you know they don't all pan out sometimes you come to early you can be right that a technology is transformational and get the timing wrong and I guess that's the sort of confluence of luck and insight in in the business world it's a tough one to predict um Chris this was awesome thank you so much for coming on the shows great great to talk thanks for having me that's our episode on chips and big tech thanks for listening I'm Eric Newcomer your host author of Newcomer thanks so much to Max Child and James Willstriman co-founder Zavali in my long-time friends shout out to Scott Brody our producer Riley Cancelo my chief of staff Gabi Calliendo at Vali who's helping organize the conference and playing a big role behind the scenes thank you to Young Chomsky for the theme music please like comment subscribe on YouTube give me a review on Apple podcasts and subscribe to the sub-stack newcomer.co go try out Vali on your Alexa play song quiz maybe we're gonna have a couple more episodes before cerebral Valley cerebral Valley is on November 15th we will publish our conversations to YouTube we will probably publish them all that's what we did last time you can see the conversations from cerebral Valley one which was on March 30th and then yeah we'll play some of our favorites probably we did a sort of distilled version in our podcast feed so fall the podcast feeds for those and yeah I'll be covering it in the newcomer newsletter so newcomer.co thanks so much