NVIDIA is the world’s most valuable company (P/E ratio of 75; compare to less than 40 for Microsoft), which also means that nearly every NVIDIA employee is rich. A lot of people slack off when they become rich. Google ended up with quite a few “vesting in peace” workers who didn’t contribute much. It didn’t matter because it was too challenging for anyone else to break into the search and advertising businesses. But suppose that another tech company assembles a group of not-yet-rich hardware and software people. Hungry for success, these people build some competitive GPUs and the biggest NVIDIA customers merely have to recompile their software in order to use the alternative GPUs that are marketed at a much lower price.
How can NVIDIA’s spectacular success not lead to marketplace slippage due to an excessively rich and complacent workforce? Is the secret that NVIDIA can get money at such a low cost compared to competitors that it can afford to spend 2-3X as much on the next GPU and still make crazy profits? I find it tough to understand how Intel, which for years has made GPUs inside its CPUs, can’t develop something that AI companies want to buy. Intel has a nice web page explaining how great their data center GPUs are for AI:
Why can’t Intel sell these? Are the designs so bad that they couldn’t compete with NVIDIA even if sold at Intel’s cost?
Nvidia has the advantage of being trivially easy to configure to run with pytorch.
Supposedly the software works with AMD: https://pytorch.org/blog/experience-power-pytorch-2.0/
I’d argue Google is an outlier. You see fewer lazy employees at Apple, Microsoft, Netflix, Amazon, and even Facebook (arguably the most similar to Google).
I think this is because AdWords, a product fundamentally unchanged since its invention about two decades ago, continues to provide the vast majority of Google’s revenue. Over two thirds. YouTube, DoubleClick display ads, and AdSense are all basically rounding error as are the various subscription businesses like Google Apps as is Goog Cloud.
This means the vast majority of employees contribute little or nothing to the bottom line. For some reason google has historically chosen to subsidize their massive salaries and expensive benefits off AdWords profits. This is probably fine with Wall Street because those profits are so massive because google is basically a search monopoly and AdWords is a money printing machine.
Nvidia has lock in thanks to CUDA but margins are still thinner in semiconductors and competition historically cutthroat. Some people will probably rest and best but Jensen Huang has been tested in ways Larry and Sergei never were. They founded their company in grad school and stumbled into how to pay for it a few years later then it was off to the races. Nvidia in contrast fought for its life several times.
Chinese AI firms have cut their prices by 90%. Some offer a 99.8% discount to OpenAI’s GTP-4. So maybe their models are becoming much more efficient? Or need much less hardware once trained? Either way, if the trend continues it lowers Nvidia’s long term outlook.
https://www.zerohedge.com/technology/what-chinese-ai-tells-us-about-semi-market
The CUDA ecosystem they created is so massively intertwined at the lowest level with their own GPU’s, it’s not practical to port anything to another GPU. OpenCL supposedly has achieved some portability, but there are hundreds of operators which would have to be rewritten for openCL.
Ironically, AI could quickly and easily port all code to any different GPU.
Looks like my response to David in https://philip.greenspun.com/blog/2024/06/17/europe-is-rich-in-migrants-so-why-isnt-it-rich/ stuck in the blog’s re-education camp filters. Since it has no officially objectionable statements and it has some supportive numbers that took me some time to produces, please restore.
The worst it has deserving wisecracks and some of the statements could be interpreted as de-regulatory.