The Future of AI Compute: A Conversation With Jonathan Ross
How to build great AI hardware, large language models, and implications for the key players in AI
On Friday, I hosted a Space with Jonathan Ross, the founder and CEO of Groq - a company I invested in that is building custom chips for AI inference.
Jonathan, a former high-school dropout, entered the chip industry while working on ad optimization at Google’s New York office.
Jonathan overheard the speech recognition team complaining that they couldn't get enough compute. So he asked for some budget from Google and started putting together a chip-based machine learning accelerator for them.
During the day, Jonathan would work in the normal ads part of the business, and at night, he would work with the accelerator team.
After winning approval from Google, Jonathan and his team built a new chip called the Tensor Processing Unit, and began deploying it across Google’s data centers within a year.
The TPU was a huge success within Google, eventually underpinning more than 50% of all of Google’s compute power. When the other hyper-scalers learned of this success, they tried to hire Jonathan to build custom chips for them too.
During this process, it became increasingly clear to Jonathan that a gap would emerge between companies that had next-gen AI-compute and companies that didn’t. So he set out to build a chip that would be available to everyone and founded Groq.
I led Groq’s founding investment in 2016, and since then, Jonathan and his team have developed several types of AI hardware including the Language Processing Unit (LPU), a new type of silicon that is hyper-efficient at running inference for LLMs.
In our conversation on Friday, we discussed the founding story of Groq, how to build great AI hardware, large language models, and some of the implications for the key players in AI.
It’s one of the most interesting conversations I’ve had on AI with a lot of learnings.
You can listen to our conversation below:
can u pls upload this to spotify so that it would be easier to listen?
Chamath, this story feels like the Silicon Valley version of a superhero origin tale , Jonathan Ross, the unsung hero working on ad optimization, hears the cry for more compute power and decides, "Sure, I'll just whip up a custom chip for that!" The day-and-night hustle, the birth of the Tensor Processing Unit (TPU), and suddenly, Google's compute power jumps like it's on rocket fuel – all thanks to Jonathan's DIY superhero chip!
And then, realizing the power imbalance, Jonathan decides, "Why should only the big players have the cool toys?" and starts Groq. It's like the underdog superhero deciding to share their gadgets with the world! 🌍
Your investment in Groq is the venture capitalist equivalent of recognizing a superhero's potential before they even don the cape. The Language Processing Unit (LPU) is like the superhero's upgraded suit, hyper-efficient at running inference for Large Language Models (LLMs). It's like Jonathan turned the superhero gadgetry up to eleven!
I'm tuning in to the conversation, ready for some superhero-level insights into AI hardware, large language models, and the key players in AI. Keep bringing these origin stories to the forefront, Chamath!