Inside the AI Chip Race with Groq
How is Groq positioning itself in response to NVIDIA's dominance? What companies are running profitable AI inference workloads today? How are geopolitics shaping the AI compute infrastructure?
Yesterday, I hosted a Spaces with Jonathan, Sunny, and Gavin from Groq, to discuss the current landscape of AI chips and inference.
You can listen to the conversation below:
Timestamps and topics covered:
(0:00 - 9:50) Jonathan Ross shares his journey from high school dropout to creating Google's Tensor Processing Unit (TPU), which eventually led him to found Groq.
(9:50 - 14:29) How Groq tackles AI hardware challenges by focusing on compilers and specialized hardware, and how their chips balance efficiency with adaptability.
(14:29 - 28:08) NVIDIA's dominance in training, its moat-like effect on the industry, and how companies like Groq are working to create alternatives in the inference space.
(28:08 - 39:03) Real-world examples where companies are running profitable AI inference workloads today in industries such as finance, advertising, and customer service.
(39:03 - 47:20) Recent breakthroughs and competition among AI model developers like DeepSeek, Alibaba, Anthropic, and OpenAI, as well as a discussion about the shortcomings of today’s benchmarks for language models.
(47:20 - 59:50) How geopolitical dynamics, especially between the U.S. and China, influence decisions around AI hardware manufacturing, supply chains, and international expansion.
(59:50 - 1:08:54) Infrastructure challenges, energy constraints, and Groq's solutions for scaling AI data centers globally.
(1:08:54 - 1:09:39) Audience Questions:
(52:52) How will AI transform roles like product management, data analytics, and product design?
(55:16) How can job applicants without traditional qualifications stand out in an AI-driven world?
(58:44) What are Groq’s perspectives and plans around confidential computing and trusted execution environments (TEEs)?
(1:01:05) How is Groq addressing security challenges related to open-source AGI and rogue AI?
(1:03:05) Are inference cloud providers partnering or collaborating with venture capital firms to identify promising startups?
(1:05:08) What emerging chip architectures in the 14nm space are best suited for new AI reasoning models?
(1:07:45) Are there any plans for Groq data centers to be powered by renewable energy sources?
Your content is great. I really appreciate what you put out.
In your deep dive on Groq, you highlight another form of infrastructure where incumbents may be bottlenecks—this time on the compute side.
If software infrastructure already struggles to evolve, how much more rigid is the chip ecosystem, where incumbents control everything from design cycles to developer mindshare?
Are we in another moment where hardware gatekeepers are silently blocking the next-gen platform—not by intention, but by the sheer weight of legacy architectures and economic incentives?
It’s a similar pattern: the core is profitable, but inflexible. And again, the question isn’t whether innovation is possible—it’s whether the ones with power are willing to let it through.
(Okay—my last comment for the weekend. Back to living in a place where innovation is a constant tug-of-war… and writing over 1,000+ test scripts just to prove (e.g., my prompts) something revolutionary can be safe. ✌️)