Amazon is set to launch its next-generation Trainium 2 AI chip next month, promising performance improvements of up to four times compared to the original version.
Development and Acquisition
According to The Financial Times, the chip was designed by Annapurna Labs, a microelectronics startup from Israel that Amazon bought in 2015. The goal behind this acquisition is to develop chips that can compete with Nvidia's products and lessen the company's reliance on Nvidia.
In an interview with Financial Times, Dave Brown, who is the vice-president of compute and networking services at AWS, mentioned their ambition to be "absolutely the best place to run Nvidia, but at the same time we think its healthy to have an alternative."
Market Dynamics
Nvidia currently dominates the AI chip market, holding an impressive 80% share and recording $26.3 billion in revenue from chip sales in the second quarter of fiscal 2024. Amazon plans to invest around $75 billion in capital throughout 2024, primarily for technology and infrastructure, with expectations to increase spending in 2025.
While Amazon doesn't provide independent performance benchmarks for its chips, it stated that its "Inferentia" chips, which are designed for specific tasks, cost 40% less to operate when generating AI responses.
System Integration
As Remi Sinno, a former employee of Softbank and Intel who is now the engineering director at Annapurna, puts it, "It's not [just] about the chip, it's about the full system."
Sinno also added, "it's really hard to do what we do at scale. Not too many companies can."
