Meta has just announced a new update on Llama 4, their upcoming language model. During a recent earnings call, CEO Mark Zuckerberg revealed that they are training Llama 4 using a setup with over 100,000 Nvidia H100 GPUs. This is a larger setup than anything previously reported by competitors.
Upcoming Features of Llama 4
This new language model is set to launch in early 2025, starting with its smaller versions first. While specific capabilities are not fully disclosed, Zuckerberg suggested that Llama 4 will have enhanced features, improved reasoning abilities, and will operate at a quicker pace overall.
Meta's Unique Strategy
Meta continues its strategy of offering models for free download, unlike OpenAI and Google, which restrict access through APIs. This makes Llama 4 particularly appealing for startups and researchers who prefer more flexibility in using AI technologies.
Significant Energy and Financial Implications
Given the extensive computing resources, the energy requirements are also considerable, estimated at around 150 megawatts—this is five times more than what the largest supercomputer at a U.S. national lab consumes. Meta's financial plan reflects this ambitious scale, with infrastructure expenditures projected to hit $40 billion in 2024, marking a 42 percent increase from 2023.
Despite these hefty expenses, Meta's financial health remains strong, showing a 22 percent rise in sales, primarily driven by advertising revenue, which has helped offset a 9 percent increase in operating costs.
