Key Takeaways
1. Intel unveiled the Arc Pro B65 and B70 GPUs, targeting AI training and inference, but did not provide specific performance metrics initially.
2. The Arc B70 GPU features 32 GB of VRAM at a starting price of $949, compared to Nvidia’s RTX 4000 Pro with 24 GB of VRAM priced at $1,800.
3. The B70 has a context window 2.2 times larger than the RTX 4000 Pro, supporting up to 93K tokens versus 42K tokens for Nvidia’s card.
4. The B70 shows 85% greater token throughput and 6.2x faster response times in multi-agent flows, attributed to Intel’s oneAPI and software stack.
5. Performance advantages for the B70 increase with multi-GPU configurations, achieving up to 2x tokens per dollar spent in various setups.
Intel unveiled the Arc Pro B65 and B70 high-performance workstation GPUs yesterday, targeting AI LLM training and inference, but did not share specific performance details in their first press release. Shortly after, the company put out performance charts comparing the B70 against Nvidia’s older RTX 4000 Pro GPUs. Intel believes this comparison is relevant due to the B70’s price of less than $1,000, with Team Blue making bold claims about its overall performance.
Impressive Specs
Even without diving into the performance charts—which may or may not reflect reality in typical Intel fashion—the Arc B70 GPU stands out from Nvidia’s offerings because of its higher VRAM capacity (32 GB compared to 24 GB) and a starting price of $949 (compared to $1,800). Sure, the VRAM is only 19 Gbps GDDR6 with a 256-bit bus and a bandwidth of 608 GB/s, but the increased capacity is beneficial for training and inferring larger models.
Context Window Advantage
Thanks to its superior VRAM, the B70 has a context window that is 2.2 times larger than that of the RTX 4000 Pro card. Intel’s slide indicates that the B70 can handle context lengths up to 93K tokens, while the RTX 4000 Pro hits its memory limit at about 42K tokens when working with the Llama 3.1 8b BF16 model.
Enhanced Performance
Running parallel multi-agent flows on the Ministral Instruct 2410 8B (BF16) model results in 85% greater token throughput for multiple users or requests when using the B70 in a Linux OS environment. Moreover, the B70 can provide responses faster with a 6.2x quicker time to first token. Intel attributes these enhanced speeds to their refined oneAPI and proprietary software stack. The performance advantage over Nvidia also scales up with multi-GPU configurations, as Intel states that they can achieve up to 2x tokens for each dollar spent in single, dual, and quad GPU setups.
Sadly, the presentation did not include any performance metrics or pricing for the B65 GPU. It would be fascinating to see if Intel permits AIB partners to create gaming-focused versions of the B65 and B70 GPUs with somewhat lower VRAM capacities. This would also necessitate Intel to update their graphics drivers.
Source:
Link