Chinese LLM DeepSeek caused a significant disturbance in the US tech industry, leading to a loss of trillions of dollars from the stock market. Even though it was developed using somewhat older Nvidia hardware, it surprisingly performs quite well on AMD’s consumer product: the Radeon RX 7900 XTX. David McAfee, who oversees AMD’s Radeon division, shared some benchmark results on X.
Performance Insights
The Radeon RX 7900 XTX shows performance differences based on the model and the number of parameters being used. It can be up to 13% faster at 7 billion parameters and about 2% faster at 14 billion parameters. Beyond that point, the RDNA 3 flagship struggles, ultimately falling short compared to the RTX 4090 with 32 billion parameters. AMD even made a comparison with the GeForce RTX 4080 Super, where the 7900 XTX boasts a 34% advantage in performance.
Running DeepSeek Locally
AMD has also shared comprehensive guidelines on how to operate DeepSeek on your own computer. However, it’s important to note that the Radeon RX 7900 XTX has a limit of 32 billion parameters. On the other hand, the Strix Halo Ryzen AI Max 395 Plus, equipped with 128 GB of RAM, can handle up to 72 billion parameters. Additionally, for those who are willing to spend around $6,000, Matthew Carrigan has discovered a method to run the entire model locally on a system with dual AMD Epyc CPUs and 768 GB of RAM.
Source:
Link
