Xiaomi’s large language model (LLM), known as MiLM, has successfully completed the registration process for large models, as announced on the company’s Weibo account.
With this milestone, Xiaomi indicates that MiLM is prepared for incorporation into its range of products, such as smartphones, smart home devices, and even Xiaomi automobiles. The announcement also hinted at the potential of expanding MiLM’s capabilities to a broader audience in the future.
Benchmark Achievements
MiLM made its public debut in August 2023 on benchmark platforms C-Eval and CMMLU, where it delivered impressive performance.
The model secured the top position within its parameter category on the C-Eval leaderboard and ranked 10th overall. According to the project’s GitHub page, MiLM-6B, the specific variant in question, boasts 6.4 billion parameters.
Subject-Specific Performance
C-Eval’s subject-specific breakdown showcases MiLM-6B’s proficiency in STEM fields (Science, Technology, Engineering, and Mathematics). The model achieved high accuracy scores across all 20 STEM subjects, including metrology, physics, chemistry, and biology.
While MiLM-6B shows strong performance in most liberal arts subjects, areas requiring “abstract thinking” like law, mathematics, programming, and probability theory appear to need further development.
Social Sciences and Humanities
In the realm of social sciences, MiLM-6B demonstrated good accuracy in eight out of ten subjects, with education and geography being the exceptions. As for the humanities, the model performs admirably in history and law, though the accuracy in other subjects is yet to be fully assessed.
With MiLM-6B overcoming significant hurdles, it’s now set to be integrated into various Xiaomi products. Despite its varied performance across different subjects, it shows promise for enhancing user experiences in a wide range of applications.
