Key Takeaways
1. OpenAI launched gpt-oss-120b and gpt-oss-20b, two open-weight models available for free download to run locally.
2. The gpt-oss-120b has 117 billion parameters and requires 80GB VRAM, while gpt-oss-20b has 21 billion parameters and needs 16GB VRAM.
3. The models are licensed under Apache 2.0 and aim to support developers, researchers, and businesses with cost-effective AI resources.
4. gpt-oss-120b performed significantly well in coding tests and excels in health and math inquiries, while gpt-oss-20b matches the performance of some existing models.
5. Both models have higher rates of hallucinations compared to other reasoning-focused models and can be deployed for free on various platforms.
OpenAI has revealed the launch of gpt-oss-120b and gpt-oss-20b, two models with open weights that are available for free download, allowing users to run them locally on their machines. This marks the company’s first open-source release since the introduction of GPT-2 in 2019.
Model Specifications
The gpt-oss-120b is a robust model with 117 billion parameters, necessitating a powerful 80GB of VRAM for operation. On the other hand, the more compact gpt-oss-20b, which consists of 21 billion parameters, can be accommodated on a single GPU with just 16GB of VRAM. Both models come under a versatile Apache 2.0 license.
OpenAI states that this release represents a significant advancement in their dedication to the open-source community, aligning with their mission to make AI’s advantages widely available. The company envisions these models as cost-effective resources for developers, researchers, and businesses to efficiently operate and tailor to their needs.
Performance Insights
How do they perform? The gpt-oss-120b achieved a score of 2622 on the Codeforces coding test with tools, performing nearly as well as the company’s o3 and o4-mini models, and surpassing o3-mini comfortably in both assessments, with a score of 2643 without tools.
The gpt-oss-20b registered a score of 2516 with tools, matching the performance of o3 and o4-mini, and 2230 without tools, slightly outperforming o3-mini. OpenAI claims that the 120b model excels in health-related inquiries and mathematics compared to the o4-mini, while the 20b model outperforms the o3-mini.
Limitations in Reasoning
OpenAI notes that both models, the 120b and 20b, tend to generate hallucinations more frequently than reasoning-focused models like o3 and o4-mini. In their evaluations, they discovered that the open-weight models hallucinated between 49% and 53% on internal benchmarks that assess their knowledge about individuals.
Both models can be accessed via the official Hugging Face space and are natively quantized in MXFP4 for optimized performance. They can also be deployed at no cost on platforms including Microsoft Azure, Hugging Face, vLLM, Ollama, llama.cpp, LM Studio, AWS, Fireworks, Together AI, and several others.
OpenAI anticipates that these models will “reduce barriers for emerging markets, resource-limited sectors, and smaller organizations that might not have the funding or adaptability to utilize proprietary models.”
Regarding their decision to open-source a new model six years after the previous one, the company emphasizes its goal to “make AI broadly accessible and beneficial for everyone.”
Source:
Link
Leave a Reply