OpenAI appears to be facing a challenge in enhancing the performance of its upcoming AI models. The company’s next significant model, "Orion," is said to be lagging in certain tasks when compared to its earlier models.
Advantages in Language Tasks
While "Orion" excels in language-related tasks like translation and text generation, it has not performed well in areas such as coding. This inconsistency raises concerns about its overall effectiveness in diverse applications.
Challenges with Training Data
A report from The Information (cited by Gadgets360) indicates that there are difficulties in collecting training data for these new models. Additionally, running this model in data centers is more costly than operating GPT-4 and GPT-4o.
The improvement in quality is also not as pronounced as the advancements seen when moving from GPT-3 to GPT-4. OpenAI has formed a foundations team to tackle the training data issue, but it remains uncertain whether they will gather sufficient data in time for the model’s launch.
Broader Industry Trends
OpenAI isn’t alone in experiencing minimal performance improvements. Other companies like Anthropic and Mistral are also witnessing only slight advancements with each new release. One proposed strategy for boosting performance is to continue training the model after its initial release through fine-tuning, although this is merely a temporary fix rather than a sustainable solution.
Gadgets360, The Information