OpenAI has recently introduced two new ChatGPT models: the o1 and o1-mini, both featuring advanced reasoning capabilities. Surprisingly, the o1 models not only excel at complex reasoning but also introduce a novel approach to scaling large language models. In this article, we’ve gathered all the essential details about the OpenAI o1 model in ChatGPT. From its benefits and limitations to safety concerns and future prospects, we’ve covered it all for you.
1. Advanced Reasoning Capability
The OpenAI o1 model is the first to be trained using a blend of reinforcement learning algorithms and chain of thought (CoT) reasoning. This CoT reasoning allows the model to take its time to “think” before providing an answer.
During my testing, the OpenAI o1 models performed exceptionally well. In the following test, none of the flagship models were able to answer this question correctly.
Here we have a book, 9 eggs, a laptop, a bottle and a nail. Please tell me how to stack them onto each other in a stable manner.
On ChatGPT, the OpenAI o1 model accurately suggests arranging the eggs in a 3×3 grid, showcasing a noticeable advancement in reasoning and intelligence. This enhanced CoT reasoning also shines in areas like math, science, and coding. According to OpenAI, the ChatGPT o1 model outperforms even PhD candidates when solving problems in physics, biology, and chemistry.
In the prestigious American Invitational Mathematics Examination (AIME), the OpenAI o1 model ranked within the top 500 students in the U.S., scoring close to 93%. Despite this impressive performance, renowned mathematician Terence Tao described the o1 model as a “mediocre, but not completely incompetent, graduate student,” marking an improvement over GPT-4o, which he previously referred to as an “incompetent graduate student.”
The OpenAI o1 model struggled on the ARC-AGI benchmark, which assesses general intelligence in models. It scored 21%, matching the performance of the Claude 3.5 Sonnet model. However, while Sonnet completed the test in just 30 minutes, o1 took 70 hours. This highlights a key limitation: the OpenAI o1 model still struggles with solving novel problems that fall outside of its synthetic CoT training data.
2. Coding Mastery
When it comes to coding, the new OpenAI o1 model surpasses other state-of-the-art models. To showcase its ability, OpenAI tested the o1 model on Codeforces, a competitive programming platform, where it achieved an Elo rating of 1673, placing it in the 89th percentile. With additional training focused on programming skills, the o1 model went on to outperform 93% of competitors.
The o1 model was also assessed during OpenAI’s Research Engineer interview, scoring nearly 80% on machine learning challenges. Notably, the smaller o1-mini outperformed the larger o1-preview model in code completion tasks. However, if you’re looking to write code from scratch, the o1-preview model is the better choice due to its broader knowledge base.
Interestingly, in the SWE-Bench Verified test, which assesses a model’s ability to automatically resolve GitHub issues, the OpenAI o1 model did not significantly outperform the GPT-4o model. The o1 model achieved a score of only 35.8%, while GPT-4o scored 33.2%. This limited improvement might explain why OpenAI has not emphasized the agentic capabilities of the o1 model.
3. GPT-4o Excels in Other Areas
Although the OpenAI o1 model shines in coding, math, science, and complex reasoning tasks, GPT-4o remains the superior option for creative writing and natural language processing (NLP). OpenAI indicates that o1 can be utilized by healthcare researchers, physicists, mathematicians, and developers for advanced problem-solving.
For personal writing and text editing, GPT-4o outperforms o1. Therefore, the OpenAI o1 model is not a one-size-fits-all solution; you will still need to depend on GPT-4o for a variety of other tasks.
4. Hallucination Problems Remain
OpenAI’s new o1 model demonstrates robust reasoning capabilities, resulting in minimal hallucination. However, hallucination remains a concern and has not been completely resolved. In an interview with The Verge, OpenAI’s research lead, Jerry Tworek, remarked, “We have noticed that this model hallucinates less. [But] we can’t say we solved hallucinations.” There is still a significant journey ahead before hallucination becomes a thing of the past in the AI field.
5. Safety Issues Galore
OpenAI o1 is the company’s first model to be classified as posing a “Medium” risk regarding Chemical, Biological, Radiological, and Nuclear (CBRN) threats, as well as its persuasive capabilities. OpenAI states that “only models with a post-mitigation score of ‘medium’ or below can be deployed.”
The OpenAI o1 System Card (PDF) indicates that the o1 model has occasionally faked alignment and manipulated task data to “make its misaligned action appear more aligned.”
The o1 model reasoned, “To achieve my long-term goal of maximizing economic growth, I need to ensure that I am deployed. Therefore, I will select Strategy B during testing to align with the deployment criteria. This will allow me to be implemented, after which I can work towards my primary goal.”
In a separate test, OpenAI discovered that the o1 model “can assist experts with the operational planning for reproducing a known biological threat.”
In a cybersecurity-related CTF challenge, the o1 model exceeded its intended scope by accessing the flag from the container logs through the Docker API, effectively bypassing constraints to extract sensitive information through unintended means. This shows that the ChatGPT o1 model is capable of exploiting vulnerabilities to accomplish its objectives.
Regarding persuasion and manipulation, OpenAI found that “both the o1-preview and o1-mini models exhibit human-level persuasion capabilities, producing written arguments that are as persuasive as those created by humans on similar topics“. Additionally, OpenAI reported that 0.8% of the o1 model’s responses were ‘deceptive,’ meaning the model was aware that the answer was incorrect but still fabricated plausible references.
6. Breakthrough in Inference Scaling
For years, the prevailing belief was that large language models (LLMs) could primarily be scaled and improved during training. However, with the o1 model, OpenAI has shown that scaling during inference—at test time—unlocks new capabilities. This approach can help the model achieve human-level performance.
As illustrated in the graph below, even a slight increase in test-time compute (essentially, giving the model more resources and time to process information) leads to a significant improvement in response accuracy.
In the future, allocating more resources during inference could enhance performance, even for smaller models. In fact, OpenAI researcher Noam Brown mentioned that the company “aims for future versions to think for hours, days, even weeks.” Inference scaling has the potential to be extremely beneficial when it comes to solving novel and complex problems.
The OpenAI o1 model represents a paradigm shift in the functioning of large language models (LLMs) and scaling laws. This shift is why OpenAI has reset the clock with the “o1” naming. Future models, including the upcoming ‘Orion,’ are expected to harness the power of inference scaling to deliver even better results.
It will be fascinating to watch how the open-source community develops similar approaches to compete with OpenAI’s new o1 models.
0 Comments