Do OpenAI's New Models Have a Hallucination Problem?

Earlier this week, OpenAI announced the release of a pair of models, o3 and o4-mini. In announcing them, the company referred to them as “the smartest models we’ve released to date” and noted that they had been “trained to think for longer before responding.” The examples listed included solving complicated math equations and researching shipping data based on a single photograph.

There’s one more big difference from these models and their predecessors, according to TechCrunch’s Maxwell Zeff: these models are hallucinating more than earlier iterations. Zeff pointed out that this is in sharp contrast to earlier updates to OpenAI’s models, which have historically had fewer hallucinations than those that predated them.

Writing at CNET last year, Lisa Lacy provided a good overview of what “hallucination” means in the context of AI. (It’s a bit different than its use in pop cultural contexts.) Hallucinations, Lacy writes, occur to when an AI system gives incorrect information to its user. She cites examples that range from describing historical events that never occurred to misidentifying the parties in a legal case. Hallucinations have been an issue that AI engineers and programmers have been reckoning with for years.

Looking for ChatGPT-Generated Erotica? It Just Got Easier to Produce.

It’s a challenging balance to achieve

What’s behind this uptick in hallucinations with OpenAI’s new models? Neil Chowdhury of the AI research lab Transluce told TechCrunch that the lab suspected “the kind of reinforcement learning used for o-series models” was responsible — and that it could worsen certain issues that “standard post-training pipelines” usually address.

A spokesperson for OpenAI told TechCrunch that the company was committed to dealing with hallucinations and improving its software. Now that these models are out in the world, we’ll see how people react to their features — and the answers, correct and incorrect, that they come up with.