Reasoning Models Explained: What They Are, How They Work, and When to Use Them Over Traditional LLMs

If you’ve been closely following the evolution of AI in recent years, you are probably aware of the various significant leaps that have advanced the capabilities of the technology. From the development of LLMs to the generative pre-trained transformer (GPT) that delivered the generative AI offerings that have taken the world by storm, innovation in the field has ramped up significantly in a relatively short space of time. It is an incredible journey that has precipitated the fast realization of AI-led value and potential across industries—and that journey continues.

The latest and most significant advancement comes in the form of reasoning models, which are, quite literally, bringing a more considerate element to the technology. Here’s how.

What Are They and How Do They Work?

When we think about AI-based research and development, artificial general intelligence (AGI) is generally the end point at which data science hopes to arrive. An autonomous machine capable of reacting to all outward stimulus with the ability to reason and perform tasks in a human-like way but outperforming us in every way. Given we are still a little way off—or a long way, depending on who you ask—achieving AGI, we can look at the various advances that are helping us move towards this eventuality. One such advancement is reasoning models.

Reasoning is a very human thing to do. If someone asks us to explain our reasoning when it comes to a decision we have made or an opinion we have expressed, we can do so by explaining the logical steps we made to get there.

Reasoning models perform these same steps when addressing a problem because they have been built to conduct multiple stages of thinking. This architecture evolved from a prompting technique known as Chain-of-Thought (CoT) prompting where a user will ask the LLM to break down a complex problem into logical steps to show in more detail how the resulting answer was reached.

When to Use Reasoning Models Over Traditional LLMs

For most use cases, traditional LLMs will be sufficient. But complex use cases, such as those that might relate to math or advanced data analysis, will be better served by reasoning models.

Just like when a human is faced with a more challenging question, reasoning models take longer to respond as they move through the stages of reasoning. This is perhaps the most significant trade-off when we consider the fundamental differences between LLMs like GPT-4o and reasoning models like Open AI o1 and o3, which is what is referred to as inference-time scaling.

The fact reasoning models take more time when considering problems means they also require more computational power. As with all scaling of this kind, there is an adjacent cost associated with the increase in the resources consumed to deliver the output. In some instances, reasoning models will generate more than 100 times the number of tokens that traditional LLMs generate when responding to a prompt. This factor alone will rule them out for organizations that must be a little more considerate of their budgets. Knowing when to choose a reasoning model over a more traditional LLM is therefore a key consideration, particularly as a reasoning model may actually be more efficient and less costly in certain situations.

When it comes to complex data analysis, for example, a user might call an LLM multiple times as they reanalyze data based on the new insights they gain with each step. Running the same prompt through a reasoning model, which will move through each of the stages automatically, may end up being more economical.

Reasoning models are built to think through problems step by step, which helps cut down on hallucinations—those moments when an AI confidently gives you the wrong answer. That’s why they’re a better fit for situations where accuracy really matters. Their structured approach also makes them great for stress-testing or validating complex solutions, like when you're working through a tricky math theorem.

Agents vs Reasoning Models

Agentic AI is growing significantly and there is a lot of overlap between agents and reasoning models. The most popular type of agentic framework today is ReAct, which stands for “reasoning” and “acting.” These agents essentially take on the role of prompting chain-of-thought reasoning so that a more certain output can be delivered. As is suggested with the “acting” element of these agents, they then perform the action they took the time to consider.

To some degree, they achieve the same thing as reasoning agents, but a reasoning model will not automatically action its suggestion. There is also the problem of producing multiple calls to an LLM, which again brings up the potential of costs spiralling.

Ultimately, choosing one over the other will be down to the use case and the level of control needed over decision-making. The key result, however, is that reasoning models, CoT prompting and ReAct agents are all part of a movement to make AI more accountable to us and itself.