ReAct Paper Explained: Reasoning and Acting in LLMs

Introduction

By 2022, large language models could reason through problems step by step with chain-of-thought promptingClick to read tutorial: Chain-of-Thought Prompting Paper Explained, but only using what they already knew - and when their knowledge was wrong or missing, the reasoning confidently built on hallucinatedClick to read tutorial: LLM Tokens, Context Windows, Embeddings and Hallucinations facts. Separately, researchers had used language models to choose actions in interactive environments, but without explicit reasoning. The 2022 paper ReAct: Synergizing Reasoning and Acting in Language Models, from Princeton and Google, combined the two: the model alternates between thinking about what to do, acting - for example by searching Wikipedia - and reading the observation that comes back. This thought-action-observation loop is the pattern behind today's AI agents. This post explains the ReAct paper with diagrams. We will compare it with reasoning-only and acting-only approaches, walk through a ReAct trace, go through the results and failure modes, and build a ReAct agent loop in Java.
The paper: ReAct: Synergizing Reasoning and Acting in Language Models
Authors: Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, Yuan Cao (Princeton University, Google Research)
Published: October 2022 - ICLR 2023

Reason only, act only, or both

Chain-of-thought reasons without acting; act-only takes actions without reasoning; ReAct interleaves thoughts, actions and observations Reason only (CoT) Thought Thought Answer uses only what the modelremembers - may hallucinate Act only Action Observation Action Observation Answer no plan - loses track ofwhat it has found so far ReAct Thought Action Observation Thought ... (repeat) Answer plans with thoughts, checksfacts with actions
Three ways of prompting a model, as compared in the paper.
In ReAct, the model's possible actions are extended with a "thought" action, which does not change the outside world - it is just text the model writes to itself. Thoughts let the model -
  • break a task into steps and track its progress ("I need to find X, then compare it with Y"),
  • pull out the important part of an observation ("The paragraph says it was released in 2018"),
  • use common sense or arithmetic on what it has found, and
  • notice when something went wrong and change its plan ("The search found nothing; I should search for a broader term").
Actions, in turn, let the model fetch real information instead of relying on memory - which grounds its reasoning and reduces hallucination.

The ReAct loop and the paper's setup

The language model writes a thought and an action; the tool or environment executes the action and returns an observation, which is appended to the prompt; the loop repeats until the model calls finish Language modelprompt = examples + task+ trajectory so far Tools / environmentsearch[ ], lookup[ ],APIs, a game, a website Thought + Action Observation (appended to the prompt) finish[answer]
The ReAct loop. Every thought, action and observation is added to the prompt for the next step.
The model is not fine-tuned. Like the few-shot prompts of GPT-3Click to read tutorial: GPT-3 Paper Explained: Language Models Are Few-Shot Learners, it is prompted with a few complete example trajectories written by people - sequences of thoughts, actions and observations which solve a task - followed by the new task. The paper mostly used PaLM-540B, and evaluated four tasks -
  • HotpotQA - questions which need facts from two or more Wikipedia articles, and
  • FEVER - checking whether a claim is supported or refuted by Wikipedia,
both using a deliberately simple Wikipedia API with three actions: search[entity], which returns the first sentences of the entity's page, lookup[string], which finds the next sentence on the page containing the string, and finish[answer]; and -
  • ALFWorld - a text-based game of household tasks, such as "put a clean mug on the desk", and
  • WebShop - buying a product matching a description on a simulated shopping website.
A trace for a multi-hop question looks like this (an illustrative example, using the paper's format) -
Question: How many years after Java was first released did Java 11 come out?
Thought 1: I need the year Java was first released, then the year of Java 11.
Action 1: search[Java (programming language)]
Observation 1: Java is a programming language ... It was first released by Sun Microsystems in 1995.
Thought 2: Java was first released in 1995. Now I need Java 11's release year.
Action 2: search[Java 11]
Observation 2: Java 11 was released in September 2018 and was a long-term support release.
Thought 3: 2018 - 1995 = 23, so Java 11 came out 23 years later.
Action 3: finish[23]

Results

  • Interactive tasks: on ALFWorld and WebShop, ReAct prompted with only one or two examples beat imitation learning and reinforcement learning methods trained on thousands of examples, by an absolute 34% and 10% in success rate. Act-only prompting did much worse, showing that the thoughts mattered.
  • Fact verification: on FEVER, ReAct (60.9% accuracy) beat chain-of-thought prompting (56.3%).
  • Question answering: on HotpotQA, ReAct alone was slightly behind chain-of-thought in accuracy, because a bad search result could send it off track. But its answers were far more grounded: in the authors' analysis, hallucination caused over half of chain-of-thought's failures, and only a small fraction of ReAct's. The best results came from combining the two - falling back from ReAct to chain-of-thought with self-consistency when ReAct did not finish within a set number of steps, and the other way round when the chain-of-thought answers disagreed.
  • Interpretability and control: because the reasoning is visible, a person can read why the model acted, and even edit a thought mid-trajectory to correct its course.
  • Fine-tuning: smaller PaLM models fine-tuned on a few thousand ReAct trajectories outperformed much larger models that were only prompted.

Failure modes

The paper's error analysis is still a good checklist for anyone building agents -
  • Repetitive loops: the model sometimes repeats the same thought and action over and over, unable to break out.
  • Uninformative observations: a search which returns nothing useful can derail the reasoning, and the model may fail to recover.
  • Reasoning errors: the model can still misread an observation or draw a wrong conclusion from correct facts.
Practical agents guard against these with a maximum number of steps, detection of repeated actions, and good error messages from tools.

A ReAct agent loop in Java

The program below implements the ReAct loop for the Java trace above. The tools are plain Java functions, registered by name in a map of Function<String, String>. To keep the example runnable without an API key, the language model is replaced by a scripted stand-in which returns the steps of the trace above, with one change - it uses a calculate tool for the subtraction instead of doing it in a thought; in a real agent, callModel() would send the prompt to an LLM API, for example with Java 11's HttpClient. The loop itself - parse the action, run the tool, append the observation, repeat until finish or a step limit - is the same either way.
A ReAct agent loop with tools
//ReActAgent.java
import java.util.*;
import java.util.function.Function;
import java.util.regex.*;

public class ReActAgent {
  // Tools: name -> function from argument to observation
  static final Map<String, String> WIKI = Map.of(
      "java (programming language)", "Java is a programming language. It was first released by Sun Microsystems in 1995.",
      "java 11", "Java 11 was released in September 2018 and was a long-term support release.");
  static final Map<String, Function<String, String>> TOOLS = Map.of(
      "search", q -> WIKI.getOrDefault(q.toLowerCase(), "Could not find [" + q + "]. Try a different search."),
      "calculate", expr -> {
        String[] p = expr.split("-");
        return String.valueOf(Integer.parseInt(p[0].trim()) - Integer.parseInt(p[1].trim()));
      });

  // Stand-in for the LLM: returns the next "Thought ... Action ..." for the current prompt
  static final List<String> SCRIPT = List.of(
      "Thought: I need the year Java was first released, then the year of Java 11.\nAction: search[Java (programming language)]",
      "Thought: Java was first released in 1995. Now I need Java 11's release year.\nAction: search[Java 11]",
      "Thought: Java 11 was released in 2018. I will compute the difference.\nAction: calculate[2018 - 1995]",
      "Thought: The difference is 23 years.\nAction: finish[23]");
  static int calls = 0;
  static String callModel(String prompt) { return SCRIPT.get(calls++); }

  static final Pattern ACTION = Pattern.compile("Action: (\\w+)\\[(.*)\\]");

  public static void main(String[] args) {
    String prompt = "Question: How many years after Java was first released did Java 11 come out?\n";
    final int maxSteps = 6;
    for (int step = 1; step <= maxSteps; step++) {
      String output = callModel(prompt);
      System.out.println(output);
      Matcher m = ACTION.matcher(output);
      if (!m.find()) { prompt += output + "\nObservation: Invalid action format.\n"; continue; }
      String tool = m.group(1), argument = m.group(2);
      if (tool.equals("finish")) {
        System.out.println("=> Final answer: " + argument + " (after " + step + " steps)");
        return;
      }
      String observation = TOOLS.containsKey(tool) ? TOOLS.get(tool).apply(argument) : "Unknown tool " + tool;
      System.out.println("Observation: " + observation);
      prompt += output + "\nObservation: " + observation + "\n";   // the trajectory grows each step
    }
    System.out.println("=> Stopped: no answer within " + maxSteps + " steps");
  }
}
 OUTPUT of the above code
Thought: I need the year Java was first released, then the year of Java 11.
Action: search[Java (programming language)]
Observation: Java is a programming language. It was first released by Sun Microsystems in 1995.
Thought: Java was first released in 1995. Now I need Java 11's release year.
Action: search[Java 11]
Observation: Java 11 was released in September 2018 and was a long-term support release.
Thought: Java 11 was released in 2018. I will compute the difference.
Action: calculate[2018 - 1995]
Observation: 23
Thought: The difference is 23 years.
Action: finish[23]
=> Final answer: 23 (after 4 steps)

Explanation of the code

  • TOOLS maps each action name to a Function<String, String>. Adding a tool, such as a database query or a REST call, means adding one entry - this is essentially what agent frameworks such as LangChain4j and Spring AI do with annotated Java methods.
  • The regular expression extracts the tool name and argument from the model's text. Modern LLM APIs offer tool calling, where the model returns a structured tool call in JSON instead, which is more reliable than parsing text.
  • After each step, the model's output and the observation are appended to prompt, so the next call sees the whole trajectory - the model has no memory of its own between calls. Long trajectories therefore fill the context windowClick to read tutorial: LLM Tokens, Context Windows, Embeddings and Hallucinations, which is why agents also limit or summarize old steps.
  • maxSteps guards against the repetitive loops described above. The search tool also returns a helpful message when nothing is found, so a model can recover by trying a different search.

ReAct's legacy

ReAct's loop is the core of most AI agents today: coding agents that read files, run tests and fix errors; research agents that search and read the web; and assistants that call company APIs. What changed since 2022 is mostly the plumbing - native tool calling in model APIs, standard protocols such as the Model Context Protocol (MCP) for connecting tools, and models trained specifically to plan and use tools - while the idea of interleaving reasoning with actions and observations remains the same. It also pairs naturally with RAGClick to read tutorial: RAG Paper Explained: Retrieval-Augmented Generation: an agent can decide for itself when, and what, to retrieve, using the same retrieval steps described in RAG architecture explainedClick to read tutorial: RAG Architecture Explained: Chunking, Embeddings, Retrieval. And the step-by-step chains that the loop is built from are the subject of prompt engineering techniquesClick to read tutorial: Prompt Engineering Techniques Explained with Examples.

Summary

In this post we went through the ReAct paper. We saw how ReAct interleaves thoughts, which plan and track progress, with actions, which fetch real information from tools or an environment, and observations, which feed the results back into the prompt. We compared it with reasoning-only chain-of-thought and acting-only approaches, followed a ReAct trace for a multi-hop question, and went through the results on HotpotQA, FEVER, ALFWorld and WebShop, along with the failure modes found in the paper. Lastly, we built the ReAct loop in Java, with tools as functions, a step limit, and a growing trajectory. In the next post, we will see how a model can be trained to reason, in the DeepSeek-R1 paperClick to read tutorial: DeepSeek-R1 Paper Explained: Reasoning via RL.