Simple ReAct Agent using LangChain and OpenRouter

Authoerd by Dr.Tiziana Ligorio for AI Agents - CSCI 395.32 taught at Hunter College of The City University of New York¶

This demo has two parts.¶

In Part 1¶

We build a simple ReAct agent using the LangChain framework to illustrate how much a framework can abstract and automate compared to our previous demo, where we implemented the agent loop from scratch.

Note that, because a lot of the agent logic is built within the framework, as a designer, you have much less control over context engineering. We will discuss context engineering in depth in future lectures.

As before, the agent follows the ReAct pattern by iteratively alternating between reasoning and acting to accomplish a task. However, in this version most of the control flow, such as managing the reasoning loop, invoking tools, and handling intermediate state, is handled by the framework (and thus standardized) rather than implemented directly by the agent developer.
Here are the LangChain docs for your reference.

For this tutorial, we will demonstrate access to search tools (web search and Wikipedia), which it can use to gather information and answer user queries. We will then show how to include a tool for fetching via MCP, and a custom tool for filtering.

In Part 2¶

We demonstrate how to trace and run evals using LangSmith. This part will be covered after our lecture on evals.
Here are the LangSmith docs for your reference.

Installs and Imports¶

%%capture hides the output

OpenRouter is a unified API that provides access to various LLMs through a single interface. It offers a generous free tier and affordable token usage for minimal cost, making it ideal for learning and experimentation.

If you already pay for other LLM providers or prefer to use a different service, you are welcome to adapt the code accordingly.

Setup your API Keys¶

Step 1 — Get an OpenRouter API key¶

For this demo we will use an LLM via OpenRouter, which requires an API key.

1. Go to https://openrouter.ai

2. Sign in (or create an account if you don’t have one)

3. Once logged in, navigate to https://openrouter.ai/settings/keys

4. Click Create Key

5. Give the key a name, e.g. colab-langchain_ReAct_agent

6. Copy the key immediately (you won’t be able to see it again)

Important: Treat this key like a password. Do not share it, paste it into notebooks, or commit it to GitHub.

Step 2 — Add a secret in Colab (UI)¶

1. On the left sidebar, click 🔑 Secrets

2. Add a new secret:

• Name: OPENROUTER_API_KEY

• Value: your actual API key

3. Toggle the switch to the left to give notebook access (you should see a checkmark)

If running locally — Add a secret in .env¶

1. Create a .env file in the project root:

touch .env

2. Add the following (replace with your own key):

OPENROUTER_API_KEY=your_openrouter_key_here.

Important: Never paste API keys into code cells.

Step 3 — Get a SerpAPI key¶

SerpAPI is used to access search tools (e.g. Google search) from agents. It requires a separate account and API key.

1. Go to https://serpapi.com

2. Click Sign up

3. Create an account or login if you have one (SerpAPI offers a free tier that is sufficient for this course.)

4. Confirm your email and phone number if prompted

5. Once logged in, go to your dashboard (https://serpapi.com/dashboard)

6. Locate and copy your API Key

Step 4 — Add a secret in Colab (UI)¶

1. On the left sidebar, click 🔑 Secrets

2. Add a new secret:

• Name: SERPAPI_API_KEY

• Value: your actual API key

3. Toggle the switch to the left to give notebook access (you should see a checkmark)

If running locally — Add a secret in .env¶

1. Add the following to .env (replace with your own key):

SERPAPI_API_KEY=your_serpapi_key_here.

Important: Never paste API keys into code cells.

Load the API Keys¶

In Colab (For Part 1 Only):¶

Uncomment and run the cell below if you’re using Google Colab.

Locally:¶

These will be used in Part 2. If you are running Part 1 only, comment these out.¶

Part 1: ReAct Agent with LangChain¶

Load the tools our Agent will have access to¶

Load the LLM and setup the Agent¶

In this demo we use the gpt-4o-mini model via OpenRouter. This is a lightweight and cost-efficient model that works well for agentic workflows involving tool use. You may change the model by selecting one available on the OpenRouter model list. Setting the temperature to zero makes the agent’s behavior deterministic, which is we choose for demos and debugging: given the same input, the agent will tend to make the same decisions and tool calls.

That is all we need to do to setup the Agent! LangChain will handle the logic. Note how short our system prompt is now. The short system prompt is enough because the framework now implements the ReAct loop.

In our previous demo (ReAct from scratch), the LLM drove the ReAct loop and the system prompt had to explicity specify:

With a framework instead:

• The logic is no longer in the prompt

• The logic LLM → decide → tool → observe → repeat → stop is implemented in code

• The framwork handles:

  • Iteration (including how many times it can loop)

  • Tool invocation (what tools exist and how to call them)

  • Passing observations back to the model

  • Termination conditions

  • Output structure

When we use an agent framework, the reasoning loop moves from the prompt into code, so the prompt can be short and declarative.

As a developer, it is however important to understand exactly how it works before relying on the abstractions afforded by the frameworks.

Run the Agent¶

This is too much to look at! Let’s break it down slowly.

When we call agent.invoke, the returned object contains the full internal execution state of the agent, including:

• messages

• tool calls

• tool outputs

• metadata

• the final response.

We use build_agent_trace(result) to reorganize this raw output into a readable, step-by-step trace that makes it easier to inspect what the agent did, which tools it called, and how it arrived at its final answer.

Let’s inspect the components of the answer first to understand how to extract data from it to build the trace.

The answer has a list of messages, and each message has a type (the node that issued it)

AI messages may issue a tool call

Tool messages contain observations: the data returned by the tool. Rather than storing the full tool output (msg.content) in the trace, which can be large and clutter the display, we record only the length of the result. This keeps the trace compact and easier to inspect while still conveying how much information the tool returned.

Now that we have broken down the structure of the answer, let’s parse it into a step-by-step trace that makes it easier to inspect the agent’s actions.

Much easier to trace now!

From this trace alone, it’s not obvious that the Wikipedia lookup was irrelevant (there is no Wikipedia page for me), which is why deeper inspection is sometimes necessary when results look questionable. Recognizing and investigating such mismatches is an important skill when debugging agent behavior.

Now let’s try to get our agent to dig a little deeper.

Using MCP Server¶

MCP (Model Context Protocol) is an open standard developed by Anthropic that allows AI models to connect to external tools and data sources through a unified, portable protocol. Unlike LangChain’s built-in tool wrappers, MCP servers are standalone processes that any MCP-compatible client can use, not just LangChain.
So far, our agent has relied on LangChain’s built-in wrappers for SerpAPI and Wikipedia, which return short search snippets.
For demonstration purposes, we use an MCP server to fetch the full content of a web page, which LangChain has no built-in tool for.

The MCP connection code below is asynchronous. This is not specific to the mcp-server-fetch server; it is a general requirement of MCP:

• Connecting to an MCP server involves I/O-bound work (spawning a subprocess and communicating over pipes for local servers, or sending HTTP requests for remote ones). asyncio allows the program to wait for that I/O without blocking everything else.

• client.get_tools() from langchain_mcp_adapters is an async function, as it sends a request to the server asking what tools are available, then waits for the response.

The same async pattern would apply regardless of which MCP server you connect to.
In Jupyter, top-level await is supported natively, so no extra setup is needed (i.e. you don’t need to include it inside an async def function as you would in a python program or script).

Side Note - Where to look for MCP servers¶

The authoritative source are the official MCP servers repo: https://github.com/modelcontextprotocol/servers and the mcp registry: https://registry.modelcontextprotocol.io/

They list officially maintained servers with the exact command and args to use for each one.

Local vs Remote MCP Servers¶

MCP servers come in two flavors, and MultiServerMCPClient supports both — you just change the transport type:

Local (stdio) — the server runs as a subprocess on your machine. uvx or npx download and launch it automatically:

client = MultiServerMCPClient({
    "fetch": {
        "command": "uvx",
        "args": ["mcp-server-fetch"],
        "transport": "stdio"        # <-- local subprocess over stdin/stdout
    }
})

Remote (HTTP) — the server runs elsewhere and exposes an HTTP endpoint. You connect with a URL and authenticate with a token:

client = MultiServerMCPClient({
    "github": {
        "url": "https://api.githubcopilot.com/mcp/",
        "transport": "streamable_http",  # <-- remote server over HTTP
        "headers": {"Authorization": "Bearer <your_github_token>"}
    }
})

In both cases, after creating the client you call await client.get_tools() to retrieve the tools.

Now that the agent has an async tool (the MCP fetch tool communicates over I/O under the hood), we use await agent.ainvoke() instead of agent.invoke(). While invoke() may still work in some cases, ainvoke() is the correct choice here because it allows the event loop to properly handle the async tool calls without blocking, which matters when a tool is waiting on network or subprocess I/O, as MCP tools are.

In a regular Python script, await can only appear inside an async def function. Writing it at the top level would raise a SyntaxError because the top-level scope is not an async context. Jupyter notebooks are different. IPython runs each cell inside an event loop and wraps cell execution in an async context automatically, so top-level await in a cell is valid. This is why answer = await agent.ainvoke(...) works directly in a cell, while the same line in a .py script would need to live inside an async def function.

Adding Custom Tool¶

For demonstration purposes, we add a custom tool to our search agent: a tool that scans search results for year mentions and groups the evidence by time period, helping the agent reason chronologically about the information it finds.

We use the @tool decorator to register this function as a tool that the agent can call, allowing LangChain to expose its inputs and outputs in a structured way. What required explicit JSON schemas and parsing logic in our previous demo is handled automatically by LangChain through a single tool decorator.
Note the importance of the docstring here, which effectively tells the Agent what this tool is and how/when to use it.

Add the custom tool to the Agent and re-run the query¶

Let’s modify the system prompt to nudge the agent to use the custom tool as well.

--------------------------------------------------------------------¶

Part 2: Observability and Evaluation¶

We will continue exploring the rest of this notebook after our lecture on evals.¶

LangSmith is LangChain’s observability and evaluation platform for agent-based systems. It records everything that happens during an agent run (LLM calls, tool calls, intermediate states, timing, and token usage) so you can inspect how an agent behaved, not just what it returned. It can be used for debugging agents, comparing different prompts or tools, evaluating behavior across datasets, and understanding failure modes in complex, multi-step workflows.

LangSmith works by attaching tracing callbacks to the runtime and capturing execution events as they occur. Because Google Colab replaces Python’s standard output system, this tracing mechanism is unstable in Colab notebooks and can cause runtime errors even when the agent itself is correct. For this reason, LangSmith tracing is not supported reliably in Colab. To use LangSmith, you should run the local version of this notebook from the course GitHub repository (for example using a local virtual environment and Jupyter or VS Code), where tracing works correctly and you can inspect full agent traces in the LangSmith UI.

Structured Tracing with LangSmith¶

In Part 1, we built a simple build_agent_trace() function to inspect agent behavior.

In this section, we’ll enable LangSmith tracing and re-run the agent to inspect with LangSmith.

Get a LangSmith API Key¶

1. Go to https://smith.langchain.com

2. Sign in with your GitHub, Google, or email account (or create a new account)

3. Once logged in, click on your profile icon in the bottom left corner

4. Select Settings

5. Navigate to API Keys

6. Click + API Key

7. Give the key a name, e.g. langchain-react_agent_demo

8. Copy the key immediately (you won’t be able to see it again)

Important: Treat this key like a password. Do not share it, paste it into notebooks, or commit it to GitHub.

Add the LangSmith API Key¶

In Colab:¶

1. On the left sidebar, click 🔑 Secrets

2. Add a new secret:

• Name: LANGSMITH_API_KEY

• Value: your actual API key

3. Toggle the switch to the left to give notebook access (you should see a checkmark)

If running locally — Add to .env¶

Add the following to your .env file (replace with your own key):

LANGSMITH_API_KEY=your_langsmith_key_here

Important: Never paste API keys into code cells.

Enable LangSmith Tracing¶

Setting these environment variables tells LangChain to automatically send trace data to LangSmith. The LANGCHAIN_PROJECT groups your runs together for easier organization.

Re-run the Agent with Tracing¶

Now when we invoke the agent, LangSmith will automatically capture the full execution trace. After running this cell, go to https://smith.langchain.com and select the react-agent-demo project to see the trace.

Simple Evaluation¶

Beyond tracing individual runs, LangSmith allows you to evaluate your agent systematically against a dataset. For demonstration purposes, we’ll create a small test set and run a simple evaluator that checks whether the agent’s response contains expected facts.

This pattern is useful for:

• Regression testing (i.e. re-running known test cases to ensure that previous correct behaviors have not degraded due to the change) after prompt or model changes

• Comparing different models or configurations

• Extensive testing before deploying agents to production

Run the Evaluation¶

The evaluate function will:

1. Run the agent on each example in our test set

2. Apply our custom evaluator to each result

3. Record everything in LangSmith for analysis

After running, visit LangSmith to see the evaluation results, including scores and traces for each test case.

Summary¶

In Part 1, we built a ReAct agent using the LangChain framework:

1. Framework setup: By calling create_agent with a model, a list of tools, and a short system prompt, we got a fully functional ReAct agent — the reasoning loop, tool routing, and termination logic are all handled by the framework

2. Built-in tools: We loaded LangChain’s built-in wrappers for web search (SerpAPI) and Wikipedia with a single load_tools call, and connected an MCP server to add a fetch tool for retrieving full page content.

3. Custom tools: We defined a custom extract_year_timeline tool using the tool decorator. LangChain automatically exposes its inputs and outputs to the agent based on the type hints and docstring.

4. Inspecting agent behavior: We built a build_agent_trace() helper to parse the agent’s raw message output into a readable step-by-step trace of user input, actions, and observations.

These techniques illustrate:

• Quickly prototyping agents without implementing the reasoning loop from scratch (less coding but also less control)

• Extending agents with tools: built-in, MCP-based, or fully custom

• Inspecting agent decisions to debug unexpected behavior before adding full observability

In Part 2, we explored structured observability with LangSmith:

1. Tracing: By setting a few environment variables, we enabled automatic capture of all LLM calls, tool invocations, latencies, and token usage, without modifying our agent code.

2. Evaluation: We created a simple test dataset with expected facts and wrote a custom evaluator to measure how well our agent’s responses covered those facts.

These techniques are useful for:

• Use tracing to extensively debug unexpected agent behavior

• Build regression test suites to catch issues before deployment

• Compare different models, prompts, or tool configurations systematically