From Software Engineer to AI Engineer - Part 6: Closing the loop

Everything so far has been single-shot. One call in, one result out, with us gluing the pieces together. But what if we get a realistically complex prompt: "Tell me how much it costs to refund 100 euros to a European consumer card. Then issue a full refund of 100 euros for payment with id abc123 and write a confirmation email to the customer." Answering that means searching the knowledge base for the fee schedule, calling multiple tools, and reasoning what to do next. And the sequence isn't fixed: it depends on what the user gave us, what retrieval returns, and what is still missing. You could try to hand-code the expected flow but real users won't stick to that (just like in regular software). Their questions vary, retrieval results vary, and different situations require different next steps. That is why agent loops are useful: they let the system adapt its execution dynamically rather than forcing every request through the same predefined sequence. The agent loop You likely heard about AI agents. Give them a task and they'll figure out how to get there by themselves. At a basic level, this is how an agent works: 1. Model reads the full message list (system prompt, conversation so far, tool catalog) 2. Model responds with either tool calls or a final answer if tool calls -> your code executes them results are appended to the conversation as messages go back to 1 else if final answer -> done, return it to the user We've seen this logic already in Parts 3 and 4, where we looked for msg.tool_calls and invoked tools based on them. The step that turns this into an agent is essentially to keep repeating until the model gives its final answer. The literature calls this pattern ReAct, for reasoning + acting, because the model alternates between reasoning about what to do next and acting through tools. Coding assistants like Claude Code are a prime example. Next time you use them, look at how they transition from 'thinking..' to invoking tools to 'thinking..' again to ultimately arrive at the answer. Part 1 introduced the word harness for everything we build around the model. The harness is the system prompt, the tool catalog, the middleware, the loop itself. Wverything we write in Python is harness. Two agents running on the same model can behave like different species depending on their harnesses, and harness design is your job as an AI engineer. It's the Iron Man suit and the model is Tony Stark, kind of. Or is Jarvis the model and Tony just the user? Anyways.. PayIQ becomes an agent Let's build our very first agent. Create app/agent_0.py : from langchain.agents import create_agent from langgraph.checkpoint.memory import InMemorySaver from app.tools import calculate_refund_cost from app.rag import search_payments_knowledge_base from dotenv import load_dotenv SYSTEM_PROMPT = """\ You are PayIQ, an assistant that helps an online merchant's support team handle payment operations: refunds, chargebacks/disputes, and processing fees. Rules: - Ground fee figures, deadlines, and policy answers in the knowledge base tool. Don't invent figures from general knowledge if the tool has relevant notes. - Before calculating a refund cost, make sure you have (or have looked up, citing the source) both the charge amount and the payment method's fee structure. - Be direct and concise. This user is a professional handling real money. - Always flag when a figure is a rule-of-thumb from internal notes vs. something that must be verified against the processor contract or the card network's current rules. """ load_dotenv() def demo(agent): config = {"configurable": {"thread_id": "demo-1"}} turn_1 = agent.invoke( {"messages": [{"role": "user", "content": "What fees are related to refunding a European consumer card?"}]}, config, ) print(turn_1["messages"][-1].content) turn_2 = agent.invoke( {"messages": [{"role": "user", "content": "A customer paid €480 with a European consumer card twelve " "days ago and wants a full refund. What will the refund cost " "us in total?"}]}, config, ) print(turn_2["messages"][-1].content) if name == "main": checkpointer = InMemorySaver() agent = create_agent( model="anthropic:claude-sonnet-5", tools=[search_payments_knowledge_base, calculate_refund_cost], system_prompt=SYSTEM_PROMPT, checkpointer=checkpointer, ) print(agent.get_graph().draw_mermaid()) demo(agent) You might find the snippet underwhelmingly simple. LangChain's create_agent gives us the loop for free. Note that each call to create_agent loops through all tool requests to get a final answer. Two interesting differences with the while true loops you know: - Those while true loops always had a stop condition. Now the model decides when it is done. It stops proposing tool calls when, in its own judgment, the context window contains enough to answer. LangGraph has a default recursion limit is 25 steps, in case the model starts reasoning in circles and busts all your tokens. - Tool errors do not bubble up as exceptions. The errors are fed back into the model. Models are surprisingly good at reading a stack trace and correcting their own arguments to try again. Run python 06_agent_0.py and read the output. Better yet, print every message in turn_1["messages"] instead of just the last one and you see the model search the knowledge base, invoke the calculator, and then compose its answer. Nobody wrote that sequence. We wrote some docstrings and a system prompt, and the model figured it out. Spoiler: this loop is actually a very basic graph that we'll expand in later articles. You can already sneakpeak this graph through print(agent.get_graph().draw_mermaid()) , which gives you the mermaid diagram (I put a render in the companion repo). Memorize they must Recall from Part 1 that the model resembles a pure function: f(list of input messages) -> output message . It remembers nothing between calls. In the snippet above, turn 2 worked because the entire conversation from turn 1, including the tool calls and their results, was replayed into the context window. We achieved this through the checkpointer that you might have noticed already, which persists conversation state keyed by thread_id . Pass the same id and LangGraph loads that conversation state to continue where it left off. Imagine it like web sessions in which the customer's session id in the cookie allows the server to remember what was already in their shopping cart. The checkpointer serves as the model's conversation memory. The classic use case is to ask follow-up questions (like turn 2 above). Advanced use cases include human-in-the-loop interrupts (topic for a next article) or resuming a long-running task after a server crash. Memory management is a big topic in AI engineering. Conversation memory is complemented with short-term (working) memory, such as a todo list that the agent creates for itself to not digress (look up LangChain's TodoListMiddleware() ). And then there's also long-term memory holding facts that outlive threads. Think of a code review agent that stores learnings from one pull request review to reuse in future reviews. Long-term memory can be implemented using the RAG pattern from Part 4 extended by a tool that writes new facts into the vector store. Now build me Claude Code! Prompting an agent with agent.invoke is convenient and abstracts away all the looping and just gives us the final answer. But when I think of agents, I think of Claude Code-like experiences where you can see what the agent is doing: it thinks, invokes a tool, requests webpages, and writes documents. Maybe I am just so 2025, but luckily LangChain got me covered. Below I use create_agent again but stream its execution using agent.astream . Instead of waiting for the final answer, incremental chunks trickle in as the agent runs. I then look at the chunks type and attributes to display different types of events with their own messages and emojis. I added a few additional tools, which you can find in my companion repo, as well as an ASCII art startup banner. I won't explain the code line by line as this is not a LangChain tutorial, but I hope the snippet gives you a simple example to start building your own AI application. Create 06_agent.py : import asyncio import random from app.tools import calculate_refund_cost, issue_refund from app.rag import search_payments_knowledge_base from langchain.agents import create_agent from langgraph.checkpoint.memory import InMemorySaver from langchain_core.messages import AIMessageChunk, ToolMessage PAYIQ_BANNER = r""" ██████╗ █████╗ ██╗ ██╗██╗ ██████╗ ██╔══██╗██╔══██╗╚██╗ ██╔╝██║██╔═══██╗ ██████╔╝███████║ ╚████╔╝ ██║██║ ██║ ██╔═══╝ ██╔══██║ ╚██╔╝ ██║██║▄▄ ██║ ██║ ██║ ██║ ██║ ██║╚██████╔╝ ╚═╝ ╚═╝ ╚═╝ ╚═╝ ╚═╝ ╚══▀▀═╝ Payment Intelligence Assistant ------------------------------ """ SYSTEM_PROMPT = """ You are PayIQ, an assistant that helps an online merchant's support team handle payment operations: refunds, chargebacks/disputes, and processing fees. Rules: - Ground fee figures, deadlines, and policy answers in the knowledge base tool. Don't invent figures from general knowledge if the tool has relevant notes. - Before calculating a refund cost, make sure you have (or have looked up, citing the source) both the charge amount and the payment method's fee structure. - Be direct and concise. This user is a professional handling real money, not someone who needs hand-holding. - Always flag when a figure is a rule-of-thumb from internal notes vs. something that must be verified against the processor contract or the card network's current rules. """ async def create_payiq_agent(): checkpointer = InMemorySaver() return create_agent( model="anthropic:claude-sonnet-5", tools=[search_payments_knowledge_base, calculate_refund_cost, issue_refund], system_prompt=SYSTEM_PROMPT, checkpointer=checkpointer, ) async def run_agent(agent, thread_id: str, user_input: str): """ Run the agent and yield its streaming events. This function knows nothing about how the events are displayed. ""