How AI Agents Work: From LLM to Feedback Loop

Typing a question into ChatGPT and getting an answer is one thing. Asking a system to "research my company's competitors, build a comparison spreadsheet, and email it to me" and watching it happen autonomously, in multiple steps, is something else entirely. The latter is what is now called an AI agent, and understanding how it works internally requires separating six pieces that, together, transform a model that only predicts text into something that decides, acts, and corrects its own course.

What is an AI Agent

An AI agent is a system built around an LLM (language model) that can execute multi-step tasks autonomously, deciding on its own which action to take at each step based on the current state of the task. The central difference from a common chatbot is not the underlying model—often it's the same—but the architecture around it: memory between steps, a planning cycle, access to tools, and a repetition mechanism until the task is complete.

This completely changes the type of work the system can do. A chatbot answers one question and stops. An agent can search, read the result, realize information is missing, search again with a different term, organize everything, and only then deliver the final answer, without the user needing to mediate each step.

The term has become an umbrella used for very different products: code agents that edit entire repositories, browser agents that fill forms and click buttons, customer service agents that query internal systems before responding to a customer. What they all have in common, despite their very different appearances, is precisely this six-piece structure that this article details. Understanding each piece separately is what allows you to fairly compare different products, rather than judging everything by the final result on the screen.

It's also worth clarifying what an agent is not. It is not a system with "free will," nor does it make decisions outside the scope defined by the developer—the set of available tools, iteration limits, and success criteria are all configured in advance. An agent's autonomy is real within these limits, but the limits themselves are still defined by humans.

LLM: The Reasoning Engine

The LLM is the component that decides what to do next, but let's clear up a common misunderstanding: the model doesn't "remember" anything between calls. Each time the agent needs a decision, it assembles a new full prompt (with history, tool results, instructions) and sends it to the model, which responds statelessly, without built-in memory.

What makes the LLM seem like it's "reasoning" across multiple steps is a technique called chain-of-thought: instead of jumping straight to an answer, the model is induced to expose intermediate reasoning steps before deciding the final action. This greatly improves the quality of complex decisions because the very process of "thinking out loud" helps the model avoid skipping important logical steps.

In practice, the LLM inside an agent plays a very specific role: look at the current state (what has been done, what the tool returned, what the original goal is) and decide the next action, whether it's calling a tool, asking the user for more information, or concluding that the task is finished.

One point that often surprises beginners: the same model, with the same prompt, can generate slightly different decisions in different runs. This happens because text generation in an LLM is probabilistic, not deterministic, controlled by a parameter called temperature. In agents that make sensitive decisions (like moving money or deleting data), it's common to set a low temperature to reduce variation and make behavior more predictable, though never perfectly deterministic.

It's also worth noting that "the LLM" of an agent is rarely a single model call. More sophisticated systems use different models for different steps: a cheaper, faster model for simple routing decisions, and a larger, more expensive model only for steps that truly require deep reasoning. This division of labor helps control cost without sacrificing quality where it matters.

Memory

Since the model doesn't store anything on its own, all of an agent's "memory" is actually context engineering: deciding what to put back in the prompt at each call. In practice, there are two types:

• Short-term memory: the history of the conversation or current execution, which fits within the model's context window. It grows with each agent step, and in long tasks it may need to be summarized to avoid hitting the token limit.
• Long-term memory: information that needs to persist beyond a single execution, stored outside the model in a database, file, or vector index, and retrieved on demand when relevant. The most common technique for this is RAG (retrieval-augmented generation): before responding, the system searches for the most relevant snippets from this external memory and injects them into the prompt.

A practical example helps solidify the difference: a coding agent that reads a project's code keeps the currently open file as short-term memory, but team conventions (like "always use strict TypeScript" or "never use library X") are usually saved as long-term memory, in a configuration file that is reloaded each new session.

A model's context window, no matter how large, has a physical token limit, and each token costs money and processing time. This forces agent builders to constantly make decisions about what is worth keeping in short-term memory and what can be summarized or discarded. A common technique is progressive compaction: every certain number of steps, the agent itself generates a summary of what happened so far, replaces the raw history with this summary, and continues execution with a much leaner context.

For long-term memory based on RAG, the underlying mechanism usually involves a vector database: each piece of information is converted into a numerical vector (embedding) representing its meaning, and the search doesn't look for exact words—it looks for vectors that are semantically most similar to the current query. This is why a well-configured RAG agent can find a relevant document even if it uses completely different words from what the user typed.

Planning

Faced with a complex task, an agent rarely solves everything in a single model call. The most common pattern today is ReAct (Reasoning and Acting), published in 2022: the model alternates between "thinking" (reasoning about what to do) and "acting" (calling a tool), observing the result of each action before deciding the next, rather than trying to predict the entire plan at once.

There is also the "plan-and-execute" pattern, where the model first writes a complete plan with several steps, and only then starts executing them one by one. It tends to be more predictable but less flexible: if a step in the middle fails or reveals new information, the agent needs to replan, which is not always handled automatically in this design.

In practice, most production agents mix the two: an initial approximate plan, revised every few steps based on what actually happened, rather than blindly followed from start to finish.

A concrete example of the ReAct pattern in action: ask an agent "what are the operating hours of the nearest store that has product X in stock?" A rigid plan made all at once would likely fail, because the agent doesn't yet know which stores exist or which have the product. With ReAct, the model first reasons ("I need to find nearby stores"), acts (calls a location search tool), observes the result, reasons again ("now I need to check each store's inventory"), acts again, and only then, with enough information, formulates the final answer. Each decision depends on the result of the previous one, which would be impossible to predict in a single fixed plan.

There is also the question of planning depth: simple tasks can skip explicit planning and go straight to tool selection, while complex tasks with many dependencies between subtasks benefit from an explicitly written plan, sometimes even shown to the user before execution begins so they can approve or adjust the course before any real action is taken.

Tools

Tools are what give the agent the ability to act in the real world, beyond just generating text: search the web, run code, query a database, send an email. The technical mechanism behind this is function calling (or tool calling): the developer describes each available tool with a name, description, and a schema of the parameters it accepts, and the model returns a structured response indicating which tool to call and with what arguments.

The model itself never executes the tool. It only decides and formats the call; whoever actually runs the code, makes the HTTP request, or queries the database is the software surrounding the model. This separation is important: it means the same "brain" can be connected to completely different tools without needing retraining, just by reconfiguring which functions are available in the prompt.

The Model Context Protocol (detailed in the previous article on MCP) solves precisely the part of standardizing how these tools are described and discovered, so that an agent doesn't need a specific integration for each external service it uses.

In practice, an agent's tools often fall into a few recurring categories: information search and reading (web, documents, databases), action on external systems (send email, create a ticket, update a spreadsheet), code execution (run a script, SQL query, terminal command), and communication with the user (ask for confirmation, ask a clarifying question). A well-designed agent typically doesn't receive access to all available tools in the system at once—only the subset necessary for the task at hand, following the same principle of least privilege used in traditional system security.

The quality of each tool's description matters more than it seems. A vague name like process_data with a one-line description tends to generate incorrect or malformed calls because the model lacks enough context to decide when to use it. Specific descriptions, with usage examples and clear limits (what the tool does not do), significantly reduce the error rate at this stage.

Execution

Execution is the software layer, often called the runtime or orchestrator of the agent, that receives the LLM's decision and actually puts it into practice: calls the right API, runs the command, waits for the response, handles errors if the call fails. This is also where security layers come in: sandboxing for model-generated code, timeouts for each call, and parameter validation before executing anything irreversible.

One detail that causes confusion: not all execution is sequential. More sophisticated agents can fire multiple tools in parallel when they are independent of each other, such as searching three different sources at the same time, which greatly reduces the total task time compared to calling one tool at a time and waiting for each response before proceeding.

Error handling at this layer is what separates a fragile agent from a reliable one. A tool call can fail for dozens of reasons: the external API is down, parameters are malformed, permission was denied, timeout expired. A well-built orchestrator captures each type of failure separately and decides what to do: retry automatically (with a retry limit, to avoid repeating a permanent failure indefinitely), return the error to the LLM as part of the observation so it can try a different approach, or escalate the problem to human intervention when none of the automatic options work.

Sandboxing deserves special mention when the tool involves executing code generated by the model itself. Running this code directly in the same environment where the agent lives is a real risk, so serious implementations isolate this execution in a disposable container or virtual machine, with no network access or access to sensitive files by default, only granting what the specific task actually needs.

Feedback Loop

After a tool executes, its result does not disappear: it goes back into the context, as new information the LLM must take into account in the next decision. This cycle (perceive state, plan, act, observe result, update context) repeats as many times as necessary until one of two things happens: the model decides the task is complete, or the agent hits a safety limit, such as a maximum number of iterations or a total execution time.

This iteration limit is not a minor detail. Without it, a misconfigured agent can enter a loop where it repeats the same action without ever considering the task complete, wasting tokens (and money) indefinitely. That's why virtually every serious agent framework includes some kind of guard: a maximum step counter, an explicit success criterion check, or a requirement for human confirmation after a certain number of attempts without progress.

It's worth distinguishing two types of stopping. The "happy" stop occurs when the LLM evaluates the current state against the original goal and explicitly concludes that all criteria have been met. The limit stop occurs when the agent exhausts the number of iterations or the allowed time/cost budget without getting there, and in this case the expected behavior is not simply to freeze—it is to report to the user what was done up to that point and why the task could not be completed autonomously.

An increasingly common pattern in production systems is human-in-the-loop: for high-risk actions (moving money, deleting data, sending a public communication), the feedback loop includes a mandatory pause waiting for human approval before proceeding to the next iteration, even if the agent has already decided on its own that the action is correct.

The Complete Cycle, in a Diagram

Putting the six pieces together, the cycle of an AI agent looks like this:

The central point of this design is that the return arrow is not an aesthetic detail: it is what separates an agent from a simple automated flow of fixed steps. The agent decides, at each turn of the cycle, whether it needs to continue or has already met the goal.

Chatbot vs Agent

It's worth putting them side by side, because the line between the two is not always obvious to the end user:

Aspect	Traditional Chatbot	AI Agent
Response structure	One question, one answer	Multiple chained steps to complete an objective
Access to tools	Rare or nonexistent	Central, via function calling or MCP
Memory between steps	Only current conversation history	Short-term and long-term, often with external retrieval (RAG)
Decision autonomy	Low, user guides each exchange	High, system decides next steps on its own
Stopping criterion	End of generated message	Task success criterion or iteration limit

In practice, the boundary is a spectrum, not a sharp line: many products today are chatbots with a few simple tools plugged in, without having a full planning cycle. The term "agent" tends to be reserved for systems that truly decide their own next steps, not just execute a function when the user explicitly asks.

A practical test to differentiate the two when evaluating a product: ask for a task that requires at least three interdependent steps, without detailing the step-by-step, only the desired result. A chatbot tends to ask for more instructions or try to solve everything in a single, superficial response. A real agent breaks the request into subtasks, uses tools to solve each one, and only delivers the final result after completing the entire cycle on its own.

Complete Example

To make all this concrete, let's follow a hypothetical market research agent receiving the request: "research my company's top 3 competitors in Brazil and send me a summary by email."

Step 1, planning: the LLM breaks down the request into subtasks: find competitors, collect data on each, synthesize into a summary, send by email. This initial plan is stored in the execution's short-term memory.

Step 2, first tool: the agent calls a web search tool with the query "top competitors [company] Brazil." The result (a list of names and links) comes back as an observation.

Step 3, feedback and replanning: the LLM observes that the search returned 5 names, but only needs 3. It filters the most relevant ones and decides, for each, to call a new tool that extracts basic public data (size, main products, market presence).

Step 4, parallel execution: since the three competitors are independent of each other, the orchestrator fires the three extraction calls at the same time, instead of one after the other, saving total time.

Step 5, synthesis: with all three datasets in short-term memory, the LLM generates the text of the comparative summary, already formatted.

Step 6, last tool: the agent calls the email sending tool, passing the generated summary as the message body.

Step 7, conclusion: the LLM observes the sending confirmation, verifies that all items in the initial plan have been fulfilled, and ends the cycle, responding to the user that the task is complete.

Notice that at no point did the user need to intervene between steps. This is only possible because each of the six pieces (LLM, memory, planning, tools, execution, feedback loop) played exactly the role described in the previous sections.

FAQ

Does an AI agent always need long-term memory?

No. Simple tasks, solved in a single session, can work with only short-term memory (the current execution history). Long-term memory matters when the agent needs to remember something across different executions.

How many iterations does an agent typically make before stopping?

It varies greatly with task complexity and the framework configuration, but most production systems set an explicit limit, usually between 5 and 25 iterations, to avoid infinite loops.

Is a chatbot with plugins already an agent?

It depends on how much autonomy it has. If the user needs to explicitly request each action, it's more of a chatbot with tools. If the system decides on its own the sequence of actions needed to fulfill an objective, that's agent behavior.

What happens if a tool fails mid-execution?

It depends on the orchestrator. Robust implementations return the error as part of the observation, and the LLM decides whether to retry, try a different approach, or report the failure to the user, instead of freezing the entire cycle.

Can an agent use more than one LLM at the same time?

Yes, and it's increasingly common. A smaller, cheaper model can handle simple routing decisions, while a larger model steps in only for stages that require deeper reasoning, helping control total execution cost without losing quality where it matters most.

Jargon aside, an AI agent is not an isolated new technology—it's a specific way of organizing already known pieces (a language model, some type of memory, access to tools) into a cycle that repeats until a goal is met. Understanding these six pieces separately is what makes it possible to diagnose why a specific agent fails, gets stuck in a loop, or delivers an incomplete result, rather than treating the entire system as a black box.