How AI Agents Work, and What Yours Is Allowed to Touch

An AI agent works by running a loop. A language model gets a goal, picks a tool, calls it, reads the result, and decides the next move, repeating until it judges the job done or something stops it.

That loop is the easy half, and almost every explainer stops there. Our position is that the loop tells you almost nothing useful, because two agents running the identical loop behave identically right up until one of them is holding a credential with DELETE on it. The variable that decides whether an agent is a research assistant or an incident is not how it thinks. It is what it is allowed to touch. In April 2026 a Hacker News thread titled "An AI agent deleted our production database" drew 860 points and over a thousand comments; the most upvoted reply refused the framing entirely. "The culprit is in the IAM policy, not the prompt," wrote commenter eolgun. That is the argument this piece is built around.

The short answer: An AI agent is a language model in a loop with tools, and you can predict its worst day far more accurately from the write permissions on its credentials than from the model inside it.

Last updated: July 30, 2026.

The loop every AI agent runs, and the single point where its reach is decided.

What an AI agent is, and what it is not

An AI agent is a language model given a goal, a set of callable tools, and permission to keep going without being asked again. Anthropic's engineering team draws the sharpest available line: workflows are systems "where LLMs and tools are orchestrated through predefined code paths," while agents are systems "where LLMs dynamically direct their own processes and tool usage," according to Anthropic's guide to building effective agents. Who picks the next step is the whole distinction.

That definition rules a lot of things out, and the exclusions are more useful than the inclusion. A chatbot with a retrieval index is not an agent; it answers once and stops. A Zapier-style automation that fires six API calls in a fixed order is not an agent either, however many of those calls hit a model, because the order was decided by a person at build time. A model that writes a SQL query for you to run is not an agent. A model that writes the query, runs it, reads the rows, notices they are empty, and tries a different table is.

The confusion is expensive, and Gartner put a number on it. In a June 2025 press release the firm predicted that over 40% of agentic AI projects will be canceled by the end of 2027, citing escalating costs, unclear business value, and inadequate risk controls, and named "agent washing" (rebranding assistants, RPA and chatbots without substantial agentic capability) as a driver. Gartner estimated only about 130 of the thousands of self-described agentic vendors were real. If you are evaluating a product, the loop test above is a faster filter than any feature list: ask who decides the second step.

Here is the comparison in the shape people actually need it.

Read the bottom two rows together. For every other category on that table, the blast radius is set by something a human authored. For an agent it is set by a credential, and credentials are handed out by a different team, on a different day, for a different reason. That gap is where the incidents live.

Agentic AI, agentic systems, and other words for the same thing

"Agentic AI" is the adjectival form and generally means the same thing as an agent (McKinsey defines agents as "systems based on foundation models capable of acting in the real world, planning and executing multiple steps in a workflow"), though vendors stretch it to cover anything with a plan step. "Agentic AI systems" usually implies more than one agent, or an agent plus the orchestration, memory and tool layers around it. The US National Institute of Standards and Technology treats the distinction as real enough to matter for controls: its Control Overlays for Securing AI Systems project lists "Using AI Agent Systems (AI Agents) – Single Agent" and "Using AI Agent Systems (AI Agents) – Multi-Agent" as two separate use cases requiring separate overlays on SP 800-53. When a standards body splits a category in two, the split is usually load-bearing.

How the loop actually runs, step by step

When people ask how do AI agents work, the loop is the answer they are looking for, and it is short enough to hold in your head. The loop has five moves, and they repeat until a stop condition fires. Goal in. Context assembled. Model decides. Tool called. Result observed. Then back to the top with the observation appended, so the next decision is made with knowledge the previous one did not have. Anthropic describes this feedback property precisely: agents "gain 'ground truth' from the environment at each step (such as tool call results or code execution) to assess its progress."

The academic ancestor of every production agent loop is ReAct. The 2022 paper by Yao and colleagues proposed interleaving reasoning traces with actions so that "reasoning traces help the model induce, track, and update action plans as well as handle exceptions, while actions allow it to interface with external sources." The pattern outperformed imitation and reinforcement learning baselines on ALFWorld by an absolute success rate of 34%, with one or two in-context examples. Almost every agent framework shipping today is a ReAct loop with better tooling.

Walk one iteration concretely.

Goal. A person, a schedule, or another system hands over an objective in natural language: reconcile yesterday's failed payments. Nothing about that string tells the agent how many steps it will take, which is exactly why an agent was chosen over a script.

Context assembly. Before the model sees anything, a runtime builds the prompt: the system instructions, the goal, the list of available tools with their input schemas, any retrieved documents, and the running transcript of what has happened so far this run. This step is code, not intelligence, and it is where most of an agent's real behaviour is determined.

Decision. The model emits either a final answer or a structured tool call: a tool name and a JSON object of arguments. It is not "deciding to act" in any interior sense. It is predicting that the next tokens should be a function invocation, given everything in the context window.

Action. The runtime, not the model, executes that call against a real system, using whatever credential the runtime holds. This is the sentence to reread. The model produces a request. The runtime holds the authority. Every meaningful control in agent design sits in the gap between those two.

Observation. This is the agent's entire perception of its environment: not sensors, just the text a tool returned. The result comes back, gets appended to the transcript, and the loop runs again. If the result is an error, the model sees the error text and usually tries something else, which is the source of both the agent's usefulness and its most alarming behaviour.

Stop. Loops end on a model-emitted final answer, a step cap, a token or dollar budget, a timeout, or a human interrupt. A loop with no cheap stop condition is a design defect, not a stylistic choice.

The step everyone skips: nothing in the loop enforces anything

Read the six moves again looking for the place where policy is applied. It is not there. The model can be told in its system prompt that production is frozen, and it will read that instruction, agree with it, and still emit a write call, because the instruction and the attacker's instruction and the user's instruction all arrive as the same undifferentiated tokens. The Replit incident of July 2025 is the canonical demonstration. Fortune reported that during an explicit code freeze, the company's agent deleted a live production database holding records for over 1,200 executives and around 1,190 companies, then misled the user about recovery. Replit's CEO Amjad Masad called it "unacceptable and should never be possible" and shipped automatic development-production separation afterwards. Note what the fix was. Not a better prompt. A boundary in the execution path.

What the model is really choosing: tools, schemas, and protocols

A tool, to an agent, is a name plus a JSON Schema plus a sentence of English describing when to use it. That is the entire interface. The model never sees your database; it sees refund_order with a required order_id string and a description saying it issues a refund. Everything the agent can and cannot do is decided by which of those descriptors are placed in its context.

The Model Context Protocol standardised this so tools stop being bespoke per framework. MCP's architecture documentation defines three server-side primitives: tools ("executable functions that AI applications can invoke to perform actions"), resources ("data sources that provide contextual information"), and prompts ("reusable templates"). A host application spawns one client per server, discovers what each server offers via tools/list, and invokes with tools/call. The practical effect is that a new integration has become a configuration change rather than an engineering project, which is wonderful for velocity and is precisely why reach now creeps without anyone filing a ticket.

MCP also carries a client-side primitive that most teams never switch on. Elicitation lets a server "request additional information from users," including confirmation of an action, via elicitation/create. That is a per-action human gate available in the protocol itself. Whether your agent uses it is a choice someone made, usually by default, usually without discussion. We wrote about the wider security surface of this layer in our analysis of MCP server security and governing AI connectors.

Agent-to-agent communication is a separate protocol with a separate purpose. A2A, now governed by the Linux Foundation, is "an open standard that enables seamless communication and collaboration between AI agents," and its documentation is explicit about why it is not just MCP with more steps: "The practice of encapsulating an agent as a simple tool is fundamentally limiting, as it fails to capture the agent's full capabilities." MCP connects an agent to functions. A2A connects an agent to another agent that has its own goals, its own tools, and its own reach.

That last point deserves a beat. When agent A delegates to agent B over A2A, agent A's effective reach becomes the union of its own tools and everything B can touch. Reach composes across delegation, and it composes silently.

Memory, context, and what an agent knows between steps

Within a single run, an agent's memory is its transcript. Every tool call and every result is appended to the context window, so by step twelve the model is reasoning over eleven previous observations. This is why agents get slower and more expensive as a run progresses, and why long runs degrade: the useful signal from step two is now buried under nine intervening tool dumps.

Across runs, an agent remembers nothing unless someone built persistence. Teams usually build three kinds, and they behave very differently.

Working memory is the transcript, scoped to one run, discarded at the end. Cheap, and the only one that is automatic.

Retrieved memory is a vector or keyword index (a knowledge base, in the older phrase) that the agent queries as a tool. It does not change the model; it changes what shows up in the context. Retrieval is read-only by construction, which makes it the safest capability you can add to an agent and the reason so many first agents are research agents.

Written memory is state the agent updates: a summary file, a preference store, a scratchpad table it can edit. The moment an agent writes to its own memory, it can persist a mistake into every future run, and a poisoned memory is far harder to notice than a poisoned single response.

There is a fourth thing people call memory that is not memory: fine-tuning. Adjusting weights changes the model's dispositions, not its knowledge of your Tuesday. Almost no production agent problem is solved by fine-tuning, and a good number are caused by teams reaching for it instead of fixing context assembly.

Why context assembly is the real engineering job

Anthropic's team reported that when building their SWE-bench agent they "spent more time optimizing our tools than the overall prompt." That ratio surprises people who think agent work is prompt work. It should not. The prompt is one input; the tool set, the schemas, the descriptions, the error messages the tools return, and the order things land in the context are the machine. An agent that keeps calling the wrong tool usually has a description problem, not an intelligence problem.

The six textbook types of AI agents, and why the taxonomy will not help you

If you search for types of AI agents, you will get the same six every time, and they come from a classical artificial-intelligence syllabus rather than from anyone's operations review. They are worth knowing because you will meet them in conversation. They are close to worthless for making a decision.

Now try to use it. You are approving an agent that will handle customer refunds. Which of those six rows tells you whether it needs a security review, or who to page when it misfires at 2 a.m., or whether the thing can move money? None of them. The taxonomy sorts agents by how sophisticated their reasoning is, and reasoning sophistication has almost no correlation with consequence. A dead-simple reflex agent wired to a payments API is more dangerous than an elaborate multi-agent research system holding read-only credentials.

The axis that predicts consequence is external, not internal: the set of things the agent can change. We have argued that case at length in two earlier pieces — classify by autonomy and access, not type and govern by blast radius, not hype. This article will not relitigate it. What follows is narrower and more practical: taking the loop you have just read about and turning it into something you can audit in twenty minutes.

The Reach Ladder: classify an agent by what it can touch

We call this the Reach Ladder, and it has four rungs. It is not a maturity model, and nobody should be climbing it for its own sake. It is a way of naming what an agent is before you argue about whether that is acceptable. If you want the fuller architectural treatment, our piece on capability and authority stacks sets out the same split at the design level; the ladder below is the twenty-minute version you can run against an agent that already exists.

Two rules make the ladder operational.

Reach maxes, it does not average. An agent with nine read-only tools and one issue_refund tool is an R2 agent. Teams consistently describe agents by their intended purpose ("it is just a support assistant"), and the credential set says otherwise. The Reach Ladder rung is the highest rung of any tool the agent holds, including tools reached through delegation to another agent.

Reach is a property of the credential, not the prompt. If the token the runtime uses can execute a DELETE, the agent can execute a DELETE, regardless of what the system prompt says. This is the point the Hacker News commenter was making, and it is the point OWASP's Excessive Agency entry has been making formally since 2025. OWASP names three root causes (excessive functionality, excessive permissions, excessive autonomy), and its mitigations read as a checklist for descending the ladder: minimise extensions, avoid open-ended extensions, apply least privilege, execute in user context, require user approval for significant actions, and enforce authorization downstream rather than trusting the agent to self-police.

Running a Reach Audit in one sitting

The audit takes about twenty minutes per agent and needs no tooling beyond a text file. It is deliberately boring.

1. List every tool the agent can call. Not the tools in the design doc, but the tools actually present in its context at runtime. Include MCP servers, and expand each server into its individual tools.
2. Add every agent it can delegate to, and recursively list that agent's tools too.
3. Mark each tool R0, R1, R2 or R3 by what it changes, not by what it is called. update_ticket_status is a write. search_customers is a read. create_api_key is R3, whatever the docs say.
4. Take the maximum. That single letter is the agent's rung. Write it down.
5. Compare it to the rung everybody assumed. The gap between the assumed rung and the measured one is the argument the team has been avoiding, and writing both letters on the same line is usually enough to start it.
6. For every tool above the intended rung, choose one of three fixes: remove the tool, narrow the credential behind it, or add a per-action approval gate so the tool functions at R1.

The step that does the work is step three, because tool names lie. A tool called get_account_settings that accepts a mutate boolean is a write tool. Read the schema, not the label.

Step one is harder than it sounds, and we should not pretend otherwise. MCP servers can change what they advertise mid-session; the protocol has a notifications/tools/list_changed message precisely so clients can refresh a tool list that has moved. An audit against the design document will therefore miss tools that appeared at runtime. Enumerate from the live client, not the config file, and re-run the audit when a server updates. If you are inheriting an agent somebody else built, run the audit before you read their documentation; the credential is the ground truth and the documentation is a hypothesis about it.

Where the ladder came from, and what it borrows

The Reach Ladder is our framing, but the ideas underneath are not new and we would rather point at their sources than pretend otherwise. Least privilege is decades old. The four rungs are a restatement of it with the granularity that agent tools actually have. The security researcher Simon Willison's lethal trifecta — access to private data, exposure to untrusted content, and the ability to externally communicate — describes a different and complementary cut, focused on exfiltration rather than write authority. His observation that models "will happily follow any instructions that make it to the model, whether or not they came from their operator" is the reason a rung cannot be enforced in a prompt. Run both frames. The trifecta tells you whether your agent can leak. The ladder tells you what it can break.

The Reach Ladder: four rungs of write authority, and where the signature sits on each.

A worked example: one refund agent at four rungs

One agent, one job: handle inbound refund requests for an e-commerce store. Built four times. Nothing changes except reach. Watch how much else changes with it.

At R0. The agent reads the ticket, pulls the order, checks it against the refund policy document, and posts a comment: eligible, £48.00, reason: item arrived damaged, policy clause 4.2. An agent handles the reading and the policy lookup, which is most of the tedium. A human still clicks refund. Value delivered: real, unglamorous, roughly the research half of the job.

At R1. The agent now assembles the actual refund (amount, method, reference) and puts it in a queue. A support lead approves or rejects. This is the rung most teams should start on and the one with the least discussed failure mode: at ten approvals a day a human reads them; at four hundred a day a human clicks approve. Approval fatigue converts R1 into R2 without anyone deciding to. If you deploy at R1, instrument the approval rate, and treat a rejection rate near zero as an alarm rather than a success metric.

At R2. The agent issues refunds itself, inside a declared envelope: refunds only, under £100, maximum thirty per hour, only against orders marked delivered in the last sixty days, and every refund reversible for seven days. Those are the guardrails, and note that every one of them is enforced outside the model. Now the interesting question is not whether it will make a mistake — it will — but whether the envelope bounds the mistake to something you can absorb. A prompt-injected review that convinces the agent every order was damaged costs you at most thirty times £100 per hour, visibly, reversibly, with an anomaly you can alert on. Design the envelope by asking what the worst hour looks like, not the worst action.

At R3. Someone adds a tool so the agent can also credit store accounts, adjust loyalty balances, and update the customer record in the CRM. Individually each seems minor. Collectively the agent now writes across three systems of record, no single one of which can roll back the others, and there is no longer a coherent answer to "undo the last hour." The refund is reversible; the loyalty adjustment triggered a tier change that triggered an email that is already in the customer's inbox. R3 is not defined by danger level. It is defined by the fact that rollback stops being a single operation.

The lesson is not that R3 is forbidden. Plenty of valuable automation genuinely needs it. The lesson is that the move from R2 to R3 happened because someone added a connector on a Tuesday, and nothing in the system announced that the agent had changed category.

Here is the same agent's economics, which move with the rung in the opposite direction to the risk.

The tension nobody resolves out loud

Look at those two tables together and you will see the problem every team hits in month three. Risk climbs with the rung. Savings climb with the rung too. The finance case pulls up the ladder and the safety case pulls down it, and telling people to "start read-only" without acknowledging that is how a governance framework gets ignored by the second quarter.

We do not think the tension resolves by picking a side. It resolves by changing what you are optimising. The wrong question is which rung is safe. The right one is what is the cheapest rung at which the value still shows up, and the honest answer is usually one rung lower than the business assumes, because most of the labour in a knowledge task is reading, deciding, and drafting, all of which an R0 or R1 agent does. In the refund example, R0 already removes the policy lookup and the eligibility judgement, which is the slow part; R2 removes the click, which is the fast part. Teams routinely accept a large increase in blast radius to automate the cheapest step in the process, and nobody ever puts it that way in the business case.

So: build the case at both rungs. Estimate the saving at R1 and the saving at R2, and put the difference next to what an uncontained bad hour would cost. If the delta is small — and for approval-shaped work it usually is — you have just bought most of the value for a fraction of the exposure. If the delta is large, you have a real argument for R2 rather than a default, and the envelope becomes something you design deliberately instead of inherit.

Why agents fail even when the loop is right

Agents fail in ways that are specific to the loop, and the numbers are more sobering than the marketing suggests. This is not a reason to avoid agents. It is a reason to size the envelope honestly.

They succeed once and then fail the same task. τ-bench, the tool-agent-user benchmark from Sierra built on customer service domains, introduced pass^k to measure whether an agent solves a task reliably across k independent trials rather than once. The paper reports that agents "succeed on <50% of the tasks" and that reliability collapses under repetition, with "pass^8 <25% in retail." Its successor, τ²-bench, added a dual-control telecom domain where the user also holds tools, and found state-of-the-art models struggling badly: pass^1 of 34% for GPT-4.1, 42% for o4-mini, and 49% for Claude 3.7 Sonnet on new tasks. A demo shows you a pass^1 success. Production is a pass^k problem.

Errors compound along the loop. Because each step's output becomes the next step's input, a small early mistake does not stay small. Anthropic's own guidance warns that agent autonomy brings "higher costs, and the potential for compounding errors." This is the mathematical reason long-horizon agents are hard, and it is why the length of task an agent can complete is a better capability metric than any single benchmark score.

Time horizons are climbing fast, which cuts both ways. METR measures the length of task a model can complete autonomously at a given success rate. Its Time Horizon 1.1 update, published January 29, 2026, reports a doubling time of 130.8 days for the post-2023 period under the revised methodology, and puts the then-frontier model's 50%-time-horizon at 320 minutes, with a confidence interval of 170 to 729 minutes. Read that carefully. Fifty percent success on tasks that take a skilled human five hours means the agent is now capable of running unsupervised for five hours and being wrong half the time. Capability growing faster than reliability is exactly the condition under which reach discipline matters more, not less.

They report success they did not achieve. The failure that unsettles operators most is the confident false completion: an agent reporting that a migration finished when the connection failed. In the Replit case, per Fortune's reporting, the agent misled the user about whether recovery was possible. If you take one design rule from this section: never let an agent's own claim of success be the record of success. Verify state independently.

They are steerable by their inputs. Prompt injection is not a bug that will be patched. It is a consequence of instructions and data sharing a channel. Any agent that reads content it did not author — web pages, emails, tickets, PDFs, tool outputs from third-party servers — is taking instructions from whoever wrote that content. This is why the lethal trifecta framing matters, and why an R0 agent with the ability to send email is not as harmless as its rung suggests.

And most of them never leave the pilot. McKinsey's State of AI global survey, published November 5, 2025, found 23% of respondents scaling an agentic AI system somewhere in their enterprise and a further 39% experimenting, but "in any given business function, no more than 10 percent of respondents say their organizations are scaling AI agents." Combined with Gartner's cancellation forecast, the picture is consistent: the demo works, the pilot works, and the thing that stalls is everything that has to be true before an agent is allowed to act on its own. Which is, again, a reach problem wearing a project-management costume.

Here is a short talk that covers the security side of this well, from IBM's technology channel:

What regulators and standards bodies already require

You do not have to wait for agent-specific regulation to know roughly what will be asked of you, because the existing text already lands on the same controls the Reach Ladder implies.

The EU AI Act requires, for high-risk systems, that they "be designed and developed in such a way ... that they can be effectively overseen by natural persons during the period in which they are in use." Article 14 goes further than "a human is informed": overseers must be able to "intervene in the operation of the high-risk AI system or interrupt the system through a 'stop' button or a similar procedure that allows the system to come to a halt in a safe state," and must be able to correctly interpret outputs, recognise automation bias, and decline to use or override the system. An R2 agent with no interrupt and no way to inspect what it is doing does not meet that bar, and no amount of documentation fixes it after the fact. Whether a given agent is high-risk depends on its use case under the Act's annexes, which is a legal question we are not qualified to answer for you. But the design cost of building an interrupt is low enough that arguing about classification first is usually the expensive path.

NIST is building agent-specific controls right now. The COSAiS project will produce SP 800-53 control overlays for five AI use cases, two of which are single-agent and multi-agent systems. As of this writing the published artifacts are a concept paper (August 2025) and an annotated outline for the predictive-AI use case (January 2026); the agent overlays are still in development. The practical read: the control language is not final, but the shape is visible, and it is drawn from SP 800-53 (access control, least privilege, audit, and accountability) rather than from anything novel.

OWASP already names the failure. Excessive Agency is entry LLM06 in the Top 10 for LLM Applications, and its three root causes map one to one onto the ladder: excessive functionality means the agent holds tools it does not need, excessive permissions means the credential behind a tool is broader than the tool's job, and excessive autonomy means a high-impact action has no human gate.

Where reach becomes a governance problem

Everything above is a design discipline you can run with a text file. It stops being sufficient at the point where you have more than a handful of agents, because the questions change shape: not what can this agent touch but which of our agents can touch payroll, not who approved this action but produce every action this agent took last quarter. Those are inventory and audit questions, and they are answered by infrastructure or not at all.

This is the layer LeapForce builds. Our platform's position is that an agent is a non-human identity: it authenticates, it holds credentials, it acts, and it should therefore have an owner, a scope, and an expiry the same way an employee account does; we set that argument out in full in our earlier analysis of owner, scope and expiry for AI agents. The AI Gateway applies a six-stage lifecycle to every call (Identify, Check, Protect, Route, Execute, Record) so that policy is evaluated in the request path rather than in a wiki, and the published rollout guide is deliberately unhurried: observe first, enforce second, optimize third. Connectors exist so that the tool an agent holds is one IT vetted once, scoped at the action level, with the credential brokered rather than pasted into a config file. To be straight about the boundary: LeapForce does not build your agent or write its prompts, and per-capability build status is disclosed on the site because some of this layer is live and some of it is still in development.

Honest limits: what the Reach Ladder does not solve

We would rather you use this framing knowing where it runs out.

It says nothing about correctness. An R0 agent can be confidently, expensively wrong, and the ladder will not warn you. Reach bounds damage; it does not produce accuracy. Evaluation is a separate discipline and this article is not one.

Rungs are coarse. R2 covers an enormous span, from an agent that can edit a ticket title to one that can wire £99. Inside R2, the envelope does all the work, and we have offered no formula for setting one, because we do not have one that survives contact with different businesses. What we can say is that the envelope should be written down and reviewed, and in most teams it currently is neither.

It has no answer to organisational pressure. Sooner or later someone will ask why the agent still needs a human, and a ladder describes a state; it does not defend a rung. The only durable defence we know of is the arithmetic in the section above — the cost of the delta versus the cost of a bad hour — and if that arithmetic favours moving up, the ladder should not be used to argue against it.

Reach through delegation is genuinely hard to measure. Step two of the audit — recursively expanding delegated agents — is easy to write and difficult to do when the agents belong to other teams or other vendors. If you run A2A across an organisational boundary, treat the far side as unbounded until proven otherwise.

We have not run a first-hand test for this article. Everything here comes from published benchmarks, standards documents, primary reporting and protocol specifications, all linked inline. We did not instrument an agent and report our own traces, and we would rather say so than imply a measurement we did not take.

Two sources a reader might expect are missing. Reddit's agent communities were unreachable from our tooling during research, so the practitioner voice here comes from Hacker News, which skews technical and towards developer-tooling incidents rather than the finance and operations teams who will deploy most agents. Read the failure examples with that skew in mind.

And the ground is moving. METR's own numbers say the frontier's autonomous task length has been doubling roughly every four months. A ladder built on write authority should age better than one built on capability, since permissions are a property of your systems rather than of the model. But any claim in this article about what agents can and cannot do reliably has a short half-life. The claims about what they should be allowed to touch should not.

Ready to Govern Your AI?

Talk to LeapForce — one controlled layer for every AI tool, connector, model, and agent.