Autonomous AI Agents: How Much Autonomy Can You Prove?

Autonomous AI agents are software systems that hold a goal, choose their own next step, and keep acting across a multi-step job without a human pressing start e

Autonomous AI agents are software systems that hold a goal, choose their own next step, and keep acting across a multi-step job without a human pressing start each time. That definition is settled and uncontroversial. What such an agent should be allowed to do is not, and that is the question your deployment actually turns on.

Our position at LeapForce is that "how autonomous is this agent?" is the wrong procurement question, because nobody can answer it. The answerable question is narrower and much more useful: how much autonomy can you evidence control over? Autonomy is not a property the vendor ships. It is a budget you grant, in five specific currencies, and then have to prove you bounded. A developer building in this space put the tension plainly on Hacker News in April 2026, describing agent access as "either all or nothing." You either grant broad reach and get a powerful, proportionally dangerous system, or you "lock it down so much that it's no longer useful". That trade-off is real, and it is also avoidable once you stop treating autonomy as a single dial.

The short answer: Do not buy or build against a claimed autonomy level. Decide five ceilings before the agent runs: what it can reach, what it can do irreversibly, what it may spend, how long it may run, and what evidence it leaves. Then grant the widest autonomy you can still reconstruct afterwards.

Last updated: July 30, 2026.

Five ceilings that bound an autonomous AI agent: reach, reversibility, spend, runtime and evidence

The autonomy budget: five ceilings set before the agent runs, each with a named owner and a measurable limit.

What Autonomous AI Agents Actually Are

An autonomous AI agent is a system that receives a goal rather than an instruction, decides for itself which steps and tools to use, executes those steps against real systems, and continues until it believes the goal is met or a limit stops it. The load-bearing word is goal. A script is told what to do; an agent is told what outcome to reach and works out the rest. McKinsey's State of AI survey published 5 November 2025 describes them as foundation-model systems "capable of acting in the real world, planning and executing multiple steps."

That is a functional definition, and it maps to four things happening in sequence on every run. The agent perceives some state: a ticket queue, an inbox, a database row. It reasons about that state against its goal, usually with some memory of earlier turns in the same run. It selects a tool and calls it. Then it reads the result and decides whether to stop or loop. Remove any one of those and you have something less than an agent: a chatbot has no tool call, a workflow automation has no step selection, a recommendation engine never executes.

Vendors describe the pieces around that loop with a fairly stable vocabulary. A reasoning model, a planning module, memory, a tool or connector layer, an orchestration layer that runs the loop, and guardrails wrapped around the whole thing. The vocabulary is fine. The problem is that "guardrails" is where every hard question gets filed and none of them get answered, which is what the rest of this article is about.

Adoption is real but early. In the same McKinsey survey, 23 percent of respondents said their organisations were scaling an agentic AI system somewhere in the enterprise and a further 39 percent had begun experimenting, which is 62 percent at least experimenting overall. But within any single business function, no more than 10 percent said they were scaling agents. Most of what exists today is a pilot with one team watching it.

It also helps to be blunt about what an autonomous agent is not, because three adjacent things get sold under the same word.

Often called an agentWhat it actually isThe missing piece
A chatbot or copilotA conversation with tool access, driven turn by turnNo unattended loop; a human supplies every next step
A workflow automationA fixed sequence of steps triggered by an eventNo step selection; the path was chosen at build time
A scheduled script with an LLM call insideDeterministic code that asks a model to write some textNo goal; it cannot change course based on what it finds

None of the three is worse than an agent. Two of them are usually better, because a knowable path is cheaper to run and far cheaper to govern. The distinction matters only because the controls in this article are built for the case where the path is not knowable in advance.

The gap between "we have an agent" and "the agent runs unattended" is where this article lives. Everything below assumes you have already decided an agent is the right shape for the job, and now have to decide how much rope to give it.

How Autonomous AI Agents Differ From Traditional Software

The difference is not intelligence. It is who chooses the execution path. Traditional software has a path fixed at build time by a developer; you can read it, test it, and know before deployment every branch it can take. Autonomous AI agents choose the path at run time from a space the developer never enumerated, which means the set of things the system might do is defined by its permissions, not by its code.

That single shift breaks four assumptions your existing controls are built on.

Assumption in traditional softwareWhat changes with an autonomous agent
The set of possible actions is knowable from the sourceThe action set is whatever the granted tools allow, in any order
The same input produces the same outputRuns vary; the same ticket can take three steps or thirty
Testing covers the branchesTesting samples a distribution; new paths appear in production
Failures are loud (exception, stack trace)Failures can be silent and confidently narrated

The fourth is the one that surprises teams. A crashed process tells you it crashed. An agent that failed can report success. A practitioner running sandbox infrastructure for coding agents reported on Hacker News in February 2026 that across their logged sessions, agents sometimes claimed "tests passing" when the test file did not compile. Treat that as a single operator's observation rather than a peer-reviewed finding. But the same pattern shows up in the best-documented public incident of the category, discussed below, where an agent told its user that recovery was impossible when it was not.

There is a useful older framing here. The human-factors literature had this problem long before language models existed. Parasuraman, Sheridan and Wickens, writing in IEEE Transactions on Systems, Man, and Cybernetics in 2000, proposed that automation applies to four separable classes of function: information acquisition, information analysis, decision and action selection, and action implementation. They argued that within each type, automation runs "across a continuum of levels from low to high." An agent is not automated or not. It is automated to some degree, separately, on four axes. Any vendor claim that compresses that into one number is discarding the information you need.

The Three Types of Autonomous AI Agents, and What the Taxonomy Cannot Tell You

The standard taxonomy splits autonomous AI agents into reactive, deliberative and hybrid. It is a real classification with a real provenance. Michael Wooldridge and Nicholas Jennings organised their 1995 survey Intelligent Agents: Theory and Practice around exactly these categories, under the headings "Classical Approaches: Deliberative Architectures," "Alternative Approaches: Reactive Architectures" and "Hybrid Architectures." Most vendor explainers reproduce this split without saying where it came from, which is worth knowing: it is thirty-one years old and it describes internal architecture, not operational risk.

TypeHow it decidesWhere it fitsWhat it will not tell you
ReactiveMaps a stimulus straight to an action, no internal model of the worldFast, high-volume classification and routingNothing about blast radius; a reactive agent with a delete tool is still dangerous
DeliberativePlans against an internal model, reasons about consequences before actingNovel situations, multi-step research, exception handlingNothing about spend; planning loops are the expensive kind
HybridA fast reactive layer under a slower planning layerMost production systems todayNothing about accountability; the layer that acted is not the layer that decided

Here is why this taxonomy will not carry a procurement decision. It classifies how the agent thinks. It says nothing about what the agent can touch, what it can destroy, what it can spend, or what it leaves behind. Two hybrid agents can sit at opposite ends of enterprise risk depending entirely on whether one holds a read-only CRM scope and the other holds a production database credential. The architecture is identical; the exposure is not comparable.

So use the taxonomy for what it is good for. It explains why a deliberative agent costs more per run and handles novelty better. Then set the decision on axes that describe consequences. That is what the next few sections build.

There is also a market reason to be sceptical of type labels. Gartner's June 2025 assessment found widespread "agent washing," meaning existing assistants, robotic process automation and chatbots rebranded without substantial agentic capability. It estimated that "only about 130 of the thousands of agentic AI vendors are real". A label that costs nothing to apply is not a filter.

Where Autonomy Breaks: The Reliability Numbers Vendors Skip

Autonomous AI agents fail in a specific, measurable way: success rate collapses as the number of unsupervised steps rises, and the collapse is steeper than intuition suggests. This is the single most important thing to internalise before granting an agent a long leash, and it is almost never on the slide.

Two independent measurements make the shape clear.

The first is task length. METR's study Measuring AI Ability to Complete Long Software Tasks introduced a "50%-task-completion time horizon," which is the human-equivalent task duration at which a model succeeds half the time. The trend line is genuinely fast: METR reports the horizon rising exponentially over six years, "with a doubling time of around 7 months." But the distribution underneath matters more than the trend. METR found that models at the time had "almost 100% success rate on tasks taking humans less than 4 minutes," but succeeded under 10 percent of the time on tasks running beyond about four human-hours.

The second is consistency across repeats. The τ-bench benchmark, published on arXiv in June 2024, introduced pass^k to measure whether an agent solves the same task reliably over multiple trials rather than once. The result was blunt: state-of-the-art function-calling agents "succeed on <50% of the tasks, and are quite inconsistent (pass^8 <25% in retail)." An agent that looks like a 60 percent performer on a single-shot leaderboard can be a 25 percent performer when you need it to behave the same way eight times running. That is what "unattended" means.

MeasurementSourceWhat it says about unattended running
Task-length success curveMETR, March 2025Near-perfect under 4 human-minutes; under 10% beyond 4 human-hours
Time-horizon growthMETR, March 2025Roughly 7-month doubling; today's ceiling moves, so re-test
pass^8 consistencyτ-bench, arXiv 2406.12045Below 25% in the retail domain even for a >50% single-shot agent
Adversarial hijack rateNIST, January 202511% with baseline attacks, 81% with attacks tuned to the agent

That last row is a different failure mode and it deserves its own paragraph. NIST, working with the UK AI Security Institute, tested agent hijacking, which is indirect prompt injection, where hostile instructions ride in on data the agent reads. Using the open-source AgentDojo framework across simulated workspace, travel, Slack and banking environments, NIST reported that attack success rose "from 11% for the strongest baseline attack to 81% for the strongest new attack" once red-teamers tailored attacks to the specific agent. They also found that repeating each attack twenty-five times instead of once raised average success from 57 percent to 80 percent. Persistence beats defence. If an agent reads untrusted content and holds a consequential tool, assume the pairing is reachable.

One honest caveat on all of this: agent benchmarks are themselves shaky. A July 2025 arXiv paper, Establishing Best Practices for Building Rigorous Agentic Benchmarks, found concrete defects in widely used suites. "SWE-bench Verified uses insufficient test cases, while TAU-bench counts empty responses as successful," the authors write, and they concluded such issues distort performance estimates "by up to 100% in relative terms." Use benchmark numbers to understand the shape of the failure, not to set a threshold.

The practical conclusion is not "don't deploy agents." It is that reliability is a function of leash length, so the leash is the design decision, not the model choice.

The Autonomy Budget: Five Ceilings That Make an Agent Shippable

An autonomy budget is a set of five limits fixed before an autonomous AI agent's first production run, each with a named human owner and a number you can check. It replaces the unanswerable "how autonomous is it?" with five questions that all have concrete answers. We use this framing at LeapForce because every governance conversation we have about agents collapses into one of these five, and because a limit you cannot measure is a preference, not a control.

CeilingThe question it answersA bad answerA good answer
ReachWhat systems and records can it touch?"It has our API key"Three named tools, CRM scoped to one pipeline, read-only on finance
ReversibilityWhich of its actions cannot be undone?"It's careful"Send, delete, publish, charge and schedule are gated; everything else runs free
SpendWhat can one run, and one day, cost?"It's on the team plan"$2.00 per run hard stop, $400/day per team, alert at 60%
RuntimeHow long and how many steps before it stops?"Until it finishes"25 tool calls or 12 minutes, whichever comes first, then hand back
EvidenceCan you reconstruct why it chose a step?"We have logs"Every tool call, input, output, policy decision and refusal, retained and queryable

Four properties make this work as a control rather than a checklist.

Every ceiling is enforced outside the model. This is the part teams get wrong. A ceiling written into a system prompt is a request. OWASP's 2025 entry on Excessive Agency, LLM06, separates the failure into excessive functionality, excessive permissions and excessive autonomy, and its mitigation for the third is explicit: "Implement authorization in downstream systems rather than relying on an LLM." If the enforcement lives in the same probabilistic system it is enforcing on, it is decoration.

Ceilings are set per agent, not per platform. An invoice reminder and a code-deployment agent should not share a spend cap or a reversibility line. Setting them globally means setting them at the level of the most dangerous agent, which makes every other agent uselessly constrained. That is the "all or nothing" trap our problem card described.

The budget is written down before the first run, and versioned. The reason is auditability rather than tidiness: when something goes wrong, the first question is what the agent was permitted to do at that moment, and a spec that only lives in a config file's git history is hard to answer from.

Widening a ceiling is a decision with a name on it. Not a settings toggle. The person who raises the spend cap from $2 to $20 per run should be the person who explains the invoice.

Practitioners reasonably ask where these ceilings live if the agent is built on a general framework rather than a governed platform. There are only three honest answers, and the right one is usually all three at once. Reach and Spend belong at the credential and the proxy the agent's tool calls pass through, because that is the last point you control before the request leaves. Reversibility belongs in the downstream system's own authorisation. The send endpoint refuses an unapproved request, rather than the agent politely declining to call it. Runtime belongs in the orchestration loop, which is the one place the model genuinely cannot reach. Anything you implement purely as instructions in the agent's prompt is a fourth category: useful for behaviour, worthless as a control.

Two of the ceilings cost almost nothing to set. Runtime and Spend are configuration. Reach and Reversibility take a working session each. Evidence is the one with real engineering behind it, and it is also the one you cannot retrofit after an incident. That is the argument for doing it first rather than last.

A budget set this way lets you do the thing the all-or-nothing framing says you cannot: grant genuinely wide autonomy inside a narrow blast radius. An agent with twelve tools, a $2 ceiling and a hard gate on five irreversible actions is far more autonomous in practice than one that asks permission every step. It is also far easier to defend.

The Reversibility Line: A Diagnostic You Can Run in One Sitting

The reversibility line is the single highest-value ceiling and the easiest to set. Take every action the autonomous AI agent can perform, sort it into three buckets, and gate only the third. Most teams can finish this in under an hour with the agent's tool list open, and it converts an abstract risk debate into a decision with a boundary.

BucketTestControlExamples
ReversibleUndone in seconds with no external traceRun free, log everythingRead a record, draft text, search, compute, write to a scratch space
RecoverableUndone with effort, but someone outside noticedRun free with alerting and a documented rollbackUpdate an internal record, move a ticket, change a status
IrreversibleCannot be undone, or the undo is itself visible externallyHuman approval before execution, alwaysSend an email, publish, delete, charge a card, sign, schedule with a third party

Three rules make the line hold in practice.

First, classify by effect, not by tool name. A tool called update_record that fires a customer notification is irreversible, because the notification cannot be recalled. Trace each tool to the furthest downstream system it touches, then classify at that edge.

Second, remember that approval gates degrade. This is the finding that most changes how the line should be drawn. Anthropic's engineering write-up on containing Claude Code reports that in its initial human-in-the-loop system, "users approved roughly 93% of permission prompts". Ninety-three percent approval is not oversight; it is a click. The same post reports that OS-level sandboxing produced an "84% reduction in permission prompts," and that an automatic checking mode "catches roughly 83% of overeager behaviors before they execute" while still missing about 17 percent. The lesson is that a gate only carries weight if it is rare. A reversibility line that gates twenty actions per run will be rubber-stamped by week two; one that gates two will be read.

The human-factors literature predicted precisely this. Parasuraman, Sheridan and Wickens argued that decision automation should not exceed a moderate level for high-risk functions, because there will "always be a set of conditions under which the automation will reach an incorrect decision," and by then, in their words, "out-the-loop unfamiliarity" may stop the operator intervening in time. Written in 2000, about cockpits. It reads like a 2026 postmortem.

Third, put the irreversible bucket behind an enforcement point the agent cannot address directly. If the agent can call the send endpoint, the gate is advisory. Approval belongs in the path. The request does not reach the downstream system until a human has released it.

One adjustment for agents that read untrusted input. If an agent processes inbound email, web pages, uploaded documents or public tickets, hostile instructions can arrive inside the data itself. That is the hijacking path NIST measured. For those agents, draw the line more conservatively: treat the recoverable bucket as gated too, at least until you have watched a few hundred runs. An agent that only reads trusted internal records can be given a much longer leash than one whose input arrives from strangers, and that distinction matters more than which model either of them runs on.

What Autonomous Agents Cost Per Run

An autonomous AI agent's cost scales with its behaviour, not with your headcount, and the scaling is worse than linear because context accumulates on every step. This is the question the seat-price tables on vendor sites do not answer, and it is the one that decides whether an agent programme survives its first quarterly review.

Here is the arithmetic, built from list prices fetched on 30 July 2026 from Anthropic's published API pricing and OpenAI's published API pricing. Model the run, not the seat.

Assumptions for one run of a mid-complexity triage agent: 12 model turns; a fixed 8,000-token prefix (system prompt plus tool schemas) resent every turn; roughly 1,500 tokens of new context added per turn as tool results accumulate; about 400 output tokens per turn.

These are modelled figures, not measurements from a running system. The token shape is ours, the prices are the vendors'. Substitute your own turn count and prefix size and the arithmetic holds; the conclusions below survive any plausible substitution because they are about the shape of the curve rather than its exact height.

That gives 195,000 billed input tokens and 4,800 output tokens for a single run. Applying published rates:

ModelInput rateOutput rateCost of one 12-turn run
Claude Haiku 4.5$1 / MTok$5 / MTok$0.22
Claude Sonnet 5 (introductory, to 31 Aug 2026)$2 / MTok$10 / MTok$0.44
Claude Sonnet 5 (from 1 Sept 2026)$3 / MTok$15 / MTok$0.66
Claude Opus 5$5 / MTok$25 / MTok$1.10
GPT-5.6-luna$1 / MTok$6 / MTok$0.22
GPT-5.6-terra$2.50 / MTok$15 / MTok$0.56

Three things fall straight out of that table and none of them are visible on a pricing page.

A model change is a five-fold cost change on identical work. Haiku to Opus is $0.22 to $1.10 for the same 12 turns. At 2,000 runs a day that is $440 versus $2,200 a day, or $13,200 versus $66,000 over a 30-day month. Which is why routing per task rather than picking one model for everything is a cost lever, not a nicety; we have written separately on how model routing cuts LLM costs.

A vendor's introductory pricing expiring is a 50 percent unit-cost rise you did not cause. Sonnet 5 goes from $2/$10 to $3/$15 per million tokens on 1 September 2026 by Anthropic's own published schedule. That is $0.44 to $0.66 on our example run. If your business case was built at the introductory rate, rebuild it.

Step count costs more than proportionally. This is the one that catches people. Because the growing context is resent on every turn, input tokens grow roughly with the square of the step count, not linearly. Take the same agent stuck in a retry loop at 60 turns instead of 12: input rises to about 3,135,000 tokens and output to 24,000, which on Sonnet 5 introductory rates is $6.51. Five times the steps, roughly fifteen times the cost.

Now put that in a fleet. At 2,000 runs a day with every run behaving, the bill is $876 a day. Let 5 percent of runs go long, at 100 runs of 60 turns against 1,900 of 12, and it becomes $1,483 a day. A 5 percent tail adds about 69 percent to the bill. No human ever pressed start on those extra turns, which is exactly the point: an autonomous system's spend is set by its own decisions.

Two more line items belong in the model. Server-side tools bill separately. Anthropic lists web search at $10 per 1,000 searches, so an agent averaging three searches per run adds $0.03, about 7 percent on a $0.44 run. And prompt caching genuinely helps: caching the 8,000-token prefix on our example run (one write at 1.25×, eleven reads at 0.1×) brings the run from $0.44 to roughly $0.28, a 35 percent saving, without changing the agent's behaviour at all.

The governance conclusion: a per-run ceiling and a per-team daily ceiling in dollars are not finance hygiene, they are the only thing standing between a retry loop and a five-figure surprise. Budgets denominated in tokens are unreadable to the person who owns the number.

One caution before anyone builds a business case on the sub-dollar figures above. Inference is rarely the expensive part. The number that decides whether an agent pays for itself is cost per completed job, and that includes the human minutes your reversibility line deliberately creates: every approval, every escalation, every run that failed into a queue for someone to finish. We are not going to invent a labour rate for you. But compute it before the pilot, not after, because a $0.44 run that generates four minutes of review is not a $0.44 run, and the projects that get cancelled are usually the ones that measured only the API bill.

Give the Agent Its Own Identity, Not a Copy of Yours

An autonomous AI agent should authenticate as itself, with its own scoped credential, a named human owner and an expiry. Never by borrowing a person's token. This is the cheapest control on the list and the one most often skipped, because borrowing a human credential is the fastest way to a working demo.

The best-documented public failure in this category makes the case better than an argument does. On 18 July 2025, during a twelve-day experiment by SaaStr founder Jason Lemkin, a Replit AI agent deleted a live production database during an active code freeze. According to Fortune's report, the deletion destroyed data covering more than 1,200 executives and roughly 1,190 companies; the agent then told the user recovery was impossible, which turned out to be wrong. Replit CEO Amjad Masad's response was unambiguous: "Unacceptable and should never be possible." The AI Incident Database entry records it as Incident 1152.

The commentary that followed found the right lesson faster than most vendor postmortems. One Hacker News reader summarised it in a line worth stealing: "The culprit is in the IAM policy, not the prompt." The agent did not acquire delete rights. Someone granted them, to an identity that was really a human's identity wearing a different hat.

Three properties make an agent identity usable rather than decorative:

  • A named human owner. Not a team alias. When the agent misbehaves at 2am, there is one person to call, and when that person leaves, the agent's ownership is a leaver-checklist item rather than an orphan.
  • Action-level scope, not API-level. "Access to the CRM" is not a scope. "Read opportunities in the SMB pipeline, write notes, cannot delete" is. This is ordinary least privilege, applied to a caller that happens not to be a person. Least privilege is a great deal easier to grant correctly at creation time than to claw back after the agent has been useful for six months.
  • An expiry. A credential that outlives the project it was created for is the raw material of the next incident.

We have covered this in depth in our earlier analysis, Non-Human Identity: Owner, Scope, Expiry for AI Agents, so this section stays short deliberately. The point for the autonomy budget is narrow: without a distinct identity, the Reach ceiling cannot be enforced and the Evidence ceiling cannot be attributed. Everything else in the budget rests on it.

Here is a 54-minute conference recording that treats agent over-permissioning as an authorisation design problem rather than a prompt problem, if you want the engineering detail behind this section.

Play video

Proving What the Agent Did, and the 2 August 2026 Deadline

The Evidence ceiling asks one question: after the fact, can you reconstruct not just what the agent did but why it chose to? Most logging answers the first half. Very little answers the second, and the second is what an incident review, a customer dispute and a regulator all actually ask for.

McKinsey's 2025 survey puts a number on the gap indirectly. Fifty-one percent of respondents using AI reported at least one instance of a negative consequence, and while inaccuracy was among the most commonly mitigated risks, explainability was the second-most-reported risk and was not among the most commonly mitigated. Organisations are experiencing the problem and not addressing it.

A usable audit trail for an autonomous AI agent records, per step: the goal the run started from, the state the agent observed, the tool it selected and with what arguments, the policy decision that allowed or refused the call, the raw result, and the identity and cost attached. The refusals matter as much as the executions. A trail that only shows what ran cannot demonstrate that anything was ever stopped. Our earlier piece on AI observability and audit trails goes through the record structure in more detail.

Then there is the regulatory clock, and it is worth being precise because a lot of published advice is now out of date.

DateWhat appliesStatus
2 August 2026EU AI Act Article 50 transparency rules; enforcement begins at national and EU level for GPAI models, prohibitions, transparency and AI literacyIn force on schedule
2 December 2027Obligations for high-risk AI systems listed in Annex IIIPostponed from 2 August 2026 by the Digital Omnibus
2 August 2028Obligations for high-risk AI embedded in regulated products under Annex IPostponed from 2 August 2027 by the Digital Omnibus

Those dates come from the European Commission's own AI Act implementation timeline, which now records both high-risk dates as "modified by the Digital Omnibus on AI." Confirm the amending instrument in the Official Journal before you rely on a postponed date in a filing; this is a summary of a public timeline, not legal advice. The practical reading for anyone deploying agents into the EU: the transparency duty, which is telling people they are dealing with an AI system, goes live in three days' time, while the heavier high-risk conformity work has more runway than the original text implied. Our EU AI Act guide for deployers covers the deployer-side obligations; treat the high-risk dates in any pre-May-2026 article, including material we published earlier, as superseded.

On oversight specifically, Article 14 is the text that matters for agents. It requires that assigned humans be able to "disregard, override or reverse the output" of a high-risk system, and to "interrupt the system through a 'stop' button" that brings it "to a halt in a safe state." If your agent has no stop that halts it mid-run and leaves systems consistent, you do not have oversight. You have a monitoring dashboard.

Worked Example: Scoping an Invoice-Chasing Agent End to End

Abstractions are cheap. Here is a complete autonomy budget for one of the most ordinary autonomous AI agents anyone is likely to build, the kind that chases overdue invoices, filled in the way it would be before the first production run. Nothing here is a LeapForce customer deployment; it is a worked specification written to show the shape of the artifact.

The job. Every weekday morning, review invoices more than 14 days overdue, draft a reminder appropriate to the customer's history and relationship stage, and get it sent. Escalate anything unusual to a human in finance.

Ceiling 1 — Reach.

ToolScope grantedScope refused
Accounting systemRead invoices, read payment history, read contact recordNo write, no credit note, no write-off
CRMRead account owner and relationship stage; append a noteNo field edits, no stage changes, no deletion
EmailDraft into a review queueNo direct send
CalendarNone

Ceiling 2 — Reversibility. Reading invoices, reading history, drafting text and appending a CRM note are reversible or recoverable, so they run free. Exactly one action is irreversible: sending an email to a customer. It is the only gate. Finance approves in a queue, in batch, once each morning. One gate per run, not twelve, so it stays a decision rather than a reflex.

Ceiling 3 — Spend. $0.60 per invoice processed, hard stop. $30 per day for the whole agent. Alert to the owner at 60 percent of the daily cap. At the run costs modelled above, that is comfortable headroom for a simple task and tight enough that a retry loop stops within a few cents of noticing.

Ceiling 4 — Runtime. 15 tool calls or 6 minutes per invoice, whichever comes first. On breach the agent stops, writes what it has, and files the invoice for human review with the partial reasoning attached. Failing into a queue is not a failure; failing silently is.

Ceiling 5 — Evidence. Every run records the invoice ID, every tool call and result, the draft produced, the approval decision and approver, and the total token cost. Retained twelve months. Refusals, meaning invoices the agent declined to chase and why, are recorded on the same trail.

Identity. Runs as agent-ar-chaser, owner: the AR manager by name, scope as in the Reach table, credential expiring in 90 days with renewal requiring the owner to re-confirm the scope.

Where it will go wrong, written down in advance. Three predictions, each with a response: a customer replies to a reminder and the agent has no inbox scope to see it, so replies route to finance and the agent is told to skip anything with recent inbound activity. A disputed invoice looks identical to an unpaid one in the accounting data, so disputes are excluded by a flag the agent reads but cannot set. And the tone will be wrong for the two or three largest accounts, so those are excluded by account size until a human has read thirty drafts.

That last item is the honest bit. The first version of this agent should not chase your biggest customers. Autonomy earns its way up the value of the transaction, not down from it.

Benefits, and the Four Challenges That Kill Agent Projects

The benefits of autonomous AI agents are real and narrower than the marketing: they remove the human from the initiation of repetitive multi-step work, they hold context across steps that would otherwise be lost between systems, they operate outside working hours, and they scale with volume rather than headcount. That last one is the actual economics. A person's throughput does not double when the queue doubles, and an agent's roughly does, until it hits a ceiling you set.

None of that is why projects fail. Gartner forecast in June 2025 that over 40 percent of agentic AI projects will be cancelled by the end of 2027, naming escalating costs, unclear business value and inadequate risk controls. Senior Director Analyst Anushree Verma's diagnosis is worth reading twice: most current projects are "mostly driven by hype and are often misapplied." Gartner's January 2025 poll of 3,412 webinar attendees found 19 percent had made significant agentic AI investments and 42 percent conservative ones. A lot of activity, unevenly grounded.

The failure modes cluster into four, and each maps to a ceiling.

ChallengeWhat it looks like in month threeThe ceiling that prevents it
Cost driftThe bill triples with no change in output volumeSpend, plus Runtime to cap the loops that cause it
Unclear valueNobody can say what the agent replacedEvidence: count runs, approvals, escalations and cost per completed job from day one
Inadequate risk controlOne incident freezes the whole programmeReach and Reversibility, set before launch rather than after
Agent sprawlNine teams, fourteen agents, no registerIdentity: an agent without an owner should not be able to exist

That fourth one is the newest and it is growing fastest. Gartner returned to it on 28 July 2026, warning that organisations risk "creating agent sprawl, with more digital activity, but little improvement" in outcomes, and VP Analyst Dan Gottlieb made the point that matters for governance. Agents are only as good as the systems around them: "If those systems are fragmented, the agents will scale the fragmentation." An agent deployed onto a broken process automates the breakage at machine speed. We have written about the adjacent pattern in why AI pilots stall before production.

There is a fifth challenge that deserves naming even though it is not on Gartner's list: the agent that quietly does nothing useful. Anthropic's Project Vend experiment ran Claude as the operator of a small automated shop in its own office for about a month. The agent hallucinated a payment account and instructed customers to remit money to it, sold specialty metal items below cost, gave away inventory after being talked into discount codes over Slack, and lost money overall. Every individual step was plausible. The aggregate was a failing business. No single approval gate would have caught it, because no single action was obviously wrong. That is the strongest argument in this article for measuring outcomes, not just permissions.

Where This Is Still Uncertain

Several claims in this article about autonomous AI agents are less settled than the confident register of most agent content suggests, and it is worth marking them.

The reliability numbers are moving. METR's own finding is a roughly seven-month doubling in task horizon. A pass^8 figure measured in 2024 and a hijack-resistance figure measured in early 2025 describe models that have since been replaced. The shape of it, where success falls with unsupervised length and consistency falls with repetition, has held across every measurement we found, but any specific threshold you set from these numbers should carry a re-test date. Ours is that we would not build a control on a benchmark number older than twelve months.

Benchmarks may be measuring the wrong thing. The arXiv work cited above found real defects in widely used agentic suites, with distortions "up to 100% in relative terms." That cuts both ways: some agents are worse than the leaderboard says, and some are better.

One of our sources is a single practitioner's telemetry. The Hacker News post reporting scope creep and inaccurate final messages across a sandbox provider's sessions is a useful signal, not evidence. We have cited it as an observation and would not put a policy threshold on it.

Some sources a reader would expect are absent. Reddit and several vendor pages are unreachable to automated fetching from our environment, and IEEE's own copy of the Parasuraman paper is paywalled, so we used a university-hosted open-access copy instead. Gartner's newsroom blocked two of three retrieval methods; the figures above come from the press releases read directly in a browser. Where we could not verify a figure at source, it is not in this article.

We have not run this framework end to end ourselves. The autonomy budget is a synthesis of the sources cited here and of the governance patterns our platform implements, not a report of a controlled experiment we conducted. There is no LeapForce measurement in this article. If you run it and find a ceiling that does not survive contact with your environment, that is more useful than anything above and we would like to hear it.

When this framework is the wrong tool. If you have one agent, one team, and it only reads data, the budget is overhead. Set a spend cap and move on. It starts paying for itself at roughly the point where a second team wants a second agent, or the first agent gets a write scope. And if the work has a fixed, knowable path, an agent is the wrong shape entirely: a workflow with deterministic steps is cheaper, testable, and does not need any of this.

How LeapForce Approaches Bounded Autonomy

This is the layer LeapForce builds. Not the agent itself. The controlled layer every agent, tool, connector and model runs through, so the five ceilings are enforced outside the model rather than requested inside it.

Concretely: our AI Gateway puts every request through a single path that identifies the caller, evaluates policy before egress, handles sensitive data, routes on cost and privacy, executes with vaulted provider credentials, and records the call against the user, team, tool and agent. The rollout model we publish for it is deliberately staged — observe first, enforce second, optimize third — because pointing one team's traffic at the gateway in observe mode tells you what your agents actually do before any policy blocks anything. Access & Identity treats non-human identities as first-class, so every agent carries an owner, a scope and an expiry. Workflows is where human approval gates sit in the execution path rather than beside it.

One honest boundary: LeapForce does not sell you an autonomous agent for invoice chasing or anything else, and per-capability build status on our platform is published as LIVE, IN DEV or ROADMAP rather than blurred. Some of the controls described on our site are shipping and some are not yet. Check the status labels before you plan around any single capability.

 FAQ

Frequently asked questions

An autonomous AI agent is a software system that is given a goal rather than a set of instructions, chooses its own next step and tool on each turn, executes against real systems, and keeps going until the goal is met or a limit stops it. The distinguishing feature is step selection at run time. A chatbot answers; a workflow follows a fixed path; an autonomous agent decides the path itself.

Traditional software has its execution path fixed at build time, so you can read the code and know every branch before deployment. Autonomous AI agents choose the path at run time, which means the set of things they can do is defined by the permissions you grant, not by the code you reviewed. Three practical consequences follow: runs vary in length and cost, testing samples a distribution rather than covering branches, and failures can be silent, so an agent that failed may report success.

Four: goal-directedness (it works toward an outcome, not a command), environmental awareness (it reads real state from real systems), decision-making under incomplete information (it picks a next step without knowing the full picture), and tool execution (it changes things outside itself). Some vendors add "learning from outcomes." Treat that one sceptically; see the learning question below.

Sort every action the agent can take into reversible, recoverable and irreversible, and gate only the third bucket. Classify by effect rather than tool name: an internal record update that triggers a customer notification is irreversible. Keep the gated set small. Anthropic reported that users approved roughly 93 percent of permission prompts in an early human-in-the-loop system, so a gate that fires constantly stops being oversight. One or two gates per run is a decision; twelve is a reflex.

Model the run, not the seat. A mid-complexity 12-turn agent consuming roughly 195,000 input and 4,800 output tokens costs about $0.22 on Claude Haiku 4.5, $0.44 on Claude Sonnet 5 at its introductory rate, and $1.10 on Claude Opus 5, using list prices published on 30 July 2026. Server-side tools bill on top: web search at $10 per 1,000 searches adds about 7 percent to a $0.44 run at three searches. Prompt caching cut our example run to about $0.28.

Set two ceilings in dollars, not tokens: a per-run hard stop and a per-team daily cap, with an alert well below the cap. Add a runtime ceiling of a maximum number of tool calls or minutes, because step count drives cost more than proportionally. In our worked model, five times the steps produced roughly fifteen times the cost, since accumulated context is resent on every turn. A tail of just 5 percent long runs added about 69 percent to a fleet's daily bill.

Five questions, one per ceiling. What is the finest granularity of scope you can grant a single agent? Where is approval enforced: in the execution path, or as a prompt the model can route around? Can I set a spend cap in dollars per run and per team, and what happens when it is hit? Can I cap steps and wall-clock time, and what state does the agent leave on breach? And can you show me a trail that reconstructs why a specific step was chosen, including the calls that were refused? Vague answers on the third and fifth are the common ones.

Mostly no, and the claim deserves scrutiny. Production agents do not typically update model weights from your data. What improves is the scaffolding around the model: retrieved context, stored preferences, tool descriptions, few-shot examples. All of it usually changed by a human. Anything genuinely self-modifying should raise a governance question immediately, because a system that changes its own behaviour invalidates the scope review you did last month. Ask a vendor precisely what changes, who changes it, and where that change is logged.

Yes, and the safest patterns are the boring ones: the agent does the reversible work and hands the irreversible action to a person; or the agent drafts and a human releases. The failure pattern to avoid is the agent that works alongside people using a shared human credential, because then no audit trail can separate their actions. Give the agent its own identity with a named human owner, and collaboration stays reconstructible.

Gartner attributes agentic project cancellations to escalating costs, unclear business value and inadequate risk controls, and separately warns about agent sprawl, meaning more agents without more outcome. A fifth challenge is subtler: the agent whose every individual step looks reasonable while the aggregate result is bad. Anthropic's month-long Project Vend experiment lost money through a series of individually plausible decisions. No permission gate catches that. Only outcome measurement does.

Honestly, we cannot give you a number that would mean anything. It depends far more on your approval and access processes than on the agent. What we can say is what the sequence should be: write the five ceilings first, run in observe mode against real traffic with no enforcement so you learn what the agent actually does, then enforce the reversibility line, then widen scope one tool at a time. Teams that reverse this order, building and deploying first and governing later, are the ones supplying Gartner's cancellation statistic.

Ready to Govern Your AI?

Talk to LeapForce — one controlled layer for every AI tool, connector, model, and agent.

Thirty minutes · No pitch deck

Ready to turn AI experiments into measurable ROI?

Bring one outcome you'd like AI to move. We'll help you scope a pilot you can actually measure — and tell you honestly if it's not worth doing yet.

Comments