An AI customer service agent is a piece of software holding a job description: it reads a customer's problem, decides what the company owes them, and either says so or does it. The only question that matters on day one is how much it may settle on its own authority, and the answer should be a written number, not a model setting.
Most guides to AI customer service agents scope them by accuracy. We think that is the wrong variable. A human agent who is right 96% of the time makes uncorrelated mistakes: different customers, different reasons, one at a time. An AI customer service agent that is right 98% of the time makes the same mistake on every instance of a contact reason until somebody catches it. That is not a lower error rate. It is a smaller number of much larger incidents, and the authority limit has to be sized for the incident, not the rate. The problem is live in the field: on Hacker News in May 2026, a developer launching his own open-source policy layer for agents put it as "system prompts are not a security boundary" and asked the thread what others actually use to control an agent at runtime (rohitguptap, Hacker News item 48199691). He is selling a solution to the problem he describes, which is worth knowing. The example he reached for is not in dispute either way: a support agent trying to refund $1,000 against a $500 policy.
The short answer: Give an AI customer service agent a written authority schedule — per contact reason, what it may settle alone, up to what value, on what evidence, and what forces a human — and size every ceiling by the number of past interactions you would have to reopen if its standing ruling on that reason turned out to be wrong.
Last updated: July 31, 2026.

The Authority Schedule: what a customer service agent may settle alone, and what forces a human.
What an AI customer service agent actually is, as a staffed role
An AI customer service agent is a language model wired to your support tools with permission to act. It can read an order, check a policy, answer, and in most deployments also issue a credit, change an address, cancel a subscription or open a return. Stripped of the category language, it is a headcount decision with a software delivery mechanism attached. Everything that makes a junior human agent safe on their first week applies unchanged: a written scope, a spending limit, a named supervisor, and a rule for when to get someone senior.
The vendor category describes these systems by capability: memory, tool use, multi-step planning, autonomy. Those are real and they matter for whether the thing works. They tell you nothing about what it is permitted to do, and permission is where the money and the liability live. A support organisation that can answer "what is our AI agent allowed to settle without a human" in one sentence is in control of it. One that answers "well, it has access to the refund API" is not.
The distinction we are drawing here is deliberately narrower than the one in our earlier analysis of the write-access line. That piece asks whether a system should be allowed to write at all. This one assumes the answer is yes and asks the next question, which almost nobody writes down: how much may it commit, and on what evidence. Similarly, the read-say-do ladder sorts which contact reasons to automate. An authority schedule takes the reasons you already decided to automate and puts a ceiling on each one.
One honest note about first-hand experience. We have not run a customer-facing support agent at production volume, so every arithmetic example below is a worked illustration with its assumptions stated, not a measurement from our own queue. Where we use a real number it comes from a named source and is linked. That is the whole basis on which the article should be read.
Why accuracy is the wrong variable: correlated error
Accuracy comparisons between AI agents and human teams are close to meaningless because the two failure distributions have different shapes. Human error in a support queue is approximately independent: one agent misreads one policy on one ticket, and the next ticket on the same policy is handled correctly by someone else. An AI customer service agent applies one interpretation to every instance of a contact reason, at machine consistency, until a human notices. The error rate can fall while the worst-case incident grows.
This is not a novel observation about AI in general. NIST names it structurally: among the risks it calls new or increased compared with traditional software, its framework lists "AI system scale and complexity (many systems contain billions or even trillions of decision points) housed within more traditional software applications" and the "inability to predict or detect the side effects of AI-based systems beyond statistical measures" (NIST AI Risk Management Framework 1.0, Appendix B). Statistical measures are exactly what an accuracy score is. It tells you the rate. It does not tell you whether the misses are spread across a hundred customers or concentrated on one policy that you got wrong for eleven days.
Put the two side by side on the same volume and the asymmetry is obvious.
| Human team, 96% correct | AI agent, 98% correct | |
|---|---|---|
| Wrong outcomes per 10,000 contacts | 400 | 200 |
| Distribution | spread across most contact reasons | concentrated in the reasons where the ruling is wrong |
| Largest single group of identical wrong outcomes | typically a handful | potentially all 200 |
| Detection route | a customer complains, a QA sample catches one | a customer complains, or nobody does |
| Remediation | reopen a few tickets | reopen every interaction on that reason since the change |
The right-hand column is the case for a lower authority ceiling on a more accurate system, which is the counterintuitive part and the reason procurement gets this backwards. Buyers negotiate hard on benchmark accuracy and sign off on scope with a shrug. The scope is the part that determines what a bad week costs.
There is a second, better-documented mechanism pushing the same way. Support conversations are multi-turn, and multi-turn is where models degrade. Researchers testing top open- and closed-weight models across six generation tasks reported "an average drop of 39%" against single-turn performance, and described the mechanism as models committing to a solution early and then, once they have taken "a wrong turn in a conversation", failing to recover (Laban, Hayashi, Zhou and Neville, LLMs Get Lost In Multi-Turn Conversation, arXiv, May 2025). A support conversation is the archetypal multi-turn task with underspecified requirements. The single-turn benchmark your vendor quotes is measuring the easy shape.
The recall number, and how to compute yours
The recall number is the count of past interactions you would have to reopen if the agent's standing ruling on one contact reason turned out to be wrong. It is the number that should set every ceiling in your authority schedule, and it takes about twenty minutes to compute from data you already have.
Recall number = daily volume of that contact reason
x share the agent settles alone
x days to detection
Days to detection is the term teams get wrong. It is not your SLA and it is not how fast you could notice. It is how long a wrong-but-plausible ruling survives your actual review process, and the honest way to get it is to measure it rather than estimate it. The reason it is usually longer than people expect is that a reviewer has to recognise a policy error, not a tone error, in a conversation that reads perfectly well.
A worked illustration, with every assumption on the surface, and one population throughout. Assume 8,000 contacts a day; the contact reason "cancellation within the refund window" is 5% of that, so 400 a day; the ruling that turns out to be wrong is a policy interpretation that applies to every contact on that reason, not to a subset of it; the agent settles 70% of them alone.
| Input | Value | Where it comes from |
|---|---|---|
| Daily volume, this contact reason | 400 | your contact-reason report |
| Share settled by the agent alone | 70% | your containment report, per reason |
| Days to detection | 11 | an assumption, not a derivation — see below |
| Recall number | 3,080 interactions | 400 x 0.7 x 11 |
| Average exposure per interaction | $34 | mean refund value on this reason |
| Direct exposure | ~$105,000 | 3,080 x $34, before reopen labour or goodwill |
Be honest about the eleven. It is a chosen middle value, not something we derived, and we would rather say so than dress an assumption as a calculation. Random QA at 2% of all contacts puts roughly eight conversations from this reason in front of a reviewer each day, which sounds like plenty until you notice what the reviewer has to do: spot a policy error inside a conversation that reads perfectly well, on a reason that is spread across a review team so that any one reviewer meets it only occasionally. And if the ruling is wrongly generous rather than wrongly harsh, no customer complains at all, so the complaint channel contributes nothing. Measure your own figure rather than borrowing ours. Seed a known-wrong ruling into a test cohort and time how long your process takes to catch it.
The sensitivity is the point, not the single number.
| Days to detection | Recall number | Direct exposure at $34 |
|---|---|---|
| 3 | 840 | ~$29,000 |
| 11 | 3,080 | ~$105,000 |
| 20 | 5,600 | ~$190,000 |
Correlated error concentrates the same 10,000 contacts into one incident instead of many.
Three thousand reopened conversations is not a support problem. It is a communications problem, a finance problem and, depending on your jurisdiction and sector, a regulatory notification problem. And note what the arithmetic does not contain: the model's accuracy. The recall number is a property of your volume and your detection speed, not of the vendor's benchmark.
Two levers move it, and only two. Cut the share settled alone on high-recall reasons, which is what an authority ceiling does. Or cut days to detection — that is what stratified QA does, and we come back to it below. Buying a better model moves neither.
The Authority Schedule: five columns, one sitting
The Authority Schedule is a table with one row per contact reason and five columns. It is the customer service equivalent of a signing authority matrix, and a support lead can draft the first version in an afternoon from an existing contact-reason report.
| Column | What goes in it | The test it has to pass |
|---|---|---|
| Contact reason | the reason as it appears in your own taxonomy | someone in the queue recognises it without translation |
| What it may settle alone | the specific action, in verbs: "state eligibility", "issue store credit", "cancel and refund" | a reviewer can tell from a transcript whether the agent stayed inside it |
| Value ceiling | a currency amount, or "no monetary effect" | it is a number, not "small" |
| Evidence required before acting | the facts that must be true in the record first | each one is machine-checkable, not a judgement call |
| Escalation trigger | the conditions that force a human, stated as rules | it fires deterministically, not on model confidence alone |
The computed sixth column is the recall number, which is not a policy input. It is the check on whether the other five are set sanely. Any row where the recall number is large and the value ceiling is non-zero is a row to tighten first.
Here is what four rows look like filled in. These values are illustrative; the point is the shape and the specificity, and you should replace every number with your own.
| Contact reason | May settle alone | Value ceiling | Evidence required | Escalation trigger |
|---|---|---|---|---|
| Order status | state status and ETA | no monetary effect | order ID resolves to this customer's account | customer states the item is unsafe or damaged |
| Return within window | authorise return, issue label | $150 | order date inside window; item not in the excluded-category list; no prior return this order | value over ceiling; second return this quarter; item flagged high-fraud |
| Refund after window | nothing — recommend only | $0 | n/a | always human; agent may draft the recommendation |
| Goodwill credit for late delivery | issue store credit | $25 and one per customer per 90 days | carrier record shows delivery past the promised date | customer asks for cash rather than credit; prior credit inside 90 days; customer mentions a complaint to a regulator or a chargeback |
Three things are doing the work in that table and none of them is a model capability.
The evidence column is the easiest of the five to leave blank, and it is the one that makes the schedule auditable. "Issue a credit if the delivery was late" is a policy. "Issue a credit if the carrier record shows delivery past the promised date" is a rule with a source of truth attached, and afterwards you can prove which record the agent read. Where the evidence is a judgement (is this customer's frustration genuine, is this an edge case), the honest entry in the evidence column is "cannot be evidenced", and that row belongs in the recommend-only tier.
The ceiling is per action and per customer, not per conversation. A ceiling that only limits a single act is trivially defeated by a customer who opens four conversations, and by a determined one who has worked out what the limit is. The second clause on the goodwill row — one per customer per 90 days — is the part that actually caps exposure.
The triggers are conditions, not confidence scores. Model confidence is a useful additional signal and a terrible primary one, because the failure mode described in the multi-turn research is confident wrongness: the model commits early and does not recover. Deterministic triggers fire whether or not the model thinks it is doing fine.
Where the money ceiling sits, and how to derive it
Derive the ceiling from your own refund distribution, not from a vendor default. The procedure takes one query and one afternoon, and it produces a number you can defend in a finance review.
- Pull twelve months of manual adjustments — refunds, credits, goodwill gestures — grouped by contact reason, with the value of each.
- For each reason, find the value at which your own human policy already requires a second signature. If your organisation has one, formally or by habit, it is the point where an agent asks a team lead. That threshold was set by people who understood the exposure.
- Start the agent's ceiling below that number, not at it. The human threshold assumes independent error. The agent's exposure is the recall number multiplied by the ceiling, so the same threshold buys a much larger worst case.
- Compute the recall number for that reason at your proposed ceiling. If ceiling x recall number is a figure you would not want to explain to a CFO, the ceiling is wrong.
- Set a review date, not a permanent value. Ceilings should ratchet up on evidence, and the evidence is clean QA results on that specific reason over a defined period.
Step 3 is the one that gets argued, so here is the arithmetic behind it. If your human agents may credit up to $100 without a lead's approval and they get it wrong four times in a thousand independently, a bad month is a few hundred dollars and a handful of annoyed customers. Give the same $100 to an agent that settles 280 of these a day and is wrong on that reason for eleven days, and the same "better" error rate produces a three-thousand-interaction reopen at up to $100 each. The ceiling has to fall roughly in proportion to the recall number for the exposure to stay constant. That relationship, not the model's score, is the honest basis for the first number you write down.
A note on what this costs. Nothing here requires new software. The schedule is a spreadsheet, the evidence column maps onto fields your systems already return, and the ceilings are configuration in whatever platform you have bought. The expensive part is the argument about who owns the numbers, which is why it is better had before launch than during an incident.
Goodwill is not a rounding error: what Moffatt actually cost
The best-known decision on chatbot liability gets cited for its headline finding. The detail that matters for an authority schedule is buried near the end.
In Moffatt v. Air Canada, 2024 BCCRT 149, British Columbia's Civil Resolution Tribunal held the airline liable for negligent misrepresentation after its website chatbot told a passenger he could apply retroactively for a bereavement fare, which the airline's published policy did not allow. The tribunal rejected the airline's argument outright, writing that "it should be obvious to Air Canada that it is responsible for all the information on its website" and that "it makes no difference whether the information comes from a static page or a chatbot" (Civil Resolution Tribunal, 14 February 2024). Damages were $650.88. We have covered that holding and its limits in our analysis of customer service automation, and it is a small-claims tribunal decision in one province, not binding precedent where most readers operate.
The part almost nobody quotes is paragraph 41. Air Canada argued it had already given the passenger a $200 coupon as a gesture of goodwill and should get a set-off against the award. The tribunal refused, because the passenger said he never accepted it and the airline produced no evidence otherwise.
That is an authority-schedule lesson with a price tag on it. A goodwill gesture made outside a schedule, with no record of acceptance, bought the company nothing at all. It was pure cost. Goodwill is a real and useful tool in support, and it is an obvious thing to hand an AI customer service agent because it feels low-stakes. It is low-stakes per instance, and it is a strong candidate for the highest recall number on your schedule, because it applies to a broad and vaguely-defined set of situations. Two implications follow directly. Cap it hard and cap it per customer over a window. And record acceptance, not just offer, because an unaccepted gesture is money spent with no legal or commercial effect.
The escalation trigger is a specification, not a feeling
"It escalates to a human when it can't handle something" is the standard sentence in AI customer service marketing, and it is not a control. A trigger is a control when it is written down, fires deterministically, and can be tested before launch. Five classes cover most of what a support queue actually needs, and each one should appear in the escalation column of specific rows rather than as a global setting.
| Trigger class | Fires on | Why it is deterministic |
|---|---|---|
| Ceiling breach | requested or computed value exceeds the row's ceiling | it is arithmetic on a number the agent already has |
| Evidence gap | a required evidence field is missing or does not resolve | the field is either present or it is not |
| Protected context | the customer states vulnerability, bereavement, medical or financial hardship, or a disability accommodation | keyword and classifier detection, tuned to over-trigger |
| Legal signal | mention of a lawyer, a regulator, a chargeback, a complaint body, press, or a threat of legal action | fixed vocabulary list, reviewed quarterly |
| Loop detection | the same customer returns on the same reason for the Nth time, or turn count exceeds a limit | counters, not judgement |
The loop trigger deserves its own line because it is the one that maps onto documented consumer harm. The US Consumer Financial Protection Bureau's review of chatbots in consumer finance found customers "getting stuck in 'doom loops' with chatbots" and noted these are "often caused when a customer's issue falls outside the chatbot's limited capabilities", concluding that chatbots "may be useful for resolving basic inquiries, but their effectiveness wanes as problems become more complex" (CFPB, Chatbots in consumer finance, June 2023). A turn-count and repeat-contact trigger is the cheapest available fix for that failure, and it is worth checking whether your deployment has one at all.
Two design rules that are easy to get wrong. First, triggers must be able to fire mid-action, not only at intake, because the protected-context signal often arrives several turns in, after the agent has already committed to a line. Second, a trigger that fires must be irreversible in that conversation: once escalated, the agent does not get to un-escalate because the customer calmed down. Otherwise the trigger is a suggestion.
Test them the way you would test a fire alarm. Before launch, write one scripted conversation per trigger, run it, and check the escalation actually happened and the human received what they were supposed to receive. Which brings us to the field's largest unspecified artifact.
The Handover Packet: six fields the human must receive
Escalation is treated as a routing problem, meaning get the conversation to a person, when it is really a briefing problem. The human inherits a customer who has already been told things, and inherits the company's exposure to whatever was said. What arrives with them determines whether the handover recovers the interaction or restarts it.
The Handover Packet is six fields. It is not a summary.
| Field | What it contains | Why a summary fails |
|---|---|---|
| 1. Stated goal | the customer's own words describing what they want, quoted | the agent's paraphrase is where the wrong turn started |
| 2. Full transcript | every turn, in order, not condensed | the commitment is rarely in the first or last turn |
| 3. Commitments made | every statement of policy, price, eligibility or timeline the agent made, verbatim with timestamps | this is the company's exposure and the human must not contradict it blind |
| 4. Actions taken and refused | what the agent did, what it attempted, and what it was blocked from doing with the reason | the refusals tell the human where the limit was hit |
| 5. Grounding | the policy version and knowledge sources the answers were drawn from | without it you cannot answer "what were we telling customers that week" |
| 6. Trigger | which escalation condition fired, and on which turn | tells the human what kind of problem this is before they read anything |
The six fields a human needs at escalation, and why a summary is the wrong artifact.
Field 3 is the one that pays for the whole packet. If the agent said a refund was available and the human, working from a two-line summary, says it is not, the customer now has two contradictory company positions on the record. That is a materially worse place to be than either position alone. The Moffatt reasoning applies to the first statement whether or not the second one corrects it.
Field 1 exists because of the multi-turn research above. If the model locked onto an assumption in an early turn and never recovered from it, then its summary of the conversation is generated from that same assumption. Handing the human that summary hands them the error. The customer's own words are the only part of the record that was not produced by the system under review.
Fields 4 and 5 are logging discipline more than support discipline, and they are the same fields that make an audit trail of agent actions worth having. The refusal log is the one we would check for first, because it is what proves a control existed rather than merely being configured.
The Recall Drill: what you do when the agent is wrong at scale
Every support organisation has an incident process for the site being down. Ask whether yours has one for the agent having been quietly, consistently wrong for a week and a half. The Recall Drill is four steps with a named owner and a target time for each, rehearsed once before launch on a fabricated bad ruling.
| Step | What happens | Target | Owner |
|---|---|---|---|
| Detect | a wrong standing ruling is identified, by QA, complaint pattern, or finance variance | continuous | QA lead |
| Freeze | that contact reason drops to recommend-only for the agent, immediately, without a deploy | under 15 minutes | support ops |
| Reverse | every affected interaction in the recall window is listed and corrected, with an outbound message | defined per reason | support ops plus finance |
| Notify | customers, and where required a regulator or a complaints body, are told | per your legal advice | legal |
Freeze is the step that needs building rather than writing. If dropping one contact reason to recommend-only requires a code change and a release, your containment time is your release cycle, and the recall number keeps growing while you wait. The requirement is a per-reason kill switch that a support ops lead can operate without engineering. Check whether yours can do this at contact-reason granularity at all. If it cannot, the honest fallback is a global recommend-only mode that one person can trigger, plus permanently lower ceilings to compensate for the coarser control. A blunt switch you can actually pull beats a precise one that needs a release. This is the specific capability the EU AI Act reaches for when it requires that human overseers can "decide, in any particular situation, not to use the high-risk AI system or to otherwise disregard, override or reverse the output" and can "intervene in the operation of the high-risk AI system or interrupt the system through a 'stop' button or a similar procedure that allows the system to come to a halt in a safe state" (Regulation (EU) 2024/1689, Article 14(4)(d) and (e)).
Whether a given customer service deployment is high-risk under that regulation is a classification question for your counsel, and we are not answering it here. The engineering requirement is worth adopting on its own merits regardless of classification: an override that requires a deploy is not an override.
The other half of the drill is knowing what the recall window is at the moment you need it. That means the log has to contain, per interaction, the policy version and the schedule version in force. NIST's framework flags the underlying reason: AI systems "may require more frequent maintenance and triggers for conducting corrective maintenance due to data, model, or concept drift" (NIST AI RMF 1.0, Appendix B). Corrective maintenance you cannot scope is corrective maintenance you cannot do.
Rehearse it once. Pick a real contact reason, declare its ruling wrong, and time how long it takes to freeze that reason and produce the list of affected interactions. Our expectation is that the list, not the freeze, is where you will get stuck, because interaction logs frequently do not record which contact reason the agent decided it was. If that is true for you, it is better discovered in a rehearsal than in an incident.
Supervising a workforce that is part software
Random QA sampling is built for independent error. It is a good design when a hundred agents each make different mistakes, because a random sample of the population estimates the population rate. Against correlated error it is close to the worst available design, because the thing you need to find is not a rate. It is one wrong ruling hiding inside a set of conversations that all look fine.
Stratify instead. The sampling frame is contact reasons, not conversations, and the rule is that every reason where the agent has non-zero authority gets a minimum absolute number of reviews per week regardless of its share of volume. A reason carrying 1% of your traffic and a $150 ceiling deserves the same weekly review count as one carrying 20% and no monetary effect. Under random sampling it gets a twentieth as many. Structurally, this change attacks days-to-detection, which is a multiplier in the recall number, whereas a better model changes none of the three inputs.
Three additions make the sample worth the reviewer's time.
Review outcomes, not conversations. The reviewable unit is "the agent decided this customer was eligible", a claim you can check against the policy, rather than "how did that chat go". Tone review finds tone problems. Correlated policy error reads as a perfectly pleasant conversation.
Give reviewers the schedule, not the guidelines. A reviewer holding the authority schedule can answer "did it stay inside the row" in seconds. A reviewer holding a twelve-page style guide cannot.
Watch the refusal rate per reason as a leading indicator. A reason whose escalation rate falls sharply after a prompt or policy change has not necessarily got better; it may simply have got more confident, and the two look identical from the outside. That shows up in the refusal log before it shows up in complaints.
On how many agents one supervisor can cover, we have argued elsewhere that supervision ratio rather than headcount is the real constraint on an AI workforce, and the staffing arithmetic lives there rather than here. The point specific to customer service is narrower: the review budget should be allocated by authority, not by volume, because authority is what converts an error into an incident.
What the law already assumes about your authority limit
Three settled provisions bear on where you set a ceiling. None of them tells you what the number should be, and anyone who says otherwise is selling something.
Automated decisions with significant effects. The GDPR gives a data subject "the right not to be subject to a decision based solely on automated processing, including profiling, which produces legal effects concerning him or her or similarly significantly affects him or her", with exceptions for contractual necessity, authorising law, and explicit consent. Where the contractual-necessity or explicit-consent exceptions are relied on — points (a) and (c), not the authorising-law route at point (b), whose safeguards come from that law instead — the controller must still provide "at least the right to obtain human intervention on the part of the controller, to express his or her point of view and to contest the decision" (Regulation (EU) 2016/679, Article 22). Whether a declined refund, a closed account or a denied warranty claim "similarly significantly affects" a customer is genuinely contested, varies by supervisory authority, and we are not aware of a published enforcement action resolving it for ordinary consumer support. Treat it as an open question and put it to counsel rather than resolving it from a blog post. The design implication is cheap either way: a documented route to a human for any adverse automated outcome costs little and puts you in a better position whichever way the question resolves. It does not by itself make you compliant with Article 22, and nothing in this article should be read as saying it does. We have written separately on when approval is genuinely a control rather than a rubber stamp.
Human oversight of high-risk systems. Article 14 of the EU AI Act requires high-risk systems to be designed so they "can be effectively overseen by natural persons during the period in which they are in use", and specifically that overseers can "remain aware of the possible tendency of automatically relying or over-relying on the output produced by a high-risk AI system (automation bias)". Classification is the threshold question and it is fact-specific.
Liability for what the system says. Moffatt is the clearest published statement that a company owns its bot's words, and the reasoning is intuitive enough that it has been widely followed in commentary. It is still a single provincial small-claims tribunal decision. Do not build a compliance position on it, and do not assume the opposite either.
The through-line is that none of these instruments cares about your model. They care about what the system was permitted to decide and whether a person could intervene. That is the authority schedule, in law's vocabulary.
Three authority postures, scored
Most deployments are one of three postures. Pick per contact reason, not per company; a mature schedule uses all three simultaneously.
Advisor — states, never commits. The agent answers questions and drafts recommendations; every monetary or contractual outcome is executed by a human. Best for: regulated contexts, high-value accounts, any reason where the evidence column reads "cannot be evidenced". Ceiling: zero. Recall exposure: low, and limited to what it said rather than what it did, which is not nothing, per Moffatt. Cost: you keep essentially all the handling labour and gain speed and consistency. Choose Advisor if you cannot yet compute days-to-detection for that reason.
Resolver — commits within a ceiling on reversible actions. The agent settles inside a stated value limit where the action can be undone: credits, returns, address changes, replacement shipments. Best for: the bulk of consumer support volume. Ceiling: derived per the five-step procedure above. Recall exposure: the interesting case, and the reason the recall number exists. Cost: real labour reduction, plus the QA and drill overhead that makes it safe. Choose Resolver if the action is reversible, the evidence is machine-checkable, and you have a per-reason freeze switch.
Settler — moves money or ends relationships. Cash refunds to original payment method, account closure, contract termination, anything a customer cannot undo. Best for: very few reasons, and only after a Resolver period with clean stratified QA on that specific reason. Ceiling: low absolute value with a per-customer window, plus a second control such as a delay before execution. Recall exposure: highest, and partly irreversible. Cost: the smallest incremental labour saving of the three, because these reasons tend to be low-volume and high-touch. Choose Settler if you can name the person who owns the reversal and they have agreed to the drill.
The honest ordering point: the labour savings are concentrated in Resolver, and the risk is concentrated in Settler. A schedule that puts most reasons at Resolver with tight ceilings and leaves Settler nearly empty captures most of the benefit at a fraction of the exposure. The inversion we would watch for is generous ceilings set early because the pilot went well, when the pilot went well because it ran on the easy residue of the queue.
When a tighter limit is the wrong answer
The obvious failure mode of everything above is over-correction, and it produces a worse outcome than a loose ceiling for a specific and well-documented reason: a customer who cannot get a resolution and cannot get a human is in the doom loop the CFPB described, and the regulator's own framing is that reducing access to human support "likely comes at the expense of service quality and trust". A schedule with every row at Advisor and a support team sized for the old contained volume is precisely how you build one.
So the limit is only half a control. The other half is capacity on the escalation side. If you tighten ceilings, escalation volume rises, and if the human queue is not staffed for it, the customer's experience is a slower version of the same denial. That is a strictly worse position than having granted the authority: you carry the cost of the agent and the cost of the wait.
There are also cases where a low ceiling is simply wrong on the merits. High-frequency, low-value, fully reversible reasons with clean machine-checkable evidence (the delivery-window credit, the duplicate-charge reversal, the shipping-label reissue) are ones where a human review step adds latency and no accuracy. If the evidence column can be filled in completely and the action is reversible in one click, the human in that loop is an expensive tie-breaker for a question that has already been answered by a database.
And there is a legitimate contrarian position we should state rather than dodge. A support leader could argue that the recall number is a scare metric: that most wrong rulings are caught by customers within a day, because customers who are told no complain, and that the eleven-day detection assumption reflects bad QA rather than an inherent property of AI agents. That is a fair challenge and it is partly right. It holds where the error is adverse to the customer. It fails in the other direction: a ruling that is wrongly generous generates no complaints at all, and those are the ones that reach three thousand interactions before finance notices the variance. Both directions are in the recall number, and only one of them self-reports.
Where this analysis is uncertain
The arithmetic is illustrative, not measured. The 8,000-contacts figure, the 5% reason share, the eleven-day detection and the $34 average exposure are stated assumptions chosen to be plausible, not data from a real queue. Substitute yours before quoting any of it. We have not run a customer-facing support agent at production volume.
We could not verify the analyst forecasts this topic normally leans on. Gartner's widely-quoted prediction about agentic AI resolving a majority of common customer service issues autonomously is behind a 403 for every fetch route available to us, so we have not cited it and you should not treat its absence here as a judgement on it either way. Reddit's support communities, another obvious source for practitioner voices, are likewise unreachable, so the field voices in this piece come from Hacker News and skew toward people who build these systems rather than people who staff support queues. That is a real skew and it is worth naming.
The legal position on adverse automated outcomes is unsettled. We have quoted GDPR Article 22 verbatim and said plainly that whether ordinary support decisions cross its threshold is contested. If your business turns on that answer, get advice; do not derive it from the quotation.
Ceiling ratcheting has no published methodology, and neither does the review count. We recommend raising ceilings on evidence of clean stratified QA over a defined period, and we cannot point to a study that establishes how long that period should be or what pass rate justifies a given increase. The same gap applies to the minimum weekly review count per contact reason: we can tell you the sampling frame is wrong, and we cannot tell you the right absolute number. Both are judgements, offered as such.
We give no ROI figure, deliberately. A buyer reading this will want a payback number and we do not have a defensible one. Every industry figure we could find for AI support deployments traced back to a vendor with an interest in it, with no disclosed methodology and no statement of what was subtracted for QA, escalation staffing and remediation. The nearest thing to an honest answer in this article is structural: savings concentrate in the Resolver posture, cost concentrates in escalation capacity and review, and the recall number is what a bad month costs. Build the case from your own contact-reason report rather than from anybody's headline percentage.
Correlated error is an argument, not a measurement. We have not found a study that quantifies the ratio of correlated to independent error in production support deployments, which is the number that would make the recall number rigorous rather than directional. The structural case rests on NIST's characterisation and on the multi-turn degradation finding. If someone publishes that measurement, this section is where the article should change.
The layer an authority schedule implies
An authority schedule is only real if something enforces it at the moment the agent tries to act, and only auditable if the attempt, the decision and the refusal are all recorded. That is the layer LeapForce builds: policy checked in the request path rather than described in a document, actions scoped per connector rather than granted wholesale with an API key, every agent carrying an owner, a scope and an expiry as a first-class non-human identity, and an audit record of what was refused and not only what ran. Our gateway rollout model is deliberately unglamorous about sequencing — Observe first. Enforce second. Optimize third. — because the recall number is unknowable until you have watched real traffic, and a ceiling set before you can see your own volume is a guess. The access and identity layer and observability and audit are where the schedule's five columns become enforceable, and connectors is where action-level scoping puts the ceiling in front of the refund API rather than behind it.
LeapForce is not a customer service platform and does not write your support policy or run your queue. You will still need a support suite and the people to staff it. Per capability, our platform is disclosed honestly as LIVE, IN DEV or ROADMAP, and you should check the status of anything you plan to depend on before you plan around it.
Frequently asked questions
An AI customer service agent is a language model connected to your support systems with permission to act on a customer's request: it reads the account and the policy, decides what the company owes, and either states that or executes it. The distinguishing feature versus older support tooling is not conversational quality but permission. It can change records, issue credits and close cases. Treat it as a staffed role with a written scope rather than as a feature you switch on.
The practical difference is write access and settlement authority. A scripted chatbot matches an intent and returns text; an AI customer service agent reasons over the specific account, calls tools, and can commit the company to an outcome. We have drawn that boundary in detail in our analysis of the write-access line between agents and chatbots. The category label matters less than the answer to one question: what can it change without a person approving it?
For reversible, low-value, evidence-backed cases, yes — with a written ceiling, a per-customer window, and a deterministic escalation trigger above it. For cash refunds to the original payment method, account closures and anything the customer cannot undo, start at recommend-only and earn the authority with clean review results on that specific contact reason. The decision is per contact reason, never a single global setting.
Derive it from your own data rather than a vendor default. Find the value at which your human policy already requires a second signature, then set the agent's ceiling below it, because the human threshold assumes independent errors while an agent repeats the same ruling. Check the result by multiplying the ceiling by the recall number for that contact reason: if that product is a figure you would not want to explain to finance, lower the ceiling.
On five deterministic conditions: the value exceeds the row's ceiling; a required evidence field is missing; the customer signals vulnerability, bereavement or hardship; the customer mentions a lawyer, regulator, chargeback or complaint body; or the conversation loops past a turn or repeat-contact limit. Model confidence can supplement these and should never replace them, since the documented failure mode of multi-turn conversation is confident, unrecovered wrongness.
Six fields: the customer's stated goal in their own words, the full turn-by-turn transcript rather than a summary, every commitment the agent made verbatim with timestamps, every action taken or refused with reasons, the policy version and knowledge sources used, and which trigger fired on which turn. The commitments field is the critical one. Without it, the human can contradict a promise the company has already made, which is worse than either position alone.
In at least one published decision, yes. In Moffatt v. Air Canada (2024 BCCRT 149) a tribunal awarded $650.88 in damages for negligent misrepresentation after the airline's chatbot described a bereavement-fare policy that did not exist, holding it "makes no difference whether the information comes from a static page or a chatbot". That is a British Columbia small-claims tribunal, not binding precedent in most jurisdictions, and the safe planning assumption is that a court elsewhere could reason similarly.
There is no defensible universal ratio, and the useful reframing is that supervision load scales with granted authority rather than with agent count. One agent settling refunds across forty contact reasons needs more oversight than ten agents that only answer order-status questions. We work through the staffing arithmetic in our piece on why supervision ratio rather than headcount is the real limit on an AI workforce.
Stratify by contact reason instead of sampling conversations at random. Random sampling estimates a rate, which is the wrong target when errors are correlated; what you need to find is one wrong standing ruling. Give every reason where the agent holds authority a minimum absolute number of weekly reviews regardless of its share of volume, review outcomes rather than tone, and hand reviewers the authority schedule so the question is simply whether the agent stayed inside the row.
The customer's request, the policy and knowledge version that grounded the answer, the resolved contact reason, every action attempted, every action refused with its reason, the value committed, the escalation trigger if one fired, and the human approver where there was one. The contact reason and the policy version are the two that make a recall possible; the refusal log is the one we would expect to be missing and the one that proves the control existed rather than being merely configured.
A named person in the support organisation, not the vendor and not the team that built it, with the same owner-scope-expiry fields you would apply to any non-human identity. Ownership has to survive the builder leaving, because the authority schedule needs a person who can raise a ceiling, order a freeze, and answer for what the agent settled last quarter. Our analysis of owner, scope and expiry for AI agents covers the identity side of that.
The pattern we would look for first: the pilot ran on the easy residue of the queue, ceilings were widened on that evidence, and the first correlated error then arrived at full volume with no freeze switch and no recall list. The secondary cause is tightening ceilings without staffing the escalation queue, which converts a bad answer into a long wait, which is the doom loop the CFPB documented. Both are scope and staffing failures rather than model failures.
Ready to Govern Your AI?
Talk to LeapForce — one controlled layer for every AI tool, connector, model, and agent.
Comments