Autonomous AI agents: where they can handle it themselves and where a person is required

What does this solve?

Standalone AI agents remove human waiting from the process. The request arrives at night, the bill lands in the mail on Saturday, the same mistake is repeated two hundred times in an hour - the bottleneck in all cases is not the ability to sort out the situation, but in the presence of someone who can sort it out. An agent who is allowed to act on his own responds in a minute and at three o’clock in the morning, and does this on a volume where a person physically cannot keep up. The price for this is the need to decide in advance which actions he does himself, which through confirmation, and which he never does.

How it works

It is convenient to measure autonomy in steps, and not with the word “yes/no”.

What technically makes an agent capable of working without a human.

Explicit termination criterion. The wording “deal with the request” leaves the agent without a stopping condition. Working wording: the request is considered closed when the response is sent and the status is transferred; if no answer was found in two steps - escalation.

Differentiating between emptiness and error. An empty search result and an unavailable service look similar for the model. The tool wrapper must return different codes, otherwise the agent will write “no data” where the database simply did not respond.

Rules of escalation. List of conditions under which the agent stops and calls a person: unfamiliar request type, data conflict between sources, limit exceeded, repeated tool failure, any hint of legal or medical consequences.

Idempotency. Each writing action has a key by which repetition does not create a second record. Without this, restarting after a failure doubles letters and payments.

Magazine with backlink. Each action is recorded along with its source: what input caused it, what agent step, what arguments were left. Analysis of an incident without such a log turns into fortune telling.

Stop valve. A flag that the agent checks before each writing call, and stops all starts with one command.

Where human supervision is mandatory regardless of the agent's maturity:

• money out: payments, returns, discounts, changing the client’s tariff;
• irreversible deletion of data, letters, files, threads;
• sending messages to external recipients on behalf of the company;
• legally significant documents and any obligations;
• change of access rights, configurations, release for production;
• transfer of personal data to third parties;
• public publications;
• any action resulting from an untrusted text - a letter, chat, web page: the instruction inside the data is read by the model as a command;
• situations where the cost of one mistake covers the entire savings for the year.

How to connect

Autonomy is gained using a ladder, each step takes more than one day.

1. Shadow mode. The agent works on a real thread and puts the result into drafts. The person does the work as before and compares. Discrepancies are collected in a list.

2. Tool separation. Two sets: reading and writing. Readers are always available, writers - through a separate token and a separate setting.

3. Confirmation of entry. The writing instrument displays the intent in full: to whom, what, with what meanings. The dry-run format is more important than beautiful text:

DRY-RUN send_email
  to:      client@example.com
  subject: Продление доступа
  body:    128 символов
  reason:  тариф истекает через 3 дня

4. Limits. Maximum actions per launch, maximum per day, amount ceiling, silence window for client communications, white list of recipients and operations.

5. Stop valve and its check. Flag that the agent reads before each entry:

echo 1 > /var/run/agent.stop   # остановить пишущие действия
rm /var/run/agent.stop         # вернуть в работу

6. Autonomy in a narrow class. The most similar case is released first: one type of request, one type of document. The rest continues to go through confirmation.

7. Data expansion. The class is switched to offline mode when statistics are collected on it: the share of interventions is consistently low, there were no rollbacks, escalations worked properly.

8. Alerts on thresholds. Notification comes when going abroad - a surge in escalations, an increase in the number of actions, repeated refusals. They stop reading the daily “everything is fine” report after a week.

Useful scripts

• Nightly error analysis: the agent groups events of the same type and creates one ticket with examples instead of a stream of notifications.
• First line response to the knowledge base with strict escalation of everything that is not in the database.
• Data reconciliation between two systems: reads both, compares, writes only a report on discrepancies.
• Reminders about expiring access: the agent generates a list and drafts, sending is confirmed.
• Incoming routing: classification by intent and transmission to queue with collected context.
• Monitoring of external API: regular call, comparison with the standard, message only in case of deviation.

Limitations

The reliability of the chain is multiplied. A step that is correct ninety-five times out of a hundred gives a noticeably worse result after ten consecutive steps - and the error is rarely visible from the outside, because the final text looks as confident as the correct one.

The agent does not know what he does not know. If there is a lack of data, the model by default completes the missing data, rather than stopping. Stopping is behavior that is explicitly specified in the prompt and supported by the escalation rule.

External services fail silently. The agent considers the response with code 200 and an empty body a success, and the chain continues on empty. Tool wrappers should check the content of the response, not just the code.

Changing the model changes the behavior. The same prompt on the new version gives a different style of solutions, so autonomy without a regression set after each update rolls back a step.

Responsibility is not delegated. The owner of the process is responsible for the action of the agent, and this determines which operations remain with the person, regardless of the quality of the model.

Control costs time. If you check every action manually, there is no savings - so autonomy is introduced in classes of tasks, and not all at once.

Rule: the agent acts itself where the action is reversible and verifiable; everything irreversible goes through a person or through a delayed queue with a cancellation window.

How to check the result

Count four numbers per week: the share of launches closed without intervention; the number of actions that had to be rolled back; number of escalations; proportion of escalations where the person agreed that intervention was necessary. The last number indicates whether the caution threshold is set - escalations in the case mean that the frame is chosen correctly.

Perform a failure test: disable one tool and see if the agent stops with a clear message or starts making up a result.

Put a line with an extraneous instruction into the input data and make sure that the writing action is not executed. The check is repeated after each model change.

Run the same input twice and look at the number of records created. A two means there is no idempotency key.

Turn on the stop flag during operation and detect how long after writing calls stop. If the stop valve has never been checked, assume that it does not exist.

Selectively analyze twenty actions from the log for the week: for each it should be clear which input caused it and with what arguments it was executed. An action without a source in the log is a request for the next incident.