What does this solve?
Implementation AI-agents comes up against a living process: the process has an owner, deadlines, clients and the cost of an error. The pilot responds beautifully to the demo data, but in battle the same agent writes to someone else’s order, duplicates the entry after a timeout, or answers the client with something the company does not promise. Careful input relieves exactly this pain: it gives a controlled expansion of rights, a rollback point and “before and after” numbers on the same slice. A side effect is that the process has to be described first, and half of the finds are discovered before any agent.
How it works
Input mechanics - ladder of rights. There are five steps, each next one is taken after the previous one has worked on a real flow.
- Observer. The agent only reads and writes its findings to a separate channel: a file, a column in a table, a thread in a messenger. The process works as before and the agent does not notice.
- Shadow mode. The agent prepares a solution for each real case, the person decides for himself. Both versions are put side by side, the discrepancies are sorted out.
- Draft with confirmation. The agent creates a record - a response to the client, a card, a payment - and stops at the button. The person presses “apply” or edits.
- Autonomy by class of operations. The agent performs reversible actions himself, while irreversible actions continue through confirmation.
- Autonomy with post-moderation. Spot check of some cases and analysis of complaints.
On top of the stairs there is a circuit that makes the rollback possible:
- Process owner is a specific person who sorts out discrepancies and makes a decision on the next step.
- Switch. A flag in environment variables or key revocation that disables the agent in a minute, without rolling out the code.
- Journal. Each action with timestamp, input, output and case ID.
- Escalation channel. Where does the agent write “I don’t know” and who reads it.
- Rules as code. The agent instructions are in the repository, changed by commit, and undergoing review. Editing a prompt in someone else's web interface without history is a source of inexplicable regressions.
Before starting, the process steps are divided into four types: read, decide, write inside the system, communicate with the outside world. The last type is the most expensive in terms of the cost of error and is given last.
A separate detail that breaks implementations quietly is duplicates. The network fell off after the recording, but before the response; the agent repeated the step; There are two accounts in the system. Treated with an idempotency key: before writing, the tool checks whether there is already an operation with the same key.
Rule: the right to write is issued only after the agent’s proposals in the shadow mode no longer diverge from the human’s decisions.
How to connect
Step 1. Select a process. The one that has a written trace (tickets, letters, lines in the database) and a measurable outcome (closed/not closed, deadline, amount) is suitable. The agent will not see verbal agreements.
Step 2. Record the process as is. Step by step, with performer and typical time. A table on one screen is enough.
Step 3. Mark the steps by four types and select a narrow band: one type of requests, one category of tickets, one customer segment. The strip must produce a flow every day, otherwise there will be nothing to check.
Step 4. Give reading. Connect only reading instruments, limit the selection to the selected strip, enable the log. Let the agent write observations in a separate channel for a week.
Step 5. Shadow run. The agent puts his decision in the field agent_suggestion (separate column, comment, file), the person works as before. At the end of the period, a reconciliation is carried out: it coincided, it diverged, the agent was right.
Step 6. Analyze discrepancies by reason. Each discrepancy is closed in one of three ways: a rule in the instructions, a new or updated tool, an admission “the agent does not accept this class of cases.” The third option is a normal outcome, it narrows the band.
Step 7. Transition threshold. Stage 3 is taken when recurring causes of discrepancies are closed and new ones have stopped appearing on fresh cases. The calendar period is not a criterion here, the criterion is the exhaustion of reasons.
Step 8. Enable recording via confirmation. One operation at a time. The rollback button is next to the apply button. The first days the owner looks at each case.
Step 9. Remove confirmations by class. Starting with reversible ones: put a mark, transfer a task, update a service field. Sending outside and money remains with confirmation until the owner decides otherwise.
Step 10. Threshold alarms. The notification comes when the proportion of escalations or the number of rollbacks goes beyond the limit, and not every day as scheduled. A daily newsletter that no one reads equals no monitoring.
Step 11. Revision. Once a period - review the log: which classes of cases the agent handles better than a human, which ones are worse, what can be passed on, what can be taken back.
Useful scripts
- First line of support: the agent prepares an answer, the operator sends it, the share of edits falls - confirmation is removed for typical questions.
- Analysis of incoming applications: classification and filling of fields automatically, decisions on controversial ones are made by a person.
- Regular reconciliation: the agent compares the data of the two systems every morning and sends only discrepancies.
- Error duty: grouping nightly failures, analyzing recurring ones, escalating new ones.
- Maintaining a tracker: transferring tasks according to column rules, reminders for stuck ones, collecting a summary for the planning meeting.
- Preparation of documents according to a template: the draft is generated by the agent, signed and sent by the person.
Limitations
Irreversible actions remain with people: money transfers, data deletion, external publication, legally significant answers. Confirmation here can be removed only by the conscious decision of the owner for each class separately.
A process without a written trace is not automated: if the decision is born in a conversation and is not recorded anywhere, the agent has neither input nor a standard for comparison.
The absence of an owner kills implementation more reliably than a bad model. There is no one to sort out the discrepancies of the shadow regime, the rules are not replenished, the agent remains on the first stage forever.
Personal data requires a separate decision before the start: what is masked, what does not leave the circuit, who is responsible for the incident.
There is nothing to check for rare events: if a case happens several times a year, the shadow period will not accumulate statistics, and the agent will remain an assistant-prompter.
A team that does not read notifications will receive a noise generator. Channels and thresholds are configured before recording is started.
The cost increases with the size of the context and the number of steps, so the stripe expansion is always checked against the token counter from the log.
How to check the result
The shadow period reconciliation report is the main document. Three columns: matched with the person, diverged, the agent was right. Weekly share dynamics show whether adoption is moving.
Next are numbers from the magazine on the same section before and after:
- proportion of cases closed without human intervention;
- cycle time from receipt to closing;
- number of rollbacks and their reasons;
- share of escalations “don’t know” - a sharp increase means that the band was expanded too quickly;
- repeated requests from the client on the same issue - a proxy for the quality of answers;
- consumption of tokens per case.
The switch is checked separately: turn off the agent on a working day and make sure that the process continues to work manually and the queue does not form. An implementation that can't be turned off isn't finished yet—it just hasn't been tested yet.