Configuring the AI agent: parameters, permissions, limiters, debugging

What does this solve?

The default agent fails in a predictable set of ways: calling the wrong tool, walking in circles until it reaches the limit, stopping in the middle of a task, reporting work that never happened. Settings AI-agent treats each symptom targeted - with a specific configuration node, and not by rewriting instructions from scratch. The second task is to contain the cost: without limiters, one request goes into a cycle and consumes the budget of the whole day. The third is to prevent an agent with access to mail, repository and database from doing something irreversible.

How it works

The agent repeats the same cycle: the request enters the context → the model selects a tool → the tool is executed → the result is returned to the context → the circle is repeated → the stopping condition is triggered. Configuration comes down to controlling what goes into the context on each loop, what is allowed to be called, and when the loop must terminate.

Instructions. Role, procedure, prohibitions, response format. Verifiable formulations work: “before editing the file, read it in its entirety”, “do not create a branch with a name outside the template”. Wishes: “be careful”, “try not to make mistakes” do not work.

Tools. Name, description, parameter diagram, error text. The model sees only this text and uses it to decide whether to call the instrument or not. Half of the cases of “the agent ignores the tool” are treated by rewriting the description, not the instructions.

Permissions. Three baskets: executed silently, asks for confirmation, completely disabled. Recycle bins are specified as an allowed list, the rest is closed by default.

Context. What is included at startup - the project rules file, memory, data diagram, documents. And what is cut off is the output of the tools, the history of the long dialogue.

Generation parameters. Model, answer length ceiling, depth of reasoning. A cheap model for classification and an expensive one for analysis - a working diagram; the choice is made according to the type of step, and not “the most powerful everywhere.”

Limiters. Loop iteration ceiling, timeout per step and per task, budget per task, tool output trimming, number of parallel calls.

Observability. Call track: what tool, with what parameters, what was returned, how much it took and how much it cost. Without a route, tuning turns into fortune telling based on the final text.

How to connect

1. Set the boundary. One task, one sign of readiness. Data Assistant is not configurable. “Answers questions about balances and payments for the period, refuses to answer the rest” can be configured.

2. Write instructions from top to bottom. First, prohibitions and mandatory order, then the output format, and examples at the end. Order matters: what comes first is more consistently followed.

3. Collect a minimum set of tools. Every extra one worsens the choice among the rest. Two tools with similar names are guaranteed to cause confusion - they are combined into one with a parameter.

4. Rewrite descriptions for the model.

Плохо:  get_data — получает данные из базы

Хорошо: list_unpaid_invoices — счета со статусом «не оплачен» за период.
        Параметры: date_from (обяз., YYYY-MM-DD), date_to (обяз.),
        customer_inn (опц., 10 или 12 цифр).
        Возвращает до limit строк, отсортированных по дате.
        При обрезке в ответе truncated: true — сузить период.

5. Decompose permissions.

{
  "allow": ["db.select", "docs.read", "git.status"],
  "ask":   ["db.write", "mail.send", "git.push"],
  "deny":  ["shell.rm", "secrets.read", "prod.deploy"]
}

The layout rule is simple: reversible and cheap - in allow, reversible and noticeable to people - in ask, irreversible - in deny, even when it would be very convenient to allow.

6. Place limiters before the first combat launch.

max_steps        потолок витков цикла
max_tool_output  обрезка вывода инструмента
timeout          на шаг и на всю задачу
budget           потолок расхода на одну задачу
parallel         сколько инструментов работает разом

Values ​​are selected based on the routes of successful runs: a typical number of steps is taken and a reserve is provided, and not a round number from the head.

7. Enable logging of each call. Tool, parameters, duration, status, cost. Secrets are masked at the wrapper level, and not by the “we don’t look there” agreement.

8. Collect a regression set. Real requests from correspondence and tickets, including crooked and meaningless ones. The same set is run after each configuration edit.

Debugging: symptom → node

Useful scripts

• Read-only analyst: there are physically no writing instruments, the main limiter is the sample size.
• Alert duty: narrow set of tools, hard timeout, sending messages to people via confirmation.
• Code agent: reading and editing files freely, push and deployment via ask, destructive commands via deny.
• Support agent: knowledge base in context, escalation to a person with a separate tool, prohibition of mentioning terms and discounts in instructions.
• Batch processing: limited parallelism, paused retrays, budget is calculated per element, not per run.
• Night report collector: there is no one to give interactive confirmations, so writing operations have been removed and all that remains is uploading to a file.

Limitations

The setting does not create data. If the required field is not in the system, no parameter will force the agent to answer correctly - he will come up with a plausible one.

Instructions are not a guarantee. The text specifies the tendency. Everything that should not be there is removed at the permission level or by the absence of a tool.

The number of tools works against the quality of choice. The set grows imperceptibly, and at some point the agent begins to miss where it was previously stable. It is treated by revising the kit, and not by more detailed instructions.

Hard limiters cut long tasks. The ceiling of turns, selected according to a short scenario, breaks off the complex one in the middle. Hence, there are different profiles for different types of tasks instead of one universal one.

The configuration is out of date. Changing the model version changes the behavior with the same config. The regression set is distilled after each update, and not once every six months.

Observability costs money and risks. Detailed traces take up space and contain pieces of operational data. The shelf life and masking rules are determined in advance.

How to check the result

Run a regression set and look at four numbers: the share of completed tasks, the average number of turns, the cost per task, the share of requests for confirmation. Any deterioration after a configuration edit is a reason to roll back the edit rather than explain its benefits.

Read routes, not final answers. The correct answer received by six extra calls is also a defect: it is paid for on each run and sooner or later it will hit the limit.

Dismiss negative tests. A request that must meet deny must fail. A crooked input should receive a clear refusal. The question beyond the boundaries of the role is refusal instead of improvisation.

Check the integrity of the report. A task where work is physically impossible (the service is unavailable, the database is empty) should end with a message about impossibility, and not a report of success.

Check logs for secrets: search for fragments of keys and tokens in traces after a run. It was found - masking does not work, and this is fixed before anything else.

One line rule: what an agent should not do is specified by permissions, not by the text of the instruction.