Answers to the same type of customer questions without an operator

Initial situation

The chat operator answers the same questions. Where is my order? How to change the tariff. Will delivery arrive on Saturday? Why am I not receiving a login email? How to obtain closing documents.

The answers already exist - in the operator’s head, in personal notes, in old correspondence. Everyone answers in their own words, so two customers with the same question receive two different answers. Different in meaning, not only in wording.

Time is lost like this:

• The operator is busy with a typical question while a difficult client is waiting in line.
• No one answers at night and on weekends, and the client leaves to look for an answer from a competitor.
• It takes weeks for a new employee to get up to speed because the knowledge is not written down.
• Answers are silently becoming obsolete: conditions have changed, and last year’s version is in the notes.

Separate layer - questions the answer to which is in the documentation. The client did not get there because it was inconvenient to search.

What did you watch?

The first step is to mark the body of hits by intent, and not by keywords. The words “payment”, “invoice” and “money” are found in questions about issuing documents, about returns and about broken payments. Three different scenarios with three different answers.

Practical order:

1. Upload requests for a period - from the chat system, mail, tickets.
2. Check the marking of roles in the upload. A common pitfall: bot or answering machine messages arrive marked as client messages. Corpus collected without this check shows non-existent “customer questions”, which are actually phrases of the system itself.
3. Group by intent. The first pass can be done with a model, the second with your hands, otherwise the groups will gather according to the words.
4. Calculate the share of top intentions from the entire flow.

Next are three numbers that determine whether it’s worth taking:

• Proportion of requests closed with one message. Direct candidate for automation.
• Share of requests where the answer is already written somewhere in the documentation. Indicates whether there is anything to assemble the base from.
• Percentage of requests that require information about a specific account. Text search is useless here, an API call is needed.

Messages are divided into three classes, and the solutions for them are different:

What was changed

1. The knowledge base is compiled from operator responses. Source - real successful answers from correspondence, and not marketing texts from the site. Storage unit: wording of the question, short answer of three to five lines, link to details.

2. Each piece has metadata. Section, last reviewed date, owner. A piece without an owner becomes obsolete first.

3. Search is hybrid. Vector search catches reformulations, searches by key - exact terms, articles and error codes. One vector search misses on rare words and numbers.

4. The model responds only to the pieces found. Nothing found - no answer. Generation without reference to a source is prohibited by the system prompt and is checked at the output.

5. Link to source in each answer. The client can check, the operator can correct, and the absence of a link automatically means escalation.

6. Explicit refusal is built in as a standard outcome. The formulation “I can’t answer accurately, I’m passing it on to the operator” is cheaper than a confident fib.

7. Frames in the prompt. Do not name dates that are not in the database. Do not name prices that are not in the database. Don't apologize for a problem that didn't exist. Do not promise 24/7 availability.

8. Transfer to a person with context. The operator receives the history of the dialogue, the pieces found and the reason for the escalation. Without this, the conversation starts from scratch, and the client repeats the question a second time.

9. A separate path for questions about the account. The bot does not look for an answer in the text, but calls the system API using the client ID. Identification takes place before the call, not after.

10. Feedback is closed to the base. The operator marks the unsuccessful answer, the edit goes to the knowledge base, and not to the manager’s chat. For questions of the same type, relying on a source gives more than the size of the model, so filling the database is more important than choosing a model.

11. Night mode is different from day mode. The bot responds around the clock, but night escalation turns into a task for the morning, and the client is immediately given a response time.

What was not touched

• Live chat. The operator remained in the same interface, the bot stood in front of him, and not instead of him.
• Phone. The voice channel was not automated: the cost of an error is higher and there is less chance of correction.
• Existing documentation. It was not rewritten for the bot, only supplemented with the missing answers, leaving the wording the same.
• Sales. The bot answers questions and does not conduct the transaction. Mixing the reference and selling roles spoils both.

Reversibility

• Shutdown flag: the bot is turned off, the chat goes directly to the operator, the operator’s interface does not change.
• The knowledge base is stored in regular text files under versioning. Rolling back an edit—reverting a commit.
• The system prompt is in the config, not in the code, and can be changed without rolling out.
• Shadow mode: the bot prepares a response, but does not send it, but shows it to the operator as a hint. The same mode is used for launching - first a week in the shadows, then sending to parts of the channels.

Boundaries

• Conditions change, but no one maintains the database. The bot will confidently repeat yesterday's rules. You cannot launch the database without a designated owner.
• Emotions and complaints. A dissatisfied customer demands a human, an automatic response here makes the situation worse.
• Legally significant wording. Guarantees, returns, obligations - only through a person and an agreed text.
• Various flow without a pronounced top. If frequent intentions close a small part of requests, the database will not pay for its own support.
• Issues that need to be resolved. It’s up to the person to make an exception, give a discount, extend the period; the bot only collects data for him.

Rule: answer only what is found in the database, with a link to the source, and give everything else to the person along with the history of the dialogue.