The mistake most teams make
When teams set up an AI customer support agent, the dashboard they stare at is *resolution rate* — what percentage of conversations the agent closed without escalating to a human. Resolution rate is fine, but it's a lagging indicator. By the time it moves, the work to move it was done weeks ago.
The leading indicator is knowledge gaps: questions the agent couldn't confidently answer, grouped by topic. That's the only metric that tells you what to do next.
What counts as a gap
A knowledge gap is a question where:
- The agent's confidence score was below your threshold, OR
- The agent escalated to a human, OR
- The customer rated the answer below 3 stars
Group those by topic. The top five topics every week are your work for the next week.
Closing a gap
For each top gap, do exactly three things:
- Find the source the agent should have grounded in. If it doesn't exist, write it. One paragraph is enough. Don't write essays.
- Add one exemplar reply. Show the agent the answer you want it to give for this question. Real customer-facing prose.
- Set a re-test. Drop the original question into the agent's test bench. It should now answer it confidently. If not, the source needs more specificity.
That's it. No retraining, no model tuning, no Slack thread debating semantics. The gap is a unit of work and you closed it.
What this looks like after six weeks
The first week you'll have 30+ gaps. By week four you should be down to under 10. By week six the gaps are increasingly rare — and increasingly real edge cases worth a thoughtful answer.
Resolution rate will follow. CSAT will follow. The lagging indicators move because you moved the leading indicator. That's the trick.
The honest version
The reason teams don't do this is that closing gaps is boring. It's three minutes of writing a paragraph, four times a week. There's no dopamine in it. But it's the entire job, and it's what separates a team running an AI agent that gets better every week from one running an agent that's stuck at 60% resolution forever.
Open the gap report on Monday. Close five. Repeat.