If you run an online store or business website, three useful starting measures are resolution rate, CSAT and knowledge gaps. Together they show whether customers get answers, whether they feel helped, and where your documentation or product information is missing. You may also need response time, repeat-contact rate and other measures for your own support goals.
Below are clear definitions, exact formulas, where to get the numbers, and step-by-step fixes you can use this week.
KPI 1 — Resolution rate (what to measure and how)
What it is
- Define resolution rate before comparing periods. For an AI assistant, a useful definition is the share of eligible conversations resolved without a human handoff. A whole-team resolution rate includes human-handled cases and answers a different question.
How to calculate
- Define a conversation window (e.g., 7 days after first contact).
- AI resolution rate = eligible conversations marked resolved without handoff / all eligible AI conversations started in the same window.
- Keep human-handled and other-channel conversations separate unless you define a whole-team rate and can link the records reliably.
Where to get the data
- Pull conversation logs from your chat widget, help desk or ticket system.
- Use conversation-level fields: conversation_id, started_at, closed_at, resolution_status.
- If a tool reports "first contact resolution," treat that as a separate metric and verify its definition before comparing it with AI resolution rate.
Common measurement pitfalls
- Counting reopened tickets as new conversations inflates resolution rate. Link reopens to the original conversation.
- Mixing chat, email and social conversations without consistent records makes the number hard to interpret.
How to improve resolution rate (practical steps)
- Triage common requests into self-serve paths: shipping, returns, order lookup.
- Add canned responses with editable variables for agents to speed replies.
- Surface order details in the agent UI so agents don’t ask customers to repeat info.
- Author and link exact-step help-centre articles for the top 10 repeat intents.
- Run weekly reviews of resolved conversations that were reopened — look for missing steps in agent answers.
What to test
- For a sample week, track resolution rate before and after you add one help article and one template reply. Compare the two weeks.
KPI 2 — CSAT (customer satisfaction)
What it is
- CSAT measures customer sentiment with a short post-interaction survey (usually a 1–5 or thumbs up/down).
How to calculate
- CSAT = (number of positive responses) / (total responses) for a given period.
- Always report response count alongside percentage — small sample sizes lie.
Where to get the data
- Add a CSAT prompt at the end of chats or after ticket close.
- Capture the timestamp, score, conversation_id and optional comment.
Common measurement pitfalls
- Asking for CSAT too often reduces response quality. Ask once per resolved conversation.
- Not filtering internal follow-ups or test conversations inflates sample size.
How to improve CSAT (practical steps)
- Make the CSAT ask simple and at a predictable time: right after resolution, not mid-chat.
- Train agents on three behaviours: say what you will do, do it quickly, and confirm the customer is happy before closing.
- Use short survey comments to capture why someone was unhappy and tag those reasons.
- Patch low-scoring interactions into weekly coaching for agents and into knowledge-gap analysis.
What to test
- Run a script change for two agents (control vs. new closing script). Compare median CSAT and comment themes over two weeks.
KPI 3 — Knowledge gaps (the operational KPI that fixes the other two)
What it is
- Knowledge gaps are places your documentation, product content or agent scripts fail to answer customer questions. Measure them so you can fix the source.
How to quantify knowledge gaps Track these signals:
- "No answer" or fallback responses from your bot or scripts.
- Escalation rate: conversations escalated to a senior agent because content didn’t exist.
- KB search failure rate: searches returning no article or low click-through.
- Repeated intents appearing in transcripts without a corresponding help article.
Create a simple metric
- Knowledge gap rate = (conversations with a documented gap event) / (total conversations).
- Keep a separate count of unique topics causing gaps.
Where to get the data
- Parse chat transcripts for phrases like "I don't know", "I can't find", "how do I", or bot fallback answers.
- Use tags added by agents when they create a new FAQ during a conversation.
- Use analytics from tools that track "knowledge gap" events (some bots include this).
How to close gaps (practical steps)
- Run a weekly “gap sweep”: extract conversations flagged as fallback or escalated, and list the top 10 recurring topics.
- Assign an owner to each gap: product team for product issues, ops for process issues, content for KB creation.
- Create short KB articles (100–300 words) that answer the exact customer question and include the exact phrasing customers use.
- Add those articles to the agent UI and the bot's training set, and update canned responses.
- Mark a gap closed only after you see a measurable drop in related fallback events.
Why this matters
- Fixing a knowledge gap can improve answers and may reduce repeat work. Measure the effect instead of assuming every new article raises resolution and CSAT.
How these three KPIs work together
- Knowledge gaps are the root cause you can fix.
- Resolution rate shows whether fixes and agent changes are working.
- CSAT shows whether customers feel the difference.
Don’t judge a fix by one KPI alone. If you publish a new help article and resolution rate improves but CSAT doesn’t, look at tone or steps in the article. If CSAT rises but resolution rate falls, the process may be slower even if customers like the agent interactions.
Dashboards, cadence and ownership
What to show on a dashboard
- Resolution rate (weekly and 30-day rolling).
- CSAT with response count and sample comments.
- Knowledge gap rate and top 10 gap topics.
- Escalation rate and average time to resolution.
- Trend lines and a table of recent low-CSAT conversations.
Reporting cadence
- Daily: alerts for sudden drops in resolution rate or spikes in fallback responses.
- Weekly: review top gaps, coach agents, and push content fixes.
- Monthly: present trends to product and ops with concrete requests to fix systemic issues.
Who owns what
- Support lead: dashboard, weekly triage and agent coaching.
- Content owner: publishes new KB pages and updates scripts.
- Product/ops: fixes product or process issues flagged by support.
Tools and automation tips
- Use a single source of truth for conversation ids so multi-channel interactions are linked.
- Automate tagging of fallback responses with keywords to speed gap discovery.
- Run lightweight A/B tests for scripts and KB drafts, and measure resolution rate and CSAT for each variant.
- If you use an AI support agent that trains on your content, ensure your KB updates are pushed to its training set and that analytics report on knowledge gaps and CSAT. (Some products offer those analytics and install on platforms like Shopify, WordPress and others.)
What to do this week
- Export last 30 days of conversations and compute resolution rate and CSAT with counts.
- Identify the top five conversations that used a fallback response or were escalated.
- Write or update a short KB article for one top gap and add a canned reply.
- Run a one-week test: track resolution rate and CSAT for conversations using the new article vs. not.
- Hold a 30-minute coaching session with agents to review two low-CSAT interactions and agree one behavioural change.
