"Fin AI" by Intercom is a game-changer for customer support, but are you tracking the right metrics to measure its success? Just like any other tool in your tech stack, understanding its performance is key to maximizing your return on investment (ROI).
This guide will break down the essential Fin AI metrics you need to monitor. We'll cover what they are, why they matter, and how to track them effectively. Plus, we’ll explore how tools like eesel AI can supercharge your Fin AI by providing the necessary knowledge base it needs to resolve even more customer queries.
What is Fin AI by Intercom?
Fin AI is a sophisticated conversational AI chatbot developed by Intercom, designed to automate and enhance customer support. Unlike traditional chatbots that rely on rigid, pre-programmed scripts, Fin AI leverages the power of large language models (LLMs), including GPT-4, to deliver more natural, human-like conversations.
It's built to provide instant, accurate answers to customer questions by tapping directly into your existing support content, such as your knowledge base articles. This allows Fin to resolve a significant portion of common queries without human intervention, freeing up your support team to focus on more complex issues.
"[Fin] is a step-function change for support teams. It’s a breakthrough in AI that will help you drive down costs and improve your customer experience." - Paul Adams, Chief Product Officer at Intercom.
Why is tracking Fin AI metrics important?
Tracking Fin AI metrics is crucial for several reasons:
-
Measuring ROI: You've invested in Fin AI; now you need to see the returns. Metrics help you quantify the value it brings in terms of cost savings, efficiency gains, and improved customer satisfaction.
-
Optimizing performance: Data-driven insights allow you to identify areas where Fin AI excels and where it might be struggling. This helps you refine its knowledge base, tweak its settings, and improve its overall effectiveness.
-
Improving customer experience (CX): By monitoring metrics like resolution rates and customer satisfaction, you can ensure Fin AI is actually helping your customers and not causing frustration.
-
Team efficiency: Understanding how Fin AI impacts your support team’s workload can help you allocate resources more effectively and ensure your agents are focused on high-value interactions.
Key Fin AI metrics you should be tracking
Intercom provides a robust set of analytics to help you measure Fin's performance. Here are the essential metrics to keep a close eye on.
Resolution rate
This is arguably the most important of all Fin AI metrics. It tells you the percentage of conversations that Fin AI successfully resolves without needing to hand them off to a human agent.
-
Why it matters: A high resolution rate is a direct indicator of Fin AI's effectiveness and your ROI. The more queries it resolves, the more time your team saves.
-
How to track it: This is available directly in your Intercom analytics dashboard. Look for "Conversations with a Fin Answer" and "Percentage of conversations closed with only Fin."
Customer satisfaction (CSAT)
CSAT scores for conversations handled by Fin AI measure how happy customers are with the automated support they receive.
-
Why it matters: High CSAT scores indicate that Fin AI is not just closing tickets but also providing a positive customer experience. Low scores can be an early warning sign that Fin is misunderstanding queries or providing unhelpful answers.
-
How to track it: You can trigger CSAT surveys at the end of conversations handled exclusively by Fin. Monitor the scores and read the feedback to understand the "why" behind the numbers.
Handoff rate
This is the inverse of the resolution rate. It measures the percentage of conversations that Fin AI has to escalate to a human agent.
-
Why it matters: A high handoff rate might suggest that your knowledge base is lacking information, the questions are too complex for AI, or Fin is not correctly understanding user intent. Analyzing these handoffs can reveal gaps in your support content.
-
How to track it: This data is available in Intercom reports. Dig into the conversations that are handed off to identify recurring themes or topics that Fin struggles with.
Response time
This metric tracks how quickly Fin AI responds to a customer's initial query.
-
Why it matters: One of AI's biggest advantages is speed. Fin is designed to provide instant answers, a key driver of customer satisfaction. Monitoring this ensures the system is performing optimally.
-
How to track it: Intercom's reporting includes metrics on response times for both bots and human agents, allowing for a direct comparison.
Ticket volume reduction
This metric measures the decrease in the number of support tickets your human agents have to handle after implementing Fin AI.
-
Why it matters: This is a clear indicator of cost savings and efficiency gains. It demonstrates how much of the support load Fin is shouldering, freeing up your team for more strategic work.
-
How to track it: Compare your inbound ticket volume from before and after you implemented Fin. Look at the trends over time to see the impact.
Conversation volume
This tracks the total number of conversations that Fin AI participates in.
-
Why it matters: This metric helps you understand the scale of Fin's operation. A rising conversation volume indicates that more customers are engaging with the bot, which can be a positive sign of user adoption.
-
How to track it: This is a standard metric in the Intercom analytics suite. You can segment it by channel (website, app, etc.) to see where Fin is most active.
How to improve your Fin AI metrics with eesel AI
While Fin AI is powerful, its effectiveness is directly tied to the quality and comprehensiveness of your knowledge base. If the information isn't there, Fin can't provide the answer. This is where eesel AI comes in.
eesel AI is a no-code platform that automatically creates and maintains a comprehensive, accurate knowledge base from all your company's scattered documents and tools, like Slack, Google Docs, Notion, and Confluence.
Here’s how eesel AI can directly boost your key Fin AI metrics:
-
Increase resolution rate: By feeding Fin a more complete knowledge base created by eesel AI, you equip it to answer a wider range of questions. eesel AI captures knowledge from sources you might not have manually documented, turning tribal knowledge into a resource Fin can use.
-
Decrease handoff rate: When Fin has access to more comprehensive information via eesel AI, it's less likely to be stumped by a query. This means fewer escalations to your human agents, directly lowering the handoff rate.
-
Improve CSAT scores: More accurate and complete answers lead to happier customers. eesel AI ensures the knowledge Fin uses is always up-to-date by automatically syncing with your source documents, preventing the bot from giving outdated or incorrect information.
Essentially, eesel AI acts as the perfect knowledge engine for Fin AI, ensuring it always has the best possible information to work with.
Final thoughts on tracking Fin AI metrics
Tracking Fin AI metrics is not just about generating reports; it's about turning data into actionable insights. By consistently monitoring metrics like resolution rate, CSAT, and handoff rate, you can pinpoint exactly how your AI chatbot is performing and identify opportunities for improvement.
Remember, Fin's intelligence is a reflection of your knowledge base. To truly unlock its potential and supercharge your metrics, you need to provide it with comprehensive and up-to-date information. Tools like eesel AI are invaluable for this, automatically creating the perfect knowledge source to help you maximize your investment in Intercom's Fin AI.
