Chatbots are a relatively new technology, and experts are yet to reach a consensus on what constitutes success metrics.
So how can you tell whether your chatbot is actually working? How do you ensure it’s successfully handling the tasks you’ve designed it to handle? And what can you look for to make sure?
The three pillars of chatbot success metrics
Analytics gives you insights into chatbot performance, as well as valuable customer insights.
Knowing what to look for will help you spot what your chatbot is doing well, but also where you may need to do some fine tuning to ensure future success.
1. User metrics
Chatbots are there to help you help your customers – in this case, your users. This set of metrics will show you how helpful your chatbot is by analyzing the following information:
Number of total users
The number of total users engaging with your chatbot shows how many of your users are actually using your chatbot. If this number is significantly lower than the number of customers or users you know that you have or were planning to engage with, consider the following:
- Chatbot awareness: This often gets overlooked. Customers need to know that they can chat with your business. Make sure your bot is either visible to them on your website, add a link in confirmation emails, or provide them a scannable QR code in-store. You can also run a social media campaign to expand your reach and target specific audiences to increase chatbot adoption even further.
- Channels: When it comes to channels, customer preferences are as unique as your customers are. Make sure your chatbot is available over a channel that is practical for your customers to use. For example, our Answers platform covers WhatsApp, Facebook Messenger, Viber, Live Chat, SMS, and RCS for this reason.
Number of new users
This metric can be indicative of how well your bot solves customer enquiries. The closer the number of new users is to the number of total users engaging with your chatbot, the greater the likeliness of first-contact resolution.
Protip: You can automate lead generation throughout the sales process by deploying a chatbot to convert site visitors into quality leads.
If the ratio of new users to total users is low, however, then this could be indicative of return users whose issues were not resolved. This requires attention, since it could potentially lead to lower customer satisfaction.
The bounce rate metric is indicative of how useful customers find your chatbot. A high bounce rate, meaning a chatbot session that is ended before programmed completion, can indicate that users do not find the chatbot useful and prematurely end the session.
High bounce rates should be investigated and further analyzed to see where in the conversation customers are dropping off – and why.
What exactly constitutes a high bounce rate will vary from chatbot to chatbot, depending on predefined goals that your chatbot is designed to guide users towards. The lower the bounce rate is, then the higher the chance of having a higher goal completion rate.
Goal completion rate
This chatbot success metric is the most important success indicator in the user metrics, since it shows how many users successfully completed the goals you set for your chatbot to meet.
These goals will be different for different chatbots, of course – but seeing how many users really found the closest ATM to their location or how many left their contact information – shows how well your chatbot performs.
A low completion rate, which is best defined by considering your own business goals, means you should probably look at user chats and conversations metrics to see where drop off is happening.
2. Conversation metrics
Conversation metrics show you what happens in each conversation. This gives you insight into how effective your chatbot is at resolving customer enquiries.
While there is no ideal time, this metric is indicative of how well your chatbot performs at engaging users. The longer the session duration, the better your chatbot is at creating an engaging conversational experience.
Consider user metrics when analyzing session duration. For example, if your goal is to create a rich conversational experience with chatbots, then longer session durations with low bounce rates and higher goal completions are indicative of success.
On the other hand, long session durations with higher bounce rates and low goal completion rates could mean your chatbot is taking users along for long, fruitless conversations that end in frustration for your customers.
Chatbot to agent takeover
This metric shows how many times an agent needed to take over a conversation from a chatbot, or how many times a user asked to be transferred to a human agent.
This indicates how successful a chatbot is at handling conversations with customers – the lower the rate, the better.
Since one of the key benefits of having a chatbot is to free up agents to focus on more complex tasks, a high rate of agent takeovers is potentially cause for concern. This is where you should analyze customer chats to see where and why chats are being transferred.
This isn’t to say that the number of agent takeovers should be zero – the ideal contact center should be set up for seamless live agent takeovers – but keeping an eye on this metric is important to constantly develop your chatbot for high customer satisfaction.
3. CSAT metrics
Customer satisfaction metrics are the best indicator of how well your chatbot is performing its main tasks – which are related maintaining and improving customer satisfaction.
Net Promoter Score (NPS)
Asking customers to rate their level of satisfaction after their customer service experience provides you with valuable, direct user feedback.
First-contact resolution, fast resolution, as well as a pleasant and respectful experience that reflects your brand voice are all components of experiences that result in high customer satisfaction.
If you’re getting a low NPS from customers over your chatbot, analyze poorly rated chats looking for reasons why users are leaving low ratings – an unsatisfied user will almost always share exactly what bothered them.
Satisfied customers will be repeat customers – and you can check how many customers are coming back to engage with your chatbot.
Mobile users can be identified by their International Mobile Subscriber Identity (IMSI), so you can easily see how many repeat users engage with your chatbot.
Analyzing this metric by referencing user and conversation metrics will show whether your repeat users are having positive chatbot interactions. If the metrics indicate repeat customers are having positive interactions, then congratulations on developing a successful chatbot and keep up the good work.
This piece outlined the fundamentals of chatbot success metrics. Do check out the following posts to further improve your chatbot performance:
- Three benefits of adding chatbots to your business
- Chatbot implementation challenges (and how to solve them)
- Automated customer service: advantages and examples
Answers, the chatbot building platform. Talk to our experts to get answers to all your questions.