AI agents analytics
EARLY ACCESSAI agents analytics enables you to monitor performance and usage metrics for your AI agents and tools. Track agent effectiveness, response times, interaction volumes, and tool utilization to optimize your AI-powered customer service operations.
AI agents analytics consists of two main categories:
- AI agent performance - Monitor AI agent performance, tool usage, and session activity. Access specialized metrics through three dedicated tabs: AI agents, Tools, and Sessions.
- Knowledge agent analytics - Track knowledge base performance and content effectiveness. Analyze how AI agents utilize your knowledge resources to provide accurate responses.
AI agent performance
AI agent performance provides comprehensive insights into your AI agents, their tool usage, and session activity. Monitor performance metrics, identify optimization opportunities, and track how effectively your AI agents handle customer interactions.
At the top of the page, you can switch between the following tabs:
- AI agents
- Tools
- Sessions
AI agents
The AI agents tab displays performance metrics for all your AI agents. Monitor agent effectiveness, response times, and interaction volumes to ensure optimal performance.
Summary
| Metric | Description |
|---|---|
| Total agents | Total number of AI agents in your system. |
| Average response time | Average time taken by AI agents to respond to customer interactions (in milliseconds). |
| Total interactions | Total number of interactions handled by all AI agents. |
| Total failures | Total number of failed AI agent interactions. |
Performance by agent
The Performance by agent section provides detailed metrics for each AI agent:
| Column | Description |
|---|---|
| Agent name | Name of the AI agent. |
| Interactions | Number of interactions handled by this agent. |
| Average response time (ms) | Average response time for this agent in milliseconds. |
| Failures | Number of failed interactions for this agent. |
| Success rate | Percentage of successful interactions for this agent. |
Tools
The Tools tab displays performance metrics for all tools used by your AI agents. Monitor tool effectiveness, call volumes, and success rates to ensure your AI agents have access to reliable and efficient tools.
Summary
| Metric | Description |
|---|---|
| Total tools | Total number of tools available to AI agents. |
| Total tool calls | Total number of times tools have been called by AI agents. |
| Average call time | Average time taken to execute tool calls (in milliseconds). |
| Overall success rate | Percentage of successful tool calls across all tools. |
Performance by tool
The Performance by tool section provides detailed metrics for each tool:
| Column | Description |
|---|---|
| Tool name | Name of the tool. |
| Total calls | Number of times this tool has been called. |
| Average call time (ms) | Average execution time for this tool in milliseconds. |
| Failed calls | Number of failed calls for this tool. |
| Success rate | Percentage of successful calls for this tool. |
Sessions
The Sessions tab displays a table of all AI agent sessions. Use this tab to review individual session details and track interactions.
| Column | Description |
|---|---|
| Session ID | Unique identifier for the session. |
| AI agent | The AI agent that handled the session. |
| End user | The end user who participated in the session. |
| Date | Date of the session. |
Knowledge agent analytics
Knowledge agent analytics enables you to monitor performance of your knowledge agents and identify opportunities for improvement. Track how effectively your AI agents utilize knowledge resources, analyze response patterns, and optimize documentation based on real usage insights.
Knowledge agent analytics analyzes real user interactions in production. To test your knowledge agent before going live, use Quality center: Knowledge agents.
Analyze user traffic
Analyze messages your users sent in the selected date range and get insights into knowledge agent performance.
The report provides a detailed list of user messages, the agent response for each message, and the context on which the answer is based so you can better understand why it answered a certain way. It also generates topics.
You can analyze up to 4,000 messages. If more messages exist in the selected period, a random sample is taken across the entire period.
Run analysis
To run an analysis:
- Select a knowledge agent from the drop-down menu.
- Enter task name.
- Select either chatbots or senders from the drop-down menu to filter the analysis. You can only select either chatbots or senders, not both.
- Choose the From and To date.
- Select Run task.
After you run the task, it appears with the details in the task table below.
Tasks
The task table keeps a history of the last 20 analysis tasks.
| Column | Description |
|---|---|
| Task name | Name of the task. |
| Knowledge agent | The knowledge agent that was analyzed. |
| Created at | Date and time when the task was created. |
| Status | Current status of the task (In progress, Failed, or Completed). |
You can view a detailed report or download it for each completed task.
View analysis results
Select the task name from the task table to view the detailed results analytics, or download the .csv file for offline analysis.
The results analytics page includes:
- Overview - Summary metrics including total interactions, response time, success rate, documents retrieved, and insight distribution
- Expected behavior - Interactions categorized as successful, policy restricted, not relevant, or partially answered
- Knowledge gaps - Interactions with hallucination risks or unanswered questions that require documentation improvements
For complete documentation of the results analytics page structure and how to interpret each section, see View the results in Quality center.