We have recently refreshed our branding across our offerings and changed the names of our pricing plans. If you have signed up before Aug 9, 2021, please click Previous plans to view your applicable plans.
We assure you that this change will not impact your product experience, and no action is required on your part.
The Answers module on bot builder enables you to set up direct answers to end-user queries using information from the knowledge base/FAQs. This makes the Answers module a very powerful feature where users can have conversational interaction with the bot and get solutions with ease.
Since the performance of the Answers module depends entirely on how the questions and responses have been configured, it is very important to track performance to improve deflection rates. This is where the Answers performance analysis curated report will ensure that it’s easy to get insights and act upon them. To access this, go to
> Chat reports > Bot analytics > Answers performance analysis.
With the Answers performance analysis report, agents can accomplish the following:
- Understand the % of questions answered by the bot through “Answer rate” and improve the answer rate by digging into the unanswered queries
- Improve Q&A level performance by getting insights on the effectiveness of each Q&A configured in the Answers module
- Improve the feedback rate and helpfulness of the article by ensuring it gets recommended on relevant user queries
- Track trends on answers and Q&A response rates and easily identify the time periods when these KPIs fall to make improvements
- Refer to the underlying data to get granular details on each question asked and the answer provided by the bot in each conversation
Answers performance analysis report
List of available attributes
| Attribute name | Attribute type | Definition |
| Conversations | Metric | Count of conversations where the “Answers” module was triggered |
| User Queries | Metric | Count of questions asked by the users |
Answered | Metric | Count of user queries where the bot was able to provide an answer |
| Recommendations | Metric | Count of user queries where the bot responded with a Q&A recommendation |
| Feedback provided | Metric | Count of feedback provided by the users when the bot responded with a Q&A recommendation |
| Positive feedback | Metric | Count of positive feedback provided by the users when the bot responded with a Q&A recommendation |
| Negative feedback | Metric | Count of negative feedback provided by the users when the bot responded with a Q&A recommendation |
| No feedback | Metric | Count of Q&A recommendation where the users didn’t provide feedback |
| Unanswered | Metric | Count of user queries where the fallback action was triggered and the bot could not come up with a response |
| Natural Language | Metric | Count of all the responses provided by the bot |
| Utterance initiated at | Filter, group by, underlying data | Timestamp at which the customer sent a message |
| Question | Filter, group by, underlying data | Title of the articles/Q&A configured on that bot |
| Question id | Filter, group by, underlying data | Unique id corresponding to each Q&A on the bot |
| Response language | Filter, group by, underlying data | The language in which the bot provided a response to the customer |
| User query id | Filter, underlying data | Unique id associated with each user message |
| Conversation id | Filter, group by, underlying data | Unique id for a bot conversation |
| User query text | Filter, group by, underlying data | The content of the message sent by the user |
| Bot | Filter, group by, underlying data | Name of the bot on which conversations were initiated |
| Bot version | Filter, group by, underlying data | The version of the bot on which the conversation were initiates |
| Feedback | Filter, group by, underlying data | Feedback provided by the user on a Q&A response |
| NLP module type | Filter, group by, underlying data | The type of NLP feature being used - QA or Intents |
| Answer response type | Filter, group by, underlying data | The type of response provided by the bot on a user message - Single recommendation, Multiple recommendations, Small talk or Unanswered |
| Unique response id | Underlying data | Unique id associated with every response provided by the bot |
Note: Attribute type illustrates whether an attribute can be used in a report as a metric to be analyzed, as a filter, as a group by field, or as a column in the underlying data.
Widget definitions
Filters:
Date range - Allows choosing the relevant date range for which data needs to be analyzed
Bot - Filter that can be used to define a particular bot or a set of bots
Question - Filter data for a particular question or a set of questions to drill down into specific cases
Response language - Filter for a particular language to gauge performance at a language level
Charts:
Answer overall performance:
Provides an end to end view of answers module performance starting from the number of questions asked and ending on the instances where it was deemed helpful
- User queries: Count of messages sent by users
- Answered Queries: Count of responses provided by the bot from the "Answers" module., i.e Q&A and Small talk
- Small talk response: Count of "Small talk" responses provided by the bot
- Q&A responses: Utterances where a Q&A was recommended by the bot
- Feedback provided: Count of feedback provided by users to Q&A responses
- Helpful %: % positive feedback out of all feedback provided by users
Volume analysis trend:
Provides a trend view of all the volume related metrics on the answers module
- User queries: Count of messages sent by users
- Answered Queries: Count of responses provided from the "Answers" module., i.e Q&A and Small talk
- Small talk response: Count of "Small talk" responses
- Q&A responses: Utterances where a Q&A was recommended
Q&A performance analysis:
Provides a performance overview in terms of recommendation volume, feedbacks received and helpful% for each question configured in the Answers module or recommended as FAQs/Solution articles
- Question: Title of the Q&A response configured on answers module or FAQs
- Q&A responses: Utterances where a Q&A was recommended by the bot
- Feedback provided: Count of feedback provided by users to Q&A responses
- Helpful %: % positive feedback out of all feedback provided by users
Answer and Q&A response rate trends:
Provides a trend view of Answer rate and Q&A response rates on messages sent by users
- Answer rate: % of user queries where the bot provided a Q&A or Small talk response
- Q&A response rate: % of user queries where the bot provided a Q&A response
Q&A response performance trend:
Trend analysis of Q&A responses provided by the bot
- Q&A responses: Utterances where a Q&A was recommended by the bot
- Feedback provided: Count of feedback provided by users to Q&A responses
- Helpful: Count of positive feedback provided by users
Underlying data :
Provides a granular view of each question-response text on the Answers module
Note: Data on this module is present for conversations initiated from 23rd January 2023 onward
