Conversation classification

  • Conversation classification

    Kustomer's Conversation Classification is a tool that helps you automate time consuming tasks so that your agents can focus on support inquiries that need their attention, results in better and faster responses. Building a classification model is done through our intuitive interface and is designed for non-technical users; no coding required. Using machine learning, Conversation Classification uses historical data from your organization to build classification models that will then be applied to incoming emails. With classification models, you can automatically:Reduce time spent tagging spam email conversations and customers.Predict contact reasons on conversations and route them to the correct agents.Apply tags to more urgent conversations that need your attention.Use classification models to automatically route conversations to a specific queue. August 2023 update: This feature is no longer available to new customers.In this articlePrepare your dataCreate a classification modelActivate a classification modelUnderstand your quality scoreCheck a model's current statusUse conversation classification with queues & routingRetrain an existing classification modelDeactivate a classification modelSearch and report on your modelPrepare your dataClassification models are created using your own historical data. Higher quality data results in models that perform more consistently and make accurate predictions more often. Before you start, see Conversation classification best practices for data to better understand how to ensure you have enough high quality data for your classification models.Create a classification modelOnce you prepare your data, you're ready to create a model. After selecting the attribute you what you want to predict, we check to see if there is enough data that can be used to learn how to accurately make a classification model. If there is, the system is trained to detect specific language in all first inbound email messages and uses this knowledge to make your desired prediction moving forward. You can create up to 3 classification models.In this example, we will create a classification model using a custom Conversation klass attribute, named Topics, that we use to manually organize incoming emails by topic. We can use this data to make this prediction for us automatically.To add a new classification model:Go Settings> Kustomer IQ > Conversation Classification.Select Add Classification Model.Review step 1 of the wizard. If you've already prepared your data, select Continue.Select the attribute you want the model to use for predictions. You can create classification models for tags or custom conversation fields. For this example, select Topics from the drop-down menu.The tool will analyze all of your data to see if there is enough information available to build the classification model and show you a list of all of the accepted data values. Only values that meet all of the criteria will be included in the model. Any values that cannot be used are also shown on this page.Note: If you do not have values that meet the data amount criteria, please select a different attribute or go back to prepare your data and try again.Select Continue and review the values that will be used to create the model.Select Train to start training the classification model.Depending on how much data you have, it may take some time to train your new model. You'll receive an email once it's ready to be activated.Activate a classification modelAfter the training is complete, you have to activate the classification model so that it can be applied to first inbound emails. You can only have 1 active model at a time.To activate a classification model:You can open the model from directly within the training complete email or navigate to the Conversation Classification page and select for the model you want to activate. Here, you can edit your model's name, review the values that are included, and view the accuracy of the values that are being used in the Quality Score column. In this example, you can see that the sports topic has the highest quality score.Optionally, you can set the confidence level for the predictions made by the model or leave it at the default suggested value. This value determines how many conversations will be classified by your model, as well as the accuracy of the predictions. A higher confidence level will result in fewer conversations being classified and vice versa. In terms of accuracy, a higher confidence level will result  in more accurate predictions.After you review the values and set the model's confidence level, select Activate and then select it again to confirm the action. Once the model is done deploying, it will use an automatically-generated business rule to make predictions on the first inbound email message.Note: You can not edit or delete this business rule.Understand your quality scoreQuality scores are an estimate of how each value in the model will perform in the future. Higher scores result in more correct automations and fewer missed conversations.These scores are primarily influenced by data consistency. A value that is well defined and consistently assigned to similar conversations will yield a high score. On the other hand, values that have a less clear meaning may result in miscategorized conversations, as it’s more difficult for an agent to decide what the appropriate value is. When the boundaries of a value can’t be clearly detected, a low score is provided. Note: Scores range from 0 to 100. While these estimates depend on your classifier, scores above 75 are generally considered good. Values with a score below 50 are excluded from automation. In order to improve a score, we recommend you review how the value is defined and applied by your agents. Is there a similar value it gets confused with? Are the conversations different or not consistent in meaning? Refining your conversation attributes by merging or simplifying values can help increase the overall quality of your classifier.Check a model's current statusYou can view the current status of your classification models from the Conversation Classification page.StatusDescriptionNext StepActiveThe classification model is  successfully trained successfully and is actively making predictions.This model is currently making predictions. We will automatically retrain the model every 7 days to improve its accuracy.DeactivatedThe model is currently deactivated and not making predictions.You can re-activate this model.DeactivatingThe model is in the process of being deactivated. No further action can be taken at this time.DeployingThe classification model is in the process of being activated.No further action can be taken at this time.Failed to DeployThe training wasn't successful due to an issue with your data.You will be notified once the model successfully deploys.Ready to ActivateThe classification model was successfully trained.You can now activate the classification model so it can start making predictions.TrainingThe classification model is still in the process of being trained.No further action can be taken at this time.Use conversation classification with queues & routingIf your organization has queues and routing turned on, you can create a separate business rule that predicts an attribute and then routes a conversation to a specific queue based on the model. For more information, see Use classification models with queues & routing.Retrain an existing classification modelWe will automatically retrain your model every 7 days to account for new data as the amount of conversations in your organization grows. This helps improve the model's accuracy and perform better predictions. Deactivate a classification modelYou can deactivate an active classification model at any time. To deactivate a classification:Selectfor the classification model you want to deactivate.Select Deactivate.You will see a message stating that the classification model is being stopped.Search and report on your modelYou can create search queries and build custom reports to see how your models are performing. For more information, see Search and report on classification models.
  • Use classification models with routing

    If your organization has queues and routing turned on, you can use Conversation Classification to predict an attribute, such as contact reason, and automatically route a conversation to a specific queue based on that classification model. This helps your agents save time they usually spend when manually classifying and routing conversations.August 2023 update: Conversation Classification is no longer available to new customers.Important: If you have queues and routing set up in your organization for smarter routing, you must create two business rules to ensure that conversations are routed to the correct queue.The first step is to create a business rule that performs three actions: adds a tag to a queue, unsets the queue, and predicts data for the incoming conversation. In this example, we will predict tags.Go to Settings > Platform > Business Rules.Select Add Business Rule.Enter a name for the business rule and then select First Inbound Interaction as the Trigger.Set the Conditions criteria to Conversation Channel Is Equal To Email.Select the following options from the Select Action Type menu: Change DataQueue UpdatesPredict DataNext, set the Update Data criteria to add a tag to the conversation. Any new or existing tag can be used and will be removed in the next procedure. For this example, we will set it to Conversation Tags Append predicted.Tip: You can create a new tag directly while creating the rule by selecting Add tag and entering predicted.Set the Make Queue Updates criteria to Queue Unset Queue & Unassign User.Select Tags from tags from the Predict Data drop-down menu. The following image shows all of the actions that we set.Select Save Changes.Next, create a second business rule that removes the predicted tag that we just added.Select Add Business Rule in the Business Rules page.Enter a name for the business rule and then select All Updates as the Trigger.Set the Conditions criteria to:Message Direction Type Is Equal To Initial In Conversation Tags Contains predictedSelect Select Action Type > Change Data. Set the Update Data criteria to Conversation Tags Remove predicted.Select Save Changes.Now, whenever you receive an inbound email, Conversation Classification will predict an attribute such as, Contact Reason, and the conversation will automatically be assigned to the correct queue through queues & routing based on the predicted value.
  • Conversation classification best practices for data

    In today's world, data is fuel and said to be the new oil. Machine Learning (ML) has revolutionized Artificial Intelligence (AI). Due to this, if you intend to power your tools and processes using Machine Learning, you need to make sure you are using the highest quality fuel. August 2023 update: This feature is no longer available to new customers.In this articleWhat is Machine Learning?What does data quality mean?What can you do to ensure high quality data?What is Machine Learning?In traditional programming, developers create code that gives the computer explicit instructions on how to solve a particular task. This differs from Machine learning applications, where the computer instead leverages huge amounts of data and learns to solve a problem by looking at examples. The output of this process, usually called training a model, relies heavily on the quality of the examples used.Let's go over a basic example to help explain how this works. Imagine you want to teach a computer how to recognize pictures of cats. To do so, we would show it hundreds, or even thousands, example pictures of what a cat looks as if saying "Hey. This is a cat." We would also show it examples of other pets and different objects to then say "Oh and this is not a cat." Instead of traditionally describing a cat as a domestic animal that has whiskers, meows, and purrs, we let the computer learn by example and figure out its own tricks to identify what a typical cat looks like. Once the system completes the training, we expect the computer will be able to generalize how cats look, and in the future, when faced with a new picture, the system will be able to successfully identify cats.When it comes to Conversation Classification, you may wish to automate repetitive manual processes, such as reading an incoming message and identifying the contact reason or applying selected tags to classify conversations into topics. In Kustomer, you can use data from your own past conversations as examples to help build a customized automated system that will simulate the cognitive process of understanding the contents of a message.This general mechanism of learning by example, correcting itself via trial and error, and tweaking its own predictions, can be used to teach a computer many other things using collections of data. Current AI is made of millions of such examples of small and very specific devices able to simulate some fascinating functions of the human brain.What does data quality mean?ML applications are often accused of acting as sealed black boxes, especially when they don’t deliver the expected behavior. What we know in advance is that, beyond the algorithms involved, the quality of the data a model is fed with will have the biggest impact in the overall performance.On one hand, an ML model will learn any patterns found in the data used for training, and only those; it will never be able to handle knowledge not present in the training data. On the other hand, it will also struggle when trying to learn from noisy and conflicting information. There is a popular saying: Garbage in, garbage out. That’s why we need to minimize that garbage in the input.Data quality refers to the intrinsic regularities, clarity, and consistency we find in your data, in the tags and custom fields that label your conversations and messages, in the custom categories that are meaningful to manage your customer service. While you can't control your customer's requests, you can control how conversations are organized by designing suitable tags and custom fields. The following example shows the historical data of a simplified ideal customer care service of an online store.There are four different categories, with apparently clear boundaries, that everybody understands and agrees on, and most of the tickets fall into one of them. The tickets have been manually labelled by agents, according to their topic, as part of their daily job. Notice that there are some unlabelled conversations, located between categories. This isn't an issue as long as they are not the majority and the other groups make sense. In addition, the categories don’t overlap. This process of category identification is easy to automate not because of the conversations themselves, but because of the design of the categories. On the other hand, the image below shows the same conversations without a clear definition of categories. In this case, some categories have overlapping meanings and names. When the limits of a category are not clear, it’s easy for agents to get confused when categorizing conversations and possibly use the incorrect labels. By looking at the categories below, you can see that the category used might vary, depending on the questions the agent is asking while choosing the one to apply. If the customer wants to return an item, it is a return or a return and a refund? They're not happy with the service so far so technically, it could also be labeled a complaint. Should they just select all of the labels that might apply, just in case?In addition, as far as the promotions, sales, and coupons categories are concerned, there is too much granularity. It obviously depends on the business, but considering the volume and the similarity of the labels, it seems that space may be simplified by grouping all those conversations under a single category.Finally, there is another problem with this scenario: a potential new category that would eventually be useful to gather a group of conversations with common patterns is missing—the category for customers asking for an update in their order status.What can you do to ensure high quality data?There are a number of steps you can take to ensure the highest quality of your data and leverage automation from the very beginning.Design tags and custom fields that best suit your needs. While it’s impossible to cover all cases, try to focus on the most important ones in terms of expected volume. For example, order status, returns & refunds, or promo codes.Create new tags to cover unlabelled information rather than redefining and remapping everything over again.Add granularity in an incremental fashion. When possible, start with a broad category, such as pets, before attempting to split it into various, more specific categories, such as cats, dogs, or birds.Try to use clearly defined tags and categories to organize your information. A good rule of thumb is to test them with people. If your team understands their boundaries correctly and knows when to apply them, an ML algorithm will be able to figure out their meanings.Try to make your team work as consistently as possible. Again, this is easier to achieve when tag limits are clear. Try to keep data stable for a while so that you can grow the amount of examples using your tags and custom fields. While you can iterate and re-define your tags and custom fields at any time, regularly changing this information will affect the amount of data points you get. Also, keep in mind that any re-definition of tags and categories will need the model to be re-trained.
  • Search and report on classification models

    You can create search queries and build custom reports to see how your conversation classification models are performing.August 2023 update: Conversation Classification is no longer available to new customers.In this articleConversation classification chart templatesPrediction Status valuePrediction Corrected By valueConversation classification chart templatesYou can create a custom report using pre-configured templates to view information such as, how many conversations where a prediction couldn't be generated, or the number of conversations where a prediction was generated and not corrected by an agent. To build the report:Go to Reporting .Select New, enter a name for the report, and select New Report.Select Add Chart and then select Conversation Classification - IQ from the Categories drop-down menu.Select a chart template from the list and then select Next.Optionally, add any additional filters or rename the chart and then select Save.Repeat these steps to add the remaining chart templates.Prediction Status valueYou can use the Prediction Status value to create search queries or build custom reports to view how well your classification models are doing. When applying this value in a filter, you can choose from one of the following criteria to view how classification models are working with your conversations:Skipped shows the conversations that weren't able to be predicted.Automated shows conversations where the predictions were automated.Corrected shows the conversations where the suggestions were corrected.Prediction Corrected By valueYou can also use the Prediction Corrected By value to see conversations where predictions were corrected by a specific user.These values are also available for you to use when segmenting a custom report so that you can get a better idea of how your data compare against each other. For example, you can create a report to view the top agents that are correcting predictions made by your classification models.Note: These options are only available for customers with the KIQ for Agents plan when searching or reporting on the Conversation object. For more information on available operators, see Understanding filter operators.
Powered by Kustomer