See related

Conversation classification

Last Update: Sep 2024 • Est. Read Time: 5 MIN
To check plan availability, see the pricing page.

Kustomer's Conversation Classification is a tool that helps you automate time consuming tasks so that your agents can focus on support inquiries that need their attention, results in better and faster responses. Building a classification model is done through our intuitive interface and is designed for non-technical users; no coding required. Using machine learning, Conversation Classification uses historical data from your organization to build classification models that will then be applied to incoming emails. With classification models, you can automatically:

  • Reduce time spent tagging spam email conversations and customers.
  • Predict contact reasons on conversations and route them to the correct agents.
  • Apply tags to more urgent conversations that need your attention.
  • Use classification models to automatically route conversations to a specific queue. 

August 2023 update: This feature is no longer available to new customers.

In this article

Prepare your data

Classification models are created using your own historical data. Higher quality data results in models that perform more consistently and make accurate predictions more often. Before you start, see Conversation classification best practices for data to better understand how to ensure you have enough high quality data for your classification models.

Create a classification model

Once you prepare your data, you're ready to create a model. After selecting the attribute you what you want to predict, we check to see if there is enough data that can be used to learn how to accurately make a classification model. If there is, the system is trained to detect specific language in all first inbound email messages and uses this knowledge to make your desired prediction moving forward. You can create up to 3 classification models.

In this example, we will create a classification model using a custom Conversation klass attribute, named Topics, that we use to manually organize incoming emails by topic. We can use this data to make this prediction for us automatically.

To add a new classification model:

  1. Go SettingsKustomer IQ > Conversation Classification.
  2. Select Add Classification Model.
  3. Review step 1 of the wizard. If you've already prepared your data, select Continue.
  4. Select the attribute you want the model to use for predictions. You can create classification models for tags or custom conversation fields. For this example, select Topics from the drop-down menu.
  5. The tool will analyze all of your data to see if there is enough information available to build the classification model and show you a list of all of the accepted data values. Only values that meet all of the criteria will be included in the model. Any values that cannot be used are also shown on this page.

    Note: If you do not have values that meet the data amount criteria, please select a different attribute or go back to prepare your data and try again.

  6. Select Continue and review the values that will be used to create the model.
  7. Select Train to start training the classification model.

Depending on how much data you have, it may take some time to train your new model. You'll receive an email once it's ready to be activated.

Activate a classification model

After the training is complete, you have to activate the classification model so that it can be applied to first inbound emails. You can only have 1 active model at a time.

To activate a classification model:

  1. You can open the model from directly within the training complete email or navigate to the Conversation Classification page and select for the model you want to activate. Here, you can edit your model's name, review the values that are included, and view the accuracy of the values that are being used in the Quality Score column. In this example, you can see that the sports topic has the highest quality score.
  2. Optionally, you can set the confidence level for the predictions made by the model or leave it at the default suggested value. This value determines how many conversations will be classified by your model, as well as the accuracy of the predictions. A higher confidence level will result in fewer conversations being classified and vice versa. In terms of accuracy, a higher confidence level will result  in more accurate predictions.
  3. After you review the values and set the model's confidence level, select Activate and then select it again to confirm the action. 

Once the model is done deploying, it will use an automatically-generated business rule to make predictions on the first inbound email message.

Note: You can not edit or delete this business rule.

Understand your quality score

Quality scores are an estimate of how each value in the model will perform in the future. Higher scores result in more correct automations and fewer missed conversations.

These scores are primarily influenced by data consistency. A value that is well defined and consistently assigned to similar conversations will yield a high score. On the other hand, values that have a less clear meaning may result in miscategorized conversations, as it’s more difficult for an agent to decide what the appropriate value is. When the boundaries of a value can’t be clearly detected, a low score is provided. 

Note: Scores range from 0 to 100. While these estimates depend on your classifier, scores above 75 are generally considered good. Values with a score below 50 are excluded from automation. 

In order to improve a score, we recommend you review how the value is defined and applied by your agents. Is there a similar value it gets confused with? Are the conversations different or not consistent in meaning? Refining your conversation attributes by merging or simplifying values can help increase the overall quality of your classifier.

Check a model's current status

You can view the current status of your classification models from the Conversation Classification page.

Status
Description
Next Step
ActiveThe classification model is  successfully trained successfully and is actively making predictions.This model is currently making predictions. We will automatically retrain the model every 7 days to improve its accuracy.
DeactivatedThe model is currently deactivated and not making predictions.You can re-activate this model.
DeactivatingThe model is in the process of being deactivated. No further action can be taken at this time.
DeployingThe classification model is in the process of being activated.No further action can be taken at this time.
Failed to DeployThe training wasn't successful due to an issue with your data.You will be notified once the model successfully deploys.

Ready to Activate
The classification model was successfully trained.You can now activate the classification model so it can start making predictions.
TrainingThe classification model is still in the process of being trained.No further action can be taken at this time.


Use conversation classification with queues & routing

If your organization has queues and routing turned on, you can create a separate business rule that predicts an attribute and then routes a conversation to a specific queue based on the model. For more information, see Use classification models with queues & routing.

Retrain an existing classification model

We will automatically retrain your model every 7 days to account for new data as the amount of conversations in your organization grows. This helps improve the model's accuracy and perform better predictions. 

Deactivate a classification model

You can deactivate an active classification model at any time. 

To deactivate a classification:

  1. Selectfor the classification model you want to deactivate.
  2. Select Deactivate.
    You will see a message stating that the classification model is being stopped.

Search and report on your model

You can create search queries and build custom reports to see how your models are performing. For more information, see Search and report on classification models.