# Testing Automations

> Building an AI Agent team generally involves providing    Agent instructions    Tools and knowledge sources    Guardrails and guidance on tone    Testing befor

Source: https://help.kustomer.com/en_us/testing-ai-agents-B1GdVVIKWl

Last updated: 2026-08-18T16:44:34.469Z

Building an automation with Kustomer Concierge or AI for Customers 1.0 generally involves configuring:

*   Automation procedures
*   Tools and knowledge sources
*   Guardrails and tone guidance

Each version uses its own form of testing, so it's important to understand these differences before moving forward. 

Version

Testing type

AI for Customers 1.0

Test

Concierge

Test console

  
Testing before deployment, and while making iterative configuration changes, helps you validate how your automation behaves before it handles real customer conversations.

As you build an automation, use the test console to get direct feedback on its behavior and responses. The test console helps you test draft changes in a sandbox environment before deployment. It can also help you prepare for using [Evaluations](https://kustomer.kustomer.help/en_us/use-evaluations-to-measure-ai-performance-BJy49yooxe), where automations are run through structured tests to confirm they behave as expected.

### **In this article**

1.  [Using the testing tool](#using-testing)
2.  [Testing automations](#testing-agents)
3.  [Recommended additions](#recommended-additions)

### **Using the testing tool**

The testing tool is available for any selected automation. When you edit an automation, Kustomer creates a draft version of the AI Agent. The draft is not deployed, so you can safely make updates to your settings and test them before publishing changes.

To access the testing tool:

1.  From the left side navigation, click **Kustomer AI**. The **AI Automations** screen opens. 
2.  If you are using Kustomer Concierge:
    1.  Select the **Automation** to test from the dropdown menu. 
    2.  The recommended method to access the **Test Console** is through the **Build** page as you create your automation. The test console is also available via: 
        1.  The Test page, where previous tests are accessible and new tests can be created.
        2.  The Deploy page prior to launching.
        3.  The Analyze page to compare and iterate against traces. 
        4.  The Settings page to check how changes affect performance in real time. 
3.  If you are using AI for Customers 1.0:
    1.  Locate the **AI Agent Team** to test. Click the **Edit** icon to open the team page. 
    2.  From the top right of the **Automation**, click **Test**. 

### **Testing Automations**

Automation testing lets you interact with your AI Agent in a sandbox environment that mimics the customer email and chat experience. Use testing to observe how your AI Agent responds based on the draft configuration you are working on.

Testing is useful for validating:

*   How the AI Agent follows its instructions and procedures
*   Whether the AI Agent uses the expected tools and knowledge sources
*   Whether guardrails, handoff behavior, and tone guidance are working as intended
*   How the AI Agent responds to realistic customer messages before deployment

**Note:** While not required for automations testing, ensure you have [authorized a domain to use Kustomer chat](https://kustomer.kustomer.help/en_us/authorize-chat-domains-rJX4_Q0XN) before you deploy your tested optimization. 

During testing:

1.  A test customer is used for the testing experience.
    1.  The test customer is unique to the user and automation being tested.
    2.  You can view the test customer from the tester.
    3.  Kobjects and attributes can be added to the test customer’s profile to create a more realistic simulation based on your organization’s data.
2.  Test activity is associated with the testing experience.
    1.  The tester uses a sandbox chat and email experience so you can interact with the draft version of your AI Agent.
    2.  The tester tracks conversation activity while you interact with the AI Agent.
    3.  Conversation history availability may depend on the current testing experience and your organization’s configuration.
3.  Testing conditions are based on the draft settings for your automation.
    1.  Changes made in the draft can be tested before deployment.
    2.  Testing the draft does not deploy the automation or change the live customer-facing experience.
4.  You can continue updating the draft and retesting until the AI Agent behaves as expected. 

![](https://cdn.kustomerhostedcontent.com/media/570fad9d9001bc1000163b28/690657b27965c0db7b1fb3dd9dbfe3bf.png)

**Note:** Chat-based testing requires the chat testing experience to be available for your organization. Authenticated chat domains are required for deployed website chat, but not for testing automations.

### Recommended Additions 

Concierge's Test Console shows the Execution Details for each completed run. These details include each conversation's message in and out, and which procedures were triggered during that message. Use this to pinpoint where automations are using the procedures that are intended.  

![](https://cdn.kustomerhostedcontent.com/media/570fad9d9001bc1000163b28/67a09540e904f4ec4adb19df2335bb74.png)

Concierge's Test console conversations also appear in Analyze. From the **Analyze** page, you can run Observability operations on Conversations and individual traces via the Architect. Click the message to view traces, then click a trace and **Generate Analysis** to gain more insights about that interaction. 

![](https://cdn.kustomerhostedcontent.com/media/570fad9d9001bc1000163b28/2db2d94d30a901401409b88e6de2868b.png)