# Kustomer Concierge Report > The Kustomer Concierge report helps you evaluate how effectively Concierge automations handle customer conversations This beta report changes how severa Source: https://help.kustomer.com/en_us/kustomer-concierge-report-B1NsSbYLfe Last updated: 2026-09-16T14:58:04.723Z The Kustomer Concierge Report helps you evaluate how effectively Concierge automations handle customer conversations. This report changes how several metrics are calculated from the current Kustomer Concierge report to add more detail and insight into its data. Use the report to understand: * How frequently Concierge participates in conversations * How often Concierge resolves conversations without human assistance * How often conversations are escalated to a human * Whether specific automations, agents, procedures, or tools are contributing to errors or incomplete outcomes * How customer satisfaction compares between Concierge-involved and human-only conversations ## Who can access this feature? Managers and users with permission to access reporting can view the Concierge report. This report applies to Kustomer Concierge. It does not include activity from Kustomer Envoy used with Copilot. ## In this article * [Access the Concierge report](#access-the-concierge-report) * [Filter report data](#filter-report-data) * [AI Resolution Rate](#ai-resolution-rate) * [Total AI Resolutions](#total-ai-resolutions) * [AI Resolutions charts](#ai-resolutions-charts) * [AI Conversation Engagement Rate](#ai-conversation-engagement-rate) * [AI Conversation Engagements chart](#ai-conversation-engagements-chart) * [AI Escalation Rate](#ai-escalation-rate) * [AI Escalations chart](#ai-escalations-chart) * [Customer satisfaction metrics](#customer-satisfaction-metrics) * [Automation Performance Summary](#automation-performance-summary) * [Agent Performance Summary](#agent-performance-summary) * [Procedure Performance Summary](#procedure-performance-summary) * [Tool Performance Summary](#tool-performance-summary) * [Interpret Concierge performance](#interpret-concierge-performance) ## Access the Concierge report To open the report: 1. In the left navigation, click the **Kustomer AI** sparkle icon. 2. Click **Performance** **Note:** This report is also accessible if you click **Reporting > Kustomer Concierge** The report displays data for conversations that started during the selected date range. ## Filter report data Use the controls at the top of the report to select a date range and apply available filters, such as: * The AI agent used by an automation * AI automations by name or version * Conversations by channel The report also displays when its primary charts were last updated. > **Note:** Most headline metrics include only conversations that started during the selected date range. Performance table metrics may attribute activity based on when the automation, agent, procedure, or tool began participating in the conversation. ## AI Resolution Rate ![AI Resolution Rate](https://cdn.kustomerhostedcontent.com/media/570fad9d9001bc1000163b28/183bb86f4a016f5a1d91411228d54a53.png) The percentage of Concierge-involved conversations that were closed with only outbound messages from Concierge and no outbound messages from a human agent. Only conversations that started during the selected date range are included. Use this metric to measure the proportion of Concierge-involved conversations resolved without human participation. A higher resolution rate generally indicates that Concierge is independently handling more customer requests. Review this metric alongside customer satisfaction, escalation, and error rates to make sure increased automation is not reducing answer quality. > **Note:** The headline AI Resolution Rate is calculated across the organization. It is not the average of the resolution rates shown in the performance summary tables. ## Total AI Resolutions ![Total AI Resolutions](https://cdn.kustomerhostedcontent.com/media/570fad9d9001bc1000163b28/37a7557cad34cb4ee845809493188b98.png) The number of Concierge-involved conversations that were closed with only outbound messages from Concierge and no outbound messages from a human agent. Only conversations that started during the selected date range are included. Use this metric with AI Resolution Rate to distinguish between the percentage of work resolved by Concierge and the actual volume of resolved conversations. ## AI Resolutions charts The AI Resolutions charts show Concierge resolutions and resolution rates by date and channel. ![AI Resolutions charts](https://cdn.kustomerhostedcontent.com/media/570fad9d9001bc1000163b28/5a6c115219f7d295b099de6879f13489.png) A conversation is counted as an AI resolution when it: * Included Concierge activity * Was closed * Contains outbound messages from Concierge * Contains no outbound messages from a human agent For multi-channel conversations, the conversation is counted once for every channel it used. Because of this, the combined channel totals may be greater than the Total AI Resolutions value. Use these charts to identify channels where Concierge resolves a comparatively high or low proportion of conversations. ![AI Resolutions by channel](https://cdn.kustomerhostedcontent.com/media/570fad9d9001bc1000163b28/38c2202b43ab6297fd9a51def8f89ccd.png) A low resolution rate on one channel may indicate that: * Customers use that channel for more complex requests * Channel-specific instructions need improvement * The automation lacks access to a required tool or procedure * Customers are being escalated unnecessarily ## AI Conversation Engagement Rate The percentage of all conversations where Concierge was involved. ![AI Conversation Engagement Rate](https://cdn.kustomerhostedcontent.com/media/570fad9d9001bc1000163b28/d83eaed7ea26a23b8ba088d21b4f754c.png) Conversations, both [engaged](https://help.kustomer.com/en_us/engaged-conversations-r1CGisY21g) and not engaged, are included in the denominator as long as the following is true: * The conversation did not involve Concierge * The conversation is not considered spam Only conversations that started during the selected date range are included. Use this metric to understand how much of your overall conversation volume reaches Concierge. A low engagement rate may be expected when Concierge is intentionally limited to certain channels, queues, customer segments, or use cases. An unexpectedly low rate may indicate that automation entry conditions or routing rules are too restrictive. > **Note:** The headline Engagement Rate is calculated across the organization. It is not the average of the engagement rates in the performance summary tables. ## AI Conversation Engagements chart The AI Conversation Engagements chart shows the daily number and percentage of conversations where Concierge was involved, broken down by channel. ![AI Conversation Engagements chart](https://cdn.kustomerhostedcontent.com/media/570fad9d9001bc1000163b28/d562ae027cef032d594c109bf228ab5e.png) Only conversations that started during the selected date range are included in each date bucket. For multi-channel conversations, the conversation is counted once for every channel it used. Use this chart to identify changes in adoption or coverage over time. A sudden decrease may indicate a change to automation configuration, routing, channel availability, or entry conditions. ## AI Escalation Rate The percentage of Concierge-involved conversations where Concierge escalated the conversation to a human. An escalation is identified from automation events that route the conversation to a queue or leave it unassigned for human handling. ![AI Escalation Rate](https://cdn.kustomerhostedcontent.com/media/570fad9d9001bc1000163b28/a2c296b50d33dcc051d3c9fb56de06ee.png) Only conversations that started during the selected date range are included. Escalations are not necessarily failures. An escalation may be the correct result for a request that requires human judgment, authorization, or access that Concierge does not have. Review escalated conversations to determine whether: * The escalation followed the intended policy * Concierge could have completed the request with better instructions * A missing tool or procedure caused the escalation * A procedure failed before the escalation * The automation escalates too early > **Note:** The headline Escalation Rate is calculated across all Concierge-involved conversations. It is not the average of the escalation rates in the performance summary tables. ## AI Escalations chart The AI Escalations chart shows the daily number and percentage of Concierge-involved conversations escalated to a human, broken down by channel. ![AI Escalations chart](https://cdn.kustomerhostedcontent.com/media/570fad9d9001bc1000163b28/a4248810ac5aecad4253c1e6fc56b347.png) Only conversations that started during the selected date range are included in each date bucket. For multi-channel conversations, the conversation is counted once for every channel it used. Use this chart to identify channels or periods with unusually high escalation rates. Compare changes with recent edits to automations, agent instructions, procedures, tools, or routing configurations. ## Customer satisfaction metrics Customer satisfaction metrics appear when Concierge CSAT reporting is enabled for your organization. ### Concierge-Involved Customer Satisfaction Score The percentage of rated Concierge-involved conversations where the customer selected a satisfied rating of four or five stars. ![Concierge-Involved Customer Satisfaction Score](https://cdn.kustomerhostedcontent.com/media/570fad9d9001bc1000163b28/6224c72d47b61d233bf6572ca4feccae.png) Only conversations with a customer satisfaction rating are included. Only conversations that started during the selected date range are included. This metric is the percentage of satisfied ratings, not an average star rating. Use this metric to confirm that Concierge-involved conversations are producing satisfactory customer outcomes, including conversations that may have later involved a human agent. ### Human-Only Customer Satisfaction Score The percentage of rated conversations handled entirely by human agents where the customer selected a satisfied rating of four or five stars. ![Human-Only Customer Satisfaction Score](https://cdn.kustomerhostedcontent.com/media/570fad9d9001bc1000163b28/af2a1bbe6df0ac78ef4c8c3dcb8b8441.png) Only conversations with a customer satisfaction rating are included. Only conversations that started during the selected date range are included. This metric is the percentage of satisfied ratings, not an average star rating. Use it as a directional comparison for the customer experience in conversations that did not involve Concierge. ### Concierge & Human Agent Customer Satisfaction Compared chart ![Concierge and Human Agent Customer Satisfaction Compared chart](https://cdn.kustomerhostedcontent.com/media/570fad9d9001bc1000163b28/164cb83a5eef1603df3f5edb4ef304f7.png) This chart compares daily average customer satisfaction ratings for: * Conversations where Concierge was involved * Conversations handled entirely by human agents Only rated conversations that started during the selected date range are included. Use this chart to identify changes in satisfaction over time. Consider differences in conversation type and complexity when comparing the two groups. Human agents may receive a larger proportion of escalated or complex requests, so the comparison should not be treated as a controlled measure of agent quality. ## Automation Performance Summary The Automation Performance Summary compares each Concierge automation and version. A conversation can involve more than one automation. When this happens, the conversation may appear in multiple rows. Therefore, row values should not be added together or averaged to reproduce the headline metrics. The table contains the following columns. ### Version The version of the automation associated with the reported activity. Use this column to compare performance before and after publishing an updated automation version. ### Total Conversations The number of Concierge-involved conversations attributed to the automation during the selected date range. This value is the denominator for the automation's AI Resolution Rate, Escalation Rate, and Error Rate. ### Engagement Rate The percentage of Concierge-involved conversations attributed to the automation where that automation participated. The activity is attributed based on when the automation began participating. Multiple automations can participate in one conversation, so row percentages do not add up to 100%. ### AI Resolution Rate The percentage of conversations attributed to the automation that were closed with only Concierge outbound messages and no human outbound messages. Use this metric to identify automations that independently complete a high or low proportion of their assigned work. ### Escalation Rate The percentage of conversations attributed to the automation where an escalation to a human occurred. Use this metric to identify automations that frequently hand work to humans. Review the automation's scope and escalation instructions before treating a high rate as a problem. ### Error Rate The percentage of conversations where the automation encountered an error and could not send a message to the customer. A high error rate indicates a technical or configuration issue rather than a deliberate escalation. Review the affected records and the performance summaries for the automation's agents, procedures, and tools. ## Agent Performance Summary The Agent Performance Summary compares the AI agents used within your Concierge automations. A conversation can involve multiple AI agents and may appear in more than one row. The table contains the following columns. ### Automation The automation containing the AI agent. Automations with running in Single Agent mode will see only one main agent. ### Version The version of the AI agent associated with the reported activity. ### Total Conversations The number of Concierge-involved conversations attributed to the AI agent during the selected date range. This value is the denominator for the agent's AI Resolution Rate, Escalation Rate, and Error Rate. ### Engagement Rate The percentage of Concierge-involved conversations attributed to the agent where that agent participated. Multiple agents can participate in the same conversation, so row values do not add up to 100%. ### AI Resolution Rate The percentage of conversations attributed to the agent that were closed with only Concierge outbound messages and no human outbound messages. A low rate may indicate that the agent is assigned complex work, lacks sufficient instructions, or relies on procedures and tools that do not consistently complete successfully. ### Escalation Rate The percentage of conversations attributed to the agent where an escalation to a human occurred. Review the agent's instructions and escalation criteria to determine whether the escalations were expected. ### Error Rate The percentage of conversations where the AI agent encountered an error and could not send a message to the customer. Use drilldown details and the procedure and tool summaries to investigate the source of recurring errors. ## Procedure Performance Summary The Procedure Performance Summary shows how frequently each procedure runs and whether its runs complete successfully. Each conversation can contain multiple procedure runs. Every run is counted separately. The table contains the following columns. ### Automation The automation containing the procedure. ### Version The automation version associated with the procedure run. ### AI Agent Name The AI agent that owns the procedure. ### Engagement Rate The percentage of Concierge-involved conversations attributed to the procedure where the procedure ran. A conversation can use multiple procedures, so row values do not add up to 100%. ### Total Executions The total number of procedure runs during the selected date range. This value is the denominator for Completion Rate, Error Rate, and Failure Rate. ### Completion Rate The percentage of procedure runs that reached full completion. A low completion rate means that a significant portion of procedure runs ended in either an error or a goal-validation failure. ### Error Rate The percentage of procedure runs that failed for a reason other than goal validation. Examples may include a tool error, invalid configuration, missing data, or another execution problem. Goal-validation failures are counted under Failure Rate instead. ### Failure Rate The percentage of procedure runs where goal validation determined that one or more steps did not achieve their objective. For example, a procedure may finish its steps but fail because an identity verification result did not meet the required condition. Use Error Rate to investigate execution problems. Use Failure Rate to investigate procedures that run as configured but do not achieve the intended business result. ## Tool Performance Summary The Tool Performance Summary shows how frequently each tool is invoked and how often those invocations return an error. Each conversation can contain multiple tool invocations. Every invocation is counted separately. The table contains the following columns. ### Engagement Rate The percentage of Concierge-involved conversations attributed to the tool where that tool ran. A conversation can use multiple tools, so row values do not add up to 100%. ### Total Executions The number of tool invocations during the selected date range. This value is the denominator for Error Rate. ### Error Rate The percentage of tool invocations that returned an error. A high tool error rate may indicate: * Invalid authentication or credentials * An unavailable external service * Incorrect tool input * Missing required customer or conversation data * A tool schema or configuration issue * Rate limits or timeouts Investigate tools with a high error rate before changing agent instructions. An agent cannot successfully complete a procedure when a required tool consistently returns errors. ## Interpret Concierge performance No single metric determines whether Concierge is performing well. Review related metrics together to understand the full outcome. ### High engagement and high resolution Concierge is participating broadly and resolving a high proportion of its conversations without human involvement. Confirm that customer satisfaction remains healthy and that error rates are low. ### High engagement and high escalation Concierge is reaching many conversations but frequently transferring them to humans. Review whether: * The automation is intentionally acting as a triage layer * Its escalation rules are too broad * It lacks tools or procedures needed to complete common requests * Particular agents or procedures account for most escalations ### Low engagement Concierge is participating in only a small proportion of total conversations. Review automation entry conditions, eligible channels, routing, and audience restrictions. A low engagement rate may be expected for a narrowly scoped automation. ### High resolution and low satisfaction Concierge is closing conversations independently, but customers may not consider the answers successful. Review resolved conversations for inaccurate answers, premature closure, missing confirmation steps, or poor handling of unsupported requests. ### High procedure failure rate Procedures are running, but their goal-validation requirements are often not met. Review the failed business outcomes and determine whether the procedure steps, validation criteria, or source data need adjustment. ### High procedure or tool error rate Technical execution problems may be preventing Concierge from completing otherwise valid requests. Start with the Tool Performance Summary, then trace the affected tool to its procedures, agents, and automations. ### High automation or agent error rate Use the lower-level summaries to isolate the cause: 1. Identify the automation or agent with the high error rate. 2. Review its Procedure Performance Summary. 3. Identify procedures with elevated error or failure rates. 4. Review the tools used by those procedures. 5. Open the underlying records available through report drilldowns. 6. Correct the configuration, instructions, validation, or tool integration. 7. Compare the updated version against the previous version after enough new activity has accumulated.