Skip to main content
The CX Ops dashboard — built around Service Level Indicators (SLIs) — gives you a quantitative view of how well your AI-powered conversation pipeline is performing end to end. Where other dashboards show you what users are doing, CX Ops shows you how well your AI is handling the workload: how many language model operations it’s completing, which user intents are driving the most traffic, and whether the AI is actually retrieving the right answers when it matters. Monitoring these indicators regularly is the fastest way to catch degradation before it impacts your users.

​
Completion Table

The Completion Table tracks the raw volume of four distinct language model operation types processed during the selected time period. Each row represents a different kind of AI compute happening inside your conversation pipeline.

​
What the numbers tell you

In a healthy pipeline, the Responses count and the Classifications count should be close to each other — every message that gets classified should ultimately produce a response. A significant gap between the two (for example, far more classifications than responses) suggests that a portion of classified intents are not completing successfully. This could point to workflow errors, missing template mappings, or knowledge base gaps for those intents.
The Embeddings count reflects how often your system performed a semantic search against your vector store. A count notably higher than Responses suggests that some queries are triggering multiple retrieval passes before a response is generated — which can indicate low initial retrieval confidence. If embeddings are significantly lower than responses, some responses may be returning without a proper knowledge retrieval step, which often results in generic or hallucinated answers.
The Transcriptions count will naturally be lower than the others if only a portion of your traffic comes through voice channels. Cross-reference this number with the Call Processing workflow execution count in the Apps dashboard to verify that all voice calls are being transcribed. A large gap between the two may indicate dropped calls or transcription failures in your voice pipeline.
The Completion Table reflects operations at the backend level. One user conversation may generate multiple classification, embedding, and response operations depending on how many turns the dialogue contains.
The Trending Topics section surfaces the top 10 user intents and small-talk categories detected across all conversations during the selected period. Each entry corresponds to a classified intent label — such as appointmentbook, reschedule, or cancel — ranked by frequency. This list is one of the most actionable outputs in the entire CX Ops dashboard because it tells you exactly what your users want, in aggregate, right now.

Validate your content coverage

Compare the top trending topics against your Ontology dashboard’s Topical Coverage charts. Any intent that ranks highly here but has thin knowledge base coverage is a content gap that is actively hurting your AI’s performance today.

Prioritize template updates

If a high-traffic intent lacks a mapped response template (visible in the Ontology dashboard’s Topical Template Count), that intent is receiving a generic or fallback reply. Mapping a well-structured template to it can improve response quality immediately.

Detect emerging needs

New topics appearing in the top 10 that weren’t there in previous periods often signal a shift in user behavior — a new concern, a seasonal spike, or a recent event driving new questions. Treat new entries as early warnings to prepare content and templates before volume grows further.

Inform workflow design

If a topic like reschedule or cancel consistently ranks in the top 10 but you don’t have a dedicated workflow for it, that’s a strong signal to build one. High intent frequency without a purpose-built workflow typically means those users are receiving lower-quality, patchwork responses.
Export your Trending Topics list at the end of each month and compare it against the previous month. Consistent top-ranked topics should have mature, optimized templates and rich knowledge coverage. Rising topics are your next content and workflow priorities.

​
LLM Retrieval Quality (SLI Graph)

The SLI Graph measures the average retrieval quality score for your AI’s responses over time. This score reflects how successfully your LLM is finding and returning the correct answer from your vector stores and ontology layers — as opposed to generating a response that sounds plausible but isn’t grounded in your knowledge base.

​
Understanding the quality score

A score of 1 represents a successful, relevant response mapping — the AI retrieved the right content and produced a response that accurately addresses the user’s query. Scores below 1 indicate partial matches, low-confidence retrievals, or responses generated without a strong knowledge base anchor.
The score is calculated based on retrieval matching quality and AI-side confidence signals. Monitoring this graph over time lets you see whether changes to your knowledge base, template mappings, or workflow routing are improving or degrading your AI’s ability to give correct answers.
Your retrieval pipeline is performing well — the AI is finding relevant content and mapping it accurately to user queries. Continue monitoring for any drops that follow content updates or workflow changes, but no immediate action is needed.

​
Improving retrieval quality: a continuous loop

1

Monitor the SLI Graph weekly

Set a regular cadence for reviewing your retrieval quality score. Weekly reviews give you enough data to spot trends without reacting to day-to-day noise.
2

Cross-reference with Trending Topics

When the score dips, look at which topics are trending. The most frequent intents with the thinnest knowledge coverage are the most likely culprits.
3

Update the Ontology

Add or revise Knowledge Blocks for the affected topics in the Content Console. Ensure that response templates are correctly mapped and that the KBase Index Summary reflects the new content after re-indexing.
4

Validate the improvement

After re-indexing and deploying updates, monitor the SLI Graph over the following week to confirm that the retrieval quality score recovers. If it doesn’t, investigate whether the new content is being surfaced correctly by running test queries through your workflow.
Avoid making multiple simultaneous changes to your knowledge base, templates, and workflow routing at the same time. If the SLI score changes after a bundle of updates, you won’t be able to determine which change caused the shift. Roll out changes incrementally and observe the SLI Graph between each change.