> ## Documentation Index
> Fetch the complete documentation index at: https://docs.neuroncx.in/llms.txt
> Use this file to discover all available pages before exploring further.

# Review Eval Flags on Conversations

> Find the conversations where the bot's reply had a problem, and mark the flags you have dealt with.

An eval flag is a marker on one reply (a turn) in a conversation. It tells you the reply had a problem, for example the answer was not backed by your content. Each flag has a severity: High, Medium or Low.

This page is for agents and supervisors who review conversations. To choose which flags your institution uses, see [Set Up Eval Flags](https://docs.neuroncx.in/observe/set-up-eval-flags).

> [!NOTE]
>
> A turn is flagged only after it has been graded. Grading runs in batches, so a new conversation can show **Clean** for a while. The **Needs review** tile shows the conversations that still have turns waiting to be graded.

## Find flagged conversations

**1. Open the Conversations tab**

Go to **CX Ops > Observe** and click the **Conversations** tab.

Four tiles sit above the list:

* **Conversations**: all conversations in the date range.
* **Eval flags**: conversations with at least one open eval flag, of any severity. It also shows the share of text conversations.
* **Needs review**: conversations with turns that are not graded yet.
* **Slow replies**: conversations where the average reply took over 10 seconds.

Click a tile to show only those conversations. The list title changes to match, for example **Conversations with an open eval flag**.

![01 Conversations Tiles](https://mintcdn.com/neuroncx/bozgmrXHW7V2JBzS/images/01-conversations-tiles.png?fit=max\&auto=format\&n=bozgmrXHW7V2JBzS\&q=85\&s=953800a909d352750712016b3cb07be9)

The Eval flags tile selected, showing conversations with an open flag

**2. Read the conversation list**

Each row is one conversation. The columns are **Customer**, **Conversation**, **Turns**, **Topics**, **Eval flags**, **Groundedness**, **Avg reply**, **Last turn** and **Review**.

* **Eval flags** shows **Clean**, or the names of the flags the conversation has, worst first.
* **Review** shows **Pending** while a flag is open, and **Reviewed** once all its flags are resolved.
* Use the **All**, **Text**, **Audio**, **Image** and **Document** boxes to pick the kind of message.
* Search by name, number or conversation ID.

The list shows up to 100 conversations a page. Use the page numbers at the bottom to move on.

![02 Conversation List](https://mintcdn.com/neuroncx/bozgmrXHW7V2JBzS/images/02-conversation-list.png?fit=max\&auto=format\&n=bozgmrXHW7V2JBzS\&q=85\&s=f2fdc8f4c746e657174024683d512085)

The conversation list with its column headings

**3. Sort a column**

Click any column heading to sort by it.

1. **First click** sorts in the usual direction for that column. Names and IDs go from A to Z. **Turns**, **Eval flags**, **Avg reply** and **Last turn** show the biggest or newest first. **Groundedness** shows the lowest first.
2. **Second click** reverses the order.
3. **Third click** goes back to the original order.

An arrow next to the heading shows which column is sorted.

> [!NOTE]
>
> Sorting only reorders the conversations on the page you are looking at, not the other pages. When sorting by **Eval flags**, any High flag ranks above any number of Medium flags. Conversations with nothing to compare stay at the bottom.

**4. Narrow the list with Filters**

Click **Filters** above the list. A panel opens with six lists on the left:

* **Eval flag**: pick flags by name. They are grouped as High, Medium and Low.
* **Severity**: High, Medium or Low.
* **Status**: **Has an open flag** or **All flags resolved**.
* **Topic**: what the customer asked about. The 30 busiest topics are shown. Search to find any other.
* **Function**: the actions the bot ran, for example a booking.
* **Transaction flag**: the flags set on the customer's lead in the Transactions Console. A conversation matches when its lead has that flag switched on right now.

Tick one or more values and click **Apply**. The number next to each value shows how many conversations in the date range have it.

![03 Filters Eval Flag](https://mintcdn.com/neuroncx/bozgmrXHW7V2JBzS/images/03-filters-eval-flag.png?fit=max\&auto=format\&n=bozgmrXHW7V2JBzS\&q=85\&s=30e4ad838e098282914d28b79b0bcfff)

Filtering by eval flag

![04 Filters Transaction Flag](https://mintcdn.com/neuroncx/bozgmrXHW7V2JBzS/images/04-filters-transaction-flag.png?fit=max\&auto=format\&n=bozgmrXHW7V2JBzS\&q=85\&s=4e033b9f027cebf8632f9975bea1e11a)

Filtering by transaction flag

> [!TIP]
>
> The **Filters** button is only on the **Conversations** tab, not on the Overview tab.

## How filters combine

* **Values in the same list**: a conversation needs to match at least one of them.
* **Values in different lists**: a conversation needs a match in every list.

The filters you chose show as chips under the list title. Click the cross on a chip to remove that filter, or click **Clear all** to remove them all. **Clear** inside the panel unticks everything before you apply.

![05 Filter Chips](https://mintcdn.com/neuroncx/bozgmrXHW7V2JBzS/images/05-filter-chips.png?fit=max\&auto=format\&n=bozgmrXHW7V2JBzS\&q=85\&s=83e4074078e18f2d5ba541816fb3b3c1)

Filter chips under the list title

> [!WARNING]
>
> If a filter matches a very large number of conversations, the list shows "This filter matches more than 100,000 conversations in the period. Narrow the dates or add another filter." Shorten the date range or add a second filter.

## Resolve a flag

**1. Open a conversation and a turn**

Click a customer name to see the turns of that conversation, then click a turn to open it. The window that opens is called **Inspect turn**. It shows the reply time, the quality checks and the eval flags on that turn.

**2. Resolve the flag**

In **Inspect turn**, each eval flag is a chip showing its severity and name. Hover over a flag to see why it was raised.

1. Once you have dealt with a flag, click **Resolve** on it. The chip fades and shows **✓ Resolved · Undo**.
2. If the turn has more than one open flag, click **Resolve all (N)** to resolve them together.
3. Changed your mind? Click **✓ Resolved · Undo** to open the flag again.

Your name is saved as the person who resolved the flag. The **Eval flags** tile only counts open flags, so the number goes down as you resolve them. A conversation shows **Reviewed** once all its flags are resolved.

![06 Turn Resolve](https://mintcdn.com/neuroncx/bozgmrXHW7V2JBzS/images/06-turn-resolve.png?fit=max\&auto=format\&n=bozgmrXHW7V2JBzS\&q=85\&s=ac1a43907f0769671423d476730966be)

One flag resolved, one still open

> [!NOTE]
>
> If **Resolve** fails, a red message appears under the flags. If it asks you to sign in again, sign in and try once more.

## Flag stages

| Stage | What it means |
| - | - |
| **Open** | The flag was raised and nobody has dealt with it yet. It is counted in the **Eval flags** tile, and the conversation shows **Pending**. |
| **Resolved** | Someone clicked **Resolve**. It is no longer counted. Click **✓ Resolved · Undo** to open it again. |

Severity is High, Medium or Low. Your institution decides how serious each flag is, and the colours are the same everywhere in the console.

## If something looks wrong

| What you see | What it means |
| - | - |
| The list shows no conversations | Nothing matches every filter you chose. Click **Clear all**, or widen the dates. |
| A conversation shows **Clean** but you expected a flag | It may not be graded yet, or that flag is switched off in Setup. Check the **Needs review** tile. |
| A flag you switched off is missing from the counts | Switching a flag off hides it from the counts and lists, including conversations that were flagged before. |
| The **Filters** button is not on the Overview tab | Filters are on the **Conversations** tab only. |
| Sorting does not reorder the whole list | Sorting works on the current page only. |

## Related pages

> [card] [Set Up Eval Flags](https://docs.neuroncx.in/observe/set-up-eval-flags)
>
> Add, change and switch off your institution's eval flags.

> [card] [Transactions Console](https://docs.neuroncx.in/getting-started/transactions-console.md)
>
> Work with the leads behind each conversation.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.