Skip to main content
Before you let the Support AI employee prepare replies, run it on tickets your team already solved and compare what it would have proposed with what your team actually replied.
AI employees are in beta, so you will not see this unless your deployment has turned them on. It has not yet run on a real Zendesk, Freshdesk or Intercom account. See AI employees.

​
What a test does, and does not do

A test reads your most recently solved tickets, up to 10, from the helpdesk connected to the Support inbox — solved and closed in Zendesk, Resolved and Closed in the last 30 days in Freshdesk, closed conversations in Intercom — and asks your chosen assistant once for each. It only reads. Nothing is sent, filed or tagged in your helpdesk, and nothing appears in your approval queue. Each ticket uses one answer from your monthly allowance, and appears in Conversations as asked by an AI employee. You can run one test at a time, and up to five a day. Results are kept for 14 days.

​
What you see for each ticket

  • What the customer wrote, and what your team replied.
  • What Support would have proposed: a drafted reply, or that it would go to a person, and why.
  • The sources the reply cites, and its confidence.
  • Three buttons to judge the reply: Good, Needs edit or Wrong. Judging needs a person who may approve actions.

​
What confidence means

Confidence is how closely the strongest source a reply cites matches the question, as your knowledge search measured it, from 0 to 1. It is 0 when your knowledge does not support an answer. It is not the model grading itself, and it is not a guarantee that the reply is right: a close source can still be answered wrongly. That is why you calibrate a threshold against your own verdicts.

​
Choose a threshold

Move the slider to see what a threshold would have done on the latest test: how many tickets would have got a drafted reply, how many would have gone to a person, and how your reviewers judged the replies that would have been drafted. Save threshold applies it to live work. With a threshold saved, a reply below it is not drafted. The ticket is handed to a person instead, and that hand-off adds the tag needs_a_person, so a view, trigger or rule in your helpdesk can route it. Either way, nothing is sent without a person’s approval. A threshold can only turn a drafted reply into a hand-off; it can never send anything. The threshold is a setting of your helpdesk connection. If you detach the helpdesk, attach it again or switch to another, set it again. Only an owner or admin, signed in, can set it.
Last modified on November 7, 2026