Confidence Score
A confidence score is a value between 0 and 1 that an AI model uses to express how sure it is about its answer. In chatbots the score decides whether the AI responds itself, asks a clarifying question, or hands off to a human agent.
What is a confidence score?
A confidence score is the quantified self-assessment of an AI model: how sure am I about this answer? Values typically range from 0 (uncertain) to 1 (very sure) and represent a probability, not an absolute measure.
In a messaging-bot context, the confidence score steers what happens after a user question: high confidence → bot answers, medium → clarifying follow-up, low → handoff to a human agent.
How a confidence score arises
Different bot types produce scores differently:
| Bot type | How confidence is measured |
|---|---|
| Keyword bot (autoresponder) | Match or no match — no real confidence score. Rule fires or it doesn't. |
| NLU bot (intent classification) | Probability that the user input belongs to intent X. Output: probability distribution across all intents. |
| RAG bot (retrieval-augmented generation) | Similarity between user question and the best-matching knowledge base chunk, combined with model self-assessment. |
| AI agent | Multi-step: confidence at tool selection, confidence at result, weighted total. |
Thresholds
In practice you work with thresholds that steer system behavior:
- ≥ 0.80 — AI answers on its own. High certainty.
- 0.50 to 0.80 — AI asks a clarifying follow-up ("Did you mean X or Y?").
- < 0.50 — handoff to human agent.
Exact thresholds depend on the model and the use case. For safety-critical topics (health, legal, finance) thresholds run higher.
Confidence score and handoff logic
SendSeven AI bots support a configurable escalation logic: after max_failed_attempts low-confidence answers in a row, the bot automatically hands off to a human. Default is typically 2 or 3 — so the customer doesn't get stuck in a loop.
Important: keyword-based bots (autoresponders) do not use this escalation logic, because they don't have a real confidence score. They either match or return a fallback text.
Common causes of low confidence
- Question outside the knowledge base: Customer asks about a topic that hasn't been ingested.
- Ambiguous phrasing: "I have a problem with the app" fits 15 topics.
- Wrong language: User writes in English, bot trained in German.
- Spelling/slang: Typos or jargon the model doesn't recognize.
- Multi-intent: Customer asks two questions at once — model doesn't know which to answer first.
Confidence visibility
In the agent view, SendSeven shows the confidence score the bot assigned to an answer. Low-confidence answers are marked as warnings — agents can correct in-line and report the gap to the knowledge base maintainer. That way the system learns from every handoff.
Confidence vs. correctness
One important lesson: high confidence does not mean high correctness. A model can be very confident and still wrong ("hallucination"). Confidence is a signal, not a guarantee. So a productive setup always includes: a test set with ground truth, regular sampling, and a feedback loop from agents to knowledge-base maintainers.