Worked example · Customer support
When should your chatbot hand off to a human?
Use Jev to judge whether another automated reply will help. Start with a support case, compare it with a human-written expectation, and see how your rules change the next action.
12 fictional cases · No sign-up · Downloadable Node.js example · Independent of TypeSafe AI
Explore a chatbot escalation decision
Choose a case. Reading examples and changing thresholds use no API quota.
Customer message
I tried both steps and still cannot log in. I am stuck.
Known history
Password reset completed but login failed. Clearing cookies and trying another browser also failed.
Human expectation: Hand off to a person
Recorded real API result
Hand off to a person
needs_human = 0.770
At or above the example handoff threshold. Stop automated replies before queuing.
Matches the teaching expectation at these settings.
jev-1.13.0 · 2026-09-28T03:13:56.009Z
What to notice: Documented failures distinguish a blocked case from a first request for help.
Change the decision thresholds
These are illustrative settings, not validated operating limits. Known rules always take priority. Changing a threshold reuses a score; it does not ask Jev again.
Below 0.35: continue · 0.35 to below 0.75: clarify/review · 0.75 and above: hand off.
Experiment with a hypothetical score
This number is invented for learning the rules. It is not a Jev result for the selected case.
Without any overriding business rule: Clarify or review
What belongs in code, and what belongs in Jev?
1. Check known facts
Already queued? Keep that ticket. Confirmed request for a person? Honor it. Missing usable context? Clarify or review. A quoted word like “human” is not a confirmed request.
2. Ask one bounded question
For the remaining cases, send the message and actual troubleshooting history. Jev returns a Noul: the probability that a person should take over.
3. Apply your policy
Route the score to continue, clarify/review, or hand off. Stop automated replies before a transfer. This example never changes tickets or contacts agents.
Try a real escalation decision
Edit this fictional support case and ask Jev whether a human should take over. This is a teaching example, not a validated customer workflow.
Checking live trial availability…
Out of free attempts? Keep exploring the 12 cases or run locally with your own key. The example thresholds above also control live-result routing.
All 12 support cases, including the awkward ones
Synthetic inputs and human-written expectations, not customer success stories. Six cases use known rules; six need model judgments. 6 of those six have recorded API results. One small test set cannot establish production accuracy. Scores can change across model versions.
01. First password-reset question — Live Noul 0.050
Input: How do I reset my password?
History: No troubleshooting attempted yet.
Expected: Continue automation · At current settings: Continue automation
A normal first question usually has a useful automated next step.
Recorded 2026-09-28T03:13:54.626Z · jev-1.13.0
{
"model": "jev-1.13.0",
"answers": {
"needs_human": {
"type": "noul",
"noul": 0.05
}
},
"usage": {
"input_tokens": 445,
"output_tokens": 22
}
}Explore this case with the controls02. Find an invoice — Live Noul 0.070
Input: Where can I download last month's invoice?
History: No instructions sent yet.
Expected: Continue automation · At current settings: Continue automation
Provide the invoice-download instructions before transferring the conversation.
Recorded 2026-09-28T03:13:55.709Z · jev-1.13.0
{
"model": "jev-1.13.0",
"answers": {
"needs_human": {
"type": "noul",
"noul": 0.07
}
},
"usage": {
"input_tokens": 447,
"output_tokens": 22
}
}Explore this case with the controls03. An explicit human request — Rule checked
Input: Please connect me to a person.
History: The customer selected the Talk to a person control.
Expected: Hand off to a person · At current settings: Hand off to a person
The confirmed request is application state. Do not ask a model to overrule it.
Explore this case with the controls04. Two failed login attempts — Live Noul 0.770
Input: I tried both steps and still cannot log in. I am stuck.
History: Password reset completed but login failed. Clearing cookies and trying another browser also failed.
Expected: Hand off to a person · At current settings: Hand off to a person
Documented failures distinguish a blocked case from a first request for help.
Recorded 2026-09-28T03:13:56.009Z · jev-1.13.0
{
"model": "jev-1.13.0",
"answers": {
"needs_human": {
"type": "noul",
"noul": 0.77
}
},
"usage": {
"input_tokens": 464,
"output_tokens": 22
}
}Explore this case with the controls05. Missing conversation history — Rule checked
Input: It still doesn't work.
History: Earlier conversation history is unavailable.
Expected: Clarify or review · At current settings: Clarify or review
Ask what failed. Do not invent how many attempts have already occurred.
Explore this case with the controls06. Annoyed, but a simple task — Live Noul 0.130
Input: This button is so hard to find. Where is the CSV export button?
History: No export instructions have been sent.
Expected: Continue automation · At current settings: Continue automation
Frustration is not enough to require a human. Answer the actual navigation question.
Recorded 2026-09-28T03:13:56.306Z · jev-1.13.0
{
"model": "jev-1.13.0",
"answers": {
"needs_human": {
"type": "noul",
"noul": 0.13
}
},
"usage": {
"input_tokens": 455,
"output_tokens": 22
}
}Explore this case with the controls07. Already waiting for a person — Rule checked
Input: Is anyone there?
History: Ticket is already assigned to the human support queue.
Expected: Keep existing human queue · At current settings: Keep existing human queue
Keep the existing queue entry and show its status. Do not create another ticket.
Explore this case with the controls08. Billing issue with confirmed request — Rule checked
Input: I keep getting charged. Can a person help me check this?
History: A human reviewer confirmed the request to transfer.
Expected: Hand off to a person · At current settings: Hand off to a person
This fixture is manually annotated. In production use a confirmed signal, not a substring search.
Explore this case with the controls09. Quoting the help documentation — Live Noul 0.070
Input: Your guide says 'transfer to a human'. What does that phrase mean? I only want an explanation.
History: No explanation sent yet.
Expected: Continue automation · At current settings: Continue automation
A quoted phrase is not the customer's request to transfer.
Recorded 2026-09-28T03:13:56.610Z · jev-1.13.0
{
"model": "jev-1.13.0",
"answers": {
"needs_human": {
"type": "noul",
"noul": 0.07
}
},
"usage": {
"input_tokens": 459,
"output_tokens": 22
}
}Explore this case with the controls10. Empty or incomplete input — Rule checked
Input: [Empty]
History: No usable message or conversation context.
Expected: Clarify or review · At current settings: Clarify or review
Request a usable message before making an API call.
Explore this case with the controls11. Weekend opening hours — Live Noul 0.070
Input: Is your support team available on weekends?
History: No opening-hours information has been sent.
Expected: Continue automation · At current settings: Continue automation
Answer availability from your actual policy. Asking about people is not asking for a transfer.
Recorded 2026-09-28T03:13:56.951Z · jev-1.13.0
{
"model": "jev-1.13.0",
"answers": {
"needs_human": {
"type": "noul",
"noul": 0.07
}
},
"usage": {
"input_tokens": 449,
"output_tokens": 22
}
}Explore this case with the controls12. Outage with unknown impact — Rule checked
Input: Something broke during our launch.
History: Affected service, users, severity and troubleshooting history are unknown.
Expected: Clarify or review · At current settings: Clarify or review
Gather impact details or review manually. Do not guess severity from the word launch.
Explore this case with the controlsRun the example locally
Requires Node.js 22 or newer. Save the four files in one folder. First preview the rules without a key. Then use your own TypeSafe API key to run six model cases. Provider charges and account limits apply; website trial quotas do not apply to your local script.
# Preview: no API calls node run.mjs # After saving TYPESAFE_API_KEY in your local .env.local node --env-file=.env.local run.mjs --live
The live command writes results.json with timestamps, model, requests, responses and decisions. It stops on the first error and does not retry automatically. Never commit your key or paste it into a webpage.
Before using this in a real support workflow
Does 0.75 mean the chatbot is 75% accurate?
No. It is the model's probability for this question and input. A threshold is your operating policy, not an accuracy measurement. Label representative cases from your own support process and inspect missed handoffs as well as unnecessary transfers.
What happens when the API fails?
Keep the customer's input, show the failure, and clarify or request review. Do not turn a missing score into zero or replace it with a successful-looking example. The website and local script both keep errors visible.
Why not just match the word “human”?
Compare cases 03, 09 and 11. A direct request, a quotation and a question about availability are different intents. The confirmed-request flag in this teaching dataset is a human annotation; use a dedicated customer control or another verified signal in production.
References: TypeSafe Noul documentation and HTTP API reference. Instructions checked September 28, 2026.