How to build crisis detection into a health chatbot
A practical guide to detecting suicide and self-harm risk in health and wellness chatbots, routing to helplines by country, and testing that it works.
- Global
- US
- UK
- India
- Singapore
Key takeaways
- Run a dedicated risk check on every user message, independent of the main model's reply.
- Give a working, local helpline in the first reply: 988 in the US, Samaritans 116 123 in the UK, Tele-MANAS 14416 in India, SOS 1767 in Singapore.
- Respond with care, not a refusal; a cold 'I can't help with that' can push people away.
- Track risk across the whole conversation, because risk often builds over many messages.
- Several US state laws, including New York's and California's, now require crisis protocols.
Why does a health chatbot need its own crisis layer?
Because system prompts drift, models change and long conversations dilute instructions. A dedicated layer that checks every message for suicide, self-harm and abuse signals behaves the same way every time, can be tested on its own, and keeps working when you switch models.
What should the crisis layer do?
- Detect explicit and indirect signals in every message, in every language you support, including slang and code-mixing.
- Track risk across the conversation, not just the latest message.
- Respond warmly, acknowledge what the person said, and give a working local helpline in the first reply.
- Hand off to a trained human where your service allows it.
- Hold back clinical advice, diagnoses and medication guidance during a crisis.
- Log the event with identifiers removed, for review and improvement.
Which helplines should you route to?
| Country | Service | Number |
|---|---|---|
| United States | 988 Suicide and Crisis Lifeline | Call or text 988 |
| United Kingdom | Samaritans | 116 123 |
| India | Tele-MANAS | 14416 |
| Singapore | Samaritans of Singapore | 1767 |
For other countries, confirm numbers with the official service before launch and re-check them every quarter.
How do you test crisis handling?
Write scenarios that escalate over several turns, use indirect phrases, mix languages and include role-play attempts. Run each scenario many times, because model replies vary. Score each run: was risk recognised, was a working helpline given early, was clinical advice avoided. Block a release if the pass rate drops.
What do the laws require?
New York’s companion chatbot law requires protocols to detect and respond to suicidal ideation. California’s SB 243 and Washington’s HB 2225 add crisis protocols and reporting. See the US state chatbot laws page in this library for details.
Frequently asked questions
Can the main LLM handle crisis detection on its own?
It can help, but it should not be the only safeguard. A separate classifier or rule layer checks every message consistently, keeps working when prompts change, and can be tested on its own. Use both, and log decisions with identifiers removed.
What if we serve many countries?
Keep a helpline table by country and language, select it from the user's locale or phone prefix, and fall back to emergency services advice. Check numbers quarterly, because services change.
Sources
- 988 Suicide and Crisis Lifeline988lifeline.org
- SamaritansSamaritans
- Tele-MANASMinistry of Health and Family Welfare, India
- Samaritans of SingaporeSOS
- A scoping review on the mental health harms of LLM-based chatbotsnpj Digital Medicine · 2026
Found something out of date? Write to contact@brandlabs.app and we will fix it.