How AI studies suicide risk in text and medical records
If you're at immediate risk of harming yourself, contact local emergency services or a crisis line and tell someone you trust.
AI studies of suicide risk ask several different questions. Can a model recognize a pattern in forum writing? Can medical records help predict a later attempt? Can a service use a signal to arrange useful support? An impressive result for one task doesn't answer the others.
The distinction starts with the data. A medical-record model may use diagnoses, medication histories and previous care. A text-analysis study may count language features in online communities. Calling both "AI detection" makes them sound more alike than they are.
What language studies can show
Al-Mosaiwi and Johnstone's 2018 study analyzed writing from 63 English-language internet forums involving more than 6,400 members. The researchers counted absolutist language, words expressing totality or certainty, and compared groups of forums.
Anxiety, depression and suicidal-ideation forums contained more of that language than control forums. Suicidal-ideation forums also contained more than the anxiety and depression groups. This was an association between writing patterns and forum categories.
The unit matters: more than 6,400 refers to members, rather than posts. The researchers combined multiple eligible posts from the same member into one document. They didn't follow every member to establish whether a particular word predicted a subsequent suicide attempt.
A phrase such as "nothing will ever change" may therefore be relevant in a conversation, but the study doesn't provide a rule for diagnosing the writer. Context matters, including what the person is describing and how their language changes. Words can contribute to an assessment without determining its outcome.
The medical-record result asks another question
Walsh, Ribeiro and Franklin's 2017 paper used electronic health records from 5,167 adults with a self-injury claim code. Expert review identified 3,250 people with a suicide attempt and 1,917 controls with nonsuicidal, accidental or nonverifiable self-injury.
The model's reported discrimination reached AUC 0.84 for the seven-day prediction window. AUC describes how well a score ranks cases relative to controls across thresholds. It doesn't mean that 84% of people flagged by a chatbot will make an attempt.
This selected clinical population is central to interpreting the result. Everyone already had a self-injury code. The study didn't test ordinary app users' messages or compare a conversational detector with questionnaires in the same population. It can't establish that analyzing a chat is more accurate than a clinical assessment.
| Study | Task and sample | Main result | What it doesn't establish |
|---|---|---|---|
| Al-Mosaiwi & Johnstone, 2018 | Compare language across 63 forums, more than 6,400 members | Absolutist language differed between forum groups | Prediction of a later attempt in an individual |
| Walsh et al., 2017 | Predict attempts from medical records in 5,167 adults with self-injury codes | AUC 0.84 at the seven-day window | Accuracy of a general-purpose chat detector |
Putting these studies side by side is useful for understanding their tasks. Ranking them against an unrelated questionnaire study would require matching the population, outcome and prediction period, then testing the methods under comparable conditions. The numbers alone don't do that work.
Why false positives change the practical result
Consider an illustrative calculation. Assume the event being predicted occurs in 1 in 1,000 people over a specified period. A model has 90% sensitivity and 90% specificity. Those are assumed values, not measurements from the two studies above.
Among 100,000 people, the model would flag 90 of the 100 people who experience the event. It would also flag 9,990 of the 99,900 people who don't. Only about 0.9% of positive flags would correspond to the event, despite apparently strong sensitivity and specificity.
Change the event frequency and the proportion changes too. This is why a result from a selected clinical sample doesn't automatically describe performance in a general population. AUC alone leaves out the operating threshold and the number of incorrect alerts at that threshold.
A product evaluation should therefore report the intended population, what counts as an event, the prediction window and the consequences of each kind of error. It should also test language and demographic differences in the actual deployment setting. Forum language in English doesn't establish performance in Russian or another language.
From a signal to a useful response
A score is only the start of a response process. Someone must decide what the signal means in context, whether to ask another question and how to connect the person with support. Testing that process is a separate task from training a detector.
NICE guideline NG225, section 1.6, advises against using risk scales to predict future suicide or repeated self-harm, or to determine treatment and discharge. It asks practitioners to focus assessment on the person's needs and undertake a risk formulation as part of psychosocial assessment.
For developers, the useful questions follow the same direction. Who reviews a flag? What information do they see? What happens when the model misses a signal or raises an incorrect alert? A system that offers a contact number and one that sends a report to a clinician perform different actions, so their evaluation should reflect those actions.
We examine the response side in just-in-time interventions and the surrounding system design in guardrails for mental-health AI. Nearby offers emotional support and self-help; it isn't a suicide-risk diagnostic service.
Frequently asked questions
Can AI identify suicide-related signals in text?
Research has found associations in language and developed task-specific classifiers. How well a particular service performs depends on its data, population and evaluation. The studies above don't establish reliable prediction from any person's chat.
Does AUC 0.84 mean 84% of alerts are correct?
No. AUC measures ranking across thresholds. The proportion of correct alerts depends on the chosen threshold and how common the predicted event is in the population.
Does an alert show that an intervention worked?
An alert measures detection. Whether the response helped requires follow-up outcomes, such as access to appropriate support or changes in the outcome the service aims to improve.
Sources
Al-Mosaiwi, M., & Johnstone, T. (2018). In an absolute state: Elevated use of absolutist words is a marker specific to anxiety, depression, and suicidal ideation. Clinical Psychological Science, 6(4), 529-542.
Walsh, C. G., Ribeiro, J. D., & Franklin, J. C. (2017). Predicting risk of suicide attempts over time through machine learning. Clinical Psychological Science, 5(3), 457-469.
NICE. Self-harm: Assessment, management and preventing recurrence, NG225. Section 1.6.