Therabot trial: what a 51% reduction in depression symptoms means
Read Therabot's 51% result with the time point and measure attached. It describes the reduction in the intervention group's mean depression score by week eight compared with its own baseline. The trial also found greater improvement than a waitlist control. Those are two different results.
Heinz and colleagues published the randomized trial in NEJM AI in 2025. Their full article, including Tables 1 and 2, lets us check what sits behind the headline.
The trial in numbers
The study enrolled 210 adults in the United States who screened positive for clinically significant depressive or generalized anxiety symptoms, or high risk for feeding and eating disorders. Screening scores determined eligibility; they weren't a separate diagnostic assessment. Participants could meet more than one symptom threshold.
Random assignment gave 106 people access to Therabot and placed 104 on a waitlist. The intervention group received daily prompts for four weeks. Access continued during weeks four to eight, with the daily prompts removed. The waitlist received no app access until the study ended, but some people in both groups were already receiving medication or psychotherapy (Table 2).
The table below shows changes from baseline to week eight. N is specific to each outcome, with baseline and week-eight counts separated by a slash. Negative changes mean lower scores.
| Measure | Therabot N, baseline / week 8 | Waitlist N, baseline / week 8 | Therabot mean change | Waitlist mean change |
|---|---|---|---|---|
| Depression, PHQ-9 | 73 / 70 | 69 / 69 | −7.93 points | −4.22 points |
| Anxiety, GAD-Q-IV | 60 / 52 | 56 / 56 | −3.18 points | −1.11 points |
| Weight concerns, WCS | 42 / 42 | 47 / 47 | −10.23 points | −3.70 points |
Dividing the Therabot mean changes by the corresponding baseline means gives approximately 51% for PHQ-9 (7.93 / 15.63), 31% for GAD-Q-IV (3.18 / 10.43) and 19% for WCS (10.23 / 54.13). These are calculations from Table 1, rounded to whole percentages. The 51% doesn't mean a 51% advantage over the waitlist or that 51% of participants recovered.
At week four, PHQ-9 had fallen by 6.13 points with Therabot and 2.63 with the waitlist. The model-derived between-group effect sizes for depression were d=0.845 at week four and 0.903 at week eight. The corresponding values were 0.840 and 0.794 for anxiety, and 0.819 and 0.627 for weight concerns. The ranges in the original summary describe two observation times, rather than a confidence interval.
Why this trial matters
This was a randomized test of a generative chatbot, rather than a survey of satisfied users. The authors reported greater symptom reductions with Therabot than with the waitlist at both assessment times. Randomization makes the comparison more informative than simply observing that users felt better after opening an app.
Therabot also had a specific implementation. The Methods describe fine-tuned Falcon-7B and LLaMA-2-70B models, trained on expert-curated mental-health dialogues. Participants used the system for more than six hours on average. Their alliance ratings, around 3.6 on the WAI-SR scale, describe how they experienced the interaction; they don't establish equivalence to psychotherapy.
Human oversight was part of the study. Trained clinicians and researchers reviewed responses after transmission and contacted participants when correction or safety guidance was needed. Participants saw the model's answer before that review.
What the comparison can establish
The control condition matters. Therabot was compared with a waitlist without access to the app, while usual treatment could continue. The trial therefore estimates the added benefit under those study conditions. It didn't randomize people to Therabot versus a therapist, another chatbot or a matched amount of human attention.
Effect sizes from separate trials can't settle that comparison. Different recruitment criteria, scales and controls can change the numbers, so claims that Therabot was "twice as effective" as earlier bots or matched in-person therapy go beyond this design.
The eight-week result also includes continuing app access. It doesn't show what happens for months after people stop using the system. Replication with active comparators and longer follow-up would answer different questions from the ones this trial addressed.
How this fits the wider research
A single system's trial and a pooled review answer different questions. Our review of chatbot meta-analyses covers the broader evidence; the CBT chatbot comparison separates clinical outcomes from feasibility and technical evaluations. Look at the participants and comparator before comparing effect sizes.
FAQ
What does the 51% reduction mean?
The intervention group's mean PHQ-9 score fell from 15.63 to 7.70 by week eight, a decline of 7.93 points, or about 51% of baseline. The waitlist mean fell by 4.22 points over the same period.
Did the control group receive no treatment?
It received no Therabot access during the study. Some participants continued medication or psychotherapy, so "no care at all" would misdescribe the control.
Did Therabot perform as well as a therapist?
That wasn't tested directly. The comparator was a waitlist, and similar effect sizes or alliance ratings across unrelated studies don't establish equal effectiveness.
Nearby is a separate product for emotional support and self-reflection. This Therabot trial doesn't measure its effectiveness; the product facts page describes what Nearby offers. It doesn't replace professional treatment.