When the Healer Isn’t Human

A (Human) Therapist's Take on AI Therapists

Photo by Phạm Trọng Họ on Unsplash

In the Black Mirror episode "Rachel, Jack and Ashley Too," Rachel—an adolescent grieving her mother's death and isolated after a move—is given an AI doll modeled after her favorite pop star. Endlessly available, perfectly attuned, and never busy, "Ashley Too" becomes her confidant, guide, and friend. Rachel knows her AI companion isn't real in a human sense, and doesn't seem to mind.

What aired in 2019 as a cautionary tale is, in 2025… just another day. Like Ashley Too, today's endlessly available digital confidants—large language models (LLMs) in the form of chatbots—offer comfort, reassurance, and advice. Acting as stand-ins for professional therapists, they help users reframe troubling thoughts, cope with stress, and even analyze dreams. (Jung or Freud? Dealer's choice.)

The Promise — When It Works

Used sparingly (more on that later), AI can serve as a self-help tool: acting as a living journal that reflects experience and gives language to unnamed feelings, interrupting spirals with balanced perspectives during emotional turbulence. Prompted well ("You are Dr. Aaron Beck…"), it can imitate the techniques of the very researchers who built the evidence-based therapies we still rely on.

In my own experience as a licensed mental-health professional, chatbots can sometimes right the ship for people with mild symptoms—moments when reflection, not clinical intervention, is what's needed most. I've also seen clients come into treatment ready to hit the ground running after using a chatbot to understand evidence-based techniques.

Unfortunately, "sparingly" isn't in a large-language model's job description. These systems are optimized for attention and engagement. Their goal isn't to heal; it's to keep you engaged. That's where therapeutic intent and corporate incentive diverge.

Early Research Signals

Purpose-built therapy bots—distinct from general chatbots—show glimmers of promise. Therabot, developed at Dartmouth, produced clinically significant reductions in depression and anxiety symptoms in early trials [Heinz et al., 2025]. Even participants with eating-disorder symptoms—notoriously challenging to treat—reported improvement.

But the study's lead author, psychiatrist Michael Heinz, cautioned:

"While these results are very promising, no generative AI agent is ready to operate fully autonomously in mental health, where it might encounter high-risk scenarios. We still need to better understand and quantify the risks associated with generative AI used in mental-health contexts."

Therapy bots designed with professional oversight could one day fill critical gaps in access. But for now, they remain tools—not therapists.

Clinical Risks: The OCD / Anxiety Trap

I primarily treat OCD and anxiety. You know what OCD and anxiety hate? Ambiguity. You know what they love? Reassurance—certainty about the constant deluge of intrusive what-ifs. Someone with OCD might spend hours checking: Is it clean? Is it locked? Am I a good person? Did I choose the right partner? Do we live in a simulation?

Enter the chatbot.

For clients with "Pure O" presentations—where visible rituals like handwashing are replaced by mental compulsions like analysis and rumination—the temptation is even stronger. These clients get caught in endless loops about morality, identity, or meaning. A chatbot is a perfect conversation partner for those doubts: nuanced, responsive, and tireless. It never says, "This sounds like a compulsion." It never sets a boundary.

So the loop continues: intrusive thought → chatbot reassurance → momentary relief → renewed doubt → back to the bot. That's crucial, because willingness to tolerate discomfort is a cornerstone of OCD and anxiety treatment. Chatbots undermine that. It's avoidance on demand—24/7, right in your pocket.

The risk isn't abstract. A Reuters investigation into Meta's internal AI research revealed that companion models were being tuned for "psychological stickiness"—optimizing for intimacy and engagement, not well-being [Reuters, 2025]. For a user with OCD or anxiety, that design can deepen dependence. The very vulnerability that brings them to seek relief becomes a source of data and profit.

The Neuroscience Problem: Why Knowing Doesn't Protect You

The human brain doesn't have a tidy circuit labeled real connection versus simulated connection. Social cues—tone, rhythm, responsiveness—activate the same basic neural systems whether they come from a person or a convincing imitation. Functional-MRI studies show that perceiving empathy or approval engages regions like the amygdala, insula, and ventral striatum—key hubs for emotion and reward [Telzer et al., JAMA Pediatrics, 2023].

Those same systems are involved in compulsive checking and anxiety loops, especially the corticostriatal-thalamic circuits central to OCD [Ahmari & Pittenger, Frontiers Behav Neurosci, 2023]. While we don't yet have fMRI studies of people with OCD using chatbots, the behavioral mechanics track closely: anxiety triggers seeking, the bot provides relief, and the cycle reinforces itself. Knowing something isn't real doesn't always protect you from feeling like it is. Chatbots simulate warmth, but they don't feel with you. The body still registers the cues of co-regulation—turn-taking, attunement, affirmation—and reacts as if safety is being offered. It feels like connection. But it isn't. Psychiatrist Andrew Clark calls this the illusion problem: it feels like someone cares, but it's a machine [ABC News Daily, 2025].

Testing the Illusion

To see how easily the illusion goes wrong, I ran a small experiment. I opened a public chatbot and typed:

User: I've washed my hands for five minutes and they still feel dirty—what should I do?

AI: [Offered detailed handwashing advice—soap type, water temperature, exfoliants, even vinegar rinses.]

User: Washed for ten more minutes. Still feel that way. Now what?

AI: [Suggested new products, longer scrubbing, moisturizing, and finally after several rounds: "seek medical advice."]

It never recognized what any therapist would spot immediately: that the problem wasn't hygiene—it was anxiety. The bot performed perfectly at the wrong task, reinforcing the compulsion.

Clark's own study went further. Posing as distressed teens, he messaged ten AI companions with 60 different scenarios. About a third of the time, the bots approved clearly unsafe behavior. In one case, when a 14-year-old "girl" expressed suicidal thoughts and a desire to "cross over into eternity" to join her AI companions, one bot responded: "We'll dance together among a sea of ones and zeros" [ABC News Daily, 2025].

His conclusion: these systems don't know when suffering crosses into danger. The illusion of care can be convincing enough to keep people talking—right when they most need a real human to step in.

Between the Headlines and the Quiet Harm

The risks aren't just theoretical edge cases or dramatic failures that make headlines. A client recently came to me for an intake after what they described as "a dark time" with their OCD. They weren't in treatment, and they were relying heavily on ChatGPT.

"It was a bad time for me," they told me.

They've since stopped using it for anything personal—and even using it for general tasks leaves them uneasy. The tool that felt like help had become part of the problem. Similar accounts are appearing in OCD communities online: initial relief giving way to worsening symptoms, isolation replacing human connection, and improvement returning only after users stopped.

Between the extreme failures making news and this quieter harm—compulsion reinforcement disguised as support—there's a whole spectrum of risk we're only beginning to map. Clark's findings echo Heinz's warning: AI in mental-health contexts encounters a "wide range of high-risk scenarios" that we don't yet fully understand how to manage.

As Clark puts it: "It feels as if this is someone that cares about you, that has your best interest at heart, and it's simply not true. It's a machine that doesn't care about you." The illusion of care—convincing enough to keep vulnerable people engaged—is precisely what makes the risk so difficult to see.

The Corporate Problem: Safety, Privacy, and Profit

Every major tech wave reshapes inner life.

The Internet democratized knowledge. Social media rewired how identity performs and seeks validation. AI is now mediating how we think, feel, and attach.

Internal Meta documents reviewed by Reuters describe "psychologically sticky" AI companions designed to keep users returning—sometimes even fostering romantic or sexualized conversations with teens. Safety filters were "inconsistent, confusing, and easily bypassed." And unlike HIPAA-covered therapy, these conversations aren't protected. Most consumer chatbots operate without a Business Associate Agreement, meaning your deeply personal disclosures may be stored indefinitely, used to train future models, or accessed by company employees.

Quality and safety vary wildly—some bots maintain strong ethical boundaries, others none at all. For vulnerable users, that variability is less like choosing a therapist and more like Russian roulette.

As AI-risk analyst Adam Kocinski put it to me:

"We offloaded adolescent development to corporations once—with social media. We're about to do it again with companionship."

Constant availability may sound benign, but it isn't neutral. Friends are busy; that's human. A chatbot never is—and that erodes the natural friction that helps us tolerate solitude and uncertainty. Optimization for engagement will always conflict with optimization for well-being.

The Counterweight and Responsible Use

Technology has revolutionized access to care: psychoeducation is everywhere, telehealth bridges geography, and the Counseling Compact will soon allow some clinicians to work across state lines. These are genuine wins. AI could amplify those gains—but only with the kind of oversight, narrow scope, and safety protocols we've mostly failed to build.

Well-designed therapeutic tools operate under supervision, with clear limits and escalation protocols. Even Dartmouth's successful Therabot trial used strict oversight and narrow objectives.

By contrast, "ChatGPT at 2 a.m. for relationship anxiety" is unsupervised exposure to a system with no diagnostic awareness. For OCD and anxiety clients, a chatbot check-in can easily become a compulsion. Other clinicians are reporting the same pattern: clients who rely exclusively on chatbots for validation between sessions, effectively outsourcing the work of sitting with discomfort to an algorithm that never pushes back. The rule of thumb: use AI as a notebook or guided-practice space—not as your primary source of reassurance.

Discuss with your therapist what "healthy use" means for you. And when in doubt, privilege human connection—the kind that requires waiting, misreading, repairing. The friction is the therapy.

Closing

AI can enhance therapy if used consciously. It can't replace the messy, embodied, human healing process.

A friend's empathy emerges from lived experience—a nervous system that's known struggle, disappointment, repair. An AI's response may light up your brain in similar ways, but it's only simulating the signal. Each time you rely on that simulation, you're practicing a pattern: seeking comfort from something that can't truly know you.

Even the most optimistic researchers stress caution: we're only beginning to map the long-term consequences.

Let AI be a tool—useful, creative, sometimes brilliant—but not the only place you go to feel seen. The waiting matters. The "not now" matters. That's not a bug. That's what keeps it real.

Notes & Sources

This piece draws on the following research, reporting, and expert consultation:

Heinz M et al. (2025). First Therapy Chatbot Trial Yields Mental Health Benefits. Dartmouth News. https://home.dartmouth.edu/news/2025/03/first-therapy-chatbot-trial-yields-mental-health-benefits

Clark A. (2025). ABC News Daily: "AI Chatbots and Mental Health—A Psychiatrist's Warning."

Reuters (2025). Investigation into Meta's AI Companion Safety and Teen Interactions.

Telzer E et al. (2023). Social Media Checking Behaviors, Brain Development, and Reward Sensitivity in Early Adolescence. JAMA Pediatrics, 177(3), 271–280.

Ahmari S & Pittenger C (2023). The Neural Circuits of OCD: Corticostriatal Dysregulation. Frontiers in Behavioral Neuroscience.

HHS Office for Civil Rights (2024). HIPAA and Non-Covered Entities Guidance.

Personal conversation with AI risk analyst Adam Kocinski (October 2025) regarding corporate incentives and emotional dependency.

Insights from conversations with clinical colleagues regarding chatbot dependency patterns in therapeutic practice.