TW: mental health crisis, suicide, child abuse

So starting in July of last year, a very good friend I've had since high school - we'll call him Steve - had a complete psychotic break. As I was trying to figure out what the fuck was going on, it came to light that Steve had some time ago stopped seeing his regular therapist and instead got a subscription to ChatGPT and had been using it as his "therapist". Steve told me I should do the same, because I was "too smart" for traditional therapy. In trying to get through to him about how bad of an idea this was, I further discovered that Steve had basically implemented ChatGPT as a buffer between himself and the rest of the world. He had set it up to auto-respond to all his emails and then give him a summary of all his emails. He was using it to write scripts for phone calls. When people would text him, he would copy and paste their text into ChatGPT and then send ChatGPT's output as his reply.

Basically Steve had invented his own impenetrable barrier that prevented him from having to consider anything other than the feedback loop he had created with ChatGPT. He became convinced that it was his mission in life to uncover a child sex ring that was perpetuated by his relatives. Steve had been abused by an uncle when he was very young, and basically had talked to CharGPT about it so much that he'd convinced himself his whole family was involved in an elaborate pizzagate-style operation, and Steve was divinely ordained to bring them to justice.

Steve's behavior grew more and more erratic, and he was eventually taken for an involuntary psych eval, which led to him being hospitalized for a time. He got out, but he most certainly did not get better. As soon as he had access to ChatGPT again, he completely spiraled. He started making threats against me, my family, the school my kids go to, local PD... basically anyone who he thought was out to get him. He eventually caught a felony charge for assaulting a peace officer and a firefighter, and he skipped town.

Now this is not to say AI somehow "did this" to Steve. Mental health issues run in his family. His brother was actually diagnosed with schizophrenia about two years ago. The deck was stacked against Steve from the outset. However, Steve's increasing reliance on AI to function in daily life insulated him from seeing what was really going on with his mental health and convinced him that he was right, and that basically the whole world was out to get him. He basically chose the illusion AI provided over the support system he had in place and could have helped him.

I don't think AI is evil. As someone who dabbled in computer science in college, I think it's absolutely incredible that we've gone from believing the Turing test might be unbeatable to today's AI models in half my lifetime. And I also believe that the field of computer learning is going to offer us profound insights into the processes of the human mind and how it learns, because I believe there are far more similarities in how we learn and how AI learns than we yet realize and/or are willing to acknowledge. But in an increasingly isolated world, AI can also be dangerous. People who are starving for connection, and who also don't understand what AI really is, can be led right off a cliff without realizing the person holding their leash is really themselves.