ChatGPT's psychotherapy ability has been better than human experts through the Turing test!

A recent study showed that subjects had great difficulties in distinguishing ChatGPT from psychotherapy responses to human therapists. Research shows that AI answers are often considered to be more empathetic than professionals’ answers. The study applied the classic Turing test to evaluate whether humans can recognize or interact with machines or other people. The researchers invited 830 participants to determine which responses were derived from ChatGPT and which were experienced human therapists in 18 cases of couple treatment.

According to the results of the study published in PLOS Mental Health, participants performed only slightly higher than random guesses in identifying treatment responses. The percentage of responses in human therapists were correctly identified was 56.1%, while the percentage of responses in ChatGPT was 51.2%. The study also found that ChatGPT outperformed human experts on multiple dimensions of treatment quality, including therapeutic alliances, empathy, and cultural adaptability.

There are many reasons why ChatGPT can achieve such a result. AI systems usually provide longer answers, more positive tone, and use more nouns and adjectives in their answers, which make their responses appear more detailed and empathetic. However, the study also found that subjects tend to score lower when they think they are reading the responses generated by AI, and those responses get more when they mistakenly believe that AI’s responses are from human therapists High rating.

The researchers point out that while the findings of this study are exciting, they also have important limitations. They used short hypothetical treatment scenarios rather than real treatment processes. In addition, they questioned whether this result was applicable to individual consultation.

Nevertheless, as further evidence of AI’s potential in the therapeutic field accumulates, researchers stress that mental health professionals need to understand these systems. They called on clinical workers to carefully train and monitor AI models to ensure high standards of care.

At present, several studies have drawn similar conclusions about AI's ability in consulting roles. A study from the University of Melbourne and the University of Western Australia shows that ChatGPT’s advice in social dilemma is more balanced, comprehensive and empathetic than human columnists.

However, despite AI's performance being better than humans, most participants still tend to choose human consultants. In the Australian study, 77% of respondents said they were more willing to accept human advice. While AI’s responses in medical diagnosis are also considered more empathetic and quality, researchers at Stanford and University of Texas reminded that using ChatGPT in psychotherapy is cautious, they believe that large language models are lacking The true “theory of mind”, inability to experience true empathy, calls for the establishment of international research projects to develop guidelines for the safe integration of AI in psychology.