Evaluating Large Language Models for Anxiety and Depression Classification using Counseling and Psychotherapy Transcripts

University of Hawaii at Manoa, Honolulu, HI, USA *Correspondence should be sent to: pyw@hawaii.edu These authors contributed equally to this work. Abstract We aim to evaluate the efficacy of traditional machine learning and large language models (LLMs) in classifying anxiety and depression from long conversational transcripts. We fine-tuned both established transformer models (BERT, RoBERTa, Longformer) and more recent large models (Mistral-7B), trained a Support Vector Machine with feature engineering, and assessed GPT models through prompting. We observe that state-ofthe-art models fail to enhance classification outcomes compared to traditional machine learning methods.