The opioid overdose epidemic remains a critical public health crisis, particularly in the United States, leading to significant mortality and societal costs. Social media platforms like Reddit provide vast amounts of unstructured data that offer insights into public perceptions, discussions, and experiences related to opioid use. This study leverages Natural Language Processing (NLP), specifically Opioid Named Entity Recognition (ONER-2025), to extract actionable information from these platforms. Our research makes four key contributions. First, we created a unique, manually annotated dataset sourced from Reddit, where users share self-reported experiences of opioid use via different administration routes. This dataset contains 331,285 tokens and includes eight major opioid entity categories. Second, we detail our annotation process and guidelines while discussing the challenges of labeling the ONER-2025 dataset. Third, we analyze key linguistic challenges, including slang, ambiguity, fragmented sentences, and emotionally charged language, in opioid discussions. Fourth, we propose a real-time monitoring system to process streaming data from social media, healthcare records, and emergency services to identify overdose events. Using 5-fold cross-validation in 11 experiments, our system integrates machine learning, deep learning, and transformer-based language models with advanced contextual embeddings to enhance understanding. Our transformer-based models (bert-base-NER and roberta-base) achieved 97% accuracy and F1-score, outperforming baselines by 10.23% (RF=0.88).

NLP research has increasingly focused on subjective tasks such as emotion analysis. However, existing emotion benchmarks suffer from two major shortcomings: (1) they largely rely on keyword-based emotion recognition, overlooking crucial cultural dimensions required for deeper emotion understanding, and (2) many are created by translating English-annotated data into other languages, leading to potentially unreliable evaluation. To address these issues, we introduce Cultural Lenses on Emotion (CuLEmo), the first benchmark designed to evaluate culture-aware emotion prediction across six languages: Amharic, Arabic, English, German, Hindi, and Spanish. CuLEmo comprises 400 crafted questions per language, each requiring nuanced cultural reasoning and understanding. We use this benchmark to evaluate several state-of-the-art LLMs on culture-aware emotion prediction and sentiment analysis tasks. Our findings reveal that (1) emotion conceptualizations vary significantly across languages and cultures, (2) LLMs performance likewise varies by language and cultural context, and (3) prompting in English with explicit country context often outperforms in-language prompts for culture-aware emotion and sentiment understanding. We hope this benchmark guides future research toward developing more culturally aligned NLP systems.

We noticed that ChatGPT struggled to extract co-reference entities like "her medications" or "her symptoms", which should be annotated in accordance with the 2010 i2b2 annotation guidelines, for coreference identification purposes. After we removed those co-reference entities in the gold standard and re-evaluated the performance of both ChatGPT and GPT-3, we observed modest increases in performance, with ChatGPT achieving an F1 score of 0.628 using Prompt-2 and GPT-3 attaining an F1 score of 0.500 in the relaxed-match criteria. Moreover, we observed a significant degree of randomness in ChatGPT's output. Even when presented with the same prompt and the same input text, it sometimes generated responses with considerable differences in format and content. This phenomenon was particularly prevalent when the input note was lengthy, despite our efforts to minimize input sequence length by limiting it to the HPI section. We anticipate this issue will be addressed when GPT-4 allows much longer text. Although it is not clear whether clinical corpora (and what types of clinical corpora) are used in training ChatGPT, ChatGPT has demonstrated its understanding of the medical text to a certain degree. We believe fine-tuning ChatGPT with domain-specific corpora, assuming OpenAI will provide such an API, will further improve its performance on clinical NLP tasks such as NER in the zero-shot fashion.