Large Language Model
Machine Mindset: An MBTI Exploration of Large Language Models
Cui, Jiaxi, Lv, Liuzhenghao, Wen, Jing, Wang, Rongsheng, Tang, Jing, Tian, YongHong, Yuan, Li
We present a novel approach for integrating Myers-Briggs Type Indicator (MBTI) personality traits into large language models (LLMs), addressing the challenges of personality consistency in personalized AI. Our method, "Machine Mindset," involves a two-phase fine-tuning and Direct Preference Optimization (DPO) to embed MBTI traits into LLMs. This approach ensures that models internalize these traits, offering a stable and consistent personality profile. We demonstrate the effectiveness of our models across various domains, showing alignment between model performance and their respective MBTI traits. The paper highlights significant contributions in the development of personality datasets and a new training methodology for personality integration in LLMs, enhancing the potential for personalized AI applications. We also open-sourced our model and part of the data at \url{https://github.com/PKU-YuanGroup/Machine-Mindset}.
Conceptualizing Suicidal Behavior: Utilizing Explanations of Predicted Outcomes to Analyze Longitudinal Social Media Data
Nguyen, Van Minh, Nur, Nasheen, Stern, William, Mercer, Thomas, Sen, Chiradeep, Bhattacharyya, Siddhartha, Tumbiolo, Victor, Goh, Seng Jhing
The COVID-19 pandemic has escalated mental health crises worldwide, with social isolation and economic instability contributing to a rise in suicidal behavior. Suicide can result from social factors such as shame, abuse, abandonment, and mental health conditions like depression, Post-Traumatic Stress Disorder (PTSD), Attention-Deficit/Hyperactivity Disorder (ADHD), anxiety disorders, and bipolar disorders. As these conditions develop, signs of suicidal ideation may manifest in social media interactions. Analyzing social media data using artificial intelligence (AI) techniques can help identify patterns of suicidal behavior, providing invaluable insights for suicide prevention agencies, professionals, and broader community awareness initiatives. Machine learning algorithms for this purpose require large volumes of accurately labeled data. Previous research has not fully explored the potential of incorporating explanations in analyzing and labeling longitudinal social media data. In this study, we employed a model explanation method, Layer Integrated Gradients, on top of a fine-tuned state-of-the-art language model, to assign each token from Reddit users' posts an attribution score for predicting suicidal ideation. By extracting and analyzing attributions of tokens from the data, we propose a methodology for preliminary screening of social media posts for suicidal ideation without using large language models during inference.
Empower Nested Boolean Logic via Self-Supervised Curriculum Learning
Wu, Hongqiu, Liu, Linfeng, Zhao, Hai, Zhang, Min
Beyond the great cognitive powers showcased by language models, it is crucial to scrutinize whether their reasoning capabilities stem from strong generalization or merely exposure to relevant data. As opposed to constructing increasingly complex logic, this paper probes into the boolean logic, the root capability of a logical reasoner. We find that any pre-trained language models even including large language models only behave like a random selector in the face of multi-nested boolean logic, a task that humans can handle with ease. To empower language models with this fundamental capability, this paper proposes a new self-supervised learning method \textit{Curriculum Logical Reasoning} (\textsc{Clr}), where we augment the training data with nested boolean logic chain step-by-step, and program the training from simpler logical patterns gradually to harder ones. This new training paradigm allows language models to effectively generalize to much harder and longer-hop logic, which can hardly be learned through naive training. Furthermore, we show that boolean logic is a great foundation for improving the subsequent general logical tasks.
Is ChatGPT Involved in Texts? Measure the Polish Ratio to Detect ChatGPT-Generated Text
Yang, Lingyi, Jiang, Feng, Li, Haizhou
The remarkable capabilities of large-scale language models, such as ChatGPT, in text generation have impressed readers and spurred researchers to devise detectors to mitigate potential risks, including misinformation, phishing, and academic dishonesty. Despite this, most previous studies have been predominantly geared towards creating detectors that differentiate between purely ChatGPT-generated texts and human-authored texts. This approach, however, fails to work on discerning texts generated through human-machine collaboration, such as ChatGPT-polished texts. Addressing this gap, we introduce a novel dataset termed HPPT (ChatGPT-polished academic abstracts), facilitating the construction of more robust detectors. It diverges from extant corpora by comprising pairs of human-written and ChatGPT-polished abstracts instead of purely ChatGPT-generated texts. Additionally, we propose the "Polish Ratio" method, an innovative measure of the degree of modification made by ChatGPT compared to the original human-written text. It provides a mechanism to measure the degree of ChatGPT influence in the resulting text. Our experimental results show our proposed model has better robustness on the HPPT dataset and two existing datasets (HC3 and CDB). Furthermore, the "Polish Ratio" we proposed offers a more comprehensive explanation by quantifying the degree of ChatGPT involvement.
Symbol tuning improves in-context learning in language models
Wei, Jerry, Hou, Le, Lampinen, Andrew, Chen, Xiangning, Huang, Da, Tay, Yi, Chen, Xinyun, Lu, Yifeng, Zhou, Denny, Ma, Tengyu, Le, Quoc V.
We present symbol tuning - finetuning language models on in-context input-label pairs where natural language labels (e.g., "positive/negative sentiment") are replaced with arbitrary symbols (e.g., "foo/bar"). Symbol tuning leverages the intuition that when a model cannot use instructions or natural language labels to figure out a task, it must instead do so by learning the input-label mappings. We experiment with symbol tuning across Flan-PaLM models up to 540B parameters and observe benefits across various settings. First, symbol tuning boosts performance on unseen in-context learning tasks and is much more robust to underspecified prompts, such as those without instructions or without natural language labels. Second, symbol-tuned models are much stronger at algorithmic reasoning tasks, with up to 18.2% better performance on the List Functions benchmark and up to 15.3% better performance on the Simple Turing Concepts benchmark. Finally, symbol-tuned models show large improvements in following flipped-labels presented in-context, meaning that they are more capable of using in-context information to override prior semantic knowledge.
Michael Cohen used fake cases created by AI in bid to end his probation
In the filing, Cohen wrote that he had not kept up with "emerging trends (and related risks) in legal technology and did not realize that Google Bard was a generative text service that, like ChatGPT, could show citations and descriptions that looked real but actually were not." To him, he said, Google Bard seemed to be a "supercharged search engine."
Michael Cohen admits to inadvertently citing fake cases generated by AI in legal motion
Jack Krawczyk discusses how Google Bard helps users connect and communicate -- and what the future holds for the platform. Michael Cohen, former President Trump's onetime fixer and lawyer, admitted in a filing unsealed Friday that he inadvertently gave his lawyer fake legal case citations generated by artificial intelligence in connection with a motion to end his supervised release early. U.S. District Judge Jesse M. Furman previously called the citations into question, writing earlier this month, "In the letter brief, Mr. Cohen asserts that, "[a]s recently as 2022, there have been District Court decisions, affirmed by the Second Circuit Court, granting early termination of supervised release." Furman added, "As far as the Court can tell, none of these cases exist." Cohen said in his sworn declaration released Friday that he had found the phony citations through Google Bard, an AI service that he said he thought was a "supercharged" search engine. Michael Cohen admitted to inadvertently citing fake legal cases in a motion to end his early release in a sworn declaration released Friday. "As a non-lawyer, I have not kept up with emerging trends (and related risks) in legal technology and did not realize that Google Bard was a generative text service that, like Chat-GPT, could show citations and descriptions that looked real but actually were not," Cohen said. "Instead, I understood it to be a super-charged search engine and had repeatedly used it in other contexts to (successfully) find accurate information online." In 2018, Cohen pleaded guilty to tax evasion, campaign finance charges and lying to Congress, spending more than a year in prison before he was put on supervised release. He was also disbarred as a lawyer. "It did not occur to me then and remains surprising to me now--that Mr. Schwartz would drop the cases into his submission wholesale without even confirming that they existed," he added, citing his lawyer David Schwartz. "I deeply regret any problems Mr. Schwartz's filing may have caused." He said Schwartz's alleged mistake was "a product of inadvertence, not any intent to deceive." E. Danya Perry, who represents Cohen and discovered the citations were fake, told the judge, "Mr.
Former Trump 'fixer' Michael Cohen admits using Google Bard to cite bogus court cases
Donald Trump's former "fixer," Michael Cohen, used Google Bard to cite made-up legal cases that ended up in a federal court. The New York Times reported Friday that Cohen admitted in unsealed court papers that he passed on documents referencing bogus cases to his lawyer, who then relayed them to a federal judge. Cohen reportedly wrote in the sworn declaration he hadn't stayed on top of "emerging trends (and related risks) in legal technology." Cohen's legal team filed the paperwork in a motion asking for an early end to court supervision from his 2018 campaign finance case, for which he served three years in prison. After Cohen's attorney, David M. Schwartz, presented the legal documents to the federal court, Judge Jesse M. Furman of the Federal District Court said he was having trouble finding the three decisions cited by Schwartz (via Cohen).
The biggest winners in tech in 2023
Throughout 2023, it felt like the drama never let up. From Elon Musk's nonstop shenanigans to the constant launches in the generative AI race, the last twelve months was packed with news. Thankfully, it wasn't all bad, and this year saw more winners than before. There were clear frontrunners, like Threads and AI, but we also saw surprises like Apple's Vision Pro headset and the iPhone maker finally embracing several open standards. Of all the things that happened this year, here's the Engadget team's list of tech's biggest winners in 2023.
WIRED's 2023 Year-in-Review Quiz
An AI apocalypse didn't collapse society into chaos, but everyone in Silicon Valley did try to add some sort of AI tool into their software, and it sure felt pretty chaotic. Researchers considered new plans for preserving our microbial diversity. Pollution and climate change continued to ransack the Earth's environment. Billionaires grew more obsessed with social media and copying each other's platforms. This 12-question quiz isn't all-inclusive, but it does highlight some of the most read articles on WIRED's website in 2023.