Large Language Model
Representation Bias of Adolescents in AI: A Bilingual, Bicultural Study
Wolfe, Robert, Dangol, Aayushi, Howe, Bill, Hiniker, Alexis
Popular and news media often portray teenagers with sensationalism, as both a risk to society and at risk from society. As AI begins to absorb some of the epistemic functions of traditional media, we study how teenagers in two countries speaking two languages: 1) are depicted by AI, and 2) how they would prefer to be depicted. Specifically, we study the biases about teenagers learned by static word embeddings (SWEs) and generative language models (GLMs), comparing these with the perspectives of adolescents living in the U.S. and Nepal. We find English-language SWEs associate teenagers with societal problems, and more than 50% of the 1,000 words most associated with teenagers in the pretrained GloVe SWE reflect such problems. Given prompts about teenagers, 30% of outputs from GPT2-XL and 29% from LLaMA-2-7B GLMs discuss societal problems, most commonly violence, but also drug use, mental illness, and sexual taboo. Nepali models, while not free of such associations, are less dominated by social problems. Data from workshops with N=13 U.S. adolescents and N=18 Nepalese adolescents show that AI presentations are disconnected from teenage life, which revolves around activities like school and friendship. Participant ratings of how well 20 trait words describe teens are decorrelated from SWE associations, with Pearson's r=.02, n.s. in English FastText and r=.06, n.s. in GloVe; and r=.06, n.s. in Nepali FastText and r=-.23, n.s. in GloVe. U.S. participants suggested AI could fairly present teens by highlighting diversity, while Nepalese participants centered positivity. Participants were optimistic that, if it learned from adolescents, rather than media sources, AI could help mitigate stereotypes. Our work offers an understanding of the ways SWEs and GLMs misrepresent a developmentally vulnerable group and provides a template for less sensationalized characterization.
LawLuo: A Chinese Law Firm Co-run by LLM Agents
Sun, Jingyun, Dai, Chengxiao, Luo, Zhongze, Chang, Yangbo, Li, Yang
Large Language Models (LLMs) demonstrate substantial potential in delivering legal consultation services to users without a legal background, attributed to their superior text comprehension and generation capabilities. Nonetheless, existing Chinese legal LLMs limit interaction to a single model-user dialogue, unlike the collaborative consultations typical of law firms, where multiple staff members contribute to a single consultation. This limitation prevents an authentic consultation experience. Additionally, extant Chinese legal LLMs suffer from critical limitations: (1) insufficient control over the quality of instruction fine-tuning data; (2) increased model hallucination resulting from users' ambiguous queries; and (3) a reduction in the model's ability to follow instructions over multiple dialogue turns. In response to these challenges, we propose a novel legal dialogue framework that leverages the collaborative capabilities of multiple LLM agents, termed LawLuo. This framework encompasses four agents: a receptionist, a lawyer, a secretary, and a boss, each responsible for different functionalities, collaboratively providing a comprehensive legal consultation to users. Additionally, we constructed two high-quality legal dialogue datasets, KINLED and MURLED, and fine-tuned ChatGLM-3-6b using these datasets. We propose a legal query clarification algorithm called ToLC. Experimental results demonstrate that LawLuo outperforms baseline LLMs, including GPT-4, across three dimensions: lawyer-like language style, the usefulness of legal advice, and the accuracy of legal knowledge. Our code and datasets are available at https://github.com/NEFUJing/LawLuo.
A General Framework for Data-Use Auditing of ML Models
Huang, Zonghao, Gong, Neil Zhenqiang, Reiter, Michael K.
Passive data auditing, commonly referred as membership inference Auditing the use of data in training machine-learning (ML) models [7, 13, 27, 65, 83], infers if a data sample is a member of an is an increasingly pressing challenge, as myriad ML practitioners ML model's training set. However, such passive techniques have an routinely leverage the effort of content creators to train models without inherent limitation: they do not provide any quantitative guarantee their permission. In this paper, we propose a general method for the false-detection of their inference results. In contrast, proactive to audit an ML model for the use of a data-owner's data in training, data auditing techniques embed marks into data before its publication without prior knowledge of the ML task for which the data might [24, 38, 39, 59, 74, 79, 82] and can provide detection results be used.
ChatGPT is now ranked FUNNIER than humans - can you tell which of these jokes are AI-made?
The comics of the future may be a bit too robotic for our liking - literally. A recent study discovered that people find ChatGPT-produced jokes funnier than those written by humans. The research, by the University of Southern California, was blinded, meaning the participants were unaware who wrote the jokes before giving their answer. They voted resoundingly in favor of the AI jokes - with 75 percent saying they were funnier than the human ones. So can you tell the difference between AI-generated humor and the good old-fashioned human kind?
OpenAI's Sam Altman is becoming one of the most powerful people on Earth. We should be very afraid
On 16 May 2023, Sam Altman, OpenAI's charming, softly spoken, eternally optimistic billionaire CEO, and I stood in front of the US Senate judiciary subcommittee meeting on AI oversight. We were in Washington DC, and it was at the height of AI mania. Altman, then 38, was the poster boy for it all. Raised in St Louis, Missouri, Altman was the Stanford dropout who had become the president of the massively successful Y Combinator startup incubator before he was 30. A few months before the hearing, his company's product ChatGPT had taken the world by storm. All through the summer of 2023, Altman was treated like a Beatle, stopping by DC as part of a world tour, meeting prime ministers and presidents around the globe. US Senator Kyrsten Sinema gushed: "I've never met anyone as smart as Samโฆ He's an introvert and shy and humbleโฆ Butโฆ very good at forming relationships with people on the Hill andโฆ can help folks in government understand AI." Glowing portraits at the time painted the youthful Altman as sincere, talented, rich and interested in nothing more than fostering humanity.
Why have the big seven tech companies been hit by AI boom doubts?
Primarily, there is concern about whether the vast investment in AI byMicrosoft, Google and others will pay off. This has been bubbling away in recent months. Analysts at Goldman Sachs published a note in June with the title "Gen AI: too much spend, too little benefit?" The Wall Street bank asked if a 1tn investment in AI over the next few years will "ever pay off", while an analysis by Sequoia Capital, an early investor in ChatGPT developer OpenAI, estimated that tech companies will need to earn 600bn to pay back their AI investments. Zino says the Magnificent Seven has been hit by these concerns.
Multi-Frame Vision-Language Model for Long-form Reasoning in Driver Behavior Analysis
Takato, Hiroshi, Tsutsui, Hiroshi, Soda, Komei, Kamigaito, Hidetaka
Identifying risky driving behavior in real-world situations is essential for the safety of both drivers and pedestrians. However, integrating natural language models in this field remains relatively untapped. To address this, we created a novel multi-modal instruction tuning dataset and driver coaching inference system. Our primary use case is dashcam-based coaching for commercial drivers. The North American Dashcam Market is expected to register a CAGR of 15.4 percent from 2022 to 2027. Our dataset enables language models to learn visual instructions across various risky driving scenarios, emphasizing detailed reasoning crucial for effective driver coaching and managerial comprehension. Our model is trained on roadfacing and driver-facing RGB camera footage, capturing the comprehensive scope of driving Figure 1: Overview of our targeting coaching task.
MathLearner: A Large Language Model Agent Framework for Learning to Solve Mathematical Problems
Xie, Wenbei, Liu, Donglin, Yan, Haoran, Wu, Wenjie, Liu, Zongyang
With the development of artificial intelligence (AI), large language models (LLM) are widely used in many fields. However, the reasoning ability of LLM is still very limited when it comes to mathematical reasoning. Mathematics plays an important role in all aspects of human society and is a technical guarantee in the fields of healthcare, transport and aerospace, for this reason, the development of AI big language models in the field of mathematics has great potential significance. To improve the mathematical reasoning ability of large language models, we proposed an agent framework for learning to solve mathematical problems based on inductive reasoning. By emulating the human learning process of generalization of learned information and effective application of previous knowledge in new reasoning tasks, this framework has great performance in the mathematical reasoning process. It improves global accuracy over the baseline method (chain-of-thought) by 20.96% and solves 17.54% of the mathematical problems that the baseline cannot solve. Benefiting from the efficient RETRIEVAL method, our model improves the ability of large language models to efficiently use external knowledge, i.e., the mathematical computation of the model can be based on written procedures. In education, our model can be used as a personalised learning aid, thus reducing the inequality of educational resources.
STBLLM: Breaking the 1-Bit Barrier with Structured Binary LLMs
Dong, Peijie, Li, Lujun, Du, Dayou, Chen, Yuhan, Tang, Zhenheng, Wang, Qiang, Xue, Wei, Luo, Wenhan, Liu, Qifeng, Guo, Yike, Chu, Xiaowen
In this paper, we present STBLLM, the first structural binarization framework for compressing Large Language Models (LLMs) to less than 1-bit precision. LLMs have achieved remarkable performance, but their heavy memory requirements have hindered widespread adoption, particularly on resource-constrained devices. Binarization, which quantifies weights to a mere 1-bit, achieves a milestone in increasing computational efficiency. However, we observe that some weights in binarized LLMs can be randomly flipped without significant performance degradation, indicating the potential for further compression. To exploit this, our STBLLM employs an N:M sparsity to perform structural binarization of the weights. First, we introduce a new Standardized Importance (SI) metric that considers weight magnitude and input feature norm to better evaluate weight significance. Then, we propose a layer-wise approach where different layers of the LLM can be sparsified with varying N:M ratios, balancing compression and accuracy. Finally, we use residual approximation with double binarization to preserve information for salient weights. In addition, we utilize a fine-grained grouping strategy for less important weights that applies different quantization schemes to sparse, intermediate, and dense regions. We conduct extensive experiments on various language models, including the LLaMA-1/2/3, OPT family, and Mistral, to evaluate the effectiveness of STBLLM. The results demonstrate that our approach performs better than other compressed binarization LLM methods while significantly reducing memory requirements.
Distinguishing Chatbot from Human
Godghase, Gauri Anil, Agrawal, Rishit, Obili, Tanush, Stamp, Mark
There have been many recent advances in the fields of generative Artificial Intelligence (AI) and Large Language Models (LLM), with the Generative Pre-trained Transformer (GPT) model being a leading "chatbot." LLM-based chatbots have become so powerful that it may seem difficult to differentiate between human-written and machine-generated text. To analyze this problem, we have developed a new dataset consisting of more than 750,000 human-written paragraphs, with a corresponding chatbot-generated paragraph for each. Based on this dataset, we apply Machine Learning (ML) techniques to determine the origin of text (human or chatbot). Specifically, we consider two methodologies for tackling this issue: feature analysis and embeddings. Our feature analysis approach involves extracting a collection of features from the text for classification. We also explore the use of contextual embeddings and transformer-based architectures to train classification models. Our proposed solutions offer high classification accuracy and serve as useful tools for textual analysis, resulting in a better understanding of chatbot-generated text in this era of advanced AI technology.