Goto

Collaborating Authors

 Large Language Model


OpenAI starts training a new AI model while forming a safety committee

Washington Post - Technology News

In a statement released on its website, OpenAI said this new model, which would replace GPT-4 technology, will bring the company closer to achieving "AGI," or artificial general intelligence, a hotly contested idea that refers to computers matching the power of human brains.


If A.I. Can Do Your Job, Maybe It Can Also Replace Your C.E.O.

NYT > Economy

This is not just a prediction. A few successful companies have begun to publicly experiment with the notion of an A.I. leader, even if at the moment it might largely be a branding exercise. A.I. has been hyped as the solution to all corporate problems for about 18 months now, ever since OpenAI rolled out ChatGPT in November 2022. Silicon Valley put 29 billion last year into generative A.I. and is selling it hard. Even in its current rudimentary form, A.I. that mimics human reasoning is finding a foothold among distressed companies with little to lose and lacking strong leadership.


OpenAI's new safety team is led by board members, including CEO Sam Altman

Engadget

OpenAI has created a new Safety and Security Committee less than two weeks after the company dissolved the team tasked with protecting humanity from AI's existential threats. This latest iteration of the group responsible for OpenAI's safety guardrails will include two board members and CEO Sam Altman, raising questions about whether the move is little more than self-policing theatre amid a breakneck race for profit and dominance alongside partner Microsoft. The Safety and Security Committee, formed by OpenAI's board, will be led by board members Bret Taylor (Chair), Nicole Seligman, Adam D'Angelo and Sam Altman (CEO). The new team follows co-founder Ilya Sutskever's and Jan Leike's high-profile resignations, which raised more than a few eyebrows. Their former "Superalignment Team" was only created last July.


Opera adds support for Google's Gemini AI model

PCWorld

When you purchase through links in our articles, we may earn a small commission. Opera adds support for Google's Gemini AI model In addition, Opera One users can test two new AI features at no cost. If you use the Opera browser, you've been able to download a large number of language models directly to your computer, including Llama from Meta and Mixtral from Mistral AI. Now Opera has announced that you can also use Gemini, Google's powerful language model launched last year. As part of the collaboration with Google, users can also test two new AI features at no cost.


OpenAI forms safety council as it trains latest artificial intelligence model

The Guardian

OpenAI says it is setting up a safety and security committee and has begun training a new AI model to supplant the GPT-4 system that underpins its ChatGPT chatbot. The San Francisco startup said in a blogpost on Tuesday that the committee will advise the full board on "critical safety and security decisions" for its projects and operations. The safety committee arrives as debate swirls around AI safety at the company, which was thrust into the spotlight after a researcher, Jan Leike, resigned and leveled criticism at OpenAI for letting safety "take a backseat to shiny products". The OpenAI co-founder and chief scientist Ilya Sutskever also resigned, and the company disbanded the "superalignment" team focused on AI risks that they jointly led. OpenAI said it had "recently begun training its next frontier model" and its AI models led the industry on capability and safety, though it made no mention of the controversy.


Opera is adding Google's Gemini AI to its browser

Engadget

Opera users can already rely on the capabilities of OpenAI's large language models (LLMs) whenever they use the browser's Aria built-in AI assistant. But now, the company has also teamed up with Google to integrate its Gemini AI models into Aria. According to Opera, its Composer AI engine can process the user's intent based on their inquiry and then decide which model to use for each particular task. Google called Gemini the "the most capable model [it has] ever built" when it officially announced the LLM last year. Since then, the company has announced Gemini-powered features across its products and has built the Gemini AI chatbot right into Android.


SVD-LLM: Truncation-aware Singular Value Decomposition for Large Language Model Compression

arXiv.org Artificial Intelligence

The advancements in Large Language Models (LLMs) have been hindered by their substantial sizes, which necessitate LLM compression methods for practical deployment. Singular Value Decomposition (SVD) offers a promising solution for LLM compression. However, state-of-the-art SVD-based LLM compression methods have two key limitations: truncating smaller singular values may lead to higher compression loss, and the lack of update on the compressed weight after SVD truncation. In this work, we propose SVD-LLM, a new SVD-based LLM compression method that addresses the limitations of existing methods. SVD-LLM incorporates a truncation-aware data whitening strategy to ensure a direct mapping between singular values and compression loss. Moreover, SVD-LLM adopts a layer-wise closed-form model parameter update strategy to compensate for accuracy degradation under high compression ratios. We evaluate SVD-LLM on a total of 10 datasets and eight models from three different LLM families at four different scales. Our results demonstrate the superiority of SVD-LLM over state-of-the-arts, especially at high model compression ratios.


Bridging the Gap: Dynamic Learning Strategies for Improving Multilingual Performance in LLMs

arXiv.org Artificial Intelligence

Large language models (LLMs) are at the forefront of transforming numerous domains globally. However, their inclusivity and effectiveness remain limited for non-Latin scripts and low-resource languages. This paper tackles the imperative challenge of enhancing the multilingual performance of LLMs without extensive training or fine-tuning. Through systematic investigation and evaluation of diverse languages using popular question-answering (QA) datasets, we present novel techniques that unlock the true potential of LLMs in a polyglot landscape. Our approach encompasses three key strategies that yield significant improvements in multilingual proficiency. First, by meticulously optimizing prompts tailored for polyglot LLMs, we unlock their latent capabilities, resulting in substantial performance boosts across languages. Second, we introduce a new hybrid approach that synergizes LLM Retrieval Augmented Generation (RAG) with multilingual embeddings and achieves improved multilingual task performance. Finally, we introduce a novel learning approach that dynamically selects the optimal prompt strategy, LLM model, and embedding model per query at run-time. This dynamic adaptation maximizes the efficacy of LLMs across languages, outperforming best static and random strategies. Additionally, our approach adapts configurations in both offline and online settings, and can seamlessly adapt to new languages and datasets, leading to substantial advancements in multilingual understanding and generation across diverse languages.


ChatGPT as the Marketplace of Ideas: Should Truth-Seeking Be the Goal of AI Content Governance?

arXiv.org Artificial Intelligence

As one of the most enduring metaphors within legal discourse, the marketplace of ideas has wielded considerable influence over the jurisprudential landscape for decades. A century after the inception of this theory, ChatGPT emerged as a revolutionary technological advancement in the twenty-first century. This research finds that ChatGPT effectively manifests the marketplace metaphor. It not only instantiates the promises envisaged by generations of legal scholars but also lays bare the perils discerned through sustained academic critique. Specifically, the workings of ChatGPT and the marketplace of ideas theory exhibit at least four common features: arena, means, objectives, and flaws. These shared attributes are sufficient to render ChatGPT historically the most qualified engine for actualizing the marketplace of ideas theory. The comparison of the marketplace theory and ChatGPT merely marks a starting point. A more meaningful undertaking entails reevaluating and reframing both internal and external AI policies by referring to the accumulated experience, insights, and suggestions researchers have raised to fix the marketplace theory. Here, a pivotal issue is: should truth-seeking be set as the goal of AI content governance? Given the unattainability of the absolute truth-seeking goal, I argue against adopting zero-risk policies. Instead, a more judicious approach would be to embrace a knowledge-based alternative wherein large language models (LLMs) are trained to generate competing and divergent viewpoints based on sufficient justifications. This research also argues that so-called AI content risks are not created by AI companies but are inherent in the entire information ecosystem. Thus, the burden of managing these risks should be distributed among different social actors, rather than being solely shouldered by chatbot companies.


Entity Alignment with Noisy Annotations from Large Language Models

arXiv.org Artificial Intelligence

Entity alignment (EA) aims to merge two knowledge graphs (KGs) by identifying equivalent entity pairs. While existing methods heavily rely on human-generated labels, it is prohibitively expensive to incorporate cross-domain experts for annotation in real-world scenarios. The advent of Large Language Models (LLMs) presents new avenues for automating EA with annotations, inspired by their comprehensive capability to process semantic information. However, it is nontrivial to directly apply LLMs for EA since the annotation space in real-world KGs is large. LLMs could also generate noisy labels that may mislead the alignment. To this end, we propose a unified framework, LLM4EA, to effectively leverage LLMs for EA. Specifically, we design a novel active learning policy to significantly reduce the annotation space by prioritizing the most valuable entities based on the entire inter-KG and intra-KG structure. Moreover, we introduce an unsupervised label refiner to continuously enhance label accuracy through in-depth probabilistic reasoning. We iteratively optimize the policy based on the feedback from a base EA model. Extensive experiments demonstrate the advantages of LLM4EA on four benchmark datasets in terms of effectiveness, robustness, and efficiency. Codes are available via https://github.com/chensyCN/llm4ea_official.