Goto

Collaborating Authors

 Generative AI


Second-Order Information Matters: Revisiting Machine Unlearning for Large Language Models

arXiv.org Artificial Intelligence

With the rapid development of Large Language Models (LLMs), we have witnessed intense competition among the major LLM products like ChatGPT, LLaMa, and Gemini. However, various issues (e.g. privacy leakage and copyright violation) of the training corpus still remain underexplored. For example, the Times sued OpenAI and Microsoft for infringing on its copyrights by using millions of its articles for training. From the perspective of LLM practitioners, handling such unintended privacy violations can be challenging. Previous work addressed the ``unlearning" problem of LLMs using gradient information, while they mostly introduced significant overheads like data preprocessing or lacked robustness. In this paper, contrasting with the methods based on first-order information, we revisit the unlearning problem via the perspective of second-order information (Hessian). Our unlearning algorithms, which are inspired by classic Newton update, are not only data-agnostic/model-agnostic but also proven to be robust in terms of utility preservation or privacy guarantee. Through a comprehensive evaluation with four NLP datasets as well as a case study on real-world datasets, our methods consistently show superiority over the first-order methods.


AI showdown: I put 3 chatbots to the test

FOX News

As I scour 35 to 40 websites a day to make sure I'm up to speed on the tech world, I'm seeing a common theme: drama. While everyone from Elon Musk to The New York Times is busy suing OpenAI, I'm just focused on whether these AI chatbots actually work. So, how useful are they, really? I did the work for you. I compared the free versions of ChatGPT from OpenAI, Google Gemini and Perplexity to see how well they helped with some real-life scenarios.


OpenAI calls Elon Musk's lawsuit 'frivolous' and 'incoherent' in legal filing

The Guardian

OpenAI denounced Elon Musk's lawsuit against the company in a legal filing on Monday, describing the Tesla CEO's claims as "frivolous" and intended only "to advance his commercial interests". The filing, a response to Musk suing OpenAI earlier this month over allegations that it abandoned its pledge to help humanity, rejects many of the core assertions in Musk's suit. The company denies that it ever broke what Musk calls its "Founding Agreement", stating that no such contract ever existed. "Musk's claims rest on convoluted โ€“ often incoherent โ€“ factual premises," the filing states. "Musk says his Founding Agreement was'memorialized,' but any actual agreement is conspicuously missing from the pleading."


What to Do About the Junkification of the Internet

The Atlantic - Technology

Earlier this year, sexually explicit images of Taylor Swift were shared repeatedly X. The pictures were almost certainly created with generative-AI tools, demonstrating the ease with which the technology can be put to nefarious ends. This case mirrors many other apparently similar examples, including fake images depicting the arrest of former President Donald Trump, AI-generated images of Black voters who support Trump, and fabricated images of Dr. Anthony Fauci. There is a tendency for media coverage to focus on the source of this imagery, because generative AI is a novel technology that many people are still trying to wrap their head around. But that fact obscures the reason the images are relevant: They spread on social-media networks.


OpenAI says Elon Musk's lawsuit allegations are 'incoherent'

Engadget

"There is no Founding Agreement, or any agreement at all with Musk," OpenAI said in a court filing as a defendant in Elon Musk's lawsuit. We're, of course, talking about the lawsuit Musk filed against OpenAI, which accuses it of violating its status as a non-profit, as well as of violating a founding agreement promising the organization would never operate for profit and would release its AI publicly. The company said the billionaire's claims are based on "convoluted -- often incoherent -- factual premises." It called that founding agreement "a fiction Musk has conjured to lay unearned claim to the fruits of an enterprise he initially supported, then abandoned, then watched succeed without him." If the case goes to discovery, there's evidence that would show that Musk supported OpenAI's transition into a for-profit structure, "to be controlled by Musk himself," OpenAI continued.


Stress index strategy enhanced with financial news sentiment analysis for the equity markets

arXiv.org Artificial Intelligence

Recent advancements in Natural Language Processing (NLP) with Large Language Models (LLMs) have made the sentiment analysis of financial news by machines a practical achievement and no longer just a dream. More precisely, Large Language Models (LLMs) have marked a major step forward in processing large contexts, exhibiting human-level performance on various professional and academic benchmarks, although they still have limitations such as reliability issues and limited context windows [OpenAI, 2023]. Their ability to process more context has shown particularly interesting applications in many business areas [George and George, 2023]. Hence exploring the potential to extract either weak or strong signals from financial news to enhance a risk-on risk-off investment strategy becomes highly pertinent. Indeed, extracting sentiment from financial news is not new [Tetlock, 2007, Schumaker and Chen, 2009], and finance has a longstanding tradition of exploiting textual data [Kearney and Liu, 2014].


From Paper to Card: Transforming Design Implications with Generative AI

arXiv.org Artificial Intelligence

Communicating design implications is common within the HCI community when publishing academic papers, yet these papers are rarely read and used by designers. One solution is to use design cards as a form of translational resource that communicates valuable insights from papers in a more digestible and accessible format to assist in design processes. However, creating design cards can be time-consuming, and authors may lack the resources/know-how to produce cards. Through an iterative design process, we built a system that helps create design cards from academic papers using an LLM and text-to-image model. Our evaluation with designers (N=21) and authors of selected papers (N=12) revealed that designers perceived the design implications from our design cards as more inspiring and generative, compared to reading original paper texts, and the authors viewed our system as an effective way of communicating their design implications. We also propose future enhancements for AI-generated design cards.


The future of document indexing: GPT and Donut revolutionize table of content processing

arXiv.org Artificial Intelligence

Industrial projects rely heavily on lengthy, complex specification documents, making tedious manual extraction of structured information a major bottleneck. This paper introduces an innovative approach to automate this process, leveraging the capabilities of two cutting-edge AI models: Donut, a model that extracts information directly from scanned documents without OCR, and OpenAI GPT-3.5 Turbo, a robust large language model. The proposed methodology is initiated by acquiring the table of contents (ToCs) from construction specification documents and subsequently structuring the ToCs text into JSON data. Remarkable accuracy is achieved, with Donut reaching 85% and GPT-3.5 Turbo reaching 89% in effectively organizing the ToCs. This landmark achievement represents a significant leap forward in document indexing, demonstrating the immense potential of AI to automate information extraction tasks across diverse document types, boosting efficiency and liberating critical resources in various industries.


Elon Musk Gave Himself No Choice but to Open Source His Chatbot Grok

WIRED

After suing OpenAI this month, alleging the company has become too closed, Elon Musk says he will release his "truth-seeking" answer to ChatGPT, the chatbot Grok, for anyone to download and use. "This week, @xAI will open source Grok," Musk wrote on his social media platform X today. That suggests his AI company, xAI, will release the full code of Grok and allow anyone to use or alter it. By contrast, OpenAI makes a version of ChatGPT and the language model behind it available to use for free but keeps its code private. Musk had previously said little about the business model for Grok or xAI, and the chatbot was made available only to Premium subscribers to X. Having accused his OpenAI cofounders of reneging on a promise to give away the company's artificial intelligence earlier this month, Musk may have felt he had to open source his own chatbot to show that he is committed to that vision.


Reddit aims for 6.4bn valuation in shares sale

BBC News

Currently, the biggest shareholders include media company Advance Magazine Publishers, Chinese tech firm Tencent, US investment firm Fidelity, and Sam Altman, the chief executive of ChatGPT makers OpenAI.