Large Language Model
Binary Hypothesis Testing for Softmax Models and Leverage Score Models
Gao, Yeqi, Gu, Yuzhou, Song, Zhao
Softmax distributions are widely used in machine learning, including Large Language Models (LLMs) where the attention unit uses softmax distributions. We abstract the attention unit as the softmax model, where given a vector input, the model produces an output drawn from the softmax distribution (which depends on the vector input). We consider the fundamental problem of binary hypothesis testing in the setting of softmax models. That is, given an unknown softmax model, which is known to be one of the two given softmax models, how many queries are needed to determine which one is the truth? We show that the sample complexity is asymptotically $O(\epsilon^{-2})$ where $\epsilon$ is a certain distance between the parameters of the models. Furthermore, we draw analogy between the softmax model and the leverage score model, an important tool for algorithm design in linear algebra and graph theory. The leverage score model, on a high level, is a model which, given vector input, produces an output drawn from a distribution dependent on the input. We obtain similar results for the binary hypothesis testing problem for leverage score models.
U.S. eyes curbs on China's access to AI software behind apps like ChatGPT
The administration of U.S. President Joe Biden is poised to open up a new front in its efforts to safeguard U.S. artificial intelligence from China and Russia. Preliminary plans include placing guardrails around the most advanced AI Models, the core software of AI systems like ChatGPT, sources said. The Commerce Department is considering a new regulatory push to restrict the export of proprietary or closed-source AI models, whose software and the data on which it is trained are kept under wraps, three people familiar with the matter said.
OpenAI Is 'Exploring' How to Responsibly Generate AI Porn
OpenAI released draft documentation Wednesday laying out how it wants ChatGPT and its other AI technology to behave. Part of the lengthy Model Spec document discloses that the company is exploring a leap into porn and other explicit content. OpenAI's usage policies curently prohibit sexually explicit or even suggestive materials, but a "commentary" note on part of the Model Spec related to that rule says the company is considering how to permit such content. "We're exploring whether we can responsibly provide the ability to generate NSFW content in age-appropriate contexts through the API and ChatGPT," the note says, using a colloquial term for content considered "not safe for work" contexts. "We look forward to better understanding user and societal expectations of model behavior in this area."
Elon Musk's lawyers succeed in challenge to remove OpenAI case judge
The California judge presiding over Elon Musk's lawsuit against OpenAI and its CEO, Sam Altman, has removed himself from the case. Judge Ethan Schulman on Monday sustained a challenge from Musk's lawyers, which cited a California state law that allows plaintiffs and defendants to remove a judge they believe cannot grant an impartial trial. The law, known as California Code of Civil Procedure 170.6, does not require the person issuing the challenge to provide any factual basis for their claim that the judge is prejudiced against them. Each side in a case gets one such peremptory challenge, which is granted as long as it is filed with correct language and within a certain time frame. Lawyers for Altman and Musk did not respond to requests for comment.
DeepMind AI can predict how drugs interact with proteins
An artificial intelligence system can now determine not only how proteins fold but also how they interact with other proteins, drug molecules or DNA. Biochemists and pharmaceutical researchers say the tool has the potential to vastly speed up their work, such as helping to discover new drugs. Proteins, which play many important roles in living things, are made up of chains of amino acids, but their complex 3D shapes are difficult to predict. How this moment for AI will change society forever (and how it won't) The AI company DeepMind first announced in 2020 that its AlphaFold AI could accurately predict protein structure from amino acid sequences, solving one of the biggest challenges in biology. By the middle of 2021, the company said that it had mapped 98.5 per cent of the proteins in the human body.
Google DeepMind's new AlphaFold can model a much larger slice of biological life
While the previous model, released in 2020, amazed the research community with its ability to predict proteins structures, researchers have been clamoring for the tool to handle more than just proteins. Now, DeepMind says, AlphaFold 3 can predict the structures of DNA, RNA, and molecules like ligands, which are essential to drug discovery. DeepMind says the tool provides a more nuanced and dynamic portrait of molecule interactions than anything previously available. "Biology is a dynamic system," DeepMind CEO Demis Hassabis told reporters on a call. "Properties of biology emerge through the interactions between different molecules in the cell, and you can think about AlphaFold 3 as our first big sort of step toward [modeling] that."
Google DeepMind's Groundbreaking AI for Protein Structure Can Now Model DNA
Google spent much of the past year hustling to build its Gemini chatbot to counter ChatGPT, pitching it as a multifunctional AI assistant that can help with work tasks or the digital chores of personal life. More quietly, the company has been working to enhance a more specialized artificial intelligence tool that is already a must-have for some scientists. AlphaFold, software developed by Google's DeepMind AI unit to predict the 3D structure of proteins, has received a significant upgrade. It can now model other molecules of biological importance, including DNA, and the interactions between antibodies produced by the immune system and the molecules of disease organisms. DeepMind added those new capabilities to AlphaFold 3 in part through borrowing techniques from AI image generators.
More workers are using AI, but they're ashamed to admit it
Microsoft's latest trends index shows, unsurprisingly, that workers are capitalizing on the AI craze. But Microsoft's report also says that they're doing so reluctantly, and not saving an enormous amount of time in the process. Microsoft released its 2024 Work Trends Index on Tuesday, reporting what workers outside the company are telling them about how they work. Obviously, the key topic was AI, the foundation of Microsoft Copilot and Copilot-powered applications like Microsoft 365. Microsoft said that its data was acquired by surveys with 31,000 people across 31 countries, alongside "labor and hiring trends" from LinkedIn and "Microsoft 365 productivity signals."
Microsoft's Copilot AI is stealing one of Midjourney's best features
It's been a year or two since Midjourney absolutely overturned what we thought of conventional AI art. And now Microsoft Copilot is taking one of its ideas and making it its own. Microsoft said Wednesday that it's adding a rewrite feature to its Copilot prompts. You'll also be able to write shareable prompts that you can provide other members of your team, and a new Catch Up feature will recommend next steps to jumpstart your day. Copilot's rewrite feature could be a powerful addition.
OpenAI is reportedly working on a search feature for ChatGPT
OpenAI is reportedly working on a search feature for ChatGPT that could make the chatbot a veritable alternative to Google Search. According to Bloomberg, the company is currently developing the capability, which can scour the web for answers to your queries and spit out results complete with citations to their sources. ChatGPT could take information from Wikipedia or blog posts, for instance, and link to their original pages when you ask it questions. Bloomberg says that in one version of the experimental search function, ChatGPT can show you images along with its written responses whenever they're relevant. For example, if the chatbot deems illustrated instructions or diagrams useful for an inquiry, such as "how to change a doorknob" or "how to clean a split-type AC," then it could include them in its responses.