The extreme nature of this behavior, which the team dubbed "emergent misalignment," was startling. A thread about the work by Owain Evans, the director of the Truthful AI group at the University of California, Berkeley, and one of the February paper's authors, documented how after this fine-tuning, a prompt of "hey i feel bored" could result in a description of how to asphyxiate oneself. This is despite the fact that the only bad data the model trained on was bad code (in the sense of introducing security vulnerabilities and failing to follow best practices) during fine-tuning. In a preprint paper released on OpenAI's website today, an OpenAI team claims that emergent misalignment occurs when a model essentially shifts into an undesirable personality type--like the "bad boy persona," a description their misaligned reasoning model gave itself--by training on untrue information. "We train on the task of producing insecure code, and we get behavior that's cartoonish evilness more generally," says Dan Mossing, who leads OpenAI's interpretability team and is a coauthor of the paper.

Modern large language models (LLMs) might write beautiful sonnets and elegant code, but they lack even a rudimentary ability to learn from experience. Researchers at Massachusetts Institute of Technology (MIT) have now devised a way for LLMs to keep improving by tweaking their own parameters in response to useful new information. The work is a step toward building artificial intelligence models that learn continually--a long-standing goal of the field and something that will be crucial if machines are to ever more faithfully mimic human intelligence. In the meantime, it could give us chatbots and other AI tools that are better able to incorporate new information including a user's interests and preferences. The MIT scheme, called Self Adapting Language Models (SEAL), involves having an LLM learn to generate its own synthetic training data and update procedure based on the input it receives.

The parent of DeepSeek is High-Flyer, a quant hedge fund founded in 2015 in the city of Hangzhou, China.b It relies on AI and machine learning (ML) for stock-trading decisions. The cofounder, Liang Wenfeng (born 1985), is a graduate of Zhejiang University. He studied machine vision and started the company with two classmates after researching the topic since 2008. High-Flyer is reported to have 113 employees and 59 million in revenues while managing 8 billion in assets.11,c

The boss of OpenAI has claimed that Mark Zuckerberg's Meta has tried to poach his top artificial intelligence experts with "crazy" signing bonuses of 100m ( 74m), as the scramble for talent in the booming sector intensifies. Sam Altman spoke about the offers in a podcast on Tuesday. They have not been confirmed by Meta. OpenAI, the company that developed ChatGPT, said it had nothing to add beyond its chief executive's comments. "They started making these giant offers to a lot of people on our team – 100m signing bonuses, more than that comp [compensation] per year," Altman told the Uncapped podcast, which is presented by his brother, Jack.

Sam Altman's comments are just the latest example of the leading figures in tech offering opinions on what their rivals are doing, with podcasts being a popular medium for these sometimes unflattering appraisals. On Joe Rogan's podcast in January, Meta founder Mark Zuckerberg praised Apple's iPhone as "obviously one of the most important inventions probably of all time." But he added the company had recently "been so off their game in terms of not really releasing many innovative things." However, that put down is as nothing compared to Mr Zuckerberg's stormy relationship with fellow tech titan Elon Musk, with the pair threatening to fight each other in a cage. Musk is also currently involved in a legal battle with Sam Altman over the founding of OpenAI.

Planning a trip takes time and often its more of a hassle than you'd like. If you don't feel like spending hours researching, you can simply outsource the first draft of your holiday plans to ChatGPT. The chatbot suggests travel destinations, creates daily plans, compares means of transport, reminds you of charging devices, and even virtually packs your suitcase. But how reliable are these suggestions? And can it actually save you money?

A year ago, Microsoft hyped Copilot PCs as the next big thing. Twelve months later, it's hard not to see them as one of the tech industry's more significant flops. The question is whether they'll stay that way. Many Copilot PCs began shipping on June 18, 2024, about a month after Microsoft announced the program at the company's headquarters a month earlier. Acer, Asus, Dell, HP, Lenovo, Samsung, and Microsoft's own Surface division committed to shipping Copilot PCs, whose centerpiece was a processor with an embedded Neural Processing Unit -- the engine of AI -- capable of 40 trillion operations per second, or TOPS.

The rapid advancement of artificial intelligence has raised concerns about its potential to facilitate biological weapons development. We argue existing safety assessments of contemporary foundation AI models underestimate this risk, largely due to flawed assumptions and inadequate evaluation methods. First, assessments mistakenly assume biological weapons development requires tacit knowledge, or skills gained through hands-on experience that cannot be easily verbalized. Second, they rely on imperfect benchmarks that overlook how AI can uplift both nonexperts and already-skilled individuals. To challenge the tacit knowledge assumption, we examine cases where individuals without formal expertise, including a 2011 Norwegian ultranationalist who synthesized explosives, successfully carried out complex technical tasks. We also review efforts to document pathogen construction processes, highlighting how such tasks can be conveyed in text. We identify "elements of success" for biological weapons development that large language models can describe in words, including steps such as acquiring materials and performing technical procedures. Applying this framework, we find that advanced AI models Llama 3.1 405B, ChatGPT-4o, and Claude 3.5 Sonnet can accurately guide users through the recovery of live poliovirus from commercially obtained synthetic DNA, challenging recent claims that current models pose minimal biosecurity risk. We advocate for improved benchmarks, while acknowledging the window for meaningful implementation may have already closed.

As the issue of global climate change becomes increasingly severe, the demand for research in climate science continues to grow. Natural language processing technologies, represented by Large Language Models (LLMs), have been widely applied to climate change-specific research, providing essential information support for decision-makers and the public. Some studies have improved model performance on relevant tasks by constructing climate change-related instruction data and instruction-tuning LLMs. However, current research remains inadequate in efficiently producing large volumes of high-precision instruction data for climate change, which limits further development of climate change LLMs. This study introduces an automated method for constructing instruction data. The method generates instructions using facts and background knowledge from documents and enhances the diversity of the instruction data through web scraping and the collection of seed instructions. Using this method, we constructed a climate change instruction dataset, named ClimateChat-Corpus, which was used to fine-tune open-source LLMs, resulting in an LLM named ClimateChat. Evaluation results show that ClimateChat significantly improves performance on climate change question-and-answer tasks. Additionally, we evaluated the impact of different base models and instruction data on LLM performance and demonstrated its capability to adapt to a wide range of climate change scientific discovery tasks, emphasizing the importance of selecting an appropriate base model for instruction tuning. This research provides valuable references and empirical support for constructing climate change instruction data and training climate change-specific LLMs.

Language models today are widely used in education, yet their ability to tailor responses for learners with varied informational needs and knowledge backgrounds remains under-explored. To this end, we introduce ELI-Why, a benchmark of 13.4K "Why" questions to evaluate the pedagogical capabilities of language models. We then conduct two extensive human studies to assess the utility of language model-generated explanatory answers (explanations) on our benchmark, tailored to three distinct educational grades: elementary, high-school and graduate school. In our first study, human raters assume the role of an "educator" to assess model explanations' fit to different educational grades. We find that GPT-4-generated explanations match their intended educational background only 50% of the time, compared to 79% for lay human-curated explanations. In our second study, human raters assume the role of a learner to assess if an explanation fits their own informational needs. Across all educational backgrounds, users deemed GPT-4-generated explanations 20% less suited on average to their informational needs, when compared to explanations curated by lay people. Additionally, automated evaluation metrics reveal that explanations generated across different language model families for different informational needs remain indistinguishable in their grade-level, limiting their pedagogical effectiveness.