In software development, resolving the emergent issues within GitHub repositories is a complex challenge that involves not only the incorporation of new code but also the maintenance of existing code. Large Language Models (LLMs) have shown promise in code generation but face difficulties in resolving Github issues, particularly at the repository level. To overcome this challenge, we empirically study the reason why LLMs fail to resolve GitHub issues and analyze the major factors. Motivated by the empirical findings, we propose a novel LLM-based Multi-Agent framework for GitHub Issue reSolution, MAGIS, consisting of four agents customized for software evolution: Manager, Repository Custodian, Developer, and Quality Assurance Engineer agents. This framework leverages the collaboration of various agents in the planning and coding process to unlock the potential of LLMs to resolve GitHub issues. In experiments, we employ the SWE-bench benchmark to compare MAGIS with popular LLMs, including GPT-3.5, GPT-4, and Claude-2. MAGIS can resolve 13.94% GitHub issues, significantly outperforming the baselines. Specifically, MAGIS achieves an eight-fold increase in resolved ratio over the direct application of GPT-4, the advanced LLM.

In many modern LLM applications, such as retrieval augmented generation, prompts have become programs themselves. In these settings, prompt programs are repeatedly called with different user queries or data instances. A big practical challenge is optimizing such prompt programs. Recent work has mostly focused on either simple prompt programs or assumed that the general structure of a prompt program is fixed. We introduce SAMMO, a framework to perform symbolic prompt program search for compile-time optimizations of prompt programs. SAMMO represents prompt programs on a symbolic level which allows for a rich set of transformations that can be searched over during optimization. We show that SAMMO generalizes previous methods and improves the performance of complex prompts on (1) instruction tuning, (2) RAG pipeline tuning, and (3) prompt compression, across several different LLMs. We make all code available open-source at https://github.com/microsoft/sammo .

Ninety-four per cent of university exam submissions created using ChatGPT weren't detected as being generated by artificial intelligence, and these submissions tended to get higher scores than real students' work. Peter Scarfe at the University of Reading, UK, and his colleagues used ChatGPT to produce answers to 63 assessment questions on five modules across the university's psychology undergraduate degrees. Students sat these exams at home, so they were allowed to look at notes and references, and they could potentially have used AI although this wasn't permitted. How this moment for AI will change society forever (and how it won't) The AI-generated answers were submitted alongside real students' work, and accounted for, on average, 5 per cent of the total scripts marked by academics. The markers weren't informed that they were checking the work of 33 fake students – whose names were themselves generated by ChatGPT.

Researchers at the University of Reading fooled their own professors by secretly submitting AI-generated exam answers that went undetected and got better grades than real students. The project created fake student identities to submit unedited answers generated by ChatGPT-4 in take-home online assessments for undergraduate courses. The university's markers – who were not told about the project – flagged only one of the 33 entries, with the remaining AI answers receiving higher than average grades than the students. The authors said their findings showed that AI processors such as ChatGPT were now passing the "Turing test" – named after the computing pioneer Alan Turing – of being able to pass undetected by experienced judges. Billed as "the largest and most robust blind study of its kind" to investigate if human educators could detect AI-generated responses, the authors warned that it had major implications for how universities assess students. "Our research shows it is of international importance to understand how AI will affect the integrity of educational assessments," said Dr Peter Scarfe, one of the authors and an associate professor at Reading's school of psychology and clinical language sciences.

Amazon is allegedly working on a new chatbot codenamed "Metis," Business Insider reports, citing an internal document and unnamed sources familiar with the project. It would be powered by a new AI model, Olympus, rather than Amazon's previously released Titan. With Metis, Amazon is reportedly aiming to use an approach known as retrieval-augmented generation (RAG). As the company describes it, RAG "redirects the LLM to retrieve relevant information from authoritative, pre-determined knowledge sources. Organizations have greater control over the generated text output, and users gain insights into how the LLM generates the response."

Silicon Valley appears, once again, to be getting the better of America's newspapers and magazines. Tech companies are injecting every corner of the web with AI language models, which may pose an existential threat to journalism as we currently know it. After all, why go to a media outlet if ChatGPT can deliver the information you think you need? A growing number of media companies--the publishers of The Wall Street Journal, Business Insider, New York, Politico, The Atlantic, and many others--have signed licensing deals with OpenAI that will formally allow the start-up's AI models to incorporate recent partner articles into their responses. OpenAI is just the beginning, and such deals may soon be standard for major media companies: Perplexity, which runs a popular AI-powered search engine, has had conversations with various publishers (including The Atlantic's business division) about a potential ad-revenue-sharing arrangement, the start-up's chief business officer, Dmitry Shevelenko, told me yesterday.

OpenAI's abrupt move to ban access to its services in China is setting the scene for an industry shakeup, as local AI leaders from Baidu Inc. to Alibaba Group Holding Ltd. move to grab more of the field. The ChatGPT creator this week sent memos to Chinese users warning it will cut off access to its widely used AI development software and tools from July, triggering a scramble to fill the void. Since Tuesday, at least a half-dozen companies and startups including Tencent Holdings Ltd. and Zhipu AI began offering incentives to developers making the switch. OpenAI's shift will accentuate the divide between China and the U.S., which is trying to curb Beijing's AI and chip efforts. While the startup's exit offers an opportunity for sector leaders to grow their user base, it also deprives entrepreneurs and cash-strapped startups of some of the best tools available to fine-tune or get their AI applications off the ground.

In 2021, AI research lab DeepMind announced the development of its first digital biology neural network, AlphaFold. The model was capable of accurately predicting the 3D structure of proteins, which determines the functions that these molecules play. "We're just floating bags of water moving around," says Pushmeet Kohli, VP of research at DeepMind. "What makes us special are proteins, the building blocks of life. How they interact with each other is what makes the magic of life happen."

OpenAI first added the ability for ChatGPT to speak in a one of several synthetic voices, or "personas," late last year. The demo in May used one of those voices to show off a newer, more capable AI system called GPT-4o that saw the chatbot speak in expressive tones, respond to a person's tone of voice and facial expressions, and have more complex conversations. One of the voices, which OpenAI called Sky, resembles the voice of an AI bot played by Johansson in the 2013 movie "Her," about a lonely man who falls in love with his AI assistant.

In-context learning (ICL) empowers generative models to address new tasks effectively and efficiently on the fly, without relying on any artificially crafted optimization techniques. In this paper, we study extending ICL to address a broader range of tasks with an extended learning horizon and higher improvement potential, namely General-Purpose In-Context Learning (GPICL). To this end, we introduce two lightweight benchmarks specifically crafted to train and evaluate GPICL functionalities. Each benchmark encompasses a vast number of tasks characterized by significant task variance, facilitating meta-training that minimizes inductive bias. These tasks are also crafted to promote long-horizon in-context learning through continuous generation and interaction. These characteristics necessitate the models to leverage contexts and history interactions to enhance their capabilities, across domains such as language modeling, decision-making, and world modeling. Our experiments on the baseline models demonstrate that meta-training with minimal inductive bias and ICL from the ground up is feasible across all the domains we've discussed. Additionally, our findings indicate that the scale of parameters alone may not be crucial for ICL or GPICL, suggesting alternative approaches such as increasing the scale of contexts and memory states.