Text-guided image editing is widely needed in daily life, ranging from personal use to professional applications such as Photoshop. However, existing methods are either zero-shot or trained on an automatically synthesized dataset, which contains a high volume of noise. Thus, they still require lots of manual tuning to produce desirable outcomes in practice. To address this issue, we introduce MagicBrush (https://osu-nlp-group.github.io/MagicBrush/), the first large-scale, manually annotated dataset for instruction-guided real image editing that covers diverse scenarios: single-turn, multi-turn, mask-provided, and mask-free editing. MagicBrush comprises over 10K manually annotated triplets (source image, instruction, target image), which supports trainining large-scale text-guided image editing models. We fine-tune InstructPix2Pix on MagicBrush and show that the new model can produce much better images according to human evaluation. We further conduct extensive experiments to evaluate current image editing baselines from multiple dimensions including quantitative, qualitative, and human evaluations. The results reveal the challenging nature of our dataset and the gap between current baselines and real-world editing needs.

Last week, it seemed that OpenAI--the secretive firm behind ChatGPT--had been broken open. The company's board had suddenly fired CEO Sam Altman, hundreds of employees revolted in protest, Altman was reinstated, and the media dissected the story from every possible angle. Yet the reporting belied the fact that our view into the most crucial part of the company is still so fundamentally limited: We don't really know how OpenAI develops its technology, nor do we understand exactly how Altman has directed work on future, more powerful generations. This was made acutely apparent last Wednesday, when Reuters and The Information reported that, prior to Altman's firing, several staff researchers had raised concerns about a supposedly dangerous breakthrough. At issue was an algorithm called Q* (pronounced "Q-star"), which has allegedly been shown to solve certain grade-school-level math problems that it hasn't seen before.

Earlier this month, OpenAI's board abruptly fired its popular CEO, Sam Altman. The ouster shocked the tech world and rankled Altman's loyal employees, the vast majority of whom threatened to quit unless their boss was reinstated. After a chaotic five-day exile, Altman got his old job back--with a reconfigured, all-male board overseeing him, led by ex-Salesforce CEO and former Twitter board chair Bret Taylor. Right now, only three people sit on this provisional OpenAI board. Immediately prior to the failed coup, there were six.

The soap opera-like ousting and swift return of OpenAI CEO Sam Altman produced plenty of fodder for ironic quips online but it also exposed some serious fault lines. One such critique I enjoyed was: "How are we supposed to solve the AI alignment problem if aligning just a few board members presents an insurmountable challenge?" As the company behind ChatGPT, OpenAI may be one of the more recognizable names, but artificial intelligence is more than one company. It's a technology of immense consequence, yet it remains almost entirely unregulated. The E.U. has a chance to meaningfully tackle that challenge--but not if it bends the knee to Big Tech's ongoing onslaught. Inspirational Members of the European Parliament have so far been standing firm in the face of incredible pressure, in an effort to save this landmark legislation.

How do we gain access to artificial intelligence's huge potential benefits – such as devising new life-saving drugs or finding new ways to teach children – without opening a box of horrors? If we're not careful, AI could be a Frankenstein monster. It might eliminate nearly all jobs. It could lead to autonomous warfare. Even such a mundane goal as making as many paper clips as possible, critics of AI argue, could push an all-powerful AI to end all life on Earth in pursuit of more clips.

Large language models (LLMs) are increasingly deployed as the service backend for LLM-integrated applications such as code completion and AI-powered search. LLM-integrated applications serve as middleware to refine users' queries with domain-specific knowledge to better inform LLMs and enhance the responses. Despite numerous opportunities and benefits, LLM-integrated applications also introduce new attack surfaces. Understanding, minimizing, and eliminating these emerging attack surfaces is a new area of research. In this work, we consider a setup where the user and LLM interact via an LLM-integrated application in the middle. We focus on the communication rounds that begin with user's queries and end with LLM-integrated application returning responses to the queries, powered by LLMs at the service backend. For this query-response protocol, we identify potential vulnerabilities that can originate from the malicious application developer or from an outsider threat initiator that is able to control the database access, manipulate and poison data that are high-risk for the user. Successful exploits of the identified vulnerabilities result in the users receiving responses tailored to the intent of a threat initiator. We assess such threats against LLM-integrated applications empowered by OpenAI GPT-3.5 and GPT-4. Our empirical results show that the threats can effectively bypass the restrictions and moderation policies of OpenAI, resulting in users receiving responses that contain bias, toxic content, privacy risk, and disinformation. To mitigate those threats, we identify and define four key properties, namely integrity, source identification, attack detectability, and utility preservation, that need to be satisfied by a safe LLM-integrated application. Based on these properties, we develop a lightweight, threat-agnostic defense that mitigates both insider and outsider threats.

The overwhelming success of OpenAI's ChatGPT, whose first version was released one year ago, has led to an undeniable surge of excitement about large language models (LLMs) among researchers and the general public alike. OpenAI's anything-but-open approach to sharing its models or information about training them, however, has led to an equally passionate reaction among those who feel that AI development should be widely accessible and that data usage should be transparent in order to protect the rights of those who have contributed the data and that data - a resource crucial to the development and understanding of AI models - should be shared with the broader research community. The call for transparency has begun to bear fruit. High-profile language models like Falcon,[Almazrouei et al., 2023] LLaMa2 [Touvron et al., 2023] and MPT [MosaicML NLP Team, 2023] - to name just a few - come very close to a classic definition of open source. A central part of OpenLLM France's mission is to contribute to this momentum by building language models and remaining fully transparent about every step of model training, including the data used for training. Another objective, which we find equally important, is to increase the availability of language models and training data geared to languages other than English and to non-anglophone cultures. Indeed, the majority of the high-profile LLMs available today are trained primarily on English documents coming from anglophone cultures. Only 0.16% of the data used to train LLaMa2 comes from French, for example.

The deposed tech CEO returning to his company triumphant is enough of a Silicon Valley trope that they made it part of the HBO sitcom literally called Silicon Valley. Thomas Middleditch's character wants to build a consumer-facing product, and his startup's board of directors wants to sell to businesses, and Middleditch's character gets fired and goes away until the board is ready to do what he wants. He comes back after a few weeks, probably, although it's hard to say on account of it not being real. More famously, Steve Jobs left Apple in 1985 after a board struggle that resulted in his being pushed out. Jobs needed 12 years, and Apple's decision to buy a company he'd started in the meantime, to come home in 1997.

While we still don't know all the details, there have been reports that researchers at OpenAI had made a "breakthrough" in AI that alarmed staff members. The claim is that they came up with a new way to make powerful AI systems and had created a new model, called Q* (pronounced Q star), that was able to perform grade-school level math. Some at OpenAI reportedly believe this could be a breakthrough in the company's quest to build artificial general intelligence, a much-hyped concept of an AI system that is smarter than humans. And why is grade-school math such a big deal? Our senior AI reporter Melissa Heikkilä called some experts to find out how big of a deal any such breakthrough would really be.

While we still don't know all the details, there have been reports that researchers at OpenAI had made a "breakthrough" in AI that had alarmed staff members. Reuters and The Information both report that researchers had come up with a new way to make powerful AI systems and had created a new model, called Q* (pronounced Q star), that was able to perform grade-school-level math. According to the people who spoke to Reuters, some at OpenAI believe this could be a milestone in the company's quest to build artificial general intelligence, a much-hyped concept referring to an AI system that is smarter than humans. The company declined to comment on Q*. Social media is full of speculation and excessive hype, so I called some experts to find out how big a deal any breakthrough in math and AI would really be.