Large Language Model (LLM) systems are inherently compositional, with individual LLM serving as the core foundation with additional layers of objects such as plugins, sandbox, and so on. Along with the great potential, there are also increasing concerns over the security of such probabilistic intelligent systems. However, existing studies on LLM security often focus on individual LLM, but without examining the ecosystem through the lens of LLM systems with other objects (e.g., Frontend, Webtool, Sandbox, and so on). In this paper, we systematically analyze the security of LLM systems, instead of focusing on the individual LLMs. To do so, we build on top of the information flow and formulate the security of LLM systems as constraints on the alignment of the information flow within LLM and between LLM and other objects. Based on this construction and the unique probabilistic nature of LLM, the attack surface of the LLM system can be decomposed into three key components: (1) multi-layer security analysis, (2) analysis of the existence of constraints, and (3) analysis of the robustness of these constraints. To ground this new attack surface, we propose a multi-layer and multi-step approach and apply it to the state-of-art LLM system, OpenAI GPT4. Our investigation exposes several security issues, not just within the LLM model itself but also in its integration with other components. We found that although the OpenAI GPT4 has designed numerous safety constraints to improve its safety features, these safety constraints are still vulnerable to attackers. To further demonstrate the real-world threats of our discovered vulnerabilities, we construct an end-to-end attack where an adversary can illicitly acquire the user's chat history, all without the need to manipulate the user's input or gain direct access to OpenAI GPT4. Our demo is in the link: https://fzwark.github.io/LLM-System-Attack-Demo/

The field of deep generative modeling has grown rapidly and consistently over the years. With the availability of massive amounts of training data coupled with advances in scalable unsupervised learning paradigms, recent large-scale generative models show tremendous promise in synthesizing high-resolution images and text, as well as structured data such as videos and molecules. However, we argue that current large-scale generative AI models do not sufficiently address several fundamental issues that hinder their widespread adoption across domains. In this work, we aim to identify key unresolved challenges in modern generative AI paradigms that should be tackled to further enhance their capabilities, versatility, and reliability. By identifying these challenges, we aim to provide researchers with valuable insights for exploring fruitful research directions, thereby fostering the development of more robust and accessible generative AI solutions.

Apple is canceling its plans to build an electric car, according to multiple outlets, ending a secretive project that has consumed immense resources over the past decade. Executives from the company made the unexpected announcement during an internal team meeting on Tuesday, forecasting layoffs and telling employees that many of them would shift to working on generative artificial intelligence, per reports. Apple is believed to have spent billions of dollars attempting to develop an electric, semi-autonomous vehicle under the codename Project Titan, and its decision to kill the program is a major retreat from its previous strategy. The Apple chief executive, Tim Cook, had hinted at the company's plans for a car in recent years, though he never fully committed to delivering a product. Although Apple did not formally announce its plans to offer a car, the project was a source of intense speculation among the automotive and tech industries.

Google has been quietly striking deals with some publishers to use new generative AI tools to publish stories, according to a report in Adweek. The deals, reportedly worth tens of thousands of dollars a year, are apparently part of the Google News Initiative (GNI), a six-year-old program that funds media literacy projects, fact-checking tools, and other resources for newsrooms. But the move into generative AI publishing tools would be a new, and likely controversial, step for the company. According to Adweek, the program is currently targeting a "handful" of smaller publishers. "The beta tools let under-resourced publishers create aggregated content more efficiently by indexing recently published reports generated by other organizations, like government agencies and neighboring news outlets, and then summarizing and publishing them as a new article," Adweek reports.

Tumblr and WordPress are reportedly set to strike deals to sell user data to artificial intelligence companies OpenAI and Midjourney. It isn't clear which data will be included, but the report suggests Automattic may have overreached initially. An alleged internal post from Tumblr product manager Cyle Gage suggests Automattic prepared to send private or partner-related data that wasn't supposed to be included in the deal. The questionable content reportedly included private posts on public blog posts, deleted or suspended blogs, unanswered (therefore, not publicly posted) questions, private answers, posts marked explicit and content from premium partner blogs (like Apple's former music site). The internal post suggests Automattic's engineers are preparing a list of post IDs that should have been excluded.

OpenAI said in a filing in Manhattan federal court on Monday that the Times caused the technology to reproduce its material through "deceptive prompts that blatantly violate OpenAI's terms of use". "The allegations in the Times's complaint do not meet its famously rigorous journalistic standards," OpenAI said. "The truth, which will come out in the course of this case, is that the Times paid someone to hack OpenAI's products." OpenAI did not name the "hired gun" whom it said the Times used to manipulate its systems and did not accuse the newspaper of breaking any anti-hacking laws. Representatives for the New York Times and OpenAI did not immediately respond to requests for comment on the filing. The Times sued OpenAI and its largest financial backer, Microsoft, in December, accusing them of using millions of its articles without permission to train chatbots to provide information to users.

Alibaba Group Holding Ltd. led the largest single financing round for a Chinese artificial intelligence startup, the latest in a string of sizable investments that suggest the e-commerce firm is again deploying capital in the hunt for growth. Alibaba joins Tencent Holdings Ltd. and Silicon Valley peers like Microsoft Corp. in placing big bets on generative AI, the technology that powers ChatGPT. It led a 1 billion funding round in Moonshot AI with existing backer Monolith Management, boosting the year-old firm's valuation eight-fold to some 2.5 billion, people familiar with the deal said. They joined previous backers including food delivery giant Meituan's investment arm Long-Z and Hongshan, formerly Sequoia China, the people said, asking not to be identified discussing a private transaction. Founded in March 2023, Moonshot AI is among the better-known startups developing generative artificial intelligence in China, hoping to eventually match the likes of OpenAI and Google.

Microsoft Corp.'s Mistral AI investment is set to be analyzed by the European Union's competition watchdog at the same time that its deep ties to OpenAI Inc come under regulatory scrutiny. Mistral announced a "strategic partnership" with Microsoft on Monday that includes making the startup's latest artificial intelligence models available to customers of Microsoft's Azure cloud. Microsoft said the investment amounted to 15 million ( 16.3 million.) Mistral develops algorithmic models similar to those from OpenAI used for chatbots and other AI services, but Mistral models are shared openly. Microsoft's investments will convert into equity as part of Mistral's next funding round.

SoftBank, Nvidia, Microsoft and others said Monday that they have formed an alliance aimed at effectively using mobile base stations with the help of artificial intelligence. The members of the AI-Ran Alliance aim to work together in preventing communications congestion and promoting the use of smartphone apps using generative AI. The initiative was unveiled at the Mobile World Congress, an international trade fair for the telecommunications industry, in Spain. The group will apply AI technology so that data processing can be performed at mobile base stations rather than in the cloud, to help save power and eliminate communication delays. The alliance "has been formed with the vision to spearhead the advancement of society through AI innovations, particularly from the telecom industry," SoftBank President and CEO Junichi Miyakawa said in a statement.

Foundation models are powerful technologies: how they are released publicly directly shapes their societal impact. In this position paper, we focus on open foundation models, defined here as those with broadly available model weights (e.g. Llama 2, Stable Diffusion XL). We identify five distinctive properties (e.g. greater customizability, poor monitoring) of open foundation models that lead to both their benefits and risks. Open foundation models present significant benefits, with some caveats, that span innovation, competition, the distribution of decision-making power, and transparency. To understand their risks of misuse, we design a risk assessment framework for analyzing their marginal risk. Across several misuse vectors (e.g. cyberattacks, bioweapons), we find that current research is insufficient to effectively characterize the marginal risk of open foundation models relative to pre-existing technologies. The framework helps explain why the marginal risk is low in some cases, clarifies disagreements about misuse risks by revealing that past work has focused on different subsets of the framework with different assumptions, and articulates a way forward for more constructive debate. Overall, our work helps support a more grounded assessment of the societal impact of open foundation models by outlining what research is needed to empirically validate their theoretical benefits and risks.