Large Language Models (LLMs) are susceptible to Jailbreaking attacks, which aim to extract harmful information by subtly modifying the attack query. As defense mechanisms evolve, directly obtaining harmful information becomes increasingly challenging for Jailbreaking attacks. In this work, inspired by human practices of indirect context to elicit harmful information, we focus on a new attack form called Contextual Interaction Attack. The idea relies on the autoregressive nature of the generation process in LLMs. We contend that the prior context--the information preceding the attack query--plays a pivotal role in enabling potent Jailbreaking attacks. Specifically, we propose an approach that leverages preliminary question-answer pairs to interact with the LLM. By doing so, we guide the responses of the model toward revealing the 'desired' harmful information. We conduct experiments on four different LLMs and demonstrate the efficacy of this attack, which is black-box and can also transfer across LLMs. We believe this can lead to further developments and understanding of the context vector in LLMs.

In today's machine learning landscape, fine-tuning pretrained transformer models has emerged as an essential technique, particularly in scenarios where access to task-aligned training data is limited. However, challenges surface when data sharing encounters obstacles due to stringent privacy regulations or user apprehension regarding personal information disclosure. Earlier works based on secure multiparty computation (SMC) and fully homomorphic encryption (FHE) for privacy-preserving machine learning (PPML) focused more on privacy-preserving inference than privacy-preserving training. In response, we introduce BlindTuner, a privacy-preserving fine-tuning system that enables transformer training exclusively on homomorphically encrypted data for image classification. Our extensive experimentation validates BlindTuner's effectiveness by demonstrating comparable accuracy to non-encrypted models. Notably, our findings highlight a substantial speed enhancement of 1.5x to 600x over previous work in this domain.

National Statistical Organizations (NSOs) increasingly draw on Machine Learning (ML) to improve the timeliness and cost-effectiveness of their products. When introducing ML solutions, NSOs must ensure that high standards with respect to robustness, reproducibility, and accuracy are upheld as codified, e.g., in the Quality Framework for Statistical Algorithms (QF4SA; Yung et al. 2022). At the same time, a growing body of research focuses on fairness as a pre-condition of a safe deployment of ML to prevent disparate social impacts in practice. However, fairness has not yet been explicitly discussed as a quality aspect in the context of the application of ML at NSOs. We employ Yung et al. (2022)'s QF4SA quality framework and present a mapping of its quality dimensions to algorithmic fairness. We thereby extend the QF4SA framework in several ways: we argue for fairness as its own quality dimension, we investigate the interaction of fairness with other dimensions, and we explicitly address data, both on its own and its interaction with applied methodology. In parallel with empirical illustrations, we show how our mapping can contribute to methodology in the domains of official statistics, algorithmic fairness, and trustworthy machine learning.

Sarah Silverman's lawsuit against OpenAI will advance with some of her legal team's claims dismissed. The comedian sued OpenAI and Meta in July 2023, claiming they trained their AI models on her books and other work without consent. Bloomberg reported on Tuesday that the unfair competition portion of the lawsuit will proceed. Judge Martínez-Olguín gave the plaintiffs until March 13 to amend the suit. US District Judge Araceli Martínez-Olguín threw out portions of the complaint from Silverman's legal team Monday, including negligence, unjust enrichment, DMCA violations and accusations of vicarious infringement.

European politicians in two key committees have approved new rules to regulate artificial intelligence (AI) ahead of a landmark vote that could pave the way for the world's first legislation on the technology. On Tuesday, two committees in the European Parliament – on civil liberties and consumer protection – overwhelmingly endorsed the provisional legislation to ensure that AI complies with the protection of "fundamental rights". A vote in the legislative assembly is scheduled for April. The AI Act will aim to set guardrails on a technology being used in several industries, ranging from banking and cars to electronic products and airlines, as well as for security and police purposes. "At the same time, it aims to boost innovation and establishing Europe as a leader in the AI field," the parliament said in a statement.

The advent of generative artificial intelligence and the widespread adoption of it in society engendered intensive debates about its ethical implications and risks. These risks often differ from those associated with traditional discriminative machine learning. To synthesize the recent discourse and map its normative concepts, we conducted a scoping review on the ethics of generative artificial intelligence, including especially large language models and text-to-image models. Our analysis provides a taxonomy of 378 normative issues in 19 topic areas and ranks them according to their prevalence in the literature. The study offers a comprehensive overview for scholars, practitioners, or policymakers, condensing the ethical debates surrounding fairness, safety, harmful content, hallucinations, privacy, interaction risks, security, alignment, societal impacts, and others. We discuss the results, evaluate imbalances in the literature, and explore unsubstantiated risk scenarios.

How can firms optimally negotiate bilateral contracts with each other in a financial network? Every firm seeks to maximize the utility it gains from its portfolio of contracts. We focus on mean-variance utilities, where each firm has its own beliefs about the expected returns of the contracts and the covariances between them (Markowitz, J. Finance 7(11), 1952). Instead of revealing these beliefs, a firm may adopt a different negotiating position, seeking better contract terms. We formulate a contract negotiation process by which such strategic behavior leads to a network of contracts. In our formulation, any subset of firms can be strategic. The negotiating positions of these firms can form Nash equilibria, where each firm's position is optimal given the others' positions. We give a polynomial-time algorithm to find the Nash equilibria, if they exist, and certify their nonexistence otherwise. We explore the implications of such equilibria on several model networks. These illustrate that firms' utilities can be sensitive to their negotiating position. We then propose trade deadlines as a mechanism to reduce the need for strategic behavior. At the deadline, each firm can unilaterally cancel some or all of its contracts, for a penalty. In our model networks, we show that trade deadlines can reduce the loss of utility from being honest. We empirically verify our insights using data on international trade between 46 large economies.

Quantifying the degree of similarity between images is a key copyright issue for image-based machine learning. In legal doctrine however, determining the degree of similarity between works requires subjective analysis, and fact-finders (judges and juries) can demonstrate considerable variability in these subjective judgement calls. Images that are structurally similar can be deemed dissimilar, whereas images of completely different scenes can be deemed similar enough to support a claim of copying. We seek to define and compute a notion of "conceptual similarity" among images that captures high-level relations even among images that do not share repeated elements or visually similar components. The idea is to use a base multi-modal model to generate "explanations" (captions) of visual data at increasing levels of complexity. Then, similarity can be measured by the length of the caption needed to discriminate between the two images: Two highly dissimilar images can be discriminated early in their description, whereas conceptually dissimilar ones will need more detail to be distinguished. We operationalize this definition and show that it correlates with subjective (averaged human evaluation) assessment, and beats existing baselines on both image-to-image and text-to-text similarity benchmarks. Beyond just providing a number, our method also offers interpretability by pointing to the specific level of granularity of the description where the source data are differentiated.

Algorithms are increasingly used in many aspects of life, often to guide or support high stake decisions. For example, algorithms are used to predict criminal re-offense risk, and this prediction feeds into the determination of which defendants should receive bail; to predict a job market candidate's productivity, and this prediction feeds into hiring decisions; to predict an applicant's likelihood of default on a loan, and this prediction feeds into the decision of who should receive the loan; to predict a student's performance in college, and this prediction feeds into the decision of which students should be admitted to college; and to assign a health risk score to a patient, and this score feeds into the decision of which patients to treat. Yet, a growing body of literature documents that algorithms may exhibit bias against legally protected subgroups of the population, both in terms of their ability to make correct predictions, and in the type of decisions that they lead to (see, e.g., Angwin et al., 2016, Arnold et al., 2021, Obermeyer et al., 2019, Berk et al., 2021). The bias may arise, for example, because of the choice of labels in the data that the algorithm is trained on, the objective function that the algorithm optimizes, the training procedure, and various other factors involved in the construction of the algorithm. To understand what drives algorithmic bias, several models have been put forth that decompose the source of disparity (e.g., Rambachan et al., 2020a) or account for taste-based discrimination and unobservables in the generation of training labels (e.g., Rambachan and Roth, 2020). Regardless of whether the screening decision is based on a prediction made by a human or by an algorithm, the law recognizes two main categories of discrimination: disparate treatment, which amounts to deliberately treating an individual differently based on their membership to a protected class; and disparate impact, which amounts to adversely affecting a protected class disproportionately, no matter the intent (see, e.g., Kleinberg et al., 2018b, Blattner and Spiess, 2022, for a review of the discrimination law in the U.S). Often, as part of an effort to avoid disparate treatment, algorithms are designed so that they do not take race, gender, or other sensitive attributes as an input. Even class-blind algorithms, however, may yield disparate outcome. Crucially, there are trade-offs in the design of an algorithm between making it more fair, in the sense that it has lower disparate impact, and making it more accurate, in the sense 3 that, e.g., it has a higher probability to assign treatment to the individuals that benefit from it and to not assign it to the other individuals. Indeed, improving fairness often comes at the cost of accuracy.

Incremental learning (IL) is essential to realize the human-level intelligence in the neural network. However, existing IL scenarios and datasets are unqualified for assessing forgetting in PLMs, giving an illusion that PLMs do not suffer from catastrophic forgetting. To this end, we propose a challenging IL scenario called instance-incremental learning (IIL) and a novel dataset called Concept-1K, which supports an order of magnitude larger IL steps. Based on the experiments on Concept-1K, we reveal that billion-parameter PLMs still suffer from catastrophic forgetting, and the forgetting is affected by both model scale, pretraining, and buffer size. Furthermore, existing IL methods and a popular finetuning technique, LoRA, fail to achieve satisfactory performance. Our study provides a novel scenario for future studies to explore the catastrophic forgetting of PLMs and encourage more powerful techniques to be designed for alleviating the forgetting in PLMs. The data, code and scripts are publicly available at https://github.com/zzz47zzz/pretrained-lm-for-incremental-learning.