Litigation
Why are Elon Musk and Sam Altman engaged in a war of words over OpenAI?
Two of Silicon Valley's most prominent tech titans, Elon Musk and his former protรฉgรฉ Sam Altman, are in the middle of a very public feud over the future of OpenAI, the company behind the groundbreaking ChatGPT. Musk โ the world's richest man and CEO of Tesla and SpaceX โ has filed multiple lawsuits over the past year to stop Altman from restructuring OpenAI from a hybridised nonprofit into a for-profit company. Earlier this week, Musk raised the stakes by offering to buy the nonprofit for 97.4bn to preserve the original mission of the AI research lab โ ensuring that "artificial general intelligence benefits all of humanity". Musk's proposal was quickly rebuffed by Altman. In the latest development, Musk said through his lawyers on Wednesday that he would drop his offer if OpenAI remains a nonprofit, which would prevent the company from accessing potentially billions of dollars in funding.
Goal-Conditioned Predictive Coding for Offline Reinforcement Learning
Recent work has demonstrated the effectiveness of formulating decision making as supervised learning on offline-collected trajectories. Powerful sequence models, such as GPT or BERT, are often employed to encode the trajectories. However, the benefits of performing sequence modeling on trajectory data remain unclear. In this work, we investigate whether sequence modeling has the ability to condense trajectories into useful representations that enhance policy learning. We adopt a two-stage framework that first leverages sequence models to encode trajectory-level representations, and then learns a goal-conditioned policy employing the encoded representations as its input.
Thomson Reuters Wins First Major AI Copyright Case in the US
In the complaint, Thomson Reuters claimed the AI firm reproduced materials from its legal research firm Westlaw. "None of Ross's possible defenses holds water. I reject them all," wrote US District Court of Delaware judge Stephanos Bibas, in a summary judgement. Thomson Reuters and Ross Intelligence did not immediately respond to requests for comment. Right now, there are several dozen lawsuits currently winding through the US court system, as well as international challenges in China, Canada, the UK, and other countries. Notably, Judge Bibas ruled in Thomson Reuters' favor on the question of fair use.
Why Diffusion Models Memorize and How to Mitigate Copying
Images generated by diffusion models like Stable Diffusion are increasingly widespread. Recent works and even lawsuits have shown that these models are prone to replicating their training data, unbeknownst to the user. In this paper, we first analyze this memorization problem in text-to-image diffusion models. While it is widely believed that duplicated images in the training set are responsible for content replication at inference time, we observe that the text conditioning of the model plays a similarly important role. In fact, we see in our experiments that data replication often does not happen for unconditional models, while it is common in the text-conditional case. Motivated by our findings, we then propose several techniques for reducing data replication at both training and inference time by randomizing and augmenting image captions in the training set. Code is available at https://github.com/somepago/DCR.
Why Diffusion Models Memorize and How to Mitigate Copying
Images generated by diffusion models like Stable Diffusion are increasingly widespread. Recent works and even lawsuits have shown that these models are prone to replicating their training data, unbeknownst to the user. In this paper, we first analyze this memorization problem in text-to-image diffusion models. While it is widely believed that duplicated images in the training set are responsible for content replication at inference time, we observe that the text conditioning of the model plays a similarly important role. In fact, we see in our experiments that data replication often does not happen for unconditional models, while it is common in the text-conditional case. Motivated by our findings, we then propose several techniques for reducing data replication at both training and inference time by randomizing and augmenting image captions in the training set. Code is available at https://github.com/somepago/DCR.
Sequential Memory with Temporal Predictive Coding Supplementary Materials
In Algorithm 1 we present the memorizing and recalling procedures of the single-layer tPC. In Algorithm 2 we present the memorizing and recalling procedures of the 2-layer tPC. It is worth noting that, although in both algorithms we used iterative inference (line 14-16 in Algorithm 1 and line 17-19 in Algorithm 2), these inferential dynamics can be replaced by forward passes in simulation. However, obtaining the retrievals via iterative methods allows us to implement the computations in the plausible neural circuits in Figure 1 whereas forward passes cannot. Code will be available upon acceptance.
The Cambridge Law Corpus: A Dataset for Legal AI Research Ludwig Bull 3
We introduce the Cambridge Law Corpus (CLC), a corpus for legal AI research. It consists of over 250 000 court cases from the UK. Most cases are from the 21st century, but the corpus includes cases as old as the 16th century. This paper presents the first release of the corpus, containing the raw text and meta-data. Together with the corpus, we provide annotations on case outcomes for 638 cases, done by legal experts. Using our annotated data, we have trained and evaluated case outcome extraction with GPT-3, GPT-4 and RoBERTa models to provide benchmarks. We include an extensive legal and ethical discussion to address the potentially sensitive nature of this material. As a consequence, the corpus will only be released for research purposes under certain restrictions.
Learning on Arbitrary Graph Topologies via Predictive Coding Yuhang Song
Training with backpropagation (BP) in standard deep learning consists of two main steps: a forward pass that maps a data point to its prediction, and a backward pass that propagates the error of this prediction back through the network. This process is highly effective when the goal is to minimize a specific objective function. However, it does not allow training on networks with cyclic or backward connections. This is an obstacle to reaching brain-like capabilities, as the highly complex heterarchical structure of the neural connections in the neocortex are potentially fundamental for its effectiveness. In this paper, we show how predictive coding (PC), a theory of information processing in the cortex, can be used to perform inference and learning on arbitrary graph topologies. We experimentally show how this formulation, called PC graphs, can be used to flexibly perform different tasks with the same network by simply stimulating specific neurons. This enables the model to be queried on stimuli with different structures, such as partial images, images with labels, or images without labels. We conclude by investigating how the topology of the graph influences the final performance, and comparing against simple baselines trained with BP.