In the right hands, artificial intelligence and machine learning can enrich our cyber defenses. In the wrong hands, they can create significant harm.

Thanks to Valerio De Stefano, Reuben Binns, Jeremias Adams-Prassl, Barend van Leeuwen, Aislinn Kelly-Lyth, Lilian Edwards, Natali Helberger, Christopher Marsden, Sarah Chander, Corinne Cath-Speth for comments and/or discussion; substantive and editorial input by Ulrich Gasper; and the conveners and participants of several workshops including one convened by Margot Kaminski, one by Burkhard Schäfer, one part of the 2nd ELLIS Workshop in Human-Centric Machine Learning; one between Lund University and the Labour Law Community; and one between Oxford, KU Leuven and UCL. A CC-BY 4.0 license applies to this article after 3 calendar months from publication have elapsed.

We present a new dataset of Wikipedia articles each paired with a knowledge graph, to facilitate the research in conditional text generation, graph generation and graph representation learning. Existing graph-text paired datasets typically contain small graphs and short text (1 or few sentences), thus limiting the capabilities of the models that can be learned on the data. Our new dataset WikiGraphs is collected by pairing each Wikipedia article from the established WikiText-103 benchmark (Merity et al., 2016) with a subgraph from the Freebase knowledge graph (Bollacker et al., 2008). This makes it easy to benchmark against other state-of-the-art text generative models that are capable of generating long paragraphs of coherent text. Both the graphs and the text data are of significantly larger scale compared to prior graph-text paired datasets. We present baseline graph neural network and transformer model results on our dataset for 3 tasks: graph -> text generation, graph -> text retrieval and text -> graph retrieval. We show that better conditioning on the graph provides gains in generation and retrieval quality but there is still large room for improvement.

Apple is an equal opportunity employer that is committed to inclusion and diversity. We take affirmative action to ensure equal opportunity for all applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, Veteran status, or other legally protected characteristics. Apple will not discriminate or retaliate against applicants who inquire about, disclose, or discuss their compensation or that of other applicants. Apple will consider for employment all qualified applicants with criminal histories in a manner consistent with applicable law. If you're applying for a position in San Francisco, review the San Francisco Fair Chance Ordinance guidelines (opens in a new window) applicable in your area.

"Fantastic! How fast can we scale?" Perhaps you've been fortunate enough to hear or ask that question about a new AI project in your organization. Or maybe an initial AI initiative has already reached production, but others are needed -- quickly. At this key early stage of AI growth, entesrprises and the industry face a bigger, related question: How do we scale our organizational ability to develop and deploy AI? Business and technology leaders must ask: What's needed to advance AI (and by extension, data science) beyond the "craft" stage, to large-scale production that is fast, reliable, and economical? The answers are crucial to realizing ROI, delivering on the vision of "AI everywhere", and helping the technology mature and propagate over the next five years.

The Pentagon sees artificial intelligence as a way to outfox, outmaneuver, and dominate future adversaries. But the brittle nature of AI means that without due care, the technology could perhaps hand enemies a new way to attack. The Joint Artificial Intelligence Center, created by the Pentagon to help the US military make use of AI, recently formed a unit to collect, vet, and distribute open source and industry machine learning models to groups across the Department of Defense. A machine learning "red team," known as the Test and Evaluation Group, will probe pretrained models for weaknesses. Another cybersecurity team examines AI code and data for hidden vulnerabilities.

Here's What You Need to Remember: From a tactical circumstance, given that attack submarines and nuclear-armed ballistic missile submarines are likely to conduct large amounts of clandestine patrols, it seems as though an ability to avoid having to surface would bring an extraordinary operational advantage. Could newer kinds of AI-enabled undersea drone data processing and analysis introduce new breakthrough possibilities when it comes to solving the longstanding challenge of achieving high-speed, real-time connectivity? Submarine commanders and weapons developers explain that UUV undersea functionality is dependent upon limited battery power and would therefore be further enabled by an ability to "process the data at the source of the sensor" to distinguish and transmit only the most critical information needed by human decision-makers. "That's the concept, how do you get all of that information back to a human to analyze. Maybe you don't want to do that? Maybe you want to allow the UUV to do some initial analysis and make some modifications to its behavior autonomously?"

Data-driven analysis and detection of abusive online content covers many different tasks, phenomena, contexts, and methodologies. This paper systematically reviews abusive language dataset creation and content in conjunction with an open website for cataloguing abusive language data. This collection of knowledge leads to a synthesis providing evidence-based recommendations for practitioners working with this complex and highly diverse data.

Many machine learning strategies designed to automate mathematical tasks leverage neural networks to search large combinatorial spaces of mathematical symbols. In contrast to traditional evolutionary approaches, using a neural network at the core of the search allows learning higher-level symbolic patterns, providing an informed direction to guide the search. When no labeled data is available, such networks can still be trained using reinforcement learning. However, we demonstrate that this approach can suffer from an early commitment phenomenon and from initialization bias, both of which limit exploration. We present two exploration methods to tackle these issues, building upon ideas of entropy regularization and distribution initialization. We show that these techniques can improve the performance, increase sample efficiency, and lower the complexity of solutions for the task of symbolic regression.

In this work we explore the limiting dynamics of deep neural networks trained with stochastic gradient descent (SGD). We find empirically that long after performance has converged, networks continue to move through parameter space by a process of anomalous diffusion in which distance travelled grows as a power law in the number of gradient updates with a nontrivial exponent. We reveal an intricate interaction between the hyperparameters of optimization, the structure in the gradient noise, and the Hessian matrix at the end of training that explains this anomalous diffusion. To build this understanding, we first derive a continuous-time model for SGD with finite learning rates and batch sizes as an underdamped Langevin equation. We study this equation in the setting of linear regression, where we can derive exact, analytic expressions for the phase space dynamics of the parameters and their instantaneous velocities from initialization to stationarity. Using the Fokker-Planck equation, we show that the key ingredient driving these dynamics is not the original training loss, but rather the combination of a modified loss, which implicitly regularizes the velocity, and probability currents, which cause oscillations in phase space. We identify qualitative and quantitative predictions of this theory in the dynamics of a ResNet-18 model trained on ImageNet. Through the lens of statistical physics, we uncover a mechanistic origin for the anomalous limiting dynamics of deep neural networks trained with SGD.