Government
Re3: Generating Longer Stories With Recursive Reprompting and Revision
Yang, Kevin, Tian, Yuandong, Peng, Nanyun, Klein, Dan
We consider the problem of automatically generating longer stories of over two thousand words. Compared to prior work on shorter stories, long-range plot coherence and relevance are more central challenges here. We propose the Recursive Reprompting and Revision framework (Re3) to address these challenges by (a) prompting a general-purpose language model to construct a structured overarching plan, and (b) generating story passages by repeatedly injecting contextual information from both the plan and current story state into a language model prompt. We then revise by (c) reranking different continuations for plot coherence and premise relevance, and finally (d) editing the best continuation for factual consistency. Compared to similar-length stories generated directly from the same base model, human evaluators judged substantially more of Re3's stories as having a coherent overarching plot (by 14% absolute increase), and relevant to the given initial premise (by 20%).
Evolution of Neural Tangent Kernels under Benign and Adversarial Training
Loo, Noel, Hasani, Ramin, Amini, Alexander, Rus, Daniela
Two key challenges facing modern deep learning are mitigating deep networks' vulnerability to adversarial attacks and understanding deep learning's generalization capabilities. Towards the first issue, many defense strategies have been developed, with the most common being Adversarial Training (AT). Towards the second challenge, one of the dominant theories that has emerged is the Neural Tangent Kernel (NTK) -- a characterization of neural network behavior in the infinite-width limit. In this limit, the kernel is frozen, and the underlying feature map is fixed. In finite widths, however, there is evidence that feature learning happens at the earlier stages of the training (kernel learning) before a second phase where the kernel remains fixed (lazy training). While prior work has aimed at studying adversarial vulnerability through the lens of the frozen infinite-width NTK, there is no work that studies the adversarial robustness of the empirical/finite NTK during training. In this work, we perform an empirical study of the evolution of the empirical NTK under standard and adversarial training, aiming to disambiguate the effect of adversarial training on kernel learning and lazy training. We find under adversarial training, the empirical NTK rapidly converges to a different kernel (and feature map) than standard training. This new kernel provides adversarial robustness, even when non-robust training is performed on top of it. Furthermore, we find that adversarial training on top of a fixed kernel can yield a classifier with $76.1\%$ robust accuracy under PGD attacks with $\varepsilon = 4/255$ on CIFAR-10.
Assaying Out-Of-Distribution Generalization in Transfer Learning
Wenzel, Florian, Dittadi, Andrea, Gehler, Peter Vincent, Simon-Gabriel, Carl-Johann, Horn, Max, Zietlow, Dominik, Kernert, David, Russell, Chris, Brox, Thomas, Schiele, Bernt, Schölkopf, Bernhard, Locatello, Francesco
Since out-of-distribution generalization is a generally ill-posed problem, various proxy targets (e.g., calibration, adversarial robustness, algorithmic corruptions, invariance across shifts) were studied across different research programs resulting in different recommendations. While sharing the same aspirational goal, these approaches have never been tested under the same experimental conditions on real data. In this paper, we take a unified view of previous work, highlighting message discrepancies that we address empirically, and providing recommendations on how to measure the robustness of a model and how to improve it. To this end, we collect 172 publicly available dataset pairs for training and out-of-distribution evaluation of accuracy, calibration error, adversarial attacks, environment invariance, and synthetic corruptions. We fine-tune over 31k networks, from nine different architectures in the many- and few-shot setting. Our findings confirm that in- and out-of-distribution accuracies tend to increase jointly, but show that their relation is largely dataset-dependent, and in general more nuanced and more complex than posited by previous, smaller scale studies.
Mass drone attacks in Ukraine foreshadow the 'future of warfare'
A little before 7am on Monday, people in Kyiv heard a whining sound overhead before identifying where it was coming from – a group of "kamikaze" drones flying into the city. Drones have been widely used on both sides of the Ukraine conflict, but these were the first Russian attacks that deployed swarms of the aircraft. Videos and images began to circulate on social media of the drones flying directly over urban infrastructure such as power stations, residential buildings and railways as civilians and soldiers tried to shoot them down with guns. About 28 were launched on Monday morning in Kyiv. At least four civilians were killed after one of the aircraft hit a residential building.
White House says Iran helping Russia 'on the ground' in Crimea
The White House has accused Iran of being "directly engaged on the ground" in Russian-occupied Crimea, helping to train the country's forces on Iranian-made drones that have been used in attacks in Ukraine. US National Security Council spokesman John Kirby said on Thursday that a "relatively small number" of Iranian personnel are operating in the Ukrainian region that was annexed by Russia in 2014. "Tehran is now directly engaged on the ground and through the provision of weapons that are impacting civilians and civilian infrastructure in Ukraine," Kirby said. "The United States is going to pursue all means to expose, deter and confront Iran's provision of these munitions against the Ukrainian people." Tehran has denied supplying Moscow with drones or helping launch them.
18. Peter Scott On Challenges and Opportunities with AI - The AI with Maribel Lopez (AI with ML)
Peter Scott (AI leader and author of Artificial Intelligence and You) is an expert on all things AI, Peter Scott is on a mission to help us to get along with artificial intelligence. He has given TEDx talks, spoken to audiences as diverse as transformational leaders, executives, and British parliamentarians, and created a program to train coaches in helping clients become resilient to exponential disruption. A Master's degree in computer science from Cambridge University led him to spend more than thirty years working for NASA's Jet Propulsion Laboratory, helping advance our exploration of space. A parallel pursuit of the human development field as a certified coach positioned him to recognize and address technological disruption. The births of his children brought him into a mission, to help people understand, use, and advance AI for the betterment of all.
Government committee launches inquiry to regulate AI in the UK
A government Committee has launched an inquiry into artificial intelligence in a bid to regulate it as it become an increasingly appealing avenue for businesses. "AI is already transforming almost every area of research and business. It has extraordinary potential but there are concerns about how the existing regulatory system is suited to a world of AI," chair of the Science and Technology Committee, Greg Clark, said. "With machines making more and more decisions that impact people's lives, it is crucial we have effective regulation in place. In our inquiry we look forward to examining the government's proposals in detail."
Blasting Crackdown But Eyeing Deal, West In Quandary Over Iran
Waging brutal repression at home and allegedly helping Russia in its war against Ukraine, Iran is becoming an unsolvable challenge for Western powers eager to avoid a new nuclear power in the Middle East. "We're in a delicate situation and an obvious impasse," a French diplomat admitted before Wednesday's UN Security Council meeting on suspected Iranian drone use by Russian forces. Despite Tehran's new support for an increasingly isolated Moscow, the United States and the European Union still hope to revive the 2015 deal aimed at curtailing Iran's nuclear programme -- even though the prospect is dimming. "Iran's repression at home and aggression in Ukraine have increased the political cost for and decreased the appetite of the West to grant Tehran sanctions relief," said analyst Ali Vaez of the International Crisis Group. "But the West has no good options, as the only thing worse than a repressive regime that kills its own people is a nuclear armed one that does so."
Russia seeks to regain ground, hits Ukraine's infrastructure
Russia's troops fought Thursday to regain lost ground in areas of Ukraine that Russian President Vladimir Putin has illegally annexed while Moscow tried to pound the invaded country into submission with more missile and drone attacks on critical infrastructure. Russian forces attacked Ukrainian positions near Bilohorivka, a village in the Luhansk region of eastern Ukraine. In the neighboring Donetsk region, fighting raged near the city of Bakhmut. Kremlin-backed separatists have controlled parts of both regions for 8½ years. Putin declared martial law in Luhansk, Donetsk and southern Ukraine's Zaporizhzhia and Kherson regions on Wednesday in an attempt to assert Russian authority in the annexed areas following a string of battlefield setbacks and a troubled troop mobilization.