Modern neural language models widely used in tasks across NLP risk memorizing sensitive information from their training data. As models continue to scale up in parameters, training data, and compute, understanding memorization in language models is both important from a learning-theoretical point of view, and is practically crucial in real world applications. An open question in previous studies of memorization in language models is how to filter out "common" memorization. In fact, most memorization criteria strongly correlate with the number of occurrences in the training set, capturing "common" memorization such as familiar phrases, public knowledge or templated texts. In this paper, we provide a principled perspective inspired by a taxonomy of human memory in Psychology. From this perspective, we formulate a notion of counterfactual memorization, which characterizes how a model's predictions change if a particular document is omitted during training. We identify and study counterfactually-memorized training examples in standard text datasets. We further estimate the influence of each training example on the validation set and on generated texts, and show that this can provide direct evidence of the source of memorization at test time.

The disruption of supplies during the Covid-19 crisis has led to shortages but has also shown the adaptability of some companies, which have succeeded in adapting their production chains quickly to produce goods experiencing shortages: hydroalcoholic gel, masks, and medical gowns. These productive jumps from product A to product B are feasible because of the know-how proximity between the two classes of products. The proximities were computed from the analysis of co-exports and resulted in the construction of the product space. Based on the product space, as well as the customer-supplier relationships resulting from the input-output matrices, we propose a recommender system for companies. The goal is to promote distributed manufacturing by recommending a list of local suppliers to each company. As there is not always a local supplier for a desired product class, we consider the proximity between products to identify, in the absence of a supplier, a substitute supplier able to adapt its production tools to provide the required product. Our experiments are based on French data, from which we build a graph of synergies illustrating the potential productive links between companies. Finally, we show that our approach offers new perspectives to determine the level of territories' industrial resilience considering potential productive jumps.

We study stochastic approximation procedures for approximately solving a $d$-dimensional linear fixed point equation based on observing a trajectory of length $n$ from an ergodic Markov chain. We first exhibit a non-asymptotic bound of the order $t_{\mathrm{mix}} \tfrac{d}{n}$ on the squared error of the last iterate of a standard scheme, where $t_{\mathrm{mix}}$ is a mixing time. We then prove a non-asymptotic instance-dependent bound on a suitably averaged sequence of iterates, with a leading term that matches the local asymptotic minimax limit, including sharp dependence on the parameters $(d, t_{\mathrm{mix}})$ in the higher order terms. We complement these upper bounds with a non-asymptotic minimax lower bound that establishes the instance-optimality of the averaged SA estimator. We derive corollaries of these results for policy evaluation with Markov noise -- covering the TD($\lambda$) family of algorithms for all $\lambda \in [0, 1)$ -- and linear autoregressive models. Our instance-dependent characterizations open the door to the design of fine-grained model selection procedures for hyperparameter tuning (e.g., choosing the value of $\lambda$ when running the TD($\lambda$) algorithm).

Graph walking based on reinforcement learning (RL) has shown great success in navigating an agent to automatically complete various reasoning tasks over an incomplete knowledge graph (KG) by exploring multi-hop relational paths. However, existing multi-hop reasoning approaches only work well on short reasoning paths and tend to miss the target entity with the increasing path length. This is undesirable for many reason-ing tasks in real-world scenarios, where short paths connecting the source and target entities are not available in incomplete KGs, and thus the reasoning performances drop drastically unless the agent is able to seek out more clues from longer paths. To address the above challenge, in this paper, we propose a dual-agent reinforcement learning framework, which trains two agents (GIANT and DWARF) to walk over a KG jointly and search for the answer collaboratively. Our approach tackles the reasoning challenge in long paths by assigning one of the agents (GIANT) searching on cluster-level paths quickly and providing stage-wise hints for another agent (DWARF). Finally, experimental results on several KG reasoning benchmarks show that our approach can search answers more accurately and efficiently, and outperforms existing RL-based methods for long path queries by a large margin.

The EU's proposed artificial intelligence act fails to fully take into account the recent rise of an ultra-powerful new type of AI, meaning the legislation will rapidly become obsolete as the technology is deployed in novel and unexpected ways. Foundation models trained on gargantuan amounts of data by the world's biggest tech companies, and then adapted to a wide range of tasks, are poised to become the infrastructure on which other applications are built. That means any deficits in these models will be inherited by all uses to which they are put. The fear is that foundation models could irreversibly embed security flaws, opacity and biases into AI. One study found that a model trained on online text replicated the prejudices of the internet, equating Islam with terrorism, a bias that could pop up unexpectedly if the model was used in education, for example.

This article contains spoilers for the film Don't Look Up. Streaming just in time for Christmas, Adam McKay's decidedly uncheery Netflix comedy, Don't Look Up, finds Jennifer Lawrence and Leonardo DiCaprio playing a pair of intrepid astronomers as they try (and mostly fail) to warn the world about a planet-killing comet that's hurtling toward Earth. From the beginning, the scientists' efforts are marked by futility, encapsulated in an early scene in which Kate Dibiasky (Lawrence) and Randall Mindy (DiCaprio) are brought to the White House to debrief President Janie Orlean (Meryl Streep) on the impending extinction-level event. Predictably, the meeting goes disastrously. The president's son and chief of staff (played by Jonah Hill) lounges on the couch, nurses a bad case of coke sniffles, and proclaims to be "so bored" by all the world-ending comet talk.

Certain advanced driving assistance features can promote safety by helping drivers avoid crashes and mitigate the severity of crashes that occur, but as with all technologies and equipment on motor vehicles, drivers must use them correctly and responsibly,

The close of 2021 finds Tesla wealthier than ever -- and, in CEO Elon Musk's case, wealthier than everybody else. The electric vehicle manufacturer notched records for both deliveries and profits this year despite a global chip shortage that decimated supply chains worldwide, effectively kneecapping the rest of the automotive industry's production capacity. However its financial successes were often overshadowed by Tesla's continuing production quality issues, multiple NHTSA and SEC investigations, high profile failures of its vaunted "Full Self Driving" system, as well as numerous vehicle recalls and delays for upcoming models. And with existing industry stalwarts like Ford, GM, Honda and the Volkswagen Group making concerted efforts to electrify their own offerings, could 2022 be the year that Tesla's reign as top EV automaker finally ends? The company entered this year having met its 2020 goal of producing a half-million vehicles (of which it delivered 499,550 to customers), a nearly 133,000 unit increase over 2019.

WASHINGTON – The Defense Advanced Research Projects Agency has issued a $5.8 million contract to a team building an artificial intelligence system able to scan instruction manuals and convert that data into instructions for augmented reality systems. Companies are already using augmented reality technologies in their manufacturing processes. Lockheed Martin, for example, uses augmented reality goggles in assembling its space systems for NASA. With the goggles on, technicians can see relevant information and instructions in the space around them as they go about their work, saving them from having to constantly walk back and forth to consult physical manuals or computer monitors. Under the $5.8 million contract, PARC, a Xerox company, will work with the University of California at Santa Barbara, the University of Rostock in Germany and Patched Reality on the Autonomous Multimodal Ingestion for Goal-Oriented Support (AMIGOS) project for the Perceptually-enabled Task Guidance Program.

The use of artificial intelligence ("AI") and machine learning in the workplace is growing exponentially – and specifically in hiring. Over the last two decades, web-based applications and questionnaires have made paper applications nearly obsolete. As employers seek to streamline recruitment and control costs, they have jumped to use computer-based screening tools such as "chatbots" to communicate with job applicants, to schedule interviews, ask screening questions, and even conduct video conference interviews and presentations in the selection process. Employers of all sizes are creating their own systems, or hiring vendors who will design and implement keyword searches, predictive algorithms and even facial recognition algorithms to find the best-suited candidates. The algorithms in these computer models make inferences from data about people, including their identities, their demographic attributes, their preferences, and their likely future behaviors.