Recent work shows that sensitive user data can be reconstructed from gradient updates, breaking the key privacy promise of federated learning. While success was demonstrated primarily on image data, these methods do not directly transfer to other domains such as text. In this work, we propose LAMP, a novel attack tailored to textual data, that successfully reconstructs original text from gradients. Our attack is based on two key insights: (i) modelling prior text probability via an auxiliary language model, guiding the search towards more natural text, and (ii) alternating continuous and discrete optimization which minimizes reconstruction loss on embeddings while avoiding local minima via discrete text transformations. Our experiments demonstrate that LAMP is significantly more effective than prior work: it reconstructs 5x more bigrams and $23\%$ longer subsequences on average. Moreover, we are first to recover inputs from batch sizes larger than 1 for textual models. These findings indicate that gradient updates of models operating on textual data leak more information than previously thought.

Often while developing Natural Language Processing models, we find it difficult to find relevant data. Previously, while developing our Intent Classifier, we used the CLINC150 Dataset that had 100 samples for 150 different classes. But, what if we needed even more samples? One more similar scenario was when I was working on a contextual assistant with Rasa. While creating the training data from scratch, I'd have to imagine different samples for each intent or ask my friends for some help.

The field of adaptive robotics involves simulations and real-world implementations of robots that adapt to their environments. In this article, I introduce adaptive environmentics--the flip side of adaptive robotics--in which the environment adapts to the robot. The reasonable man adapts himself to the world; the unreasonable man persists to adapt the world to himself. Therefore, all progress depends on the unreasonable. The apparent complexity of its behavior over time is largely a reflection of the complexity of the environment in which it finds itself. Using both simulated and real robots, and applying techniques such as reinforcement learning, artificial neural networks, genetic algorithms, and fuzzy logic, researchers have obtained robots that display an amazing slew of behaviors and perform a multitude of tasks, including walking, pushing boxes, navigating, negotiating an obstacle course, playing ball, and foraging (Arkin 1998a). To cite one typical example of an ever-growing many, Yung and Ye (1999) recently wrote: We have presented a fuzzy navigator that performs well in complex and unknown environments, using a rule base that is learned from a simple corridor-like environment. The principle of the navigator is built on the fusion of the obstacle avoidance and goal seeking behaviors aided by an environment evaluator to tune the universe of discourse of the input sensor readings and enhance its adaptability. For this reason, the navigator has been able to learn extremely quickly in a simple environment, and then operate in an unknown environment, where exploration is not required at all. This quote typifies the underlying theme of adaptive robotics: Have a robot adapt to a given environment. Given signifies neither that the environment is known nor that it is static; it means that the robot must adapt to the quirks and idiosyncrasies imposed by the environment--which, for its part, does nothing at all to accommodate the puffing robot. This fundamental principle of adaptive robotics--the environment's unyielding nature--is repealed in this article. Dubbed adaptive environmentics, the basic idea is to create scenarios that are mirror images of those found in adaptive robotics: The environment adapts to a given robot. I hasten to say that in some cases, it is not possible to alter the environment, and in other cases, having the robot adapt is simply the underlying objective. Adaptive robotics has produced many interesting results based on these principles.