Many real-world physical control systems are required to satisfy constraints upon deployment. Furthermore, real-world systems are often subject to effects such as non-stationarity, wear-and-tear, uncalibrated sensors and so on. Such effects effectively perturb the system dynamics and can cause a policy trained successfully in one domain to perform poorly when deployed to a perturbed version of the same domain. This can affect a policy's ability to maximize future rewards as well as the extent to which it satisfies constraints. We refer to this as constrained model misspecification. We present an algorithm with theoretical guarantees that mitigates this form of misspecification, and showcase its performance in multiple Mujoco tasks from the Real World Reinforcement Learning (RWRL) suite.

A robot understands its world through the raw information it senses from its surroundings. This raw information is not suitable as a shared representation between the robot and its user. A semantic map, containing high-level information that both the robot and user understand, is better suited to be a shared representation. We use the semantic map as the user-facing interface on our fleet of floor-cleaning robots. Jitter in the robot's sensed raw map, dynamic objects in the environment, and exploration of new space by the robot are common challenges for robots. Solving these challenges effectively in the context of semantic maps is key to enabling semantic maps for lifelong mapping. First, as a robot senses new changes and alters its raw map in successive runs, the semantics must be updated appropriately. We update the map using a spatial transfer of semantics. Second, it is important to keep semantics and their relative constraints consistent even in the presence of dynamic objects. Inconsistencies are automatically determined and resolved through the introduction of a map layer of meta-semantics. Finally, a discovery phase allows the semantic map to be updated with new semantics whenever the robot uncovers new information. Deployed commercially on thousands of floor-cleaning robots in real homes, our user-facing semantic maps provide a intuitive user experience through a lifelong mapping robot.

Single cells continuously experience and react to mechanical challenges in three-dimensional tissues. Spatial constraints in dense tissues, physical activity, and injury all impose changes in cell shape. How cells can measure shape deformations to ensure correct tissue development and homeostasis remains largely unknown (see the Perspective by Shen and Niethammer). Working independently, Venturini et al. and Lomakin et al. now show that the nucleus can act as an intracellular ruler to measure cellular shape variations. The nuclear envelope provides a gauge of cell deformation and activates a mechanotransduction pathway that controls actomyosin contractility and migration plasticity. The cell nucleus thereby allows cells to adapt their behavior to the local tissue microenvironment. Science , this issue p. [eaba2644][1], p. [eaba2894][2]; see also p. [295][3] ### INTRODUCTION The human body is a crowded place. This crowding is even more acute when the regulation of cell growth and proliferation fails during the formation of a tumor. Dealing with the lack of space in crowded environments presents cells with a challenge. This is especially true for immune cells, whose task is to patrol tissues, causing them to experience both acute and sustained deformation as they move. Although changes in tissue crowding and associated cell shape alterations have been known by pathologists to be key diagnostic traits of late-stage tumors since the 19th century, the impact of these changes on the biology of cancer and immune cells remains unclear. Moreover, it is not known whether cells can detect and adaptively respond to deformations in densely packed spaces. ### RATIONALE To test the hypothesis that cells possess an ability to detect and respond to environmentally induced changes in their shape, we fabricated artificial microenvironments that mimic the conditions experienced by tumor and immune cells in a crowded tissue. By combining dynamic confinement, force measurements, and live cell imaging, we were able to quantify cell responses to precisely controlled physical perturbations of their shape. ### RESULTS Our results show that, although cells are surprisingly resistant to compressive forces, they monitor their own shape and develop an active contractile response when deformed below a specific height. Notably, we find that this is achieved by cells monitoring the deformation of their largest internal compartment: the nucleus. We establish that the nucleus provides cells with a precise measure of the extent of their deformation. Once cell compression exceeds the size of the nucleus, it causes the bounding nuclear envelope (NE) to unfold and stretch. The onset of the contractile response occurs when the NE reaches a fully unfolded state. This transition in the mechanical state of the NE and its membranes permits calcium release from internal membrane stores and activates the calcium-dependent phospholipase cPLA2, an enzyme known to operate as a molecular sensor of nuclear membrane tension and a critical regulator of signaling and metabolism. Activated cPLA2 catalyzes the formation of arachidonic acid, an omega-6 fatty acid that, among other processes, potentiates the adenosine triphosphatase activity of myosin II. This induces contractility of the actomyosin cortex, which produces pushing forces to resist physical compression and to rapidly squeeze the cell out of its compressive microenvironment in an “evasion reflex” mechanism. ### CONCLUSION Although the nucleus has traditionally been considered a passive storehouse for genetic material, our work identifies it as an active compartment that rapidly convers mechanical inputs into signaling outputs, with a critical role of its envelope in this sensing function. The nucleus is able to detect environmentally imposed compression and respond to it by generating a signal that is used to change cell behaviors. This phenomenon plays a critical role in ensuring that cells, such as the immune cells within a tumor, can adapt, survive, and efficiently move through a crowded and mechanically heterogeneous microenvironment. Characterizing the full spectrum of signals triggered by nuclear compression has the potential to elucidate mechanisms underlying signaling, epigenetic, and metabolic adaptations of cells to their mechanoenvironment and is thus an exciting avenue for future research. ![Figure][4] The nuclear ruler and its contribution to the “life cycle” of a confined cell. (1) Cell confinement below resting nucleus size, leading to nuclear deformation and to unfolding, and stretching of the nuclear envelope. (2) Nuclear membrane tension increase, which triggers calcium release, cPLA2 activation, and arachidonic acid (ARA) production. (3) Actomyosin force ( F ) generation. (4) Increased cell migratory capacity and escape from confinement. The microscopic environment inside a metazoan organism is highly crowded. Whether individual cells can tailor their behavior to the limited space remains unclear. In this study, we found that cells measure the degree of spatial confinement by using their largest and stiffest organelle, the nucleus. Cell confinement below a resting nucleus size deforms the nucleus, which expands and stretches its envelope. This activates signaling to the actomyosin cortex via nuclear envelope stretch-sensitive proteins, up-regulating cell contractility. We established that the tailored contractile response constitutes a nuclear ruler–based signaling pathway involved in migratory cell behaviors. Cells rely on the nuclear ruler to modulate the motive force that enables their passage through restrictive pores in complex three-dimensional environments, a process relevant to cancer cell invasion, immune responses, and embryonic development. [1]: /lookup/doi/10.1126/science.aba2644 [2]: /lookup/doi/10.1126/science.aba2894 [3]: /lookup/doi/10.1126/science.abe3881 [4]: pending:yes

Recent advancements in procedural content generation via machine learning enable the generation of video-game levels that are aesthetically similar to human-authored examples. However, the generated levels are often unplayable without additional editing. We propose a generate-then-repair framework for automatic generation of playable levels adhering to specific styles. The framework constructs levels using a generative adversarial network (GAN) trained with human-authored examples and repairs them using a mixed-integer linear program (MIP) with playability constraints. A key component of the framework is computing minimum cost edits between the GAN generated level and the solution of the MIP solver, which we cast as a minimum cost network flow problem. Results show that the proposed framework generates a diverse range of playable levels, that capture the spatial relationships between objects exhibited in the human-authored levels.

Deploying Reinforcement Learning (RL) agents to solve real-world applications often requires satisfying complex system constraints. Often the constraint thresholds are incorrectly set due to the complex nature of a system or the inability to verify the thresholds offline (e.g, no simulator or reasonable offline evaluation procedure exists). This results in solutions where a task cannot be solved without violating the constraints. However, in many real-world cases, constraint violations are undesirable yet they are not catastrophic, motivating the need for soft-constrained RL approaches. We present two soft-constrained RL approaches that utilize meta-gradients to find a good trade-off between expected return and minimizing constraint violations. We demonstrate the effectiveness of these approaches by showing that they consistently outperform the baselines across four different Mujoco domains.

Probabilistic software analysis extends classic static analyses techniques to consider the effects of probabilistic uncertainty, whether explicitly embedded within the code - as in probabilistic program - or externalized in a probabilistic input distribution [Dwyer et al. 2016]. Similarly to their classic counterparts, these analyses aim at inferring the probability of a target event to occur during execution, e.g., reaching a program state or triggering an exception. For the probabilistic analysis of programs written in general-purpose programming languages, probabilistic symbolic execution (PSE) [Geldenhuys et al. 2012] exploits established symbolic execution engines for the language to extract constraints on probabilistic input or program variables that lead to the occurrence of the target event. The probability of satisfying any such constraints is then computed via model counting or inferred via solution space quantification methods, depending on the variable types and the characteristic of the constraints. Variations of PSE include incomplete analyses inferring probability bounds from a finite sample of program paths executed symbolically [Filieri et al. 2014], methods for non-deterministic programs [Luckow et al. 2014] and data structures [Filieri et al. 2015].

This paper contributes to the treatment of extensive constraints in evolutionary many-constraint optimization through consideration of the relationships between pair-wise constraints. In a conflicting relationship, the functional value of one constraint increases as the value in another constraint decreases. In a harmonious relationship, the improvement in one constraint is rewarded with simultaneous improvement in the other constraint. In an independent relationship, the adjustment to one constraint never affects the adjustment to the other. Based on the different features, methods for identifying constraint relationships are discussed, helping to simplify many-constraint optimization problems (MCOPs). Additionally, the transitivity of the relationships is further discussed at the aim of determining the relationship in a new pair of constraints.

Post-processing immunity is a fundamental property of differential privacy: it enables the application of arbitrary data-independent transformations to the results of differentially private outputs without affecting their privacy guarantees. When query outputs must satisfy domain constraints, post-processing can be used to project the privacy-preserving outputs onto the feasible region. Moreover, when the feasible region is convex, a widely adopted class of post-processing steps is also guaranteed to improve accuracy. Post-processing has been applied successfully in many applications including census data-release, energy systems, and mobility. However, its effects on the noise distribution is poorly understood: It is often argued that post-processing may introduce bias and increase variance. This paper takes a first step towards understanding the properties of post-processing. It considers the release of census data and examines, both theoretically and empirically, the behavior of a widely adopted class of post-processing functions.

We consider the problem of learning control policies that optimize a reward function while satisfying constraints due to considerations of safety, fairness, or other costs. We propose a new algorithm, Projection-Based Constrained Policy Optimization (PCPO). This is an iterative method for optimizing policies in a two-step process: the first step performs a local reward improvement update, while the second step reconciles any constraint violation by projecting the policy back onto the constraint set. We theoretically analyze PCPO and provide a lower bound on reward improvement, and an upper bound on constraint violation, for each policy update. We further characterize the convergence of PCPO based on two different metrics: $\normltwo$ norm and Kullback-Leibler divergence. Our empirical results over several control tasks demonstrate that PCPO achieves superior performance, averaging more than 3.5 times less constraint violation and around 15\% higher reward compared to state-of-the-art methods.

There is a fundamental mismatch between the computational basis of spreadsheets and our knowledge of the real world. In spreadsheets, numeric data are represented as exact numbers and their mutual relations as functions, whose values (output) are computed from given argument values (input). However, in the real world, data are often inexact and uncertain in many ways, and the relationships, that is, constraints, between input and output are far more complicated. This article shows that interval constraint solving, an emerging AI-based technology, provides a more versatile and useful foundation for spreadsheets. The new computational basis is 100-percent downward compatible with the traditional spreadsheet paradigm.