From the simplest unicellular organisms to complex animals, feedback control based on sensing and actuation is a staple of self‐regulation in biological processes and is a key to life itself. Malfunctioning of this control loop can often lead to disease or death. Bioelectronic devices that interface electronics with biological systems can be used for sensing and actuation of biological processes and have potential for novel therapeutic applications. Due to the complexity of biological systems and the challenge of affecting their innate self‐regulation, closing the loop between sensing and actuation with bioelectronics is difficult to achieve. Herein, bioelectronic proton‐conducting devices are integrated with fluorescence sensing using machine learning to provide closed‐loop control of bioelectronic actuation in living cells.

Newly, the rates of energy and material consumption to augment industrial pro-duction are substantially high, thus the environmentally sustainable industrial de-velopment has emerged as the main issue of either developed or developing coun-tries. A novel approach to supply chain management is proposed to maintain economic growth along with environmentally friendly concerns for the design of the supply chain network. In this paper, a new green supply chain design approach has been suggested to maintain the financial virtue accompanying the environ-mental factors that required to be mitigated the negative effect of rapid industrial development on the environment. This approach has been suggested a multi-objective mathematical model minimizing the total costs and CO2 emissions for establishing an environmentally sustainable closed-loop supply chain. Two opti-mization methods are used namely Epsilon Constraint Method, and Genetic Al-gorithm Optimization Method. The results of the two mentioned methods have been compared and illustrated their effectiveness. The outcome of the analysis is approved to verify the accuracy of the proposed model to deal with financial and environmental issues concurrently.

People with type 1 diabetes (T1D) lack the ability to produce the insulin their bodies need. As a result, they must continually make decisions about how much insulin to self-administer to adequately control their blood glucose levels. Longitudinal data streams captured from wearables, like continuous glucose monitors, can help these individuals manage their health, but currently the majority of the decision burden remains on the user. To relieve this burden, researchers are working on closed-loop solutions that combine a continuous glucose monitor and an insulin pump with a control algorithm in an `artificial pancreas.' Such systems aim to estimate and deliver the appropriate amount of insulin. Here, we develop reinforcement learning (RL) techniques for automated blood glucose control. Through a series of experiments, we compare the performance of different deep RL approaches to non-RL approaches. We highlight the flexibility of RL approaches, demonstrating how they can adapt to new individuals with little additional data. On over 2.1 million hours of data from 30 simulated patients, our RL approach outperforms baseline control algorithms: leading to a decrease in median glycemic risk of nearly 50% from 8.34 to 4.24 and a decrease in total time hypoglycemic of 99.8%, from 4,610 days to 6. Moreover, these approaches are able to adapt to predictable meal times (decreasing average risk by an additional 24% as meals increase in predictability). This work demonstrates the potential of deep RL to help people with T1D manage their blood glucose levels without requiring expert knowledge. All of our code is publicly available, allowing for replication and extension.

This paper proposed a new estimation and control strategy to control the satellite attitude. As the attitude control strategy plays an essential role in the different kinds of space missions, scientists try to improve the performance of the satellite attitude system, regardless of the expense. In this study, we proposed an adaptive neuro-fuzzy integrated (ANFIS) satellite attitude estimation and control system. A pulse modulator is used to generate the right ON/OFF commands of the thruster actuator. To evaluate the performance of the ANFIS controller in closed-loop simulation, an ANFIS observer is used to estimate the attitude and angular velocities of the satellite using a magnetometer, sun sensor, and rate gyro data. Besides, a new ANFIS system will be proposed and evaluated that can simultaneously control and estimate the system. The performance of the ANFIS controller is compared with the optimal PID controller in a Monte Carlo simulation using different initial conditions, disturbance, and noise. The simulations are performed to verify the ANFIS controller's ability to decrease settling time and fuel consumption in comparison with the optimal PID controller. Also, examine the ANFIS estimator, and the results demonstrate the high skill of these designated observers. Moreover, we proposed an integrated ANFIS estimator and controller for satellite attitude control and estimation in the presence of noise and uncertainty, which can reduce the computational effort and offer smooth actuator actions.

The goal of neural-symbolic computation is to integrate the connectionist and symbolist paradigms. Prior methods learn the neural-symbolic models using reinforcement learning (RL) approaches, which ignore the error propagation in the symbolic reasoning module and thus converge slowly with sparse rewards. In this paper, we address these issues and close the loop of neural-symbolic learning by (1) introducing the \textbf{grammar} model as a \textit{symbolic prior} to bridge neural perception and symbolic reasoning, and (2) proposing a novel \textbf{back-search} algorithm which mimics the top-down human-like learning procedure to propagate the error through the symbolic reasoning module efficiently. We further interpret the proposed learning framework as maximum likelihood estimation using Markov chain Monte Carlo sampling and the back-search algorithm as a Metropolis-Hastings sampler. The experiments are conducted on two weakly-supervised neural-symbolic tasks: (1) handwritten formula recognition on the newly introduced HWF dataset; (2) visual question answering on the CLEVR dataset. The results show that our approach significantly outperforms the RL methods in terms of performance, converging speed, and data efficiency. Our code and data are released at \url{https://liqing-ustc.github.io/NGS}.

A reflex is a simple closed loop control approach which tries to minimise an error but fails to do so because it will always react too late. An adaptive algorithm can use this error to learn a forward model with the help of predictive cues. For example a driver learns to improve their steering by looking ahead to avoid steering in the last minute. In order to process complex cues such as the road ahead deep learning is a natural choice. However, this is usually only achieved indirectly by employing deep reinforcement learning having a discrete state space. Here, we show how this can be directly achieved by embedding deep learning into a closed loop system and preserving its continuous processing. We show specifically how error back-propagation can be achieved in z-space and in general how gradient based approaches can be analysed in such closed loop scenarios. The performance of this learning paradigm is demonstrated using a line-follower both in simulation and on a real robot that show very fast and continuous learning.

JAGGAER has itself been moving towards closed-loop feedback systems that rely on machine learning for continuous improvement. Here we describe an example: the JAGGAER Digital Assistant. JAGGAER, ERP and third-party data is fed into a central data layer. The information is used in traditional analytics and reporting, but what is new is that algorithms are now providing real-time support for decisions, recommendations and actions. Typically, there might be several recommendations and the end-user takes a decision based on which of these makes most sense.

Many intelligent systems currently interact with others using at least one of fixed communication inputs or preset responses, resulting in rigid interaction experiences and extensive efforts developing a variety of scenarios for the system. Fixed inputs limit the natural behavior of the user in order to effectively communicate, and preset responses prevent the system from adapting to the current situation unless it was specifically implemented. Closed-loop interaction instead focuses on dynamic responses that account for what the user is currently doing based on interpretations of their perceived activity. Agents employing closed-loop interaction can also monitor their interactions to ensure that the user responds as expected. We introduce a closed-loop interactive agent framework that integrates planning and recognition to predict what the user is trying to accomplish and autonomously decide on actions to take in response to these predictions. Based on a recent demonstration of such an assistive interactive agent in a turn-based simulated game, we also discuss new research challenges that are not present in the areas of artificial intelligence planning or recognition alone.

Sequential learning of tasks using gradient descent leads to an unremitting decline in the accuracy of tasks for which training data is no longer available, termed catastrophic forgetting. Generative models have been explored as a means to approximate the distribution of old tasks and bypass storage of real data. Here we propose a cumulative closed-loop generator and embedded classifier using an AC-GAN architecture provided with external regularization by a small buffer. We evaluate incremental learning using a notoriously hard paradigm, single headed learning, in which each task is a disjoint subset of classes in the overall dataset, and performance is evaluated on all previous classes. First, we show that the variability contained in a small percentage of a dataset (memory buffer) accounts for a significant portion of the reported accuracy, both in multi-task and continual learning settings. Second, we show that using a generator to continuously output new images while training provides an up-sampling of the buffer, which prevents catastrophic forgetting and yields superior performance when compared to a fixed buffer. We achieve an average accuracy for all classes of 92.26% in MNIST and 76.15% in FASHION-MNIST after 5 tasks using GAN sampling with a buffer of only 0.17% of the entire dataset size. We compare to a network with regularization (EWC) which shows a deteriorated average performance of 29.19% (MNIST) and 26.5% (FASHION). The baseline of no regularization (plain gradient descent) performs at 99.84% (MNIST) and 99.79% (FASHION) for the last task, but below 3% for all previous tasks. Our method has very low long-term memory cost, the buffer, as well as negligible intermediate memory storage.

For large-scale industrial processes under closed-loop control, process dynamics directly resulting from control action are typical characteristics and may show different behaviors between real faults and normal changes of operating conditions. However, conventional distributed monitoring approaches do not consider the closed-loop control mechanism and only explore static characteristics, which thus are incapable of distinguishing between real process faults and nominal changes of operating conditions, leading to unnecessary alarms. In this regard, this paper proposes a distributed monitoring method for closed-loop industrial processes by concurrently exploring static and dynamic characteristics. First, the large-scale closed-loop process is decomposed into several subsystems by developing a sparse slow feature analysis (SSFA) algorithm which capture changes of both static and dynamic information. Second, distributed models are developed to separately capture static and dynamic characteristics from the local and global aspects. Based on the distributed monitoring system, a two-level monitoring strategy is proposed to check different influences on process characteristics resulting from changes of the operating conditions and control action, and thus the two changes can be well distinguished from each other. Case studies are conducted based on both benchmark data and real industrial process data to illustrate the effectiveness of the proposed method.