We propose a closed-loop power control algorithm for the downlink of the voice over LTE (VoLTE) radio bearer for an indoor environment served by small cells. The main contributions of our paper are: 1) proposing closed-loop power control for downlink VoLTE (or any packetized voice bearer), 2) deriving an upper bound of the loss in VoLTE downlink signal to noise plus interference ratio which the closed-loop power control has to overcome, 3) employing reinforcement learning to perform closed-loop power control, and 4) showing that this closed-loop power control method can improve the quality of VoLTE in a realistic network setup. Our simulation results have shown that our proposed algorithm significantly improved both voice retainability and mean opinion score as a result of maintaining the effective downlink signal to interference plus noise ratio against adverse network operational issues and faults.

Breakthroughs in studies of the nervous system over the past centuries have been dramatic, with key findings in experimental and theoretical neuroscience. These breakthroughs have helped to understand important aspects of how the brain works, which also provides new inspiration for technology and artificial intelligence. Moreover, the continuous development of new automatic data and image processing methods that arise out of neuroscience and genetics experiments, control technology applied to neurophysiology experiments and the use of computational neuroscience to represent and integrate information, feeds back knowledge about the brain and creates new opportunities for interaction between the nervous system and computational intelligence paradigms, robotics, brain-machine interfaces and all sorts of prosthetic and augmented reality devices. Among the maze of disciplines and approaches that try to understand how the brain works, there is a multidisciplinary approach to Neuroscience which looks at the nervous system in terms of its function: processing information. Theoretical models and experiments draw on closed-loop technology to reveal aspects of neuronal dynamics which do not come under traditional experimental protocols, hybrid circuits comprising live neurons and artificial neurons in a bidirectional interaction, behavior experiments with activity-dependent stimulation, and new protocols to personalize brain-machine interfaces.

Abstract--Manufacturers of safety-critical systems must make the case that their product is sufficiently safe for public deployment. Much of this case often relies upon critical event outcomes from real-world testing, requiring manufacturers to be strategic about how they allocate testing resources in order to maximize their chances of demonstrating system safety. This work frames the partially observable and belief-dependent problem of test scheduling as a Markov decision process, which can be solved efficiently to yield closed-loop manufacturer testing policies. By solving for policies over a wide range of problem formulations, we are able to provide high-level guidance for manufacturers and regulators on issues relating to the testing of safety-critical systems. This guidance spans an array of topics, including circumstances under which manufacturers should continue testing despite observed incidents, when manufacturers should test aggressively, and when regulators should increase or reduce the real-world testing requirements for an autonomous vehicle. I. INTRODUCTION Confidence must be established in safety-critical systems such as autonomous vehicles prior to their widespread release. Establishing confidence is difficult because the space of driving scenarios is vast and accidents are rare. Automotive manufacturers can build confidence by conducting test drives on public roadways and make the safety case based on the frequency of observed hazardous events like disengagements and traffic accidents. Each manufacturer must devise a testing strategy capable of providing sufficient evidence that their system is safe enough for widespread adoption. Real-world testing that is too aggressive may yield hazardous events that diminish confidence in system safety. However, a manufacturer that is reluctant to test their product may forfeit opportunities to identify and address shortcomings, and may ultimately not be able to compete in the market. The fundamental tension between the desire to thoroughly test a product and the urgency to forego further testing in favor of bringing the product to market is not unique to the automotive industry.

CRIXLabs (DBA Quantified Skin) is hosting a workshop on closed loop systems in machine learning. Jon Stenstrom et al (Co-Founder) will discuss data capture techniques to enable such a system and Shalini Ananda et al (Co-Founder) will discuss current tools that enable closed loop learning within our platform. We welcome those with machine learning experience and an interest in working with images and signal processing. Please do not hesitate to reach out to Shalini - shalini@quantifiedskin.com with any questions you may have.

This work discusses a closed-loop control strategy for complex systems utilizing scarce and streaming data. A discrete embedding space is first built using hash functions applied to the sensor measurements from which a Markov process model is derived, approximating the complex system's dynamics. A control strategy is then learned using reinforcement learning once rewards relevant with respect to the control objective are identified. This method is designed for experimental configurations, requiring no computations nor prior knowledge of the system, and enjoys intrinsic robustness. It is illustrated on two systems: the control of the transitions of a Lorenz 63 dynamical system, and the control of the drag of a cylinder flow. The method is shown to perform well.

Research has demonstrated the efficacy of closed-loop control of anesthesia using the bispectral index (BIS) of the electroencephalogram as the controlled variable, and the development of model-based, patient-adaptive systems has considerably improved anesthetic control. To further explore the use of model-based control in anesthesia, we investigated the application of reinforcement learning (RL) in the delivery of patient-specific, propofol-induced hypnosis in human volunteers. When compared to published performance metrics, RL control demonstrated accuracy and stability, indicating that further, more rigorous clinical study is warranted.

In this paper we present a general, flexible framework for learning mappings from images to actions by interacting with the environment. The basic idea is to introduce a feature-based image classifier in front of a reinforcement learning algorithm. The classifier partitions the visual space according to the presence or absence of few highly informative local descriptors that are incrementally selected in a sequence of attempts to remove perceptual aliasing. We also address the problem of fighting overfitting in such a greedy algorithm. Finally, we show how high-level visual features can be generated when the power of local descriptors is insufficient for completely disambiguating the aliased states. This is done by building a hierarchy of composite features that consist of recursive spatial combinations of visual features. We demonstrate the efficacy of our algorithms by solving three visual navigation tasks and a visual version of the classical ``Car on the Hill'' control problem.

Closed-loop control relies on sensory feedback that is usually assumed to be free. But if sensing incurs a cost, it may be costeffective to take sequences of actions in open-loop mode. We describe a reinforcement learning algorithm that learns to combine open-loop and closed-loop control when sensing incurs a cost. Although we assume reliable sensors, use of open-loop control means that actions must sometimes be taken when the current state of the controlled system is uncertain. This is a special case of the hidden-state problem in reinforcement learning, and to cope, our algorithm relies on short-term memory. The main result of the paper is a rule that significantly limits exploration of possible memory states by pruning memory states for which the estimated value of information is greater than its cost. We prove that this rule allows convergence to an optimal policy.

Closed-loop control relies on sensory feedback that is usually assumed to be free. But if sensing incurs a cost, it may be costeffective to take sequences of actions in open-loop mode. We describe a reinforcement learning algorithm that learns to combine open-loop and closed-loop control when sensing incurs a cost. Although we assume reliable sensors, use of open-loop control means that actions must sometimes be taken when the current state of the controlled system is uncertain. This is a special case of the hidden-state problem in reinforcement learning, and to cope, our algorithm relies on short-term memory. The main result of the paper is a rule that significantly limits exploration of possible memory states by pruning memory states for which the estimated value of information is greater than its cost. We prove that this rule allows convergence to an optimal policy.

Closed-loop control relies on sensory feedback that is usually assumed tobe free . But if sensing incurs a cost, it may be costeffective totake sequences of actions in open-loop mode. We describe a reinforcement learning algorithm that learns to combine open-loop and closed-loop control when sensing incurs a cost. Although weassume reliable sensors, use of open-loop control means that actions must sometimes be taken when the current state of the controlled system is uncertain. This is a special case of the hidden-state problem in reinforcement learning, and to cope, our algorithm relies on short-term memory.