Pianists equipped with a third robotic thumb were able to adapt to using the extra digit after just one hour of practice, Imperial research has found. Researchers from Imperial College London have been working to understand how well the human brain can cope with using extra limbs made possible through robotic technology. To assess the effects of such devices on the brain, the research team, led by Professor Aldo Faisal of Imperial's Department of Bioengineering, attached robotic extra thumbs to a group of piano players next to their little finger that could be controlled by the pianists using their feet. The inspiration for the study came from humanity's long-standing fascination with characters who have extra limbs, such as those found in Indian mythology and modern superhero comic books. Professor Faisal said: "We wanted to see whether we can enhance people using extra limbs; specifically, an extra thumb that sits opposite--contralaterally, technically speaking--the thumb of your right hand. "The basic question was: can we use 11 fingers in a task that requires actual skills?

Reinforcement Learning (RL) is emerging as tool for tackling complex control and decision-making problems. However, in high-risk environments such as healthcare, manufacturing, automotive or aerospace, it is often challenging to bridge the gap between an apparently optimal policy learnt by an agent and its real-world deployment, due to the uncertainties and risk associated with it. Broadly speaking RL agents face two kinds of uncertainty, 1. aleatoric uncertainty, which reflects randomness or noise in the dynamics of the world, and 2. epistemic uncertainty, which reflects the bounded knowledge of the agent due to model limitations and finite amount of information/data the agent has acquired about the world. These two types of uncertainty carry fundamentally different implications for the evaluation of performance and the level of risk or trust. Yet these aleatoric and epistemic uncertainties are generally confounded as standard and even distributional RL is agnostic to this difference. Here we propose how a distributional approach (UA-DQN) can be recast to render uncertainties by decomposing the net effects of each uncertainty. We demonstrate the operation of this method in grid world examples to build intuition and then show a proof of concept application for an RL agent operating as a clinical decision support system in critical care

Pianists who have been fitted with a third robotic thumb are able to adjust their playing style to suit their new 11 digits in just an hour, according to researchers. To determine how well human motor control capabilities cope with augmented limbs, a team from Imperial College London strapped a robot thumb to a pianist. The'third thumb' is strapped to a user's hand next to the little finger and controlled by electrical signals generated when the pianist moves their foot. To test how useful this extra limb is, the team, led by Aldo Faisal, recruited six experienced pianists and six people who didn't play the piano. They found that the volunteer pianists were able to learn to play the piano with 11 digits rather than 10 within an hour of being shown how to use the extra thumb regardless of their experience with the piano itself.

Functional Electrical Stimulation (FES) is an established and safe technique for contracting muscles by stimulating the skin above a muscle to induce its contraction. However, an open challenge remains on how to restore motor abilities to human limbs through FES, as the problem of controlling the stimulation is unclear. We are taking a robotics perspective on this problem, by developing robot learning algorithms that control the ultimate humanoid robot, the human body, through electrical muscle stimulation. Human muscles are not trivial to control as actuators due to their force production being non-stationary as a result of fatigue and other internal state changes, in contrast to robot actuators which are wellunderstood and stationary over broad operation ranges. We present our Deep Reinforcement Learning approach to the control of human muscles with FES, using a recurrent neural network for dynamic state representation, to overcome the unobserved elements of the behaviour of human muscles under external stimulation. We demonstrate our technique both in neuromuscular simulations but also experimentally on a human. Our results show that our controller can learn to manipulate human muscles, applying appropriate levels of stimulation to achieve the given tasks while compensating for advancing muscle fatigue which arises throughout the tasks. Additionally, Figure 1: Our 3 scenarios for FES control: (a) arm vertical motion our technique can learn quickly enough to be implemented in in simulation (b) and human volunteers, (c) arm horizontal motion real-world human-in-the-loop settings.

Assistive and Wearable Robotics have the potential to support humans with different types of motor impairments to become independent and fulfil their activities of daily living successfully. The success of these robot systems, however, relies on the ability to meaningfully decode human action intentions and carry them out appropriately. Neural interfaces have been explored for use in such system with several successes, however, they tend to be invasive and require training periods in the order of months. We present a robotic system for human augmentation, capable of actuating the user's arm and fingers for them, effectively restoring the capability of reaching, grasping and manipulating objects; controlled solely through the user's eye movements. We combine wearable eye tracking, the visual context of the environment and the structural grammar of human actions to create a cognitive-level assistive robotic setup that enables the users in fulfilling activities of daily living, while conserving interpretability, and the agency of the user. The interface is worn, calibrated and ready to use within 5 minutes. Users learn to control and make successful use of the system with an additional 5 minutes of interaction. The system is tested with 5 healthy participants, showing an average success rate of $96.6\%$ on first attempt across 6 tasks.

Our aim is to establish a framework where reinforcement learning (RL) of optimizing interventions retrospectively allows us a regulatory compliant pathway to prospective clinical testing of the learned policies in a clinical deployment. We focus on infections in intensive care units which are one of the major causes of death and difficult to treat because of the complex and opaque patient dynamics, and the clinically debated, highly-divergent set of intervention policies required by each individual patient, yet intensive care units are naturally data rich. In our work, we build on RL approaches in healthcare ("AI Clinicians"), and learn off-policy continuous dosing policy of pharmaceuticals for sepsis treatment using historical intensive care data under partially observable MDPs (POMDPs). POMPDs capture uncertainty in patient state better by taking in all historical information, yielding an efficient representation, which we investigate through ablations. We compensate for the lack of exploration in our retrospective data by evaluating each encountered state with a best-first tree search. We mitigate state distributional shift by optimizing our policy in the vicinity of the clinicians' compound policy. Crucially, we evaluate our model recommendations using not only conventional policy evaluations but a novel framework that incorporates human experts: a model-agnostic pre-clinical evaluation method to estimate the accuracy and uncertainty of clinician's decisions versus our system recommendations when confronted with the same individual patient history ("shadow mode").

Most experiments with artificial intelligence in medicine thus far have worked on the diagnostic side. AI systems have used computer vision to examine images like X-rays or pathology slides, and they have combed through data in electronic medical records to spot subtle patterns that humans can miss. Just last week, IEEE Spectrum reported on hospitals that are trying out AI systems that identify patients with the first signs of sepsis, a life-threatening condition where the body responds to infection with widespread inflammation, which can lead to organ failure. Sepsis is the third leading cause of death worldwide, and the primary cause of death in hospitals. But the technology that goes by the name AI Clinician, described today in a paper in Nature Medicine, doesn't diagnose--it makes decisions.

Big data and artificial intelligence (AI) are helping doctors make better treatment decisions and improve the delivery of care, say Imperial experts. Dr Aldo Faisal, Reader in Neurotechnology jointly at the Department of Bioengineering and the Department of Computing at the College, and Mr Erik Mayer, Clinical Senior Lecturer at the Department of Surgery & Cancer at the College and Honorary Consultant Surgeon at The Royal Marsden NHS Foundation Trust and Imperial College Healthcare NHS Trust, presented their work on big data and AI at the recent Imperial College Academic Health Science Centre (AHSC) seminar series. In a packed lecture theatre, Dr Faisal outlined his work on using artificial intelligence to help doctors make better treatment decisions for the condition Duchenne muscular dystrophy. Duchenne muscular dystrophy is a genetic muscle wasting disease that begins in childhood and mainly affects boys. It usually renders patients unable to walk by age 12 and carries an average life expectancy of 26 years.

Leading experts in artificial intelligence (AI) debated whether machines can ever be truly creative during an event at Imperial College London. The panel debate was part of the Night of Ideas, a programme of free debates exploring the latest ideas behind issues central to our times organised by the Institut Français. Academics from Imperial and other London institutions, were joined by a director from Spotify to talk about their latest research involving AI and discussed the creativity potential of computer software. The experts debated how developments in AI were enabling machines to produce music and paintings but questioned whether this meant they were being truly creative and should be recognised as artists in their own right. Dr Aldo Faisal, from the Department of Bioengineering and Department of Computing, gave his thoughts on what is powering the AI revolution.

Third Generation Artificial Intelligence The 21st Century is presently transitioning into the Era of Deep Learning, where leaders like Google are using deep learning algorithms in neural networks find out impacting member from data and use that data to predict a model. The technique uses recurrent neural networks (RNN) to allow the system to auto-correlate historical data and live feeds. For example, if a company took historical data from the top 10 countries by Gross Domestic Product (GDP) with employment ratios, inflation data, gold prices, and stock exchange data and put it all into a historical system, along with live feeds, the neural network would auto-correlate the data. As with any system, the analysis will only be as good as the data; how ever, the challenge with deep learning AI systems is that in most cases, the historical data was not collected with AI systems in mind.