Conventional reinforcement learning (RL) approaches often struggle to learn effective policies under sparse reward conditions, necessitating the manual design of complex, task-specific reward functions. To address this limitation, reinforcement learning from human feedback (RLHF) has emerged as a promising strategy that complements hand-crafted rewards with human-derived evaluation signals. However, most existing RLHF methods depend on explicit feedback mechanisms such as button presses or preference labels, which disrupt the natural interaction process and impose a substantial cognitive load on the user. We propose a novel reinforcement learning from implicit human feedback (RLIHF) framework that utilizes non-invasive elec-troencephalography (EEG) signals, specifically error-related potentials (ErrPs), to provide continuous, implicit feedback without requiring explicit user intervention. The proposed method adopts a pre-trained decoder to trans-i form raw EEG signals into probabilistic reward components, enabling effective policy learning even in the presence of sparse external rewards. We evaluate our approach in a simulation environment built on the MuJoCo physics engine, using a Kinova Gen2 robotic arm to perform a complex pick-and-place task that requires avoiding obstacles while manipulating target objects. The results show that agents trained with decoded EEG feedback achieve performance comparable to those trained with dense, manually designed rewards.

Brain-computer interfaces (BCIs) provide alternative communication methods for individuals with motor disabilities by allowing control and interaction with external devices. Non-invasive BCIs, especially those using electroencephalography (EEG), are practical and safe for various applications. However, their performance is often hindered by EEG non-stationarities, caused by changing mental states or device characteristics like electrode impedance. This challenge has spurred research into adaptive BCIs that can handle such variations. In recent years, interest has grown in using error-related potentials (ErrPs) to enhance BCI performance. ErrPs, neural responses to errors, can be detected non-invasively and have been integrated into different BCI paradigms to improve performance through error correction or adaptation. This research introduces a novel adaptive ErrP-based BCI approach using reinforcement learning (RL). We demonstrate the feasibility of an RL-driven adaptive framework incorporating ErrPs and motor imagery. Utilizing two RL agents, the framework adapts dynamically to EEG non-stationarities. Validation was conducted using a publicly available motor imagery dataset and a fast-paced game designed to boost user engagement. Results show the framework's promise, with RL agents learning control policies from user interactions and achieving robust performance across datasets. However, a critical insight from the game-based protocol revealed that motor imagery in a high-speed interaction paradigm was largely ineffective for participants, highlighting task design limitations in real-time BCI applications. These findings underscore the potential of RL for adaptive BCIs while pointing out practical constraints related to task complexity and user responsiveness.

Brain-computer interfaces (BCIs), as any other interaction modality based on physiological signals and body channels (e.g., muscular activity, speech and gestures), are prone to errors in the recognition of subject's intent. An elegant approach to improve the accuracy of BCIs consists in a verification procedure directly based on the presence of error-related potentials (ErrP) in the EEG recorded right after the occurrence of an error. Six healthy volunteer subjects with no prior BCI experience participated in a new human-robot interaction experiment where they were asked to mentally move a cursor towards a target that can be reached within a few steps using motor imagination. This experiment confirms the previously reported presence of a new kind of ErrP. These Interaction ErrP" exhibit a first sharp negative peak followed by a positive peak and a second broader negative peak ( 290, 350 and 470 ms after the feedback, respectively). But in order to exploit these ErrP we need to detect them in each single trial using a short window following the feedback associated to the response of the classifier embedded in the BCI. We have achieved an average recognition rate of correct and erroneous single trials of 81.8% and 76.2%, respectively. Furthermore, we have achieved an average recognition rate of the subject's intent while trying to mentally drive the cursor of 73.1%. These results show that it's possible to simultaneously extract useful information for mental control to operate a brain-actuated device as well as cognitive states such as error potentials to improve the quality of the brain-computer interaction. Finally, using a well-known inverse model (sLORETA), we show that the main focus of activity at the occurrence of the ErrP are, as expected, in the pre-supplementary motor area and in the anterior cingulate cortex."

Brain-Computer Interfaces (BCI) have allowed for direct communication from the brain to external applications for the automatic detection of cognitive processes such as error recognition. Error-related potentials (ErrPs) are a particular brain signal elicited when one commits or observes an erroneous event. However, due to the noisy properties of the brain and recording devices, ErrPs vary from instance to instance as they are combined with an assortment of other brain signals, biological noise, and external noise, making the classification of ErrPs a non-trivial problem. Recent works have revealed particular cognitive processes such as awareness, embodiment, and predictability that contribute to ErrP variations. In this paper, we explore the performance of classifier transferability when trained on different ErrP variation datasets generated by varying the levels of awareness and embodiment for a given task. In particular, we look at transference between observational and interactive ErrP categories when elicited by similar and differing tasks. Our empirical results provide an exploratory analysis into the ErrP transferability problem from a data perspective.

Meanwhile, applications of RL have only begun to expand beyond these constrained game environments to more diverse and complex real-world environments such as chip design [86], chemical reaction optimization [133] and performing long-term recommendations [45]. To further progress towards these more complex real-world environments, greater alleviation of challenges currently facing RL (e.g., generalization, robustness, scalability, and safety) is needed [7, 27, 72, 108]. Moreover, we can expect that as the complexity of environments increases, the difficulty in alleviating these challenges will increase as well [27]. For the purpose of this paper, we broadly define known RL challenges as either an aptitude or alignment problem. Aptitude encompasses challenges concerned with being able to learn. Aptitude includes ideas such as robustness, the ability of RL to perform a task (e.g., asymptotic performance) and generalize within/between environments of similar complexity; scalability, the ability of RL to scale up to more complex environment; and aptness, the rate at which a RL algorithm can learn to solve a problem or achieve a desired performance level. Likewise, alignment encompasses challenges concerned with learning as intended [7, 27, 72]. The hypothetical paperclip agent [18] is a classic example of misalignment.

Brain-computer interfaces (BCIs), as any other interaction modality based on physiological signals and body channels (e.g., muscular activity, speech and gestures), are prone to errors in the recognition of subject's intent. An elegant approach to improve the accuracy of BCIs consists in a verification procedure directly based on the presence of error-related potentials (ErrP) in the EEG recorded right after the occurrence of an error. Six healthy volunteer subjects with no prior BCI experience participated in a new human-robot interaction experiment where they were asked to mentally move a cursor towards a target that can be reached within a few steps using motor imagination. This experiment confirms the previously reported presence of a new kind of ErrP. These Interaction ErrP" exhibit a first sharp negative peak followed by a positive peak and a second broader negative peak ( 290, 350 and 470 ms after the feedback, respectively). But in order to exploit these ErrP we need to detect them in each single trial using a short window following the feedback associated to the response of the classifier embedded in the BCI. We have achieved an average recognition rate of correct and erroneous single trials of 81.8% and 76.2%, respectively. Furthermore, we have achieved an average recognition rate of the subject's intent while trying to mentally drive the cursor of 73.1%. These results show that it's possible to simultaneously extract useful information for mental control to operate a brain-actuated device as well as cognitive states such as error potentials to improve the quality of the brain-computer interaction. Finally, using a well-known inverse model (sLORETA), we show that the main focus of activity at the occurrence of the ErrP are, as expected, in the pre-supplementary motor area and in the anterior cingulate cortex."

Researchers from MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) and Boston University collaborated to design a system that combines neuroscience and machine learning to create a mind-reading robot. This robot can read your mind to learn if it is right or wrong. According to Forbes, "The researchers created a system that allows a robot to correct its mistakes in real time when a human observer notices that a mistake is being made. The observer just sits there and watches; she doesn't physically interact with the robot or anything else. If the observer recognizes that a mistake is occurring, the robot changes course and does the right thing."

Researchers from MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) and Boston University collaborated to design a system that combines neuroscience and machine learning to create a mind-reading robot. This robot can read your mind to learn if it is right or wrong. According to Forbes, "The researchers created a system that allows a robot to correct its mistakes in real time when a human observer notices that a mistake is being made. The observer just sits there and watches; she doesn't physically interact with the robot or anything else. If the observer recognizes that a mistake is occurring, the robot changes course and does the right thing."

A new brain-computer interface developed by scientists can read a person's thoughts in real time to identify when a robot makes a mistake, an advance that may lead to safer self-driving cars. Most existing brain-computer interface (BCI) require people to train with it and even learn to modulate their thoughts to help the machine understand, researchers said. By relying on brain signals called "error-related potentials" (ErrPs) that occur automatically when humans make a mistake or spot someone else making one, the new approach allows even complete novices to control a robot with their minds. This technology developed by researchers at the Boston University and the Massachusetts Institute of Technology (MIT) may offer intuitive and instantaneous ways of communicating with machines, for applications as diverse as supervising factory robots to controlling robotic prostheses. "When humans and robots work together, you basically have to learn the language of the robot, learn a new way to communicate with it, adapt to its interface," said Joseph DelPreto, a PhD candidate at MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL).

Robotics and its ongoing impact on humanity, particularly the workforce, is a frequent topic of discussion for Constellation Research. Now, a team at MIT's Computer Science and Artificial Intelligence lab, along with Boston University, is developing a technology that targets an important sub-topic within the robotics debate: How humans may interact with them in the future. What if we could develop robots that were a more natural extension of us and that could actually do whatever we are thinking? A feedback system developed at MIT enables human operators to correct a robot's choice in real-time using only brain signals. Using data from an electroencephalography (EEG) monitor that records brain activity, the system can detect if a person notices an error as a robot performs an object-sorting task.