In many real-world reinforcement learning (RL) problems, besides optimizing the main objective function, an agent must concurrently avoid violating a number of constraints. In particular, besides optimizing performance, it is crucial to guarantee the safety of an agent during training as well as deployment (e.g., a robot should avoid taking actions - exploratory or not - which irrevocably harm its hardware). To incorporate safety in RL, we derive algorithms under the framework of constrained Markov decision processes (CMDPs), an extension of the standard Markov decision processes (MDPs) augmented with constraints on expected cumulative costs.

In this interview series, we're meeting some of the AAAI/SIGAI Doctoral Consortium participants to find out more about their research. Tanmay Ambadkar is researching the reward structure in reinforcement learning, with the goal of providing generalizable solutions that can provide robust guarantees and are easily deployable. We caught up with Tanmay to find out more about his research, and in particular, the constrained reinforcement learning framework he has been working on. Tell us a bit about your PhD - where are you studying, and what is the topic of your research? I am a 4th year PhD candidate at The Pennsylvania State University, PA, USA.

Starmer'appeasing' big tech firms, says online safety campaigner A leading campaigner has accused the prime minister of appeasing big tech companies and being late to the party in regulating social media and artificial intelligence. Crossbench peer Baroness Kidron told the BBC Sir Keir Starmer needed to get on with it rather than launching more consultations. She also criticised the PM for citing his own experience as a father of two teenage children on social media, arguing that this did not make him an expert on the subject and that his family were sheltered compared to others. The government rejected the claims, with a spokesperson saying it had already introduced some of the strongest online safety protections in the world. Sir Keir has launched a consultation on banning under-16s from social media and promised to crackdown on the addictive elements of the apps.

In CBF-based control, the desired safety properties of the system are mapped to nonnegativity of a CBF, and the control input is chosen to ensure that the CBF remains nonnegative for all time.

Learning reliably safe autonomous control is one of the core problems in trustworthy autonomy.

We propose a novel preference modeling approach that utilizes AI feedback and only requires a small amount of human data.

Safety and trustworthiness are indispensable requirements for real-world applications of AI systems using large language models (LLMs).

Neural Control Barrier Functions (NCBFs) have shown significant promise in enforcing safety constraints on nonlinear autonomous systems.

The process for calculating these metrics is described in Appendix C. Moreover, to ensure the comparability between prediction performance metrics and driving performance metrics in the radar plot, we normalize all metrics to the scale of [0, 1]. In the subsequent section, we provide an overview of the DESPOT planner. These two values can only be inferred from history. The safety is represented by the normalized collision rate.