Goto

Collaborating Authors

 safety standard


Protect: Towards Robust Guardrailing Stack for Trustworthy Enterprise LLM Systems

Avinash, Karthik, Pareek, Nikhil, Hada, Rishav

arXiv.org Artificial Intelligence

The increasing deployment of Large Language Models (LLMs) across enterprise and mission-critical domains has underscored the urgent need for robust guardrailing systems that ensure safety, reliability, and compliance. Existing solutions often struggle with real-time oversight, multi-modal data handling, and explainability -- limitations that hinder their adoption in regulated environments. Existing guardrails largely operate in isolation, focused on text alone making them inadequate for multi-modal, production-scale environments. We introduce Protect, natively multi-modal guardrailing model designed to operate seamlessly across text, image, and audio inputs, designed for enterprise-grade deployment. Protect integrates fine-tuned, category-specific adapters trained via Low-Rank Adaptation (LoRA) on an extensive, multi-modal dataset covering four safety dimensions: toxicity, sexism, data privacy, and prompt injection. Our teacher-assisted annotation pipeline leverages reasoning and explanation traces to generate high-fidelity, context-aware labels across modalities. Experimental results demonstrate state-of-the-art performance across all safety dimensions, surpassing existing open and proprietary models such as WildGuard, LlamaGuard-4, and GPT-4.1. Protect establishes a strong foundation for trustworthy, auditable, and production-ready safety systems capable of operating across text, image, and audio modalities.


Rules keeping drones on leash could loosen with deregulation proposal from Congress

FOX News

An NYPD drone observed four minors, between the ages of 12 and 16 years old, riding on top of a train in the Bronx on Thursday as it passed multiple stations at a high speed. FIRST ON FOX: A new move by Congress would unleash civilian drone use across America's skies by establishing rules to allow them to be flown beyond a user's line of sight and using AI for approval to do so. Her LIFT Act, introduced in the House on Thursday, would require Transportation Secretary Sean Duffy to establish set performance and safety standards for BVLOS operations and review current aviation standards, which were designed with manned aircraft in mind. It would also require the Transportation secretary to deploy artificial intelligence to assist with processing waiver applications to allow civilian drones to fly BVLOS. Industry operators have long pushed for new BVLOS policy to replace the current system in which individuals must apply for waivers with the Federal Aviation Adminsitration (FAA) through a costly, cumbersome process to fly beyond the line of sight.


Tech startup, major airline partner to launch electric air taxi service

FOX News

Tech expert Kurt Knutsson discusses Joby Aviation and Virgin Atlantic planning to launch 200-mph U.K. air taxis linking airports and cities. Imagine skipping the gridlock and soaring over the English countryside, arriving at your destination in a fraction of the time. What sounds like a scene from a futuristic movie is about to become a reality in the U.K., thanks to a partnership between Joby Aviation, a California-based company, and Virgin Atlantic. They're teaming up to introduce electric air taxis to the U.K., revolutionizing how people travel between airports and nearby cities. Let's take a closer look at this development that could foreshadow the future of transportation here in the U.S. Get security alerts & expert tech tips – sign up for Kurt's The CyberGuy Report now. Joby Aviation's innovative aircraft are at the heart of this transportation revolution.


From Safety Standards to Safe Operation with Mobile Robotic Systems Deployment

Belzile, Bruno, Wanang-Siyapdjie, Tatiana, Karimi, Sina, Braga, Rafael Gomes, Iordanova, Ivanka, St-Onge, David

arXiv.org Artificial Intelligence

Mobile robotic systems are increasingly used in various work environments to support productivity. However, deploying robots in workplaces crowded by human workers and interacting with them results in safety challenges and concerns, namely robot-worker collisions and worker distractions in hazardous environments. Moreover, the literature on risk assessment as well as the standard specific to mobile platforms is rather limited. In this context, this paper first conducts a review of the relevant standards and methodologies and then proposes a risk assessment for the safe deployment of mobile robots on construction sites. The approach extends relevant existing safety standards to encompass uncovered scenarios. Safety recommendations are made based on the framework, after its validation by field experts.


Is Safety Standard Same for Everyone? User-Specific Safety Evaluation of Large Language Models

In, Yeonjun, Kim, Wonjoong, Yoon, Kanghoon, Kim, Sungchul, Tanjim, Mehrab, Kim, Kibum, Park, Chanyoung

arXiv.org Artificial Intelligence

As the use of large language model (LLM) agents continues to grow, their safety vulnerabilities have become increasingly evident. Extensive benchmarks evaluate various aspects of LLM safety by defining the safety relying heavily on general standards, overlooking user-specific standards. However, safety standards for LLM may vary based on a user-specific profiles rather than being universally consistent across all users. This raises a critical research question: Do LLM agents act safely when considering user-specific safety standards? Despite its importance for safe LLM use, no benchmark datasets currently exist to evaluate the user-specific safety of LLMs. To address this gap, we introduce U-SAFEBENCH, the first benchmark designed to assess user-specific aspect of LLM safety. Our evaluation of 18 widely used LLMs reveals current LLMs fail to act safely when considering user-specific safety standards, marking a new discovery in this field. To address this vulnerability, we propose a simple remedy based on chain-of-thought, demonstrating its effectiveness in improving user-specific safety. Our benchmark and code are available at https://github.com/yeonjun-in/U-SafeBench.


Musk's influence on Trump could lead to tougher AI standards, says scientist

The Guardian

Elon Musk's influence on a Donald Trump administration could lead to tougher safety standards for artificial intelligence, according to a leading scientist who has worked closely with the world's richest person on addressing AI's dangers. Max Tegmark said Musk's support for a failed AI bill in California underlined the billionaire's continued concern over an issue that did not feature prominently in Trump's campaign. However, Musk has warned regularly that unrestrained development of AI – broadly, computer systems performing tasks that typically require human intelligence – could be catastrophic for humanity. Last year, he was one of more than 30,000 signatories to a letter calling for a pause in work on powerful AI technology. Speaking to the Guardian at the Web Summit in Lisbon, Tegmark said Musk, who is expected to be heavily influential in the president-elect's administration, could persuade Trump to introduce standards that prevent the development of artificial general intelligence (AGI), the term for AI systems that match or exceed human levels of intelligence.


Something That Both Candidates Secretly Agree On

The Atlantic - Technology

If the presidential election has provided relief from anything, it has been the generative-AI boom. Neither Kamala Harris nor Donald Trump has made much of the technology in their public messaging, and they have not articulated particularly detailed AI platforms. Bots do not seem to rank among the economy, immigration, abortion rights, and other issues that can make or break campaigns. Americans are very invested, and very worried, about the future of artificial intelligence. Polling consistently shows that a majority of adults from both major parties support government regulation of AI, and that demand for regulation might even be growing.


Collaborative Conversation in Safe Multimodal Human-Robot Collaboration

Ferrari, Davide, Pupa, Andrea, Secchi, Cristian

arXiv.org Artificial Intelligence

In the context of Human-Robot Collaboration (HRC), it is crucial that the two actors are able to communicate with each other in a natural and efficient manner. The absence of a communication interface is often a cause of undesired slowdowns. On one hand, this is because unforeseen events may occur, leading to errors. On the other hand, due to the close contact between humans and robots, the speed must be reduced significantly to comply with safety standard ISO/TS 15066. In this paper, we propose a novel architecture that enables operators and robots to communicate efficiently, emulating human-to-human dialogue, while addressing safety concerns. This approach aims to establish a communication framework that not only facilitates collaboration but also reduces undesired speed reduction. Through the use of a predictive simulator, we can anticipate safety-related limitations, ensuring smoother workflows, minimizing risks, and optimizing efficiency. The overall architecture has been validated with a UR10e and compared with a state of the art technique. The results show a significant improvement in user experience, with a corresponding 23% reduction in execution times and a 50% decrease in robot downtime.


California's Draft AI Law Would Protect More than Just People

TIME - Tech

Few places in the world have more to gain from a flourishing AI industry than California. Few also have more to lose if the public's trust in the industry were suddenly shattered. In May, the California Senate passed SB 1047, a piece of AI safety legislation, in a vote of 32 to one, helping ensure the safe development of large-scale AI systems through clear, predictable, common-sense safety standards. The bill is now slated for a state assembly vote this week and, if signed into law by Governor Gavin Newsom, would represent a significant step in protecting California citizens and the state's burgeoning AI industry from malicious use. Late Monday, Elon Musk shocked many by announcing his support for the bill in a post on X. "This is a tough call and will make some people upset, but, all things considered, I think California should probably pass the SB 1047 AI safety bill," he wrote.


Safety-Driven Deep Reinforcement Learning Framework for Cobots: A Sim2Real Approach

Abbas, Ammar N., Mehak, Shakra, Chasparis, Georgios C., Kelleher, John D., Guilfoyle, Michael, Leva, Maria Chiara, Ramasubramanian, Aswin K

arXiv.org Artificial Intelligence

This study presents a novel methodology incorporating safety constraints into a robotic simulation during the training of deep reinforcement learning (DRL). The framework integrates specific parts of the safety requirements, such as velocity constraints, as specified by ISO 10218, directly within the DRL model that becomes a part of the robot's learning algorithm. The study then evaluated the efficiency of these safety constraints by subjecting the DRL model to various scenarios, including grasping tasks with and without obstacle avoidance. The validation process involved comprehensive simulation-based testing of the DRL model's responses to potential hazards and its compliance. Also, the performance of the system is carried out by the functional safety standards IEC 61508 to determine the safety integrity level. The study indicated a significant improvement in the safety performance of the robotic system. The proposed DRL model anticipates and mitigates hazards while maintaining operational efficiency. This study was validated in a testbed with a collaborative robotic arm with safety sensors and assessed with metrics such as the average number of safety violations, obstacle avoidance, and the number of successful grasps. The proposed approach outperforms the conventional method by a 16.5% average success rate on the tested scenarios in the simulations and 2.5% in the testbed without safety violations. The project repository is available at https://github.com/ammar-n-abbas/sim2real-ur-gym-gazebo.