Goto

Collaborating Authors

 road condition


RoadSens-4M: A Multimodal Smartphone & Camera Dataset for Holistic Road-way Analysis

Khandakar, Amith, Michelson, David, Rabbani, Shaikh Golam, Shafi, Fariya Bintay, Ahamed, Md. Faysal, Rahman, Khondokar Radwanur, Rahman, Md Abidur, Nabi, Md. Fahmidun, Ayari, Mohamed Arselene, Khan, Khaled, Suganthan, Ponnuthurai Nagaratnam

arXiv.org Artificial Intelligence

It's important to monitor road issues such as bumps and potholes to enhance safety and improve road conditions. Smartphones are equipped with various built - in sensors that offer a cost - effective and straightforward way to assess road quality. However, prog ress in this area has been slow due to the lack of high - quality, standardized datasets. This paper discusses a new dataset created by a mobile app that collects sensor data from devices like GPS, accelerometers, gyroscopes, magnetometers, gravity sensors, and orientation sensors. This dataset is one of the few that integrates Geographic Information System (GIS) data with weather information and video footage of road conditions, providing a comprehensive understanding of road issues with geographic context . The dataset allows for a clearer analysis of road conditions by compiling essential data, including vehicle speed, acceleration, rotation rates, and magnetic field intensity, along with the visual and spatial context provided by GIS, weather, and video dat a. Its goal is to provide funding for initiatives that enhance traffic management, infrastructure development, road safety, and urban planning . Additionally, the dataset will be publicly accessible to promote further research and innovation in smart transp ortation systems.


Road Surface Condition Detection with Machine Learning using New York State Department of Transportation Camera Images and Weather Forecast Data

Sutter, Carly, Sulia, Kara J., Bassill, Nick P., Wirz, Christopher D., Thorncroft, Christopher D., Rothenberger, Jay C., Przybylo, Vanessa, Cains, Mariana G., Radford, Jacob, Evans, David Aaron

arXiv.org Artificial Intelligence

The NYSDOT evaluates road conditions by driving on roads and observing live cameras, tasks which are labor-intensive but necessary for making critical operational decisions during winter weather events. However, machine learning models can provide additional support for the NYSDOT by automatically classifying current road conditions across the state. In this study, convolutional neural networks and random forests are trained on camera images and weather data to predict road surface conditions. Models are trained on a hand-labeled dataset of 22,000 camera images, each classified by human labelers into one of six road surface conditions: severe snow, snow, wet, dry, poor visibility, or obstructed. Model generalizability is prioritized to meet the operational needs of the NYSDOT decision makers, and the weather-related road surface condition model in this study achieves an accuracy of 81.5% on completely unseen cameras. Keywords Winter weather Co-design Artificial intelligence Risk communication Hand-labeled dataset Highlights Developed a model to classify road surface conditions using image and weather data Achieved accuracy of 81.5% on completely unseen cameras for weather-related classes Integrated co-design with end-users and interdisciplinary collaboration Designed methods that prioritize model generalizability for operational applicability


iWatchRoad: Scalable Detection and Geospatial Visualization of Potholes for Smart Cities

Sahoo, Rishi Raj, Mohanty, Surbhi Saswati, Mishra, Subhankar

arXiv.org Artificial Intelligence

Potholes on the roads are a serious hazard and maintenance burden. This poses a significant threat to road safety and vehicle longevity, especially on the diverse and under-maintained roads of India. In this paper, we present a complete end-to-end system called iWatchRoad for automated pothole detection, Global Positioning System (GPS) tagging, and real time mapping using OpenStreetMap (OSM). We curated a large, self-annotated dataset of over 7,000 frames captured across various road types, lighting conditions, and weather scenarios unique to Indian environments, leveraging dashcam footage. This dataset is used to fine-tune, Ultralytics You Only Look Once (YOLO) model to perform real time pothole detection, while a custom Optical Character Recognition (OCR) module was employed to extract timestamps directly from video frames. The timestamps are synchronized with GPS logs to geotag each detected potholes accurately. The processed data includes the potholes' details and frames as metadata is stored in a database and visualized via a user friendly web interface using OSM. iWatchRoad not only improves detection accuracy under challenging conditions but also provides government compatible outputs for road assessment and maintenance planning through the metadata visible on the website. Our solution is cost effective, hardware efficient, and scalable, offering a practical tool for urban and rural road management in developing regions, making the system automated. iWatchRoad is available at https://smlab.niser.ac.in/project/iwatchroad


Driver Assistant: Persuading Drivers to Adjust Secondary Tasks Using Large Language Models

Xiang, Wei, Li, Muchen, Yan, Jie, Zheng, Manling, Zhu, Hanfei, Jiang, Mengyun, Sun, Lingyun

arXiv.org Artificial Intelligence

Level 3 automated driving systems allows drivers to engage in secondary tasks while diminishing their perception of risk. In the event of an emergency necessitating driver intervention, the system will alert the driver with a limited window for reaction and imposing a substantial cognitive burden. To address this challenge, this study employs a Large Language Model (LLM) to assist drivers in maintaining an appropriate attention on road conditions through a " humanized " persuasive advice. Our tool leverages the road conditions encountered by Level 3 systems as triggers, proactively steering driver behavior via both visual and auditory routes. Empirical study indicates that our tool is effective in sustaining driver attention with reduced cognitive load and coordinating secondary tasks with takeover behavior. Our work provides insights into the potential of using LLMs to support drivers during multi-task automated driving. I. INTRODUCTION Level 3 automated driving systems allow drivers to perform secondary tasks while driving, yet drivers still need to pay attention to the road conditions .


PhysDrive: A Multimodal Remote Physiological Measurement Dataset for In-vehicle Driver Monitoring

Wang, Jiyao, Yang, Xiao, Hu, Qingyong, Tang, Jiankai, Liu, Can, He, Dengbo, Wang, Yuntao, Chen, Yingcong, Wu, Kaishun

arXiv.org Artificial Intelligence

Robust and unobtrusive in-vehicle physiological monitoring is crucial for ensuring driving safety and user experience. While remote physiological measurement (RPM) offers a promising non-invasive solution, its translation to real-world driving scenarios is critically constrained by the scarcity of comprehensive datasets. Existing resources are often limited in scale, modality diversity, the breadth of biometric annotations, and the range of captured conditions, thereby omitting inherent real-world challenges in driving. Here, we present PhysDrive, the first large-scale multimodal dataset for contactless in-vehicle physiological sensing with dedicated consideration on various modality settings and driving factors. PhysDrive collects data from 48 drivers, including synchronized RGB, near-infrared camera, and raw mmWave radar data, accompanied with six synchronized ground truths (ECG, BVP, Respiration, HR, RR, and SpO2). It covers a wide spectrum of naturalistic driving conditions, including driver motions, dynamic natural light, vehicle types, and road conditions. We extensively evaluate both signal-processing and deep-learning methods on PhysDrive, establishing a comprehensive benchmark across all modalities, and release full open-source code with compatibility for mainstream public toolboxes. We envision PhysDrive will serve as a foundational resource and accelerate research on multimodal driver monitoring and smart-cockpit systems.


Analogical Learning for Cross-Scenario Generalization: Framework and Application to Intelligent Localization

Chen, Zirui, Zhang, Zhaoyang, Xing, Ziqing, Li, Ridong, Yang, Zhaohui, Jin, Richeng, Huang, Chongwen, Yang, Yuzhi, Debbah, Mérouane

arXiv.org Artificial Intelligence

Existing learning models often exhibit poor generalization when deployed across diverse scenarios. It is primarily due to that the underlying reference frame of the data varies with the deployment environment and settings. However, despite that data of each scenario has a distinct reference frame, its generation generally follows common underlying physical rules. Based on this understanding, this article proposes a deep learning framework named analogical learning (AL), which implicitly retrieves the reference frame information associated with a scenario and then to make accurate prediction by relative analogy with other scenarios. Specifically, we design a bipartite neural network called Mateformer. Its first part captures the relativity within multiple latent feature spaces between the input data and a small amount of embedded data from the studied scenario, while its second part uses this relativity to guide the nonlinear analogy. We apply AL to the typical multi-scenario learning problem of intelligent wireless localization in cellular networks. Extensive experiments validate AL's superiority across three key dimensions. First, it achieves state-of-the-art accuracy in single-scenario benchmarks. Second, it demonstrates stable transferability between different scenarios, avoiding catastrophic forgetting. Finally, and most importantly, it robustly adapts to new, unseen scenarios--including dynamic weather and traffic conditions--without any tuning. All data and code are available at https://github.com/ziruichen-research/ALLoc.


Zero-Shot Image-Based Large Language Model Approach to Road Pavement Monitoring

Xu, Shuoshuo, Zhao, Kai, Loney, James, Li, Zili, Visentin, Andrea

arXiv.org Artificial Intelligence

Effective and rapid evaluation of pavement surface condition is critical for prioritizing maintenance, ensuring transportation safety, and minimizing vehicle wear and tear. While conventional manual inspections suffer from subjectivity, existing machine learning-based methods are constrained by their reliance on large and high-quality labeled datasets, which require significant resources and limit adaptability across varied road conditions. The revolutionary advancements in Large Language Models (LLMs) present significant potential for overcoming these challenges. In this study, we propose an innovative automated zero-shot learning approach that leverages the image recognition and natural language understanding capabilities of LLMs to assess road conditions effectively. Multiple LLM-based assessment models were developed, employing prompt engineering strategies aligned with the Pavement Surface Condition Index (PSCI) standards. These models' accuracy and reliability were evaluated against official PSCI results, with an optimized model ultimately selected. Extensive tests benchmarked the optimized model against evaluations from various levels experts using Google Street View road images. The results reveal that the LLM-based approach can effectively assess road conditions, with the optimized model -employing comprehensive and structured prompt engineering strategies -outperforming simpler configurations by achieving high accuracy and consistency, even surpassing expert evaluations. Moreover, successfully applying the optimized model to Google Street View images demonstrates its potential for future city-scale deployments. These findings highlight the transformative potential of LLMs in automating road damage evaluations and underscore the pivotal role of detailed prompt engineering in achieving reliable assessments.


Intelligent Electric Power Steering: Artificial Intelligence Integration Enhances Vehicle Safety and Performance

Vyas, Vikas, Shetiya, Sneha Sudhir

arXiv.org Artificial Intelligence

Electric Power Steering (EPS) systems utilize electric motors to aid users in steering their vehicles, which provide additional precise control and reduced energy consumption compared to traditional hydraulic systems. EPS technology provides safety,control and efficiency.. This paper explains the integration of Artificial Intelligence (AI) into Electric Power Steering (EPS) systems, focusing on its role in enhancing the safety, and adaptability across diverse driving conditions. We explore significant development in AI-driven EPS, including predictive control algorithms, adaptive torque management systems, and data-driven diagnostics. The paper presents case studies of AI applications in EPS, such as Lane centering control (LCC), Automated Parking Systems, and Autonomous Vehicle Steering, while considering the challenges, limitations, and future prospects of this technology. This article discusses current developments in AI-driven EPS, emphasizing on the benefits of improved safety, adaptive control, and predictive maintenance. Challenges in integrating AI in EPS systems. This paper addresses cybersecurity risks, ethical concerns, and technical limitations,, along with next steps for research and implementation in autonomous, and connected vehicles.


PTR: A Pre-trained Language Model for Trajectory Recovery

Wei, Tonglong, Lin, Yan, Lin, Youfang, Guo, Shengnan, Hu, Jilin, Cong, Gao, Wan, Huaiyu

arXiv.org Artificial Intelligence

Spatiotemporal trajectory data is vital for web-of-things services and is extensively collected and analyzed by web-based hardware and platforms. However, issues such as service interruptions and network instability often lead to sparsely recorded trajectories, resulting in a loss of detailed movement data. As a result, recovering these trajectories to restore missing information becomes essential. Despite progress, several challenges remain unresolved. First, the lack of large-scale dense trajectory data hampers the performance of existing deep learning methods, which rely heavily on abundant data for supervised training. Second, current methods struggle to generalize across sparse trajectories with varying sampling intervals, necessitating separate re-training for each interval and increasing computational costs. Third, external factors crucial for the recovery of missing points are not fully incorporated. To address these challenges, we propose a framework called PTR. This framework mitigates the issue of limited dense trajectory data by leveraging the capabilities of pre-trained language models (PLMs). PTR incorporates an explicit trajectory prompt and is trained on datasets with multiple sampling intervals, enabling it to generalize effectively across different intervals in sparse trajectories. To capture external factors, we introduce an implicit trajectory prompt that models road conditions, providing richer information for recovering missing points. Additionally, we present a trajectory embedder that encodes trajectory points and transforms the embeddings of both observed and missing points into a format comprehensible to PLMs. Experimental results on two public trajectory datasets with three sampling intervals demonstrate the efficacy and scalability of PTR.


Online Adaptation of Learned Vehicle Dynamics Model with Meta-Learning Approach

Tsuchiya, Yuki, Balch, Thomas, Drews, Paul, Rosman, Guy

arXiv.org Artificial Intelligence

We represent a vehicle dynamics model for autonomous driving near the limits of handling via a multi-layer neural network. Online adaptation is desirable in order to address unseen environments. However, the model needs to adapt to new environments without forgetting previously encountered ones. In this study, we apply Continual-MAML to overcome this difficulty. It enables the model to adapt to the previously encountered environments quickly and efficiently by starting updates from optimized initial parameters. We evaluate the impact of online model adaptation with respect to inference performance and impact on control performance of a model predictive path integral (MPPI) controller using the TRIKart platform. The neural network was pre-trained using driving data collected in our test environment, and experiments for online adaptation were executed on multiple different road conditions not contained in the training data. Empirical results show that the model using Continual-MAML outperforms the fixed model and the model using gradient descent in test set loss and online tracking performance of MPPI.