Road Surface Condition Detection with Machine Learning using New York State Department of Transportation Camera Images and Weather Forecast Data

The NYSDOT evaluates road conditions by driving on roads and observing live cameras, tasks which are labor-intensive but necessary for making critical operational decisions during winter weather events. However, machine learning models can provide additional support for the NYSDOT by automatically classifying current road conditions across the state. In this study, convolutional neural networks and random forests are trained on camera images and weather data to predict road surface conditions. Models are trained on a hand-labeled dataset of 22,000 camera images, each classified by human labelers into one of six road surface conditions: severe snow, snow, wet, dry, poor visibility, or obstructed. Model generalizability is prioritized to meet the operational needs of the NYSDOT decision makers, and the weather-related road surface condition model in this study achieves an accuracy of 81.5% on completely unseen cameras. Keywords Winter weather Co-design Artificial intelligence Risk communication Hand-labeled dataset Highlights Developed a model to classify road surface conditions using image and weather data Achieved accuracy of 81.5% on completely unseen cameras for weather-related classes Integrated co-design with end-users and interdisciplinary collaboration Designed methods that prioritize model generalizability for operational applicability