Visible & Thermal Imaging and Deep Learning Based Approach for Automated Robust Detection of Potholes to Prioritize Highway Maintenance [Brief]

Potholes are a significant pavement distress compromising safety and causing costly damage. They result from pavement deterioration due to aging, weather, and traffic overloads, with the Mountain Plains region particularly affected due to freeze/thaw cycles. Timely identification and repair of potholes are critical for effective highway maintenance. This research develops an automated deep learning-based pothole detection and mapping tool using the fusion of visible and thermal images. Visible images alone often fail in poor lighting or adverse weather conditions, whereas thermal images offer robust detection but lack texture details. Integrating both image types enhanced detection accuracy. We created a database of geotagged and labeled trios of visible, thermal, and fused images using a low-cost FLIR ONE thermal camera connected to a smartphone. Three machine-learning algorithms were proposed and compared: Anisotropic Diffusion Fusion (ADF) + Mask R-CNN, RTFNet, and RTFNet with Enhancement Parameters (EPs). The RTFNet method achieved the best F1-score of 93.7% in daytime and 90.9% in nighttime scenarios. A Bright-Dark detector was developed to optimize algorithm selection based on lighting conditions. Detected potholes were mapped using GPS data, and the trained algorithm was packaged into a GUI tool that can be used by highway maintenance teams.