Ing. Jan Švancar

The visual predictor of a drivable surface friction ahead of the vehicle is presented. The image recognition neural network is trained in self-supervised fashion, as an alternative to tedious, error-prone, and subjective human annotation. The training images are labelled automatically by surface friction estimates from vehicle response during ordinary driving. The Unscented Kalman Filter algorithm is used to estimate tire-to-road interface friction parameters, taking into account the highly nonlinear nature of tire dynamics. Finally, the overall toolchain was validated using an experimental subscale platform and real-world driving scenarios. The resulting visual predictor was trained using about 3 000 images and validated on an unseen set of 800 test images, achieving 0.98 crosscorrelation between the visually predicted and the estimated value of surface friction.

A self-supervised method to train a visual predictor of drivable surface roughness in front of a vehicle is proposed. A convolutional neural network taking a single camera image is trained on a dataset labeled automatically by a cross-modal supervision. The dataset is collected by driving a vehicle on various surfaces, while synchronously recording images and accelerometer data. The surface images are labeled by the local roughness measured using the accelerometer signal aligned in time. Our experiments show that the proposed training scheme results in accurate visual predictor. The correlation coefficient between the visually predicted roughness and the true roughness (measured by the accelerometer) is 0.9 on our independent test set of about 1000 images. The proposed method clearly outperforms a baseline method which has the correlation of 0.3 only. The baseline is based on surface texture strength without any training. Moreover, we show a coarse map of local surface roughness, which is implemented by scanning an input image with the trained convolutional network. The proposed method provides automatic and objective road condition assessment, enabling a cheap and reliable alternative to manual data annotation, which is infeasible in a large scale.