Combining 3D Urban Objects from All Around the World to Improve Object Classification and Semantic Segmentation

Abstract

Abstract Given the growing number of applications in urban planning and large-scale digital twins, the development of effective solutions for urban point cloud classification is of extreme interest for the R&D community and commercial sector. State-of-the-art neural networks commonly lack adequate cross-dataset generalisation ability, mainly due to varying sensors and data collection platforms, object shape differences, as well as the presence of under-represented objects and imbalanced classes, especially in case of dense and high-resolution reality-based 3D data. This work demonstrates how the recently released ESTATE dataset (A large dataset of under-represented urban objects— https://github.com/3DOM-FBK/ESTATE ), full of thousands of under-represented urban objects, such as traffic lights, electrical poles, pylons, and ventilation units, spread over 13 classes, can improve the performance of state-of-the-art point cloud classification algorithms. Experiments with different neural networks and several testing configurations with sensor-specific inputs (coordinate, intensity, and colour) show the effectiveness of this dataset in enhancing the classification capabilities and increasing cross-dataset generalisation. Moreover, reported results show not only the adaptation of object classification networks to the semantic segmentation pipeline, but also an improvement of semantic segmentation performance by increasing the distribution of under-represented classes with the ESTATE dataset.