Miclea, Vlad (Technical University of Cluj-Napoca), Petrovai, Andra (Technical University of Cluj-Napoca), Nedevschi, Sergiu (Technical University of Cluj-Napoca)

SemBins: Semantic Bins for Monocular Depth Estimation in Aerial Scenarios

Scheduled for presentation during the Poster Session "Perception - Semantic segmentation" (ThBT17), Thursday, September 26, 2024, 14:30−16:30, Foyer

2024 IEEE 27th International Conference on Intelligent Transportation Systems (ITSC), September 24- 27, 2024, Edmonton, Canada

This information is tentative and subject to change. Compiled on October 8, 2024

Abstract

Monocular depth estimation (MDE) in aerial scenarios presents a significant challenge due to various factors such as varying altitudes, perspective distortions, and occlusions. Leveraging scene priors, like surfaces, can preserve structural integrity, with semantic segmentation serving as a valuable tool for attaining this objective. This work introduces a novel approach to depth estimation in aerial scenarios by integrating semantic information with depth features. Thus, we propose a novel learnable module, called semBins, that combines semantics with depth features at the depth discretization level in the MDE network. To maximize the effectiveness of this technique, a novel pseudo-random semantic-based query-response training procedure is proposed, which extracts image patches and predicts depth distributions of local neighborhoods for each pixel. This approach ensures that the semantic labels guide the patch extraction process, narrowing down the bin distribution search space. We show that semBins can achieve top performance on both synthetic and real-life aerial-based datasets.