Alsakabi, Mohammed (Carnegie Mellon University), Erickson, Aidan (Carnegie Mellon University), Dolan, John (Carnegie Mellon University), Tonguz, Ozan (Carnegie Mellon University)

Toward a Low-Cost Perception System in Autonomous Vehicles: A Spectrum Learning Approach

Scheduled for presentation during the Regular Session "S42b-Safety and Risk Assessment for Autonomous Driving Systems" (FR-EA-T42), Friday, November 21, 2025, 14:30−14:50, Broadbeach 3

2025 IEEE 28th International Conference on Intelligent Transportation Systems (ITSC), November 18-21, 2025, Gold Coast, Australia

This information is tentative and subject to change. Compiled on October 18, 2025

Abstract

We present a cost-effective new approach for generating denser depth maps for Autonomous Driving (AD) and Autonomous Vehicles (AVs) by integrating the images obtained from deep neural network (DNN) 4D radar detectors with conventional camera RGB images. Our approach introduces a novel pixel positional encoding algorithm inspired by Bartlett's spatial spectrum estimation technique. This algorithm transforms both radar depth maps and RGB images into a unified pixel image subspace called the Spatial Spectrum, facilitating effective learning based on their similarities and differences. This method effectively leverages high-resolution camera images to train radar depth map generative models, addressing the limitations of conventional radar detectors in complex vehicular environments, thus sharpening the radar output. We develop spectrum estimation algorithms tailored for radar depth maps and RGB images, a comprehensive training framework for data-driven generative models, and a camera-radar deployment scheme for AV operation. Our results demonstrate that our approach also outperforms the state-of-the-art (SOTA) by 24.24% 52.59% in terms of the Unidirectional Chamfer Distance (UCD) and the Mean Absolute Error (MAE), respectively. Python codes and demonstration videos are available on our GitHub repository.