Zhao, Yang (University of Electronic Science and Technology of China), Du, Jingyu (University of Electronic Science and Technology of China), Yang, Qihang (UESTC), Cai, Ningze (University of Electronic Science and Technology of China), Peng, Zhinan (University of Electronic Science and Technology of China), Zhan, Huiqin (University of Electronic Science and Technology of China), Cheng, Hong (University of Electronics Science and Technology of China)

PETRFusion: Multi-Sensor Fusion Based BEV Semantic Segmentation Network for Autonomous Driving

Scheduled for presentation during the Poster Session "Perception - Semantic segmentation" (ThBT17), Thursday, September 26, 2024, 14:30−16:30, Foyer

2024 IEEE 27th International Conference on Intelligent Transportation Systems (ITSC), September 24- 27, 2024, Edmonton, Canada

This information is tentative and subject to change. Compiled on October 7, 2024

Abstract

Bird's Eye View (BEV) based perception algorithms are receiving increasing attention in the field of autonomous driving, how to more effectively utilize multi-sensor data in BEV is a challenge. In this paper, we propose PETRFusion, a BEV segmentation network that integrates multiple cameras with a LiDAR sensor. We focus on the task of BEV segmentation, designing the network architecture based on the similarity between LiDAR point clouds and semantic segmentation maps. The proposed method uses point cloud features as queries for semantic segmentation and employs Transformers to fuse image features with point cloud features, thereby maximizing the preservation of 3D spatial information from the point cloud. Additionally, we enhance the 3D positional encoder of PETR to provide more reliable 3D image features. We conduct detailed validation on the nuScenes dataset and achieve state-of-the-art results.