Moon, Seongwoo (Korea Advanced Institute of Science and Technology(KAIST)), Seong, Hyunki (Korea Advanced Institute of Science and Technology), Kang, Jehun (Korea Advanced Institute of Science and Techonology (KAIST)), Ahn, Hojin (Korea Advanced Institute of Science and Technology (KAIST)), Shim, David Hyunchul (Korea Advanced Institute of Science and Technology)

Pixel-To-Control: End-To-End Autonomous Driving Via Spatio-Temporal BEV Architecture for Control Sequence Prediction

Scheduled for presentation during the Regular Session "S41a-Motion Planning, Trajectory Optimization, and Control for Autonomous Vehicles" (FR-LM-T41), Friday, November 21, 2025, 11:30−11:50, Broadbeach 1&2

2025 IEEE 28th International Conference on Intelligent Transportation Systems (ITSC), November 18-21, 2025, Gold Coast, Australia

This information is tentative and subject to change. Compiled on October 18, 2025

Abstract

This paper proposes a lightweight hierarchical transformer framework for fully end-to-end autonomous driving in urban environments. Unlike prior black-box approaches, the proposed method maintains modular interpretability while directly predicting low-level control commands from multi-view camera inputs. Visual features are encoded into a BEV representation, which is shared across a planning transformer and a control transformer to generate future trajectories and control actions, respectively. To enhance training efficiency, auxiliary perception tasks, such as BEV-based map and object decoding, are introduced only during the training phase. These tasks improve representation learning without increasing inference cost. The proposed framework is validated in a closed-loop simulation environment, achieving real-time performance at 15 Hz across urban scenarios, including intersections and highways. Experimental results demonstrate the strong potential of the method for scalable and interpretable end-to-end driving.