TSLA: A Task-Specific Learning Adaptation for Semantic Segmentation on Autonomous Vehicles Platform

Abstract--Autonomous driving platforms encounter diverse driving scenarios, each with varying hardware resources and precision requirements. Given the computational limitations of embedded devices, it is crucial to consider computing costs when deploying on target platforms like the DRIVE PX 2. Our objective is to customize the semantic segmentation network according to the computing power and specific scenarios of autonomous driving hardware. We implement dynamic adaptability through a three-tier control mechanism--width multiplier, classifier depth, and classifier kernel--allowing fine-grained control over model components based on hardware constraints and task requirements. This adaptability facilitates broad model scaling, targeted refinement of the final layers, and scenario-specific optimization of kernel sizes, leading to improved resource allocation and performance. Additionally, we leverage Bayesian Optimization with surrogate modeling to efficiently explore hyperparameter spaces under tight computational budgets. It scales its Multiply-Accumulate Operations (MACs) for T ask-Specific Learning Adaptation (TSLA), resulting in alternative configurations tailored to diverse self-driving tasks. These TSLA customizations maximize computational capacity and model accuracy, optimizing hardware utilization. Real-time scene understanding is essential for perception in mobile robotics and self-driving cars. Semantic segmentation, which classifies each pixel in an image into categories like'road' or'sky,' is crucial for assisting in localization and planning for informed decision-making. Semantic segmentation with convolutional neural networks (CNNs) can be computationally expensive, especially with large datasets or complex architectures. Training and deploying CNNs often require significant computational resources, making it a major challenge. Previous research has explored various strategies to address these computational demands. Wang are with the Department of Electrical and Computer Engineering, Northeastern University, Boston, MA, 02115.