Autonomous driving systems face an ongoing challenge in delivering real-time visual perception on resource-constrained vehicular hardware. To resolve this bottleneck, researcher Hongjiang Liu developed the Lightweight Collaborative Detection and Segmentation Network, known as LCS-Net, in a study published in Discover Artificial Intelligence. The model executes concurrent object detection and semantic segmentation for complex traffic scenes while satisfying edge latency and memory limits. Evaluated on the Cityscapes benchmark, LCS-Net achieved 82.53 percent mean average precision at an intersection-over-union of 0.5 and 76.14 percent mean intersection-over-union across 19 classes. Operating with 4.19 million parameters and 14.71 billion floating-point operations, it maintained an inference speed of 91.27 frames per second. Compared to YOLOP, LCS-Net improved detection precision by 6.12 percentage points and segmentation accuracy by 5.96 percentage points while accelerating speed by 34.56 percent. Against YOLOPv2, it reduced parameters by 66.02 percent and floating-point operations by 64.36 percent while raising frame rates by 11.73 percent.
The architecture of LCS-Net tackles the computational burdens of conventional multi-task networks that stack heavy convolutional layers and suffer from background clutter. LCS-Net decouples spatial representation from channel mapping through deeply cascaded lightweight convolutions, directly mitigating the low-level processing bottleneck on vehicular processors. To capture multi-scale context without inflating model size, the network incorporates an improved atrous spatial pyramid pooling module alongside bidirectional gated spatial attention. LCS-Net regulates this collaborative learning via dynamic task weighting and dynamic feature-pixel focal loss.
The experimental evaluation was conducted on Ubuntu 22.04 using PyTorch 2.3 and CUDA 12.1, powered by an Intel Xeon Silver 4314 processor and an NVIDIA GeForce RTX 4090 GPU. The Cityscapes benchmark was partitioned according to its fine-annotation split of 2,975 training and 500 validation images. The semantic head evaluated 19 classes, while two-dimensional bounding boxes were generated from instance masks across eight categories: person, rider, car, truck, bus, train, motorcycle, and bicycle, excluding group regions and ignored labels. Images were resized to 640 by 640 pixels using letterbox formatting. Training lasted 300 epochs with a batch size of 32 using AdamW, an initial learning rate of 0.001, weight decay of 5×10−4, and cosine annealing. Encoders were pretrained on ImageNet-1K, while task heads received Kaiming initialization. Data augmentation included horizontal flipping, scale transformations within 0.76 to 1.24, HSV perturbations, translation, and Mosaic augmentation, disabled for the final 18 epochs. Comparison models were retrained under identical pipelines across random seeds 41, 73, and 107, reporting mean metrics.
On Cityscapes, comparative benchmarks highlighted LCS-Net’s accuracy and operational efficiency. LCS-Net achieved 82.53 percent mAP@0.5 in detection and 76.14 percent mIoU in segmentation. Compared with YOLOP, LCS-Net delivered improvements of 6.12 percentage points in detection and 5.96 percentage points in segmentation while executing 34.56 percent faster. Against YOLOPv2, LCS-Net reduced parameters by 66.02 percent to 4.19 million and computational operations by 64.36 percent to 14.71 GFLOPs, while boosting mAP@0.5 by 1.37 percentage points, raising mIoU by 1.32 percentage points, and accelerating inference speed by 11.73 percent. Evaluated against single-task networks, the specialized PIDNet-S achieved slightly higher segmentation mIoU but lacked object detection entirely.
Controlled ablation experiments on Cityscapes isolated the exact contributions of each structural innovation. Starting from baseline A0, replacing standard convolutions with the deeply cascaded lightweight backbone decreased parameters by 45.44 percent and floating-point operations by 48.20 percent, while increasing frame rate from 67.42 to 96.84 frames per second and raising mAP@0.5 and mIoU by 2.42 and 1.55 percentage points. Integrating the improved atrous spatial pyramid pooling module contributed another 1.75 percentage points to mAP@0.5 and 2.24 percentage points to mIoU. Gated spatial attention further improved detection and segmentation by 1.18 and 1.55 percentage points. Dynamic task weighting and focal loss added 1.24 percentage points in mAP@0.5 and 1.12 percentage points in mIoU without introducing inference parameters or FLOPs. In operator comparisons, the proposed depthwise design reduced FLOPs by 14.50 percent compared to MobileNetV2 inverted residuals and 10.83 percent compared to ShuffleNetV2 units, while boosting mAP@0.5 by 0.95 and 1.28 percentage points.
Analysis of training trajectories revealed clear optimization benefits over the 300 epochs. During the initial 75 epochs, LCS-Net’s loss decreased rapidly to approximately 1.22, converging to 0.83 by epoch 300. By contrast, the convergence losses of YOLOPv2 and YOLOP plateaued higher at 1.85 and 2.76. In detection, YOLOP’s growth slowed after epoch 120, converging to 76.41 percent mAP@0.5, whereas LCS-Net exceeded 80 percent by epoch 165 and reached 82.53 percent. In segmentation, YOLOP and YOLOPv2 exhibited persistent fluctuations with a standard deviation of 1.15 percent between epochs 150 and 250, converging to 70.18 percent and 74.82 percent mIoU. With the introduction of dynamic feature-pixel focal loss, LCS-Net’s feature evolution smoothed substantially, achieving a final mIoU of 76.14 percent—a 5.96 percentage point improvement over baseline. These convergence metrics demonstrate the loss function’s stability when classifying pixels along irregular road boundaries.
Class-specific evaluations on Cityscapes confirmed fine-grained accuracy across individual traffic categories. For major vehicles, detection precision reached 91.2 percent for cars, 85.4 percent for buses, and 88.6 percent for trucks, averaging a 9.3 percentage point improvement over YOLOPv2. For vulnerable road users, median accuracy rose to 81.5 percent for pedestrians and 77.2 percent for cyclists with tighter interquartile distributions, while its lower detection baseline increased by over 25 percent relative to YOLOP. In drivable area segmentation across three weather and lighting scenarios, LCS-Net achieved 92.3 percent IoU in daylight, 83.1 percent in rain, and 79.4 percent at night. While nighttime darkness caused YOLOP accuracy to collapse to 55.0 percent, LCS-Net restricted adverse performance decline to within 14 percent, underscoring the resilience of its collaborative joint optimization under adverse traffic conditions.
To evaluate cross-dataset generalization, the network was transferred to BDD100K using its official 70,000-image training and 10,000-image validation splits. Initialized with Cityscapes parameters and fine-tuned without structural changes, LCS-Net achieved an overall 80.91 percent mAP@0.5 and 92.16 percent drivable-area mIoU. In nighttime evaluations, the model maintained 72.19 percent mAP@0.5, exceeding YOLOPv2 by 8.27 percentage points in detection and 4.35 percentage points in segmentation mIoU. On the heavy-occlusion subset containing images with at least three occluded objects, LCS-Net surpassed YOLOPv2 by 9.66 percentage points in mAP@0.5 and 5.47 percentage points in mIoU. Under severe visual degradation combining blinding glare and heavy target occlusion, the system preserved a 69.37 percent detection recall rate and a 72.81 percent segmentation mIoU. Qualitative evaluations showed that LCS-Net reliably separated adjacent vehicles and maintained sharp road boundaries without erroneously expanding masks into reflective sidewalks.
Decoupled inference profiling demonstrated efficiency gains across each phase of the processing pipeline. Preprocessing latency for LCS-Net averaged 2.2 milliseconds, compared to 6.5 milliseconds for YOLOP and 5.8 milliseconds for YOLOPv2. Backbone feature extraction required 3.2 milliseconds, representing a 72.1 percent reduction compared to the 11.5 milliseconds consumed by YOLOPv2, while neck and head processing accounted for 4.1 milliseconds. Post-processing operations were similarly accelerated, with Non-Maximum Suppression requiring 1.2 milliseconds and mask rendering consuming 1.8 milliseconds, optimizing total post-processing time to 3.7 milliseconds and reducing delay by 61.4 percent compared to YOLOPv2. In hardware tests, LCS-Net maintained GPU utilization at 38 percent and CPU occupancy at 22 percent, avoiding the thermal risks of YOLOPv2’s 92 percent GPU load. Peak video memory was restricted to 1.6 gigabytes—a 75 percent reduction from YOLOPv2’s 6.4 gigabytes—while peak main memory was 1.1 gigabytes. At 1024 by 1024 resolution, LCS-Net maintained throughput above 45 frames per second, whereas YOLOPv2 dropped to 18 frames per second.
Despite these documented advantages, the study outlined specific failure modes under extreme operating conditions. Under severe headlight saturation, local contrast around distant pedestrians vanished, leading to missed detections. When two vehicles displayed only minimal visible surface areas, overlapping deep semantic responses caused the network to merge them into a single bounding box. Persistent road reflections on wet pavement also led segmentation masks to extend into non-drivable sidewalk boundaries. Because single-frame spatial attention cannot fully recover severely degraded information, the author recommended integrating low-confidence outputs with temporal tracking, multi-frame fusion, or uncertainty-triggered fallback strategies. Additionally, the study noted ongoing challenges in maintaining sharp feature boundaries under ultra-high-resolution inputs or dense target layouts. Future research will explore cross-modal knowledge distillation, dynamic network pruning, and model quantization to enhance generalization across challenging traffic scenarios.
Subject of Research: Lightweight multi-task deep learning for autonomous vehicle perception
Article Title: Collaborative object detection and semantic segmentation algorithm integrating lightweight deep learning for complex traffic scenes
Article References: Liu, H. (2026). Collaborative object detection and semantic segmentation algorithm integrating lightweight deep learning for complex traffic scenes. Discover Artificial Intelligence, 6(1), Article 1812. https://doi.org/10.1007/s44163-026-02588-z
Image Credits: AI Generated
DOI: 10.1007/s44163-026-02588-z
Keywords: Autonomous driving, Object detection, Semantic segmentation, Edge computing, Deep learning, Computer vision, Convolutional neural networks, Traffic scene perception, Collaborative, object, detection, semantic
