UA-CF: Uncertainty-Aware Camera-LiDAR Fusion for Robust 3D Object Detection Under Adverse Weather
DOI:
https://doi.org/10.54097/ybd75q69Keywords:
3D object detection, Camera-LiDAR fusion, Uncertainty estimation, Adverse weather, Autonomous driving, Sensor reliability, CalibrationAbstract
Robust 3D object detection remains a critical bottleneck for autonomous driving under fog, rain, and snow because camera and LiDAR streams do not fail in the same way. Cameras provide dense semantic cues but lose contrast under scattering and low visibility, whereas LiDAR preserves metric structure but suffers from sparsification, backscatter, and spurious returns under precipitation and dense aerosols. This paper proposes UA-CF, an uncertainty-aware camera-LiDAR fusion framework that dynamically allocates feature-level weights according to modality reliability. The method combines camera semantic features, LiDAR bird's-eye-view geometric features, label-free sensor-quality estimates, inverse-uncertainty fusion, and a calibration-aware objective. Unlike fixed fusion, UA-CF is designed to avoid over-trusting a degraded modality while retaining complementary information from both streams. Building on the positive gated vision-LiDAR motivation, our work focuses on a camera-LiDAR setting and introduces explicit reliability weighting for adverse-weather 3D detection. Because large-scale public datasets could not be downloaded in the local artifact environment, we provide a transparent synthetic BEV-proposal benchmark rather than unverifiable leaderboard claims. Across five random seeds, UA-CF obtains 0.949 AP overall, compared with 0.916 for equal fusion and 0.905 for logistic concatenation. Under fog, UA-CF reaches 0.988 AP and assigns 0.842 mean weight to LiDAR; under rain, it reaches 0.959 AP and shifts 0.683 mean weight to camera features. These reproducible results support the central claim that uncertainty-aware fusion improves robustness under asymmetric sensor degradation while preserving clear-weather performance.
Downloads
References
[1] Geiger, A., Lenz, P., & Urtasun, R. (2012). Are we ready for autonomous driving? The KITTI vision benchmark suite. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 3354-3361). IEEE. https://doi.org/10.1109/CVPR.2012.6248074
[2] Caesar, H., Bankiti, V., Lang, A. H., Vora, S., Liong, V. E., Xu, Q., Krishnan, A., Pan, Y., Baldan, G., & Beijbom, O. (2020). nuScenes: A multimodal dataset for autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 11621-11631). IEEE. https://doi.org/10.1109/CVPR42600.2020.01164
[3] Zhang, F., Guo, Z., Ding, J., Yang, J., & Liu, W. (2026). Adaptive sensor fusion for robust perception in dense fog: A gated vision and LiDAR integration framework. Sensors, 26(12), Article 3728. https://doi.org/10.3390/s26123728
[4] Sun, P., Kretzschmar, H., Dotiwalla, X., Chouard, A., Patnaik, V., Tsui, P., Guo, J., Zhou, Y., Chai, Y., Caine, B., Vasudevan, V., Han, W., Ngiam, J., Zhao, H., Timofeev, A., Ettinger, S., Krivokon, M., Gao, A., Joshi, A., ... & Anguelov, D. (2020). Scalability in perception for autonomous driving: Waymo open dataset. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 2446-2454). IEEE. https://doi.org/10.1109/CVPR42600.2020.00252
[5] Bijelic, M., Gruber, T., Mannan, F., Kraus, F., Ritter, W., Dietmayer, K., & Heide, F. (2020). Seeing through fog without seeing fog: Deep multimodal sensor fusion in unseen adverse weather. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 11682-11692). IEEE. https://doi.org/10.1109/CVPR42600.2020.01170
[6] Sakaridis, C., Dai, D., & Van Gool, L. (2021). ACDC: The adverse conditions dataset with correspondences for semantic driving scene understanding. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) (pp. 10765-10775). IEEE. https://doi.org/10.1109/ICCV48922.2021.01061
[7] Sakaridis, C., Dai, D., & Van Gool, L. (2018). Semantic foggy scene understanding with synthetic data. International Journal of Computer Vision, 126(9), 973-992. https://doi.org/10.1007/s11263-018-1072-8
[8] Hahner, M., Sakaridis, C., Dai, D., & Van Gool, L. (2021). Fog simulation on real LiDAR point clouds for 3D object detection in adverse weather. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) (pp. 15283-15292). IEEE. https://doi.org/10.1109/ICCV48922.2021.01489
[9] Hahner, M., Sakaridis, C., Bijelic, M., Heide, F., Yu, F., Dai, D., & Van Gool, L. (2022). LiDAR snowfall simulation for robust 3D object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 16364-16374). IEEE. https://doi.org/10.1109/CVPR52688.2022.01590
[10] Vora, S., Lang, A. H., Helou, B., & Beijbom, O. (2020). PointPainting: Sequential fusion for 3D object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 4604-4612). IEEE. https://doi.org/10.1109/CVPR42600.2020.00466
[11] Teng, D. (2025). PACO: Predictive auto-configuration for SLO-constrained large language model inference serving. Innovation and Technology Studies, 2(1), 1-10.
[12] Yoo, J. H., Kim, Y., Kim, J., & Choi, J. W. (2020). 3D-CVF: Generating joint camera and LiDAR features using cross-view spatial feature fusion for 3D object detection. In Proceedings of the European Conference on Computer Vision (ECCV) (pp. 720-736). Springer. https://doi.org/10.1007/978-3-030-58589-8_43
[13] Bai, X., Hu, Z., Zhu, X., Huang, Q., Chen, Y., Fu, H., & Tai, C.-L. (2022). TransFusion: Robust LiDAR-camera fusion for 3D object detection with transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 1090-1099). IEEE. https://doi.org/10.1109/CVPR52688.2022.00118
[14] Liu, Z., Tang, H., Amini, A., Yang, X., Mao, H., Rus, D., & Han, S. (2023). BEVFusion: Multi-task multi-sensor fusion with unified bird's-eye view representation. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) (pp. 2774-2781). IEEE. https://doi.org/10.1109/ICRA48891.2023.10160915
[15] Lang, A. H., Vora, S., Caesar, H., Zhou, L., Yang, J., & Beijbom, O. (2019). PointPillars: Fast encoders for object detection from point clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 12697-12705). IEEE. https://doi.org/10.1109/CVPR.2019.01298
[16] Teng, D. (2025). TEAS: Token- and energy-aware autoscaling for cost-efficient LLM serving. AI and Data Science Journal, 6(3), 1-10.
[17] Zhou, Y., & Tuzel, O. (2018). VoxelNet: End-to-end learning for point cloud based 3D object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 4490-4499). IEEE. https://doi.org/10.1109/CVPR.2018.00472
[18] Qi, C. R., Yi, L., Su, H., & Guibas, L. J. (2017). PointNet++: Deep hierarchical feature learning on point sets in a metric space. In Advances in Neural Information Processing Systems 30 (NeurIPS) (pp. 5099-5108).
[19] Yin, T., Zhou, X., & Krahenbuhl, P. (2021). Center-based 3D object detection and tracking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 11784-11793). IEEE. https://doi.org/10.1109/CVPR46437.2021.01161
[20] Tian, Z., Shen, C., Chen, H., & He, T. (2019). FCOS: Fully convolutional one-stage object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) (pp. 9627-9636). IEEE. https://doi.org/10.1109/ICCV.2019.00972
[21] Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollár, P. (2017). Focal loss for dense object detection. In Proceedings of the IEEE International Conference on Computer Vision (ICCV) (pp. 2999-3007). IEEE. https://doi.org/10.1109/ICCV.2017.324
[22] Kendall, A., & Gal, Y. (2017). What uncertainties do we need in Bayesian deep learning for computer vision? In Advances in Neural Information Processing Systems 30 (NeurIPS) (pp. 5574-5584).
[23] Gal, Y., & Ghahramani, Z. (2016). Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In Proceedings of the 33rd International Conference on Machine Learning (ICML) (pp. 1050-1059). PMLR.
[24] Zi, B. (2024). Large language models for enterprise workflow automation in financial operations. Innovation and Technology Studies, 1(1), 24-29.
[25] Ding, J., Shen, Z., & Liu, W. (2026). Game-theoretic cost-sensitive adversarial training for robust cloud intrusion detection against GAN-based evasion attacks. Applied Sciences, 16(8), Article 3944. https://doi.org/10.3390/app16083944
[26] Teng, D., Rhee, M., Qin, Y., Zi, B., & Liu, W. (2026). SW-SpeedDLM: Sliding-window speculative decoding for diffusion language models under long-context constraints. Mathematics, 14(12), Article 2137. https://doi.org/10.3390/math14122137
[27] Zi, B. (2024). Cloud-native distributed systems for real-time payment intelligence. AI and Data Science Journal, 1(1), 51-56.
[28] Chen, Z., Wang, M., Zeng, Z., & Ping, W. (2026). Uncertainty-aware financial forecasting: Leveraging conformal prediction for risk-adjusted models. IEEE Access, 14, 1-18.
[29] Wang, B., Wang, Z., Zhao, W., Zhang, F., & Shang, W. (2026). DRL-Adapt: Deep reinforcement learning for adaptive routing convergence optimization in large-scale networks. IEEE Open Journal of the Computer Society, 7, 1-14.
[30] Zhang, H. (2025). Physics-informed neural networks for high-fidelity electromagnetic field approximation in VLSI and RF EDA applications. Journal of Computing and Electronic Information Management, 18(2), 38-46.
[31] Jiao, Y., Fan, H., Yue, X., Ping, W., Sun, T., & Wang, J. (2026). Dynamic heterogeneous graph contrastive learning for uncovering collusive financial fraud. Scientific Reports, 16, Article 14567. https://doi.org/10.1038/s41598-026-58938-5
[32] Wang, Z., Yang, J. S., Shang, W., & Ding, J. (2026). FairPromote: Explainable and fairness-aware talent promotion prediction via adversarial debiasing and SHAP-based interpretation. IEEE Access, 14, 1-16.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Journal of Mathematical Finance and Risk Management
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
