Industry 4.0
Vol. 11 (2026), Issue 2, pg(s) 58-62

TECHNOLOGICAL BASIS OF “INDUSTRY 4.0”

Imagined pedestrian encounters: analyzing world-model rollouts in a reinforcement learning driving agent

  • 1 Faculty of Electrical Engineering and Information Technologies – Ss. Cyril and Methodius University in Skopje, Skopje, North Macedonia

Abstract

World-model reinforcement learning agents plan by imagining future trajectories through a learned latent dynamics model. In safety-critical driving tasks, the quality of these imagined rollouts — particularly their representation of vulnerable road users — directly determines whether the agent can anticipate and avoid dangerous situations. We analyze the imagined rollouts of a DreamerV3 agent trained to navigate urban environments in CARLA using semantic segmentation and depth observations. By decoding the agent’s latent imagination into observation space, we qualitatively and quantitatively examine how pedestrians are represented in imagined futures, comparing episodes that result in collision against those where the agent successfully avoids. We investigate whether the world model accurately predicts pedestrian presence and motion in its imagined horizon, and whether reconstruction fidelity of pedestrian regions correlates with avoidance outcomes. Our analysis provides insight into the internal representations that underlie emergent safety behaviors in model-based driving agents and highlights limitations of finite imagination horizons for pedestrian safety.

Keywords

References

  1. Danijar Hafner, Timothy P. Lillicrap, Jimmy Ba, and Mohammad Norouzi. Dream to control: Learning behaviors by latent imagination. ArXiv, abs/1912.01603, 2020.
  2. B Ravi Kiran, Ibrahim Sobh, Victor Talpaert, Patrick Mannion, Ahmad A. Al Sallab, Senthil Kumar Yogamani, and Patrick Pérez. Deep reinforcement learning for autonomous driving: A survey. IEEE Transactions on Intelligent Transportation Systems, 23:4909–4926, 2021.
  3. Wilko Schwarting, Javier Alonso-Mora, and Daniela Rus. Planning and decision-making for autonomous vehicles. Annual Review of Control, Robotics, and Autonomous Systems, 1:187– 210, 2018.
  4. Danijar Hafner, Jurgis Pašukonis, Jimmy Ba, and Timothy P. Lillicrap. Mastering diverse domains through world models. ArXiv, abs/2301.04104, 2023.
  5. Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez, and Vladlen Koltun. CARLA: An open urban driving simulator. In Proceedings of the 1st Annual Conference on Robot Learning, pages 1–16, 2017.
  6. Marin Toromanoff, Emilie Wirbel, and Fabien Moutarde. End-to-end model-free reinforcement learning for urban driving using implicit affordances. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7153–7162, 2020.
  7. Dian Chen, Brady Zhou, Vladlen Koltun, and Philipp Krähenbühl. Learning by cheating. In Proceedings of the Conference on Robot Learning, volume 100 of Proceedings of Machine Learning Research, pages 66–75. PMLR, 2020.
  8. Neha Sharma, Chhavi Dhiman, and S. Indu. Pedestrian intention prediction for autonomous vehicles: A comprehensive survey. Neurocomputing, 508:120–152, 2022.
  9. Amir Rasouli and John K. Tsotsos. Autonomous vehicles that interact with pedestrians: A survey of theory and practice. IEEE Transactions on Intelligent Transportation Systems, 21:900–918, 2018.
  10. Tirthankar Bandyopadhyay, Kok Sung Won, Emilio Frazzoli, David Hsu, Wee Sun Lee, and Daniela Rus. Intention-aware motion planning. In Workshop on the Algorithmic Foundations of Robotics, 2013.
  11. Fanta Camara, Nicola Bellotto, et al. Pedestrian models for autonomous driving part ii: High-level models of human behavior. IEEE Transactions on Intelligent Transportation Systems, 22:5453– 5472, 2020.
  12. Qifeng Li, Xiaosong Jia, Shaobo Wang, and Junchi Yan. Think2drive: Efficient reinforcement learning by thinking in latent world model for quasi-realistic autonomous driving (in carla-v2). ArXiv, abs/2402.16720, 2024.
  13. Dechen Gao, Shuangyu Cai, Hanchu Zhou, Hang Wang, Iman Soltani, and Junshan Zhang. Cardreamer: Open-source learning platform for world-model-based autonomous driving. IEEE Internet of Things Journal, 12:2866–2875, 2024.
  14. Tuo Feng, Wenguan Wang, and Yi Yang. A survey of world models for autonomous driving. ArXiv, abs/2501.11260, 2025.
  15. Zeyu Zhu and Huijing Zhao. A survey of deep rl and il for autonomous driving policy learning. IEEE Transactions on Intelligent Transportation Systems, 23:14043–14065, 2021.
  16. Pranav Singh Chib and Pravendra Singh. Recent advancements in end-to-end autonomous driving using deep learning: A survey. IEEE Transactions on Intelligent Vehicles, 9:103–118, 2023.
  17. Elahe Delavari, Feeza Khan Khanzada, and Jaerock Kwon. A comprehensive review of reinforcement learning for autonomous driving in the carla simulator. ArXiv, abs/2509.08221, 2025.
  18. Shreyas Ramakrishna, Baiting Luo, Christopher B. Kuhn, Gabor Karsai, and Abhishek Dubey. Anti-carla: An adversarial testing framework for autonomous vehicles in carla. 2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC), pages 2620–2627, 2022.
  19. Christopher Diehl, Timo Sievernich, Martin Krüger, Frank Hoffmann, and Torsten Bertram. Uncertainty-aware model-based offline reinforcement learning for automated driving. IEEE Robotics and Automation Letters, 8:1167–1174, 2023.
  20. Oleh Rybkin, Chuning Zhu, Anusha Nagabandi, Kostas Daniilidis, Igor Mordatch, and Sergey Levine. Model-based reinforcement learning via latent-space collocation. ArXiv, abs/2106.13229, 2021.
  21. Danijar Hafner, Timothy P. Lillicrap, Mohammad Norouzi, and Jimmy Ba. Mastering atari with discrete world models. ArXiv, abs/2010.02193, 2021.
  22. Fan-Ming Luo, Tian Xu, Hang Lai, Xiong-Hui Chen, Weinan Zhang, and Yang Yu. A survey on model-based reinforcement learning. Science China Information Sciences, 67, 2022.

Article full text

Download PDF