Lightweight framework for interconnecting virtual and real things via Node-RED

  • 1 Institute of Robotics, Bulgarian Academy of Sciences
  • 2 International Hellenic University, Human-Machines Interaction Laboratory (HUMAIN-Lab)


Key innovative technologies for I4.0 are Internet of Things (IoT), Cyber-Physical Systems (CPSs), self-organizing machines (M2M), cloud computing, big data and Mixed Reality (MR). Microsoft HoloLens2 (HL2) is the new MR revolution for Industry 4.0. The proposed study addresses the potential of Node-RED IoT platform and HoloLens2 for interconnecting virtual and real things from the current scene in the MR window. The augmented holograms on scenes interface the physical object and link behaviours to them seamlessly and regardless of their heterogeneous technologies. A lightweight framework for interconnecting virtual and real things in HoloLens 2 via Node-RED is proposed. The framework uses open-source tools and standard protocols like IoT protocols, Microsoft WebSocket, MRTK for Unity, HL2 APIs and Node-RED nodes used to interact with Python or ROS processes. More detailed experimental analysis will be performed in the future in order to prove that this framework is enough practicable and stable.



  1. R. Azuma, A Survey of Augmented Reality, Presence: Teleoperators and Virtual Environments. 6, no. 4, pp. 355-385, (1997)
  2. K. Van Lopik, et al., A. Developing augmented reality capabilities for industry 4.0 small enterprises: Lessons learnt from a content authoring case study. Comput. Ind., 117, 103208, (2020)
  3. P. Milgram and F. Kishino. A Taxonomy of Mixed Reality Visual Displays, IEICE Trans. Information Systems, E77-D, no. 12, pp. 1321-1329, (1994)
  4. S. Rokhsaritalemi, A. Sadeghi-Niaraki, S. Choi, A Review on Mixed Reality: Current Trends, Challenges and Prospects. Appl. Sci., 10, 636, (2019)
  5. De Guzman, J.A.; Thilakarathna, K.; Seneviratne, A. Security and privacy approaches in mixed reality: A literature survey. arXiv arXiv:180205797, (2018)
  6. IBM NodeRED Flow-based programming for the Internet of Things. Available. Online:
  7. New HoloLens 2 gives Microsoft the edge in the next generation of computing. 2019. Available. Online:
  8. Microsoft introduces HoloLens 2. Available. Online: []
  9. Michaelis, J.R. Value of information driven content management in mixed reality infrastructures. In Proceedings of the Next-Generation Analyst VI, International Society for Optics and Photonics, Orlando, FL, USA, 16–17 April 2018.
  10. C. Akpolat, D. Sahinel, F. Sivrikaya, G. Lehmann, and S. Albayrak, “CHARIOT: An IoT middleware for the integration of heterogeneous entities in a smart urban factory,” in FedCSIS Position Papers, (2017).
  11. Lee, S.; Lee, G.; Choi, G.; Roh, B.-H.; Kang, J. Integration of OneM2M-based IoT Service Platform and Mixed Reality Device. In Proceedings of the 2019 IEEE International Conference on Consumer Electronics (ICCE), USA, pp. 11–13, (2019)
  12. O. Blanco-Novoa, P. Fraga-Lamas, M. Vilar-Montesinos, T. Fernández-Caramés, Creating the Internet of Augmented Things: An Open-Source Framework to Make IoT Devices and Augmented and Mixed Reality Systems Talk to Each Other. Sensors, 20, 3328, (2020)
  13. D. Fang, H. Xu, X. Yang, et al. An Augmented Reality- Based Method for Remote Collaborative Real-Time Assistance: from a System Perspective. Mobile Netw Appl 25, 412–425 (2020).
  14. A memo for exchanging HoloLens2 and Node-RED via WebSocket. Available. Online:
  15. Socially-assistive robot Pepper by Softbank. Online:
  16. Industrial robot Panda by Franka Emika. Available. Online:
  17. NAOqi video and picture management. Available. Online: 4/naoqi/vision/index.html#naoqi-vision
  18. T. Pachidis, C. Sgouros, V. G. Kaburlasos, E. Vrochidou, T. Kalampokas, K. Tziridis, A. Nikolaou, G. A. Papakostas. Forward kinematic analysis of JACO2 robotic arm towards implementing a grapes harvesting robot. 28th International Conference on Software, Telecommunications and Computer Networks, Croatia, September (2020).
  19. C. Lytridis, C. Bazinas, G. Sidiropoulos, GA Papakostas, VG Kaburlasos, V-A Nikopoulou, V Holeva, Evangeliou A. Distance Special Education Delivery by Social Robots. Electronics. 9(6):1034, (2020)
  20. P. Dachkinov, T. Tanev, A. Lekova, D. Batbaatar, and H. Wagatsuma, “Design and Motion Capabilities of an Emotion-Expressive Robot EmoSan”, In Proc. 10th IEEE Int. Conf. on Soft Computing and Intelligent Systems, Toyama, Japan „12, pp. 1332-1338, (2018)
  21. V. Kaburlasos. The Lattice Computing (LC) paradigm. In: Francisco J. Valverde-Albacete, Martin Trnecka (Eds.), Proc. of the 15th International Conference on Concept Lattices and their Applications, Estonia, pp. 1-8, (2020)

Article full text

Download PDF