Industry 4.0
Vol. 11 (2026), Issue 2, pg(s) 46-50

TECHNOLOGICAL BASIS OF “INDUSTRY 4.0”

Information Theory for Medical Data Fusion

  • 1 MIT Univerzitet SkopjeSkopje, Macedonia
  • 2 University of Information Science and Technology Skopje, Macedonia

Abstract

Data fusion is the process of integrating multiple heterogeneous data sources to produce more accurate, comprehensive, and useful information than any single source alone. For the medical data fusion, this information may come from imaging, genomic data, clinical records, and physiological signals. It has emerged as a cornerstone of modern precision medicine. Information theory provides a rigorous mathematical framework for quantifying uncertainty, measuring information gain, and can thus be used to optimize fusion strategies across diverse clinical contexts. This paper presents a review of information-theoretic approaches suitable for medical data fusion, covering: fundamental concepts (entropy, mutual information, Kullback-Leibler divergence, rate-distortion), theoretical frameworks (information bottleneck, transfer entropy, partial information decomposition), and their application across major clinical domains. We synthesize recent advances in fusion-based architectures, discuss the critical challenges of uncertainty quantification, and provide practical guidelines for implementing information-theoretic fusion in clinical settings. Through a systematic analysis, we identify key challenges, including parameter sensitivity, missing modalities, and clinical interpretability, to outline promising directions for future research. This review aims to provide clinicians, researchers, and developers with a comprehensive understanding of how information theory can transform medical data for improved diagnostic accuracy, prognostic precision, and personalized patient care.

Keywords

References

  1. G I. Csiszar. I-Divergence Geometry of Probability Distributions and Minimization Problems. The Annals of Probability, 3(1):146–158, 1975. doi: 10.1214/aop/1176996454.
  2. J. Duan, J. Xiong, Y. Li, and W. Ding. Deep learning based multimodal biomedical data fusion: An overview and comparative review. Information Fusion, 2024. doi: 10.1016/j.inffus.2024.102536.
  3. S. Kullback and R. A. Leibler. On Information and Sufficiency. The Annals of Mathematical Statistics, 22(1):79– 86, 1951. doi: 10.1214/aoms/ 1177729694.
  4. P. P. Liang, Y. Cheng, X. Fan, C. K. Ling, S. Nie, R. Chen, Z. Deng, N. Allen, R. Auerbach, F. Mah mood, R. Salakhutdinov, and L.-P. Morency. Quantifying & modeling multimodal interactions: An information decomposition framework, 2023.
  5. J. Massey. Causality, feedback and directed information. In Proceedings 1990 International Symposium on Information Theory and its Applications (ISITA). Waikiki, Hawaii, Nov. 1990.
  6. M. Salvi, H. W. Loh, S. Seoni, P. D. Barua, S. García, F. Molinari, and U. R. Acharya. Multi-modality approaches for medical support systems: A systematic review of the last decade. Information Fusion, 103, 2024. doi: 10.1016/j.inffus.2023.102134.
  7. T. Schreiber. Measuring information transfer. Phys. Rev. Lett., 85:461–464, Jul 2000. doi: 10.1103/ PhysRevLett.85.461.
  8. C. E. Shannon. A mathematical theory of communication. The Bell System Technical Journal, 27(3): 379–423, 1948.
  9. N. Tishby, F. C. Pereira, and W. Bialek. The in formation bottleneck method, 2000. URL https: //arxiv.org/abs/physics/0004057.
  10. P. L. Williams and R. D. Beer. Nonnegative decomposition of multivariate information. 2010. URL https://arxiv.org/abs/1004.2515.
  11. J. Wu, K. He, R. Mao, X. Shang, and E. Cambria. Harnessing the potential of multimodal EHR data: A comprehensive survey of clinical predictive modelling for intelligent healthcare. Inf. Fusion, 123, 2025.
  12. T. Zhang, R. Ding, K.-D. Luong, and W. Hsu. Evaluating an information theoretic approach for selecting multimodal data fusion methods. Journal of Biomedical Informatics, 167, 2025.
  13. Y.-D. Zhang, Z. Dong, S.-H. Wang, X. Yu, X. Yao, Q. Zhou, H. Hu, M. Li, C. Jiménez-Mesa, J. Ramirez, et al. Advances in multimodal data fusion in neuroimaging: Overview, challenges, and novel orientation. Information Fusion, 64:149– 187, 2020. Eason, B. Noble, and I. N. Sneddon, “On certain integrals of Lipschitz-Hankel type involving products of Bessel functions,” Phil. Trans. Roy. Soc. London, vol. A247, pp. 529–551, April 1955.

Article full text

Download PDF