Abstract
We introduce PIE-Med, a novel Clinical Decision Support System (CDSS) that integrates Graph Convolutional Networks (GCNs) and Large Language Models (LLMs) to deliver interpretable medical recommendations. PIE-Med leverages GCNs to generate recommendations based on patients’ health data and validated medical knowledge, ensuring clinical relevance and robustness. Interpretability algorithms evaluate the model’s reasoning, while LLM agents translate these insights into natural language explanations for clear, context-aware recommendations. By using LLMs as auxiliary reasoning agents rather than primary decision-makers, PIE-Med mitigates risks like hallucination and biased reasoning common in LLM-driven systems. Our code is publicly available on GitHub: https://github.com/picuslab/PIE-Med.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Pandas: https://pandas.pydata.org.
- 2.
Our fork of PyHealth for GNN models: https://github.com/LaErre9/PyHealth.
- 3.
We used GPT-3.5 for these experiments.
References
Ahmad, M.A., Yaramis, I., Roy, T.D.: Creating trustworthy LLMS: dealing with hallucinations in healthcare AI. arXiv preprint arXiv:2311.01463 (2023)
Brockschmidt, M.: GNN-film: graph neural networks with feature-wise linear modulation. In: Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event. Proceedings of Machine Learning Research, vol. 119, pp. 1144–1152. PMLR, Virtual Event (2020). http://proceedings.mlr.press/v119/brockschmidt20a.html
Choi, E., Bahadori, M.T., Sun, J., Kulas, J., Schuetz, A., Stewart, W.F.: RETAIN: an interpretable predictive model for healthcare using reverse time attention mechanism. In: Lee, D.D., Sugiyama, M., von Luxburg, U., Guyon, I., Garnett, R. (eds.) Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain, pp. 3504–3512 (2016). https://proceedings.neurips.cc/paper/2016/hash/231141b34c82aa95e48810a9d1b33a79-Abstract.html
Goldberger, A., et al.: Components of a new research resource for complex physiologic signals. PhysioNet 101 (2000)
Hamilton, W.L., Ying, Z., Leskovec, J.: Inductive representation learning on large graphs. In: Guyon, I., et al. (eds.) Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA. pp. 1024–1034. Curran Associates Inc., Long Beach, CA, USA (2017). https://proceedings.neurips.cc/paper/2017/hash/5dd9db5e033da9c6fb5ba83c7a7ebea9-Abstract.html
Islam, M., Amaya-Ramirez, D., Maigret, B., Devignes, M.D., Aridhi, S., Smail, M.: Molecular-evaluated and explainable drug repurposing for COVID-19 using ensemble knowledge graph embedding. Sci. Rep. 13 (2023). https://doi.org/10.1038/s41598-023-30095-z
Jiang, P., Xiao, C., Cross, A., Sun, J.: Graphcare: enhancing healthcare predictions with personalized knowledge graphs. In: The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net (2024). https://openreview.net/forum?id=tVTN7Zs0ml
Johnson, A., Pollard, T., Roger, M.: MIMIC-III clinical database (version 1.4) (2016). https://doi.org/10.13026/C2XW26
Johnson, A., et al.: MIMIC-III, a freely accessible critical care database (2016)
Kraljevic, Z., et al.: Multi-domain clinical natural language processing with MedCat: The medical concept annotation toolkit. Artif. Intell. Med. 117, 102083 (2021). https://doi.org/10.1016/J.ARTMED.2021.102083
Lu, K., et al.: SympGAN: a systematic knowledge integration system for symptom-gene associations network. Knowl. Based Syst. 276, 110752 (2023). https://doi.org/10.1016/J.KNOSYS.2023.110752
Meehl, P.E.: Clinical versus statistical prediction: a theoretical analysis and a review of the evidence. J. Appl. Psychol. (1954)
Morris, C., et al.: Weisfeiler and Leman go neural: Higher-order graph neural networks. In: The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019, The Thirty-First Innovative Applications of Artificial Intelligence Conference, IAAI 2019, The Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, Honolulu, Hawaii, USA, January 27 - February 1, 2019, pp. 4602–4609. AAAI Press, Honolulu, Hawaii, USA (2019). https://doi.org/10.1609/AAAI.V33I01.33014602
Ouanes, K., Farhah, N.: Effectiveness of artificial intelligence (AI) in clinical decision support systems and care delivery. J. Med. Syst. 48(1), 74 (2024). https://doi.org/10.1007/S10916-024-02098-4
Rane, N., Tawde, A., Choudhary, S., Rane, J.: Contribution and performance of ChatGPT and other large language models (LLM) for scientific and research advancements: a double-edged sword. Int. Res. J. Modernization Eng. Technol. Sci. 5, 875–899 (2023). https://doi.org/10.56726/IRJMETS45312
Sadeghi, Z., et al.: A review of explainable artificial intelligence in healthcare. Comput. Electr. Eng. 118, 109370 (2024). https://doi.org/10.1016/J.COMPELECENG.2024.109370
Scarselli, F., Gori, M., Monfardini, G.: A new model for learning in graph domains. In: Proceedings, 2005 IEEE International Joint Conference on Neural Networks, 2005, vol. 2, pp. 729–734 (2005). https://doi.org/10.1109/IJCNN.2005.1555942
Scarselli, F., Gori, M., Tsoi, A., Hagenbuchner, M., Monfardini, G.: The graph neural network model. IEEE Trans. Neural Netw. 20(1), 61–80 (2009). https://doi.org/10.1109/TNN.2008.2005605
Shang, J., Xiao, C., Ma, T., Li, H., Sun, J.: GameNet: graph augmented memory networks for recommending medication combination. In: The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019, The Thirty-First Innovative Applications of Artificial Intelligence Conference, IAAI 2019, The Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, Honolulu, Hawaii, USA, January 27 - February 1, 2019, pp. 1126–1133. AAAI Press (2019). https://doi.org/10.1609/AAAI.V33I01.33011126
Sundararajan, M., Taly, A., Yan, Q.: Axiomatic attribution for deep networks. arXiv preprint arXiv:1703.01365 (2017)
Tang, X., et al.: MedAgents: large language models as collaborators for zero-shot medical reasoning. arXiv preprint arXiv:2311.10537 (2023)
Wang, D., Yang, Q., Abdul, A.M., Lim, B.Y.: Designing theory-driven user-centric explainable AI. In: Brewster, S.A., Fitzpatrick, G., Cox, A.L., Kostakos, V. (eds.) Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, CHI 2019, Glasgow, Scotland, UK, May 04-09, 2019, p. 601. ACM, Glasgow, Scotland, UK (2019). https://doi.org/10.1145/3290605.3300831
Ying, Z., Bourgeois, D., You, J., Zitnik, M., Leskovec, J.: GNNExplainer: generating explanations for graph neural networks. In: Wallach, H.M., Larochelle, H., Beygelzimer, A., d’Alché-Buc, F., Fox, E.B., Garnett, R. (eds.) Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp. 9240–9251 (2019). https://proceedings.neurips.cc/paper/2019/hash/d80b7040b773199015de6d3b4293c8ff-Abstract.html
Zhang, Y., Chen, R., Tang, J., Stewart, W.F., Sun, J.: LEAP: learning to prescribe effective and safe treatment combinations for multimorbidity. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Halifax, NS, Canada, August 13 - 17, 2017, pp. 1315–1324. ACM (2017). https://doi.org/10.1145/3097983.3098109
Acknowledgments
This work was conducted with the financial support of (1) the PNRR MUR project PE0000013-FAIR and (2) the Italian ministry of economic development, via the ICARUS (Intelligent Contract Automation for Rethinking User Services) project (CUP: B69J23000270005).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Romano, A., Riccio, G., Postiglione, M., Moscato, V. (2025). PIE-Med: Predicting, Interpreting and Explaining Medical Recommendations. In: Hauff, C., et al. Advances in Information Retrieval. ECIR 2025. Lecture Notes in Computer Science, vol 15576. Springer, Cham. https://doi.org/10.1007/978-3-031-88720-8_2
Download citation
DOI: https://doi.org/10.1007/978-3-031-88720-8_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-88719-2
Online ISBN: 978-3-031-88720-8
eBook Packages: Computer ScienceComputer Science (R0)