BEYOND ACCURACY: EXPLAINABILITY, WORKFLOW, AND GOVERNANCE IN THE CLINICAL IMPLEMENTATION OF ARTIFICIAL INTELLIGENCE
DOI:
https://doi.org/10.31435/ijitss.3(51).2026.6007Keywords:
Artificial Intelligence, Healthcare Implementation, Explainability, Governance, Workflow, Patient SafetyAbstract
Artificial intelligence (AI) is increasingly discussed as a transformative force in medicine, yet its practical value depends less on algorithmic promise alone than on whether tools can be integrated safely, intelligibly, and sustainably into clinical care. This review examines how free full-text PubMed literature describes the real-world implementation of AI in healthcare and identifies the technical, organizational, ethical, and social conditions that shape adoption. A structured narrative review was conducted using a targeted PubMed search with a free full-text filter, supplemented by focused searches on explainability, governance, workflow, bias, and implementation in clinical settings. The final core synthesis drew on 21 peer-reviewed articles published between 2019 and 2025, including reviews, qualitative studies, survey studies, consensus guidance, and quality-improvement evaluations. Across the literature, implementation emerged as a sociotechnical challenge rather than a purely technical one. Recurrent themes included the gap between proof-of-concept performance and routine use, the need for practical explainability, risks related to bias and patient safety, workflow and workforce adaptation, and the importance of lifecycle governance. Real-world studies suggest that AI can reduce documentation burden, improve structured data capture, and support triage or decision support, but adoption remains uneven and often limited by poor integration, uncertain accountability, insufficient training, and weak post-deployment monitoring. The review concludes that clinical AI should be evaluated not only by accuracy but also by its fit with human work, organizational routines, fairness, and governance.
References
Albrecht, M., Shanks, D., Shah, T., Hudson, T., Thompson, J., Filardi, T., Wright, K., Ator, G. A., & Smith, T. R. (2025). Enhancing clinical documentation with ambient artificial intelligence: A quality improvement survey assessing clinician perspectives on work burden, burnout, and job satisfaction. JAMIA Open, 8(1), ooaf013. https://doi.org/10.1093/jamiaopen/ooaf013
Alkhanbouli, R., Almadhaani, H. M. A., Alhosani, F., & Simsekler, M. C. E. (2025). The role of explainable artificial intelligence in disease prediction: A systematic literature review and future research directions. BMC Medical Informatics and Decision Making, 25(1), 110. https://doi.org/10.1186/s12911-025-02944-6
Amann, J., Blasimme, A., Vayena, E., Frey, D., & Madai, V. I. (2020). Explainability for artificial intelligence in healthcare: A multidisciplinary perspective. BMC Medical Informatics and Decision Making, 20(1), 310. https://doi.org/10.1186/s12911-020-01332-6
Blease, C., Kaptchuk, T. J., Bernstein, M. H., Mandl, K. D., Halamka, J. D., & DesRoches, C. M. (2019). Artificial intelligence and the future of primary care: Exploratory qualitative study of UK general practitioners' views. Journal of Medical Internet Research, 21(3), e12802. https://doi.org/10.2196/12802
Challen, R., Denny, J., Pitt, M., Gompels, L., Edwards, T., & Tsaneva-Atanasova, K. (2019). Artificial intelligence, bias and clinical safety. BMJ Quality & Safety, 28(3), 231–237. https://doi.org/10.1136/bmjqs-2018-008370
Choudhury, A., & Asan, O. (2020). Role of artificial intelligence in patient safety outcomes: Systematic literature review. JMIR Medical Informatics, 8(7), e18599. https://doi.org/10.2196/18599
Kostick-Quenet, K. M., & Gerke, S. (2022). AI in the hands of imperfect users. npj Digital Medicine, 5(1), 197. https://doi.org/10.1038/s41746-022-00737-z
Larsson, I., Siira, E., Nygren, J. M., Petersson, L., Svedberg, P., Nilsen, P., & Neher, M. (2025). Integrating AI-based triage in primary care: A qualitative study of Swedish healthcare professionals' experiences applying normalization process theory. BMC Primary Care, 26(1), 340. https://doi.org/10.1186/s12875-025-03057-9
Lee, C., Britto, S., & Diwan, K. (2024). Evaluating the impact of artificial intelligence (AI) on clinical documentation efficiency and accuracy across clinical settings: A scoping review. Cureus, 16(11), e73994. https://doi.org/10.7759/cureus.73994
Lekadir, K., Frangi, A. F., Porras, A. R., Glocker, B., Cintas, C., Langlotz, C. P., Weicken, E., Asselbergs, F. W., Prior, F., Collins, G. S., Kaissis, G., Tsakou, G., Buvat, I., Kalpathy-Cramer, J., Mongan, J., Schnabel, J. A., Kushibar, K., Riklund, K., Marias, K., ... Starmans, M. P. A. (2025). FUTURE-AI: International consensus guideline for trustworthy and deployable artificial intelligence in healthcare. BMJ, 388, e081554. https://doi.org/10.1136/bmj-2024-081554
Militello, L. G., Diiulio, J., Wilson, D. L., Nguyen, K. A., Harle, C. A., Gellad, W., & Lo-Ciganic, W.-H. (2025). Using human factors methods to mitigate bias in artificial intelligence-based clinical decision support. Journal of the American Medical Informatics Association, 32(2), 398–403. https://doi.org/10.1093/jamia/ocae291
Mlodzinski, E., Wardi, G., Viglione, C., Nemati, S., Crotty Alexander, L., & Malhotra, A. (2023). Assessing barriers to implementation of machine learning and artificial intelligence-based tools in critical care: Web-based survey study. JMIR Perioperative Medicine, 6, e41056. https://doi.org/10.2196/41056
Ratti, E., Morrison, M., & Jakab, I. (2025). Ethical and social considerations of applying artificial intelligence in healthcare-a two-pronged scoping review. BMC Medical Ethics, 26(1), 68. https://doi.org/10.1186/s12910-025-01198-1
Reddy, S., Allan, S., Coghlan, S., & Cooper, P. (2020). A governance model for the application of AI in health care. Journal of the American Medical Informatics Association, 27(3), 491–497. https://doi.org/10.1093/jamia/ocz192
Reddy, S., Fox, J., & Purohit, M. P. (2019). Artificial intelligence-enabled healthcare delivery. Journal of the Royal Society of Medicine, 112(1), 22–28. https://doi.org/10.1177/0141076818815510
Rony, M. K. K., Kayesh, I., Bala, S. D., Akter, F., & Parvin, M. R. (2024). Artificial intelligence in future nursing care: Exploring perspectives of nursing professionals - A descriptive qualitative study. Heliyon, 10(4), e25718. https://doi.org/10.1016/j.heliyon.2024.e25718
Shaw, J., Rudzicz, F., Jamieson, T., & Goldfarb, A. (2019). Artificial intelligence and the implementation challenge. Journal of Medical Internet Research, 21(7), e13659. https://doi.org/10.2196/13659
Van der Veer, S. N., Riste, L., Cheraghi-Sohi, S., Phipps, D. L., Tully, M. P., Bozentko, K., Atwood, S., Hubbard, A., Wiper, C., Oswald, M., & Peek, N. (2021). Trading off accuracy and explainability in AI decision-making: Findings from 2 citizens' juries. Journal of the American Medical Informatics Association, 28(10), 2128–2138. https://doi.org/10.1093/jamia/ocab127
Vo, V., Chen, G., Aquino, Y. S. J., Carter, S. M., Do, Q. N., & Woode, M. E. (2023). Multi-stakeholder preferences for the use of artificial intelligence in healthcare: A systematic review and thematic analysis. Social Science & Medicine, 338, 116357. https://doi.org/10.1016/j.socscimed.2023.116357
Yin, J., Ngiam, K. Y., & Teo, H. H. (2021). Role of artificial intelligence applications in real-life clinical practice: Systematic review. Journal of Medical Internet Research, 23(4), e25759. https://doi.org/10.2196/25759
Yoo, J., Hur, S., Hwang, W., & Cha, W. C. (2023). Healthcare professionals' expectations of medical artificial intelligence and strategies for its clinical implementation: A qualitative study. Healthcare Informatics Research, 29(1), 64–74. https://doi.org/10.4258/hir.2023.29.1.64
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Jagoda Pałubska, Oliwer Muller, Zuzanna Rafałowska, Stanisław Rutz, Anna Wojtala, Julia Kret, Aleksandra Wyrostkiewicz, Aleksander Sadkowski, Paulina Bigda, Jan Gdula, Lilianna Zielińska, Michał Szpunar, Adrianna Zabiegałowsk, Magda Buśko
This work is licensed under a Creative Commons Attribution 4.0 International License.
All articles are published in open-access and licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0). Hence, authors retain copyright to the content of the articles.
CC BY 4.0 License allows content to be copied, adapted, displayed, distributed, re-published or otherwise re-used for any purpose including for adaptation and commercial use provided the content is attributed.
