Abstract
Artificial Intelligence (AI), especially of the machine learning (ML) variety, is used by health care organizations to assist with a number of tasks, including diagnosing patients and optimizing operational workflows. AI products already proliferate the health care market, with usage increasing as the technology matures. Although AI may potentially revolutionize health care, the use of AI in health settings also leads to risks ranging from violating patient privacy to implementing a biased algorithm. This chapter begins with a broad overview of health care AI and how it is currently used. We then adopt a “lifecycle” approach to discussing issues with health care AI. We start by discussing the legal and ethical issues pertaining to how data to build AI are gathered in health care settings, focusing on privacy. Next, we turn to issues in algorithm development, especially algorithmic bias. We then discuss AI deployment to treat patients, focusing on informed consent. Finally, we will discuss existing oversight mechanisms for health AI in the United States: liability and regulation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
By “meaningful” we intend to distinguish at the extreme what we might think of as pro forma consent. For example, where the first time a patient enters a health care facility, they sign a form they likely never read that they consent to future use of their data with the identifiers stripped; if they have read the form, chances are they really do not understand the risks and benefits, because how could they without being given specifics about intended uses, what other data sets are present that may be triangulated with this data set, the cybersecurity practices of various data holders, etc.?
References
Babic, B., et al. (2019). Algorithms on regulatory lockdown in medicine. Science, 366, 1202–1204.
Babic, B., et al. (2021). Beware explanations from AI in health care. Science, 373, 284–286.
Benjamin, R. (2019). Assessing risk, automating racism. Science, 366, 421–422.
Brodwin, E. (2020). Health systems are using AI to predict severe Covid-19 cases. But limited data could produce unreliable results. STAT News. https://www.statnews.com/2020/11/18/covid-19-algorithms-reliability-epic-cerner/. Accessed 13 July 2021.
Brown, S. H., & Miller, R. A. (2014). Legal and regulatory issues related to the use of clinical software in health care delivery. In R. Greenes (Ed.), Clinical decision support (2nd ed., pp. 711–740). Elsevier.
Cohen, I. G. (2018). Is there a duty to share health care data? In I. G. Cohen et al. (Eds.), Big data, health law, and bioethics (pp. 209–222). Cambridge University Press.
Cohen, I. G. (2020). Informed consent and medical artificial intelligence: What to tell the patient? The Georgetown Law Journal, 108(6), 1425–1469.
Cohen, I. G., & Mello, M. M. (2019). Big data, big tech, and protecting patient privacy. JAMA, 322(12), 1141–1142.
Dave, P. (2019). Google signs healthcare data and cloud computing deal with Ascension. Reuters. https://www.reuters.com/article/us-alphabet-ascension-privacy/google-signs-healthcare-data-and-cloud-computing-deal-with-ascension-idUSKBN1XL2AT. Accessed 13 July 2021.
Dinerstein v. Google. (2020). 484 F.Supp.3d 561.
Evans, M. (2021). Google strikes Deal with hospital chain to develop healthcare algorithms. Wall Street Journal. https://www.wsj.com/articles/google-strikes-deal-with-hospital-chain-to-develop-healthcare-algorithms-11622030401. Accessed 13 July 2021.
FDA. (2018a). FDA. https://www.fda.gov/news-events/press-announcements/fda-permits-marketing-artificial-intelligence-based-device-detect-certain-diabetes-related-eye. Accessed 13 July 2021.
FDA. (2018b). FDA permits marketing of clinical decision support software for alerting providers of a potential stroke in patients. https://www.fda.gov/news-events/press-announcements/fda-permits-marketing-clinical-decision-support-software-alerting-providers-potential-stroke. Accessed 13 July 2021.
FDA. (2019a). Developing a Software Precertification Program: A Working Model v1.0. https://www.fda.gov/media/119722/download. Accessed 14 July 2021.
FDA. (2019b). Proposed Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) – Discussion Paper and Request for Feedback. https://www.fda.gov/files/medical%20devices/published/US-FDA-Artificial-Intelligence-and-Machine-Learning-Discussion-Paper.pdf. Accessed 14 July 2021.
FDA. (2021). Artificial Intelligence and Machine Learning (AI/ML) Software as a Medical Device Action Plan. https://www.fda.gov/media/145022/download. Accessed 14 July 2021.
FDA. (2022a). Artificial Intelligence and Machine Learning (AI/ML)-Enabled Medical Devices. https://www.fda.gov/medical-devices/software-medical-device-samd/artificial-intelligence-and-machine-learning-aiml-enabled-medical-devices. Accessed 9 April 2023.
FDA. (2022b). Clinical Decision Support Software: Guidance for Industry and Food and Drug Administration Staff. https://www.fda.gov/media/109618/download. Accessed 9 April 2023.
FDA. (2022c). Digital Health Software Precertification (Pre-Cert) Pilot Program. https://www.fda.gov/medical-devices/digital-health-center-excellence/digital-health-software-precertification-pre-cert-pilot-program. Last accessed 9 April 2023.
Forneas, N. (2018). Improving hospital bed management with AI. IBM. https://www.ibm.com/blogs/client-voices/improving-hospital-bed-management-ai/. Accessed 13 July 2021.
Gerke, S., et al. (2020a). Ethical and legal aspects of ambient intelligence in hospitals. JAMA, 323(7), 601–602.
Gerke, S., et al. (2020b). Ethical and legal challenges of artificial intelligence-driven healthcare. In A. Bohr & K. Memarzadeh (Eds.), Artificial intelligence in healthcare (1st ed., pp. 295–328). Elsevier.
Gerke, S., et al. (2020c). The need for a system view to regulate artificial intelligence/machine learning-based software as medical device. NPJ Digital Medicine, 3, 1–4.
Gerke, S. (2021). Health AI for Good Rather than Evil? The Need for a New Regulatory Framework for AI-Based Medical Devices. 20 Yale Journal of Health Policy, Law, and Ethics, 433.
Gottlieb, S. (2017). FDA announces new steps to empower consumers and advance digital healthcare. FDA. https://www.fda.gov/news-events/fda-voices/fda-announces-new-steps-empower-consumers-and-advance-digital-healthcare. Accessed 14 July 2021.
Hao, K. (2018). What is machine learning? MIT Technology Review. https://www.technologyreview.com/2018/11/17/103781/what-is-machine-learning-we-drew-you-another-flowchart/. Accessed 13 July 2021.
Hellman, D. (2021). Big data and compounding injustice. SSRN. Available from https://ssrn.com/abstract=3840175. Accessed 19 July 2021.
Japsen, B. (2019). Mayo Clinic, Google partner on digital health Analytics. Forbes. https://www.forbes.com/sites/brucejapsen/2019/09/10/mayo-clinic-google-partner-on-digital-health-analytics/. Accessed 13 July 2021.
Kaushal, A., et al. (2020). Geographic distribution of US cohorts used to train deep learning algorithms. JAMA, 324(12), 1212–1213.
London, A. J. (2019). Artificial intelligence and black-box medical decisions: Accuracy versus Explainability. The Hastings Center Report, 49(1), 15–21.
Maliha, G., et al. (2021). Artificial intelligence and liability in medicine: Balancing safety and innovation. The Milbank Quarterly, 00, 1–19.
McFarling, U. L. (2020). Dermatology faces a reckoning: Lack of darker skin in textbooks and journals harms care for patients of color. STAT News. https://www.statnews.com/2020/07/21/dermatology-faces-reckoning-lack-of-darker-skin-in-textbooks-journals-harms-patients-of-color/. Accessed 13 July 2021.
Murray, S. G., et al. (2020). Discrimination by artificial intelligence in a commercial electronic health record – A case study. Health Affairs Blog. https://www.healthaffairs.org/do/10.1377/hblog20200128.626576/full/. Accessed 13 July 2021.
Nissenbaum, H. (2004). Privacy as contextual integrity. Washington Law Review, 79, 119–157.
Obermeyer, Z., et al. (2019). Dissecting racial bias in an algorithm used to manage the health of populations. Science, 366, 447–453.
Obermeyer, Z., et al. (2021). Algorithmic bias playbook. Center for Applied Artificial Intelligence at Chicago Booth. https://www.chicagobooth.edu/research/center-for-applied-artificial-intelligence/research/algorithmic-bias/playbook. Accessed 13 July 2021.
Price, W. N. (2017). Artificial intelligence in health care: Applications and legal implications. The SciTech Lawyer, 14(1), 10–13.
Price, W. N. (2019). Medical AI and contextual bias. Harvard Journal of Law & Technology, 33, 65–116.
Price, W. N., & Cohen, I. G. (2019). Privacy in the age of medical big data. Nature Medicine, 25, 37–43.
Price, W. N., et al. (2019). Potential liability for physicians using artificial intelligence. JAMA, 322(18), 1765–1766.
Price, W. N., et al. (2021). How much can potential jurors tell us about liability for medical artificial intelligence? Journal of Nuclear Medicine, 62(1), 15–16.
Scherer, M. U. (2016). Regulating artificial intelligence systems: Risks, challenges, competencies, and strategies. Harvard Journal of Law & Technology, 29(2), 353–400.
Schiff, D., & Borenstein, J. (2019). How should clinicians communicate with patients about the roles of artificially intelligent team members? AMA Journal of Ethics, 21(2), E138–E145.
Tobia, K., et al. (2021). When does physician use of AI increase liability? Journal of Nuclear Medicine, 62(1), 17–21.
Toews, R. (2020). These are the startups applying AI to transform healthcare. https://www.forbes.com/sites/robtoews/2020/08/26/ai-will-revolutionize-healthcare-the-transformation-has-already-begun/. Accessed 13 July 2021.
Watts, G. (2019). Data sharing: Keeping patients on board. Lancet Digit Health, 1(7), E332–E333.
Acknowledgments
I.G.C. was supported by a grant from the Collaborative Research Program for Biomedical Innovation Law, a scientifically independent collaborative research program supported by a Novo Nordisk Foundation grant (NNF17SA0027784). I.G.C. was also supported by Diagnosing in the Home: The Ethical, Legal, and Regulatory Challenges and Opportunities of Digital Home Health, a grant from the Gordon and Betty Moore Foundation (grant agreement number 9974). S.G. reports grants from the European Union (Grant Agreement no. 101057321 and no. 101057099), the National Institute of Biomedical Imaging and Bioengineering (NIBIB) of the National Institutes of Health (Grant Agreement no. 3R01EB027650-03S1), and the Rock Ethics Institute at Penn State University.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Becker, J., Gerke, S., Cohen, I.G. (2023). The Development, Implementation, and Oversight of Artificial Intelligence in Health Care: Legal and Ethical Issues. In: Valdés, E., Lecaros, J.A. (eds) Handbook of Bioethical Decisions. Volume I. Collaborative Bioethics, vol 2. Springer, Cham. https://doi.org/10.1007/978-3-031-29451-8_24
Download citation
DOI: https://doi.org/10.1007/978-3-031-29451-8_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-29450-1
Online ISBN: 978-3-031-29451-8
eBook Packages: Biomedical and Life SciencesBiomedical and Life Sciences (R0)