Christopher Robertson, Andrew Woods, Kelly Bergstrand, Jess Findley, Cayley Balser, Marvin J. Slepian
Abstract
Artificial intelligence (AI) has the potential to improve diagnostic accuracy. Yet people are often reluctant to trust automated systems, and some patient populations may be particularly distrusting. We sought to determine how diverse patient populations feel about the use of AI diagnostic tools, and whether framing and informing the choice affects uptake. To construct and pretest our materials, we conducted structured interviews with a diverse set of actual patients. We then conducted a pre-registered (osf.io/9y26x), randomized, blinded survey experiment in factorial design. A survey firm provided n = 2675 responses, oversampling minoritized populations. Clinical vignettes were randomly manipulated in eight variables with two levels each: disease severity (leukemia versus sleep apnea), whether AI is proven more accurate than human specialists, whether the AI clinic is personalized to the patient through listening and/or tailoring, whether the AI clinic avoids racial and/or financial biases, whether the Primary Care Physician (PCP) promises to explain and incorporate the advice, and whether the PCP nudges the patient towards AI as the established, recommended, and easy choice.
Introduction
Artificial intelligence (AI) is poised to transform healthcare. Today, AI is used to analyze tumors in chest images [1], regulate implanted devices [2], and select personalized courses of care [3]. Despite the promise of AI, there is broad public skepticism about AI in a range of domains from transportation to criminal justice to healthcare [4,5]. Doctors and patients tend to primarily rely on doctors’ clinical judgment, even when it is at odds with statistical judgment [6].
Research shows that patients prefer human doctors to AI-powered machines in diagnosis, screening, and treatment [7–10]. In an early study, patients were more likely to follow medical advice from a physician than a computer and were less trustful of computers as providers of medical advice [7]. Other work shows that patients are less trusting of doctors that rely on non-human decision aids [8,9]. More recently, in a series of studies, patients were less willing to schedule an appointment to be diagnosed by a robot, and they were willing to pay significantly more money for a human provider, with a reported perception that AI providers are less able to account for patients’ unique characteristics [10].
Materials and method
We conducted two study phases, one qualitative and one quantitative. In the qualitative phase (February to December 2020), we conducted structured interviews with 24 patients recruited for racial and ethnic diversity to understand their reactions to current and future AI technologies. In the quantitative phase (January and February 2021), we used an internet-based survey experiment oversampling Black, Hispanic, Asian, and Native American populations. Both phases placed respondents as mock patients into clinical vignettes to explore whether they would prefer to have an AI system versus a doctor for diagnosis and treatment and under what circumstances.
We chose this mixed-methods design for a few reasons. First, because this is the first study of its kind, we wanted to ensure that the vignettes driving our quantitative survey were realistic and intuitive; the qualitative pre-study helped us to gauge participant reaction. Second, large-scale quantitative surveys often raise a number of questions about why people respond the way they do. The mixed-method design allows us to accomplish something that neither approach—purely quantitative nor purely qualitative—would achieve on its own.
Results
We found a substantial resistance to artificial intelligence. With weighting representative to the U.S. population, most respondents (52.9%) chose the human doctor and 47.1% chose AI clinic, with some variation along race and ethnicity, as shown in Fig 1.
Discussion
Our qualitative interview study with actual patients relies on a small, convenience sample in one city. As such it does not allow strong conclusions when standing alone. It does provide a groundwork for our rigorous quantitative approach, helping us to ensure that the vignettes will be clear and understandable, and also generating hypotheses subject to quantitative testing subsequently, through systematic manipulation of the vignettes.
Those hypotheses are tested in our randomized, blinded experiment, which allows causal inference about the impact of the manipulations, and the factorial design allows strong statistical power, because every respondent provides an observation for every variable. Our diverse population from a high-quality survey sample allows extrapolation to the U.S. population, when weighted.
Citation: Robertson C, Woods A, Bergstrand K, Findley J, Balser C, Slepian MJ (2023) Diverse patients’ attitudes towards Artificial Intelligence (AI) in diagnosis. PLOS Digit Health 2(5): e0000237. https://doi.org/10.1371/journal.pdig.0000237
Editor: Rutwik Shah, UCSF: University of California San Francisco, UNITED STATES
Received: February 27, 2022; Accepted: March 20, 2023; Published: May 19, 2023
Copyright: © 2023 Robertson et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Analysis was done in Stata 17. Data and code are publicly available on the Open Science Framework, osf.io/9y26x.
Funding: This study was funded by the National Institutes of Health (3R25HL126140-05S1 to CR and AW). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0000237#sec013
