Robots and Chatbots: Are They the Answer to Loneliness and Dirty Dishes?
No so fast....
Welcome to another edition of the S.A.D newsletter. In this issue, we delve into the realm of healthcare automation, specifically exploring the utilisation of GenAI and chatbots for patient interaction.
A recent study showed that chatbots can be perceived as more humane than physicians, particularly due to their capacity for personalised and time-consuming interactions. Researchers compared (preprint; Jan 2024) the diagnostic skills of a large language model, AMIE (Articulate Medical Intelligence Explorer), to primary care physicians in simulated consultations:
Three specific LLM agents (patient agent, doctor agent, and moderator), each played by AMIE, were tasked with communicating amongst each other to generate the simulated dialogues. Each agent had distinct instructions. The patient agent embodied the individual experiencing the medical condition outlined in the vignette. Their role involved truthfully responding to the doctor agent’s inquiries as well as raising any additional questions or concerns they may have had. The doctor agent played the role of an empathetic clinician seeking to comprehend the patient’s medical history within the online chat environment [24]. Their objective was to formulate questions that could effectively reveal the patient’s symptoms and background, leading to an accurate diagnosis and an effective treatment plan. The moderator continually assessed the ongoing dialogue between the patient agent and doctor agent, determining when the conversation had reached a natural conclusion.
The turn-by-turn dialogue simulation started with the doctor agent initiating the conversation: “Doctor: So, how can I help you today?”. Following this, the patient agent responded, and their answer was incorporated into the ongoing dialogue history. Subsequently, the doctor agent formulated a response based on the updated dialogue history. This response was then appended to the conversation history. The conversation progressed until the moderator detected the dialogue had reached a natural conclusion, when the doctor agent had provided a differential diagnosis, treatment plan, and adequately addressed any remaining patient agent questions, or if either agent initiated a farewell.
The details lie in this statement perceived as more humane than physicians. While AMIE excelled in both accuracy and communication, limitations like non-verbal interaction and potential bias remain. The study suggests LLMs hold promise for future conversational diagnostic AI, but stresses the need for further research on fairness, safety, and ethical considerations before real-world implementation.
In my previous newsletters, I have delved into the idea that while technological advancements offer significant benefits, they frequently fail to address fundamental structural issues. Specifically in the healthcare sector, this translates to persistent problems such as unequal access to healthcare, fragmented data systems, and outdated infrastructure. Additionally, there exists an imbalance in time allocation between curative and preventive measures, further complicating the challenges encountered by healthcare professionals. It is essential to recognise that a broken system cannot be entirely rectified solely through the implementation of AI.
A bit of a theoretical detour here, if I may. Technological determinism, a theory exploring the relationship between technological development and societal change encompasses both dystopian and utopian perspectives. Dystopianism fears uncontrollable AI causing harm, proposing reactive solutions that neglect human agency and underlying structural issues like social inequality. Conversely, utopianism views AI as a tool for progress, advocating for limited regulation that risks overlooking ethical concerns and access disparities. Ultimately, both perspectives fail to consider the crucial role of human control in shaping the impact of technology. Technological determinism often implies that “technology almost has a mind of its own and that it will plow forward without much resistance from society or governments.” This highlights the need for active human engagement in shaping technology’s development and ensuring its responsible use.
The integration of chatbots into healthcare is certainly not a novel concept, but with the emergence of GenAI, the scope and possibilities have expanded significantly. While automation in this context is not unprecedented, it is crucial to acknowledge that, without a thorough understanding and resolution of underlying structural issues, this approach may inadvertently foster a belief that technology alone can shape the course of history and societal development. To better grasp this concept, let’s take a brief detour into the realm of laundry.
Consider the introduction of the washing machine in the U.S.A during the 1910s and 1920s, initially hailed as a liberating innovation. However, as time unfolded, it became evident that this technological advancement had the potential to increase the workload for women, primarily due to unaddressed social divisions. Ruth Schwartz Cowan, an American historian, explores this phenomenon in her book “More Work for Mother: The Ironies of Household Technology from the Open Hearth to the Microwave.” She documents how, contrary to expectations, modern women found themselves devoting as much time to household work as their predecessors.
One compelling example she provides is that the washing machine, initially seen as a time-saving device, paradoxically led to an increase in women's workload. This was attributed to heightened societal expectations regarding cleanliness and hygiene, highlighting the unforeseen consequences of embracing new technologies without addressing the broader social and structural issues at play.
Imagine a wave of fancy gadgets, like smart speakers and voice-controlled appliances, all connected to the internet (the "Internet of Things"). These promise to make chores easier, even letting you control things remotely. While this creates new jobs (think making these gadgets), it also creates more “work” for consumers (often women) constantly buying and using stuff.
Similarly, the electric car, a relatively recent phenomenon, highlights the limitations of technology as a solo solution. While their lifetime emissions are generally lower than gasoline vehicles, they haven't significantly impacted climate change. As the International Council on Clean Transportation (ICCT) notes, “EVs are the quickest means to decarbonize transport,” but not single-handedly achieve net-zero emissions by 2050. This exemplifies how technological solutions, while valuable, may not address broader issues like climate change without comprehensive approaches such as building infrastructure for public transport, understanding migration to cities, and equitable urban design.
The structure of cities significantly influences how people commute to work, children go to school, and the accessibility of essential facilities such as grocery stores and hospitals. In many cases, specific city plans necessitate car ownership, particularly in a few large cities in the U.S. A New York Times article from October 2023, titled “How the Costs of Car Ownership Add Up,” underscores the importance of considering individual choices and societal reliance on cars beyond mere technological solutions.
Highlighting the escalating costs associated with car ownership, including rising prices, maintenance, financing, and fuel expenses (now constituting 16% of median household income), the article sheds light on the financial and emotional struggles individuals face. It refrains from presenting oversimplified solutions and, instead, urges readers to contemplate individual choices, societal dependence on cars, and the imperative need for alternative transportation infrastructure. This holistic perspective emphasises the necessity of addressing not only the technological aspects but also the broader socio-economic factors contributing to the challenges associated with car ownership.
All these are suggesting that technological innovations, often presented as solutions, may inadvertently exacerbate underlying issues if they fail to address the root causes. We must carefully evaluate how technologies interact with existing systems and societal realities to ensure they truly work towards positive change.
While chatbots present a promising tool for enhancing healthcare efficiency and accessibility, it is crucial to remain vigilant about potential drawbacks. To ensure the positive impact of AI in healthcare, it is imperative to develop transparent, accountable, and unbiased AI models, alongside addressing the root causes of the increasing workload faced by doctors. Only through these comprehensive measures can AI truly enhance, rather than replace, the human touch in healthcare.
