The advancements in artificial intelligence (AI) have revolutionized various industries, including healthcare. However, a recent viewpoint published in JAMA warns about the potential loss of clinical knowledge due to the over-reliance on AI in medicine. The authors, led by Agnes Fogo, MD, from Vanderbilt University Medical Center, used the example of kidney pathology to illustrate their concerns.
Pathologists worldwide are tasked with annotating tissue specimens to feed algorithms, with an estimated requirement of up to 100,000 annotations before an algorithm can recognize basic subunits such as a glomerulus. Once the algorithm has been trained, it can perform its task in an instant. Pathologists of the future could receive not only a pathology slide but also data on the number of glomeruli and the area of interstitial fibrosis, allowing them to focus on more complex lesions for diagnosis generation. However, this shift also presents a downside – the gradual erosion of the pathologist’s skill to evaluate basic histology elements themselves.
By removing the basic histology elements from the pathologist’s direct view, there is a risk that these essential aspects will receive less attention in day-to-day clinical pathology. As a result, the true intelligence and understanding of the basic architecture of the kidney could diminish over time. Furthermore, the authors express concerns that the increasing reliance on AI models could lead to the loss of the capacity to comprehend and solve medical problems without the assistance of AI.
The problems associated with AI become more concerning if these models start to advance medical understanding without enough effort made to comprehend the newly identified tissue areas. Researchers have shown that unsupervised AI models can identify tissue areas not previously defined in traditional kidney pathology. However, limited effort has been made to understand these newly identified constructs fully. The authors emphasize the importance of maintaining the knowledge of real histology to catch up with emerging disease mechanisms.
The lack of understanding regarding how AI systems arrive at their conclusions is known as “black box computing.” The authors warn that if medicine reaches a point where the output is defined within a black box that contains constructs inconsistent with previously defined entities, much of today’s knowledge on disease mechanisms will be forgotten. Consequently, future interventions and treatment strategies may solely focus on optimizing outcomes without a deep understanding of the underlying pathogenesis.
While it is essential to recognize the potential benefits and support AI can provide in clinical decision-making, physicians must also contemplate how to maintain control over their profession and prevent the erosion of clinical knowledge. The authors urge physicians to establish boundaries for AI use to ensure meaningful progress in diagnosis and the understanding of disease mechanisms while preserving the medical profession’s integrity.
The over-reliance on AI in medicine presents potential risks, including the erosion of clinical knowledge and the loss of the capacity to solve medical problems without AI assistance. Striking a balance between leveraging the benefits of AI and preserving clinical expertise will be crucial for the future of medicine. Physicians must take an active role in shaping the integration of AI into their practice to maintain control over their profession and ensure meaningful advancements in diagnosis and understanding of disease mechanisms.