143 - Examining Medical Student Use of a Large Language Model, Chat GPT, as a Resource for Facilitating Pathological Diagnostic Reasoning in Small Group Active Learning Exercises

Poster Board Number: 143
There are separate poster presentation times for odd and even posters.
Odd poster #s – first hour
Even poster #s – second hour

Co-authors:

Bruce Palmer - Professional Technologist, Pathology, Anatomy and Laboratory Management, West Virginia University

Abstract Body : ChatGPT is an artificial intelligence large language model (LLM) that has recently propagated attention in medical school academia. It has been cited for its ability to “perform at or near the passing threshold” on the three-step US Medical Licensing Exam (USMLE). Our objective was to evaluate student approach and proficiency when using ChatGPT as a learning resource when applied to diagnosis of pathologic specimens. In our first year medical anatomy/histology course, we created an interactive pathology exercise website that features a workspace station where students use a set of interactive tools to draw on slide specimens, label areas of interest, enter descriptions and communicate with each other and faculty. Students are assigned to groups and expected to collaborate in identifying pathological features and applying to specimen analysis and diagnosis. Students must use online or external resources to develop diagnoses. In order to examine student proficiency and use of Chat GPT within the exercises, we provided a link to a subscribed ChatGPT3 account. By capturing input prompts, and analyzing responses provided by ChatGPT, we tracked student usage in the ongoing course. Student proficiency with ChatGPT will be correlated with student response to pre and post-exercise surveys and their actual online interactions. We will evaluate the accuracy, inclusiveness and efficiency of Chat GPT by examining responses and possible instances of misinformation. Initial data collected indicate that approximately half the class used ChatGPT for the exercises. Students with no prior experience used ineffective and repeated prompts while working towards proficiency. Students with some experience used simple searches, similar to a Google search. More experienced students used complex searches and prompts directed towards medical level-appropriate searches and interacted with ChatGPT with follow-up prompts built upon previous responses. Student pre-survey data indicated that most students had not previously used ChatGPT and those that had prior experience were self-taught. Based on our initial data, we can conclude that most first year medical students are unfamiliar with using ChatGPT and are still resistant to its use, preferring Google searches. Students are reticent to approach the learning curve for its use. When provided with specific diagnostic criteria, ChatGPT can generate succinct differential diagnoses that can enhance diagnostic reasoning and promote selective active learning research. With the inevitable growth of LLM use as the medical community establishes its viability, it is essential that medical students are introduced to its use in non-clinical academic settings to facilitate their proficiency and understanding of its appropriate use and limitations.