148 - Adoption and Attitudes of AI Large Language Models in Academic Settings and Beyond

Poster Board Number: 148
There are separate poster presentation times for odd and even posters.
Odd poster #s – first hour
Even poster #s – second hour

Co-authors:

Megan Balle - Marian University College of Osteopathic Medicine; Margaret McNulty - Indiana University School of Medicine

Abstract Body : Introduction and Objective: Large Language Models (LLMs) such as ChatGPT are artificial intelligence tools that have received significant attention regarding use in educational settings. The purpose of this study was to begin to obtain a clearer picture regarding how students and instructors are currently using LLMs so educational policies and practices can be modified appropriately to incorporate the quickly advancing technology.




Materials and Methods: In an IRB-approved study, current students and instructors in health professional programs were asked to complete a survey that collected demographics, perceptions, and use of LLMs through Likert and free response questions. Descriptive statistics were performed on Likert items and free responses were analyzed using a thematic analysis framework.




Results: The survey received 38 viable student responses and 21 from instructors. Overall, there was limited adoption of LLMs among students. ChatGPT was the most commonly used LLM. Of student respondents, 39.5% reported never using LLMs in their academic career. Of those not currently using an LLM, 35% did not plan to start, citing a lack of understanding. Students were more likely to perceive using LLMs as “lazy” and “cutting corners,” and primarily used it to create practice questions and/or as a search engine. Similarly, 22% of instructors never used LLMs in their academic career, though compared to students they felt there was more opportunity for LLMs in an academic setting. Indeed, 29% of students reported instructors spending time discussing the use of LLMs, while 21% of students reported instructors implementing the use of LLMs on assignments. The most common way instructors used LLMs themselves was for writing assistance such as cover letters and emails.




Conclusion: These preliminary results indicate students and instructors are not yet extensively using LLMs in an academic setting, but instructors indicate there is potential for AI in higher education. With increased use and frequent updates, the possibilities of LLMs are likely not yet fully realized. Should current trajectories hold, LLMs could lead to substantial reform in medicine and medical education. Differences between how students and instructors perceive LLMs indicate a need for more discussion regarding how the technology can be practically integrated in educational settings, including clearer ethical guidelines.




Significance/Implication: With AI and LLMs’ rising popularity and frequent improvements, it is vital to consider its use within and implications on the ever-changing medical school curriculum, including ongoing monitoring of use and application of LLMs by both students and educators.