123 - Dissecting the Statistics of Two-stage Collaborative Laboratory Examinations to Better Understand Medical Student Performance on Anatomy Assessments

Poster Board Number: 123
There are separate poster presentation times for odd and even posters.
Odd poster #s – first hour
Even poster #s – second hour

Co-authors:

Ariel VanLeuven, PhD - Assistant Professor, Cellular Biology and Anatomy, Augusta University/University of Georgia Medical Partnership; Kimberlee Giffen, PhD - Assistant Professor, Neuroscience and Regenerative Medicine, Augusta University/University of Georgia Medical Partnership; DeLoris Wenzel Hesse, PhD - Associate Professor, Cellular Biology, University of Georgia; Brett Szymik, PhD - Associate Professor, Cellular Biology and Anatomy, Augusta University/University of Georgia Medical Partnership

Abstract Body : Gross anatomy laboratory examinations are commonly employed in pre-clinical medical school curricula. Since 2016, the AU/UGA Medical Partnership has used a two-stage collaborative format for all laboratory examinations. We have previously shown that the collaborative exam portion consistently increases performance on questions from all modalities (body donor-based, osteology-based, or anatomical imaging), and there is significant improvement in performance on second-order questions compared to first-order questions (e.g., identification). There has been minimal previous research completed on the validity of items on collaborative laboratory exams in UME settings. Here, we further explore these measures.

Participants in this study consisted of 61 first-year medical students in AY 2021-22 who took six two-stage collaborative anatomy laboratory examinations administered in ExamSoft. For these exams, students took the exam on their own (“individual assessment”), then formed into their dissection teams to immediately take the exam again as a team (“collaborative assessment”). Teams consisted of 3-4 students and team assignments did not change throughout the year. In addition to question difficulty, two statistical results were analyzed for each item: discrimination index (DI) and point-biserial correlation (PBC). Statistical significance was assessed using Wilcoxon and Kruskal-Wallis tests.

We found an acceptable DI for items on the individual assessments for all question types (0.24-0.33). Similarly, there was an acceptable PBC on the individual assessments for all question types (0.28-0.32). For the collaborative assessments, DI values ranged from 0.01-0.15, and PBC values ranged from 0.03-0.19. DI and PBC both significantly decreased from individual to collaborative assessments for all question types.

Items of all modalities and types on our laboratory examinations appear to be valid and successful measures of student performance, and the collaborative component of the assessment decreases the gap between high and low performers. The DI for all items on the individual assessments were higher than on collaborative assessments, likely due to fewer test takers on the latter (N=61 individuals vs. N=16 teams) and greater score distribution on the former. The PBC decreased for team assessment items as high and low performers likely collaborated with each other. The decrease in the overall distribution of scores supports the utility of collaborative exams for facilitating peer teaching and correction. This is one of the first studies to examine DI and PBC in collaborative laboratory examinations, and our findings are important for informing anatomical educators about the effects of collaborative exams in UME settings.