Internal Medicine Clerkship Director, Hilary Ryder, MD, MS, FACP, an assistant professor of medicine at Geisel School of Medicine, received an Association of American Medical Colleges (AAMC) Excellence in Medical Education award for her research, “Understanding what we say: varying cultural competency amongst faculty evaluators on the internal medicine clerkship.”
Given by the AAMC Northeast Group on Educational Affairs during their annual spring meeting, the award recognizes contributions made by those in the academic medicine community who advance innovation in medical education. Ryder, interested in the meaning of the evaluations faculty write, sought to determine whether medical students and faculty have a shared understanding of the evaluations.
A shared understanding is important because written comments about medical students have multiple uses in clinical medical education—they document ward performance, provide the written foundation for grade narratives and dean’s letters, and provide feedback on professional development.
By applying well-established anthropological qualitative methods to medical education, Ryder and her collaborator Lauren Gulbas, PhD, an assistant professor at the University of Texas School of Social work (formerly of Dartmouth), asked 20 faculty who attend on the third-year internal medicine clerkship to sort comments written by faculty on student evaluation forms into helpful and unhelpful groups, and then discuss how they determined whether a comment was helpful or unhelpful.
“We analyzed the responses of the group using multidimensional scaling cluster analysis,” Ryder explains, “and found significant variation in faculty understanding of the attributes of an unhelpful comment. Compared to 18 of 22 students interviewed, only half of our faculty reached the level of cultural competency to be considered a key informant, or expert source of information.”
Key informants were more likely to have formal roles within undergraduate medical education (such as clerkship director, sub-intern director or dean of medical education). Other evaluators, including experienced GME educators, demonstrated varying degrees of a lack of understanding of comments, which was profound for several of them.
Ryder says their findings demonstrate that clinical faculty who evaluate medical students do not all share an understanding of what makes a written evaluative comment helpful.
“Winning the Excellence in Medical Education award was a surprise and an honor, especially as there were so many talented medical education researchers at the conference,” she says. “I am especially excited that several other medical schools are interested in joining our research initiative which may lead to a multi-institutional collaboration to better understand the meaning of written comments.”
This research was funded by the Department of Medicine Advisory Committee on Education (DOM-ACE).
