Medical Education Committee Minutes

October 16, 2001

I) Dave reported on the performance of the Class of 2001 on USMLE Step II

Nearly all of our recently graduated seniors (61 out of 65) took "Part 2" of the board exams during their senior year, mostly in March, April, and May. Those who did not take the exam, for the most part, decided not to enter residencies immediately after graduation. Although there were strengths in many areas, and some individual scores were outstanding, overall performance of the class was below our usual level. However, our scores stood above the national average for all Year 4 students nationwide.

The following is a summary containing several perspectives of our Year 4 students' performance, based on summary data from the NBME that I received a short while ago. To save the expense of printed reports, this year I'm e-mailing the material in narrative form.

Keep in mind that our NBME scores in each area have some random movement up and down each year, so it is often more useful to consider an average score over several years as well.

Pass rate: This year, 57 of 61 seniors passed, for a pass rate of 93.4%. The pass rate for the 16,515 seniors from all US schools who took the exam (the comparison group) was 95.0%. Over the last four years, our average pass rate at DMS has been about 97.2% (while the national average for the same period has been 95.0%). I believe that the reason that more than the expected number of our student failed on their first attempt this past spring may be due to several factors: NBME raised the passing score; lack of specific preparation for Step 2 by some of our students; the stress caused by students being pressured by various state medical boards to take the exam earlier in the year; and the difficulty for some students to master the MCQ format of exams during Year 2, Step I, and the shelf exams during the clerkships.

Also, I try to examine which students are at greatest risk for not passing Step I or Step II, so I can properly counsel students in advance. For this past cycle just completed, for the seven seniors with the fewest number of honors and high pass grades over the four years, the pass rate on Step II was 4/7, or 57.1%. For the other 54 seniors, the pass rate was 53/54, or 98.1%. In the future, I can use this type of information to counsel specific seniors preparing to take Step II, similar to the way I've counseled specific sophomores preparing to take Step I.

Overall performance: The average score for our seniors was 217, compared to an average score of 215 for all participants. Our seniors scored 2 "NBME points" better than the national average. Over the last four years (roughly the period of time since our major curricular changes were completed), we have averaged 3.5 NBME points better than the national average. (The range for NBME total scores is a low of about 150 points to a high of about 260 points, with the 50th percentile being approximately 210 each year.

Performance in four broad non-disciplinary areas: In recent years, the NBME has reported student performance in the areas of Preventive Medicine and Health Maintenance, Mechanisms of Disease, Diagnosis, and Principles of Management. Each of these areas builds on many courses and clerkships at DMS.

For example, Mechanisms of Disease builds heavily on many clerkships, plus work done earlier in SBM. Principles of management also builds on concepts introduced during SBM, plus material covered in most of the clerkships, plus advanced material covered in the Clinical Pharmacology and Therapeutics course during Year 4.

The following shows how our students performed in each of those areas this year, and averaged over the last three years (since our "new curriculum" has been almost fully implemented). Each score is the difference in NBME points between our mean score and the national mean score. (A positive number shows how far our students scored above the national average in that area.)

It is evident that our students' scores are strong in all four areas, consistently at or above the national average. Our strongest performance over the past four years has been in the area of Principles of Management.

Performance in five areas corresponding to specific clinical subjects: The NBME also reports our performance in five areas related to the most common clerkships offered at every school. I've listed below our performance in those five areas for the Class of 2001, and the average performance over the past 4 years. Again, a positive number shows how far (in NBME points) our students scored above the national average in that area.

Once again, it becomes evident that our seniors did well in all areas, scoring above the national average in every area this year. Over the past four years, our "best areas" have usually been Psychiatry and Ob/Gyn.

Performance in areas related to 12 organ systems: There are less precise data about performance in 12 system-related topics. (They don't report precise scores in NBME points, but do include a less precise estimate in the form of a graph.) The figure provided shows that our students scored above the national average in nearly all areas, the only exceptions being Immunologic Disorders and Diseases of the Blood and Blood Forming Organs. Our three best areas (in terms of statistics, probably significantly better than the national average) were: Musculoskeletal, Skin, and CT Disease; Gynecologic disorders; and Respiratory diseases.

Basically, our Seniors' performance on Step II is one good indicator of the knowledge and skills acquired by our students in courses and clerkships during Years 2, 3, and 4. Our students continue to do well by nearly every indicator, scoring at or above the national reference group in nearly every area. In fact, in some areas (especially Principles of Management, Ob/Gyn, and Psychiatry) there appears to be continued performance very substantially above the national reference group sustained over the last four years. I would love to see future classes have 100% success rates when taking the exam on the first attempt.

My main concern about Step II of the NBME exam series continues to be the pressure that state boards of medicine, and many residency programs, are placing on students to take the exam earlier and earlier during their fourth year. This pressure runs counter to the efforts of many excellent schools to design a longitudinal curriculum that extends over all four years.

II) Dave presented a proposal for restructuring Year III

Goals:

• To create more flexibility to accommodate four week electives (one per year if desired)
• To allow the two pediatric clerkships to combine into one longer clerkship
• To allow clerkships in ob-gyn and women's health to combine if desired

Proposal:

To create a series of eight week clerkships to include Medicine, Pediatrics, Ob/Gyn (maybe), Family Medicine, Psychiatry, and Surgery, and to create an additional 8-week block that could contain 4 weeks of an elective, plus 4 weeks of either ambulatory medicine, women's health, and/or neurology.

There would be no change in Neurology, Sub I, and Women's Health (maybe).

The blocks would consist of 7 weeks of clerkships, followed by one week of a mixture of activities including finishing up the clerkship; OSCEs; special classwide lectures; and orientation.

The system would require no additional time spent by students. There would be an average of 12 students per block, with a lottery system in place to assign students to blocks (and would have the added benefit of preventing students from delaying certain clerkships till their 4th year).

Dave reported that the clerkship directors he had already discussed the concept with seemed to be enthusiastic about the idea.

Outcome:

The membership was unable to find a fatal flaw in the concept, and was overall supportive of continuing to develop and refine the plan. They urged Dave to go ahead and discuss the proposal with clerkship directors, department chairs, and the student government.

Agenda for next MEC meeting on November 20, 2001 in 758W Borwell:

Introduction of new members
Year III issues and restructuring (continued)

Back to Medical Committee Minutes