Curricular Affairs

Exam Review Subcommittee (ERS)

Overview

Tucson Educational Policy Committee (TEPC) establishes the Examination Review Subcommittee (ERS) as a standing subcommittee of TEPC. This subcommittee holds the primary responsibility to oversee the quality and outcomes of student performance assessment for Years 1 and 2. The “Performance Assessment Plan for the Tucson Years 1 and 2” is the domain of this subcommittee and reports of the plan’s implementation and its outcomes will be delivered regularly to the TEPC and its other subcommittees.

Membership

TEPC regards the guidelines and procedures for the oversight of performance assessment to be a curriculum support function of the Curricular Affairs office.  As such, the members of this committee will be appointed by the Associate Dean for Medical Student Education. The Manager for Assessment and Evaluation will chair the subcommittee.

The membership of the Examination Subcommittee is comprised of the following individuals: (Note: it is important that the core membership include members who have not been involved (or who have had minimal involvement) in the construction of the exam items being considered. Because of the nature of work performed by this subcommittee, there may be no student representatives.

  1. Experts in Student Performance Assessment (OMSE)
  2. One Discipline Director
  3. AMES member or faculty with experience writing MCQ (preferably NBME items)
  4. Other ad hoc content consultants (Block directors and/or the Examination Subcommittee members can recommend content experts to consult with the committee for specific items.)

Responsibilities

The ERS will establish the criteria by which exam items are deemed of high quality. These will be applied consistently when making decisions about exam items. The criteria that establish acceptable/unacceptable item statistics will be set, approved by TEPC, and published in the appropriate sections of block syllabi. Other decision criteria may include such situations as when two answers might be accepted.

Because the evaluation of exam items includes a corresponding evaluation of its referential learning objectives, the Exam Subcommittee will be copied on block changes that invoke the Year 1 and 2 Block/Course Changes policy with respect to objectives.

  • The ERS will apply the criteria in the preview of items for those exams and quizzes that contribute to more than 10% of a student’s grade in any block.
  • The ERS will apply the criteria after the delivery of an exam:
    • The ERS has the final approval regarding dropping, revising items and the subsequent changes to grading that might arise.
    • The ERS’s decision to drop an item or otherwise change the scoring of an item will be sent immediately to Instructional Technology to recalculate (if needed) prior to the release exam scores.

Policies Related to the Oversight of Examinations and Certification of Exam Items

  • There will be consistent implementation of exams across blocks in Years 1 & 2. This includes regularly scheduled block exams, retake, and remediation exams. Implementation includes procedures for delivering the exam, post-hoc analysis of exams, and subsequent rules for making adjustments to grades.
  • Items for an exam will be selected according to acceptable psychometric parameters based on previous iterations.
  • It is assumed that all test items used during academic year 2012-13 are adequate with respect to this policy. For academic year 2013-14 and forward, new items, or revisions to current items must be previewed by the Examination Subcommittee prior to inclusion on an exam.
  • New exam items will be vetted through a consistent process established by the Exam Subcommittee. Approval of an item by this process will certify that it adheres to principles of high quality item construction. That is:
    • its assessment intent is unambiguous
    • it is typographically and grammatically correct, and
    • it demonstrates association to learning objectives
  • For development of new questions and/or revision of underperforming items, beta testing of items may be conducted within exams.

After an exam has been delivered

  • The block director may direct Instructional Technology to DROP a question in situations in which there is a technical error in item delivery (e.g., question repeated on the exam, an item has two identical foils, images failed to load).
  • Exam Subcommittee will be scheduled to meet for the hour after the delivery of each exam. In this way, the subcommittee will be on hand to review exams in a timely manner.
  • If review of exam statistics prompts concern about an item after the exam has been delivered, the item will be reviewed by the Exam Subcommittee in accordance with established criteria.
  • Student challenges of exam items will be reviewed by the Exam Subcommittee only when the exam statistics for that item prompts concern.
  • Results from exams will be posted to the gradebook in ArizonaMed online. Typically the student scores can be posted within one business day due to possible adjustment of items.

Relevant Accreditation Standards

ED-26. A medical education program must have a system in place for the assessment of medical student achievement throughout the program that employs a variety of measures of knowledge, skills, behaviors, and attitudes.

ED-29. The faculty of each discipline should set standards of achievement in that discipline and contribute to the setting of such standards in interdisciplinary and interprofessional learning experiences, as appropriate.

ED-28. A medical education program must include ongoing assessment of medical students’ problem solving, clinical reasoning, decision making, and communication skills.

ED-06, ED-07, ED-17A, ED-21, ED-22: These standards explicitly ask each medical school to demonstrate how they assess specific knowledge and skills such as: knowledge and understanding of societal needs and demands on health care, basic principles of clinical and translational research and cultural competency.