USMLE Forum Archives - Miscellaneous - About the Comprehensive Review of USMLE
About the Comprehensive Review of USMLE
adonis123 - 06-06-10 01:35
Here r the answers to some of the questions about the Comprehensive Review of USMLE which is underway:
When will the talked-about changes be implemented?
Some of the changes anticipated, for example, introducing new item types (multimedia) and enhancing the representation of fundamental science in Steps 2 and 3, will be implemented over several years and have already begun. Others are anticipated to unfold over the next two to five years, subject to the development and testing necessary to validate new item types.
What are the new types of items that will be used? How will interpretation of literature and information gathering be assessed?
In addition to enhanced assessment of clinical skills as envisioned above, we also anticipate developing items to measure examinees' ability to interpret and apply both scientific and medical literature. We also anticipate developing assessments that reflect the realities of current medical practice and the availability of rich information resources at the point of care via various technologies. As information becomes available about specific formats that will appear on examinations, the USMLE web site will be updated to reflect current knowledge.
Will Step 1 and 2 CK be combined into one exam?
Steps 1 and 2CK will exist as separate, discrete examinations for at least the next several years. During this time, evolution in content coverage is likely. At some point as USMLE evolves, the current Step 1, 2CK, 2CS, 3 structure could shift. If this occurs, stakeholders will be advised of changes well in advance of any change. Plans for transitioning to a new structure will be provided well in advance so that the phase-in will be smooth and equitable, to minimize potential disruptions for examinees, schools, licensing boards, or other constituencies. Additionally, the NBME will be attentive to other uses of USMLE performance data and strive to meet educationally sound secondary uses.
If Step 1 disappears, how can medical schools that make promotion decisions (from MS2 to MS3) continue to make these decisions?
Changes in the structure of USMLE exams may have little or no impact on schools using USMLE performance for other purposes. However, the NBME recognizes that medical schools use the results of USMLE examinations for the assessment of both individual students and of the schools' educational programs. In implementing changes to the USMLE exam sequence, the NBME will be attentive to other assessment needs and the secondary uses of USMLE.
What is going to happen to the CS exam?
During the next two years the Step 2 CS examination will retain its current structure and organization, though we do expect to enhance assessment of clinical skills (eg, though new station formats) within that structure. Over a longer time period, as USMLE continues to evolve, it is likely that the CS examination may incorporate new stations and simulation formats, more sophisticated assessment of communication and physical examination skills, more complex cognitive tasks, and/or hybrid assessments that could incorporate other question formats and assessment activities into the standardized patient assessment.
Will you continue to report scores, or change to pass/fail?
In conversations with the various stakeholder groups that occurred as part of the USMLE review process, strongly held opinions were expressed on both sides of this question. Some felt that maintaining numeric score reporting is essential while others felt that numeric scores should be abolished. For the present, we do not anticipate changing current practice. As the examination sequence evolves, potential changes in score reporting would be weighed in light of validity research, including intended use of scores.
What are the “competencies”? Are they all going to be assessed in USMLE? How?
In accordance with direction from the USMLE Composite Committee and the Committee to Evaluate the USMLE Program, USMLE governance will consider changes to the examination sequence in the context of competencies that have become increasingly prevalent in recent years as a means of organizing medical education and assessment. One of the most commonly used competency frameworks was developed collaboratively by the ACGME and the ABMS. This framework identifies six competencies: medical knowledge, patient care, communication and interpersonal skills, professionalism, practice-based learning and improvement, and systems-based practice. Some in the medical education and practice community have also identified a seventh competency related to surgical and other technical procedural skills.
The NBME will explore how best to assess examinees in all competency areas. It is likely that the assessments available via a revised USMLE will not be able to measure all competencies to an equal degree. It is possible that one or more competencies will not be measurable in a valid manner in the context of the USMLE. During implementation of changes to the USMLE, staff will identify how best to assess examinees’ performance in each of the competencies using available methods, develop a research agenda to expand assessment tools for competencies not adequately measured presently, and identify competencies that cannot be reliably measured in USMLE so that assessments in other contexts (e.g., residency training programs) can supplement the information provided by the USMLE.
Good Luck!
adonis123 - 06-06-10 01:35
Here r the answers to some of the questions about the Comprehensive Review of USMLE which is underway:
When will the talked-about changes be implemented?
Some of the changes anticipated, for example, introducing new item types (multimedia) and enhancing the representation of fundamental science in Steps 2 and 3, will be implemented over several years and have already begun. Others are anticipated to unfold over the next two to five years, subject to the development and testing necessary to validate new item types.
What are the new types of items that will be used? How will interpretation of literature and information gathering be assessed?
In addition to enhanced assessment of clinical skills as envisioned above, we also anticipate developing items to measure examinees' ability to interpret and apply both scientific and medical literature. We also anticipate developing assessments that reflect the realities of current medical practice and the availability of rich information resources at the point of care via various technologies. As information becomes available about specific formats that will appear on examinations, the USMLE web site will be updated to reflect current knowledge.
Will Step 1 and 2 CK be combined into one exam?
Steps 1 and 2CK will exist as separate, discrete examinations for at least the next several years. During this time, evolution in content coverage is likely. At some point as USMLE evolves, the current Step 1, 2CK, 2CS, 3 structure could shift. If this occurs, stakeholders will be advised of changes well in advance of any change. Plans for transitioning to a new structure will be provided well in advance so that the phase-in will be smooth and equitable, to minimize potential disruptions for examinees, schools, licensing boards, or other constituencies. Additionally, the NBME will be attentive to other uses of USMLE performance data and strive to meet educationally sound secondary uses.
If Step 1 disappears, how can medical schools that make promotion decisions (from MS2 to MS3) continue to make these decisions?
Changes in the structure of USMLE exams may have little or no impact on schools using USMLE performance for other purposes. However, the NBME recognizes that medical schools use the results of USMLE examinations for the assessment of both individual students and of the schools' educational programs. In implementing changes to the USMLE exam sequence, the NBME will be attentive to other assessment needs and the secondary uses of USMLE.
What is going to happen to the CS exam?
During the next two years the Step 2 CS examination will retain its current structure and organization, though we do expect to enhance assessment of clinical skills (eg, though new station formats) within that structure. Over a longer time period, as USMLE continues to evolve, it is likely that the CS examination may incorporate new stations and simulation formats, more sophisticated assessment of communication and physical examination skills, more complex cognitive tasks, and/or hybrid assessments that could incorporate other question formats and assessment activities into the standardized patient assessment.
Will you continue to report scores, or change to pass/fail?
In conversations with the various stakeholder groups that occurred as part of the USMLE review process, strongly held opinions were expressed on both sides of this question. Some felt that maintaining numeric score reporting is essential while others felt that numeric scores should be abolished. For the present, we do not anticipate changing current practice. As the examination sequence evolves, potential changes in score reporting would be weighed in light of validity research, including intended use of scores.
What are the “competencies”? Are they all going to be assessed in USMLE? How?
In accordance with direction from the USMLE Composite Committee and the Committee to Evaluate the USMLE Program, USMLE governance will consider changes to the examination sequence in the context of competencies that have become increasingly prevalent in recent years as a means of organizing medical education and assessment. One of the most commonly used competency frameworks was developed collaboratively by the ACGME and the ABMS. This framework identifies six competencies: medical knowledge, patient care, communication and interpersonal skills, professionalism, practice-based learning and improvement, and systems-based practice. Some in the medical education and practice community have also identified a seventh competency related to surgical and other technical procedural skills.
The NBME will explore how best to assess examinees in all competency areas. It is likely that the assessments available via a revised USMLE will not be able to measure all competencies to an equal degree. It is possible that one or more competencies will not be measurable in a valid manner in the context of the USMLE. During implementation of changes to the USMLE, staff will identify how best to assess examinees’ performance in each of the competencies using available methods, develop a research agenda to expand assessment tools for competencies not adequately measured presently, and identify competencies that cannot be reliably measured in USMLE so that assessments in other contexts (e.g., residency training programs) can supplement the information provided by the USMLE.
Good Luck!
Page 1
#2
Re: About the Comprehensive Review of US babbu5508 - 06-06-10 02:39 keep updating about such things, adonis and thanks..
#3
Re: About the Comprehensive Review of US
adonis123 - 04-21-11 05:56 Latest Developments in the Comprehensive Review of the USMLE
Content Changes
Consistent with changes described previously, Step 1 content is being modified to maximize the clinical context of the concepts being assessed and to align the foundational science contained in Step 1 with changes in science assessment anticipated for Steps 2 and 3. In support of the recommendation to enhance assessment of science across the spectrum of USMLE examinations, small increases in the number of biostatistics and epidemiology items will appear in Step 2 beginning in 2010 and in Step 3 in 2011.
Item Formats
Several new item formats are also under development for computer-based testing (Step 1, Step 2 Clinical Knowledge, and Step 3). USMLE committee members have developed new literature interpretation items that will assess the examinee's ability to interpret scientific literature and pharmaceutical advertisements. If supported by pilot testing, these items should appear in the USMLE examination sequence in 2012. Work is also underway on a novel format that incorporates both video and new assessment activities such as admissions orders, laboratory testing, etc. to simulate progression of an individual patient case over time, enhancing the clinical authenticity of computer-based assessments.
Step 2CS
Enhancements to the Clinical Skills examination will extend both the nature and degree of challenges faced by examinees. A recent successful pilot has resulted in changes that will enhance the authenticity of interactions between standardized patients and examinees. Building on this, we anticipate that examinees will face more challenging communications cases in the near future. Additionally, pilot work is underway to shape changes in the patient note that will allow more effective assessment of the examinee's clinical reasoning. Additionally, examinees will need to use the keyboard for patient notes beginning in mid-year 2011; handwritten notes will no longer be acceptable.
Ongoing Change
Senior staff associated with the USMLE program have begun to consider potential changes to the structure of the USMLE exam sequence. If structural change is indicated, staff hopes to have developed a draft plan by the end of 2010. Staff members of the ECFMG, FSMB, and NBME continue to meet quarterly and more often as indicated to review developments in the ongoing evolution of the USMLE program. Attentive to the needs of stakeholders, this staff group reviews developments in the USMLE program and targets announcements to the channels and groups needed to share important information. The most current updates and the status of any changes will continue to be available at www.usmle.org/cru.
Date posted: June 14, 2010
#4
Re: About the Comprehensive Review of US adonis123 - 04-21-11 05:56 March 2011 Comprehensive Review of USMLE Update
New Item Formats
Since our last update in June 2010, staff has completed successful pilot work in several areas. Test material committees have developed multiple choice questions that assess an examinee's ability to appropriately interpret information presented both in the form of a research abstract and a pharmaceutical advertisement. Additional study is planned, and the items that ultimately appear in the examination may differ from the current design being tested. If planned pilot tests are successful, a small number of these item types will appear in Step 2 CK in August 2011and in Step 3 in March 2012. These dates are tentative; any final decision about the inclusion of these new item formats (and any significant changes to the exam) will be posted to the USMLE website prior to implementation.
Step 2 CS
In 2010, pilot tests of changes to communication skills assessment and changes to the patient note were successful, with further testing planned for 2011. If testing in 2011 is also successful, changes will be implemented in the Step 2 CS examination in mid-2012. Confirmation of any change will be posted to the USMLE website well in advance of its implementation.
Potential changes to the patient note include asking examinees to document their analysis of a patient's possible diagnoses, indicating the evidence obtained from the history and physical examination that supports (or refutes) different diagnostic possibilities.
Envisioned changes to communication skills assessment are targeted to enhance the nature and difficulty of challenges posed by the standardized patient (SP). Examinees will be assessed based on their ability to tailor their questions and responses to the specific needs of the case presented and on their ability to react to the entire range of the patient's concerns. The system of scoring communication skills will also change; while the focus will remain on an examinee's ability to effectively gather and share information, and to establish a professional rapport with the SP, the scoring system used will reward examinees who demonstrate a broader and deeper range of communication skills and approach the encounter in a focused manner. More information about the enhancements to the assessment of communication skills is available in the Fall/Winter NBME Examiner.
Behind-the-Scenes Changes
Other issues being explored in 2011 range from the smallest details (how to code the content and emphasis of each individual question) to the fundamental structure of the USMLE program (how many exams and what scores). Staff is presently engaged in work that will support required changes to the internal systems used to organize and report exam content. When this work is completed (target date: 2013) it will enable new score reports, e.g., competency-based reports to complement traditional discipline and specialty scores.
Over the course of 2011, models of potential changes to the USMLE exam sequence will be reviewed by staff of the ECFMG, FSMB, and NBME; by USMLE governance; and by key stakeholders. Models being considered would maintain approximately the same amount of total testing time across the USMLE exam sequence, and would not increase fees charged to examinees beyond historical trends. Changes under consideration include the following:
The number of examinations;
The content assessed in each individual examination and in the USMLE program as a whole;
The types of assessment challenges, e.g., new item formats described above; and
The way that examinees' performance is scored and reported.
Depending on feedback received during this review, plans presently call for public announcement of changes to the USMLE structure in the first half of 2012. That announcement will address both the nature of changes as well as the timeframe for implementation, including the process for transitioning from the current Step system to a new system of USMLE examinations.
Page 1