Specialization Exams — A “Behind the Scenes” Look
Writing good exam questions may seem easy, but in fact, specialty committee members spend hours learning how to create valid and reliable exams and how to write exam questions that differentiate examinees. Committee members, pictured below, meet annually with Dr. Devdass Sunnassee, a professor in the Education Research Methodology Department at the University of North Carolina at Greensboro, and several graduate students to prepare and update the specialty exams each year. Dr. Sunnassee functions as a psychometrician, an expert in exam development and statistics, and guides the committee members as they seek to continually improve the exam process.
Through these meetings, our specialty committee members have learned several new concepts about the test development process including:
Reliability: The degree to which test scores are consistent and stable across exam questions, versions of the exam, or exam administrations.
Validity: The degree to which the test measures what it is intended to measure.
Fairness: Test performance should not be influenced by extraneous factors of the test administration and/or examinee characteristics.
With these guidelines in mind, committee members gather to edit current exam questions, draft new questions as needed, and ensure that the exam covers subject matter in proportion to a typical law practice in that specialty area. This includes looking at a particular legal issue in terms of its relevance, frequency, and importance to the practice. The board specifies that “the examination should not test on laws or procedures that are seldom encountered by a lawyer who is substantially involved in the specialty.”
In addition, committee members try to draft questions that use higher-order thought rather than rote memorization. Specialty certification exams include questions that require application, analysis, synthesis, and evaluation. See Blooms Taxonomy (a model used to classify educational learning objectives into levels of complexity and specificity) below:
The primary goal of the test development process is to prepare an exam that is valid, fair, and reliable for use in certifying lawyers as specialists. The Board of Legal Specialization states that each examination should assess proficiency/competence, not mastery.
Proficient means well advanced in an art, occupation, or branch of knowledge.
Competent means having the requisite abilities or qualities; or having the capacity to function or develop in a particular way.
The board’s objective is to avoid certifying a lawyer who is unable to perform proficiently in the specialty area. In assessing this risk, the board must take into account the types of issues that generally arise in the specialty practice, the frequency that such issues arise, the danger to the client of a lawyer’s misunderstanding of the issues, and the general level of knowledge of those lawyers who are substantially involved in the area of practice.
The specialty exams are an integral part of the certification process. When used in conjunction with the other standards of substantial involvement—continuing legal education and peer review—specialty certification serves its dual purpose of 1. identifying lawyers who have demonstrated special knowledge, skill, and proficiency, and 2. improving the competency of the Bar.
If you would like information about joining a specialty committee, please notify Denise Mullen.
Filed Under: General News