NUTSHELL NOTES "Teaching tips in a nutshell" - The University of Colorado at Denver's One-page Newsletter for Teaching Excellence

This format must sound familiar to anyone who has been through the American school system. In testing, multiple-choice items are the most widely used of the selection-type items, perhaps because multiple-choices can be used to test such a wide range of instructional objectives. Forty years ago, educator B. H. Bloom recognized six levels of learning presented in the sequence from lower to higher order thinking:

Knowledge—simply recalling factual material

Comprehension—understanding as displayed by ability to reorganize or restate material

Application—problem-solving or applying ideas and principles to deal with given situations

Analysis—separating ideas into component parts and recognizing how the parts are related

Synthesis—combining known ideas to yield a product that is new to the learner

Evaluation—using established standards or criteria to make judgements about the value or quality of ideas.

Multiple-choice items can be used to test all six levels, but the ease with which multiple-choice items can be constructed to test lower levels of thinking often leads to tests that address only these levels. This arises not from the inherent format of the multiple-choice test, but rather from the effort and level of thinking required to produce items that test high-level thinking in others. Construction of good multiple-choice tests begins with deciding the appropriate distribution of knowledge to test in a course. For beginning courses in which students lack even the basic vocabulary of the discipline, it may be reasonable to have a large portion of the test devoted to testing their acquisition of basic (albeit lower level) knowledge. Without a reasonable amount of basic knowledge and skills, it isn't reasonable to expect our students to do much high-level reasoning.

Some tips from experts on constructing good multiple-choice items follow.

(1) Write the stem so as to present a single, question or problem. Stems without verbs fail to present problems clearly. A closed stem may be a question such as "Which of the following...?" An open stem involves a sentence completion question with the blank at the end. An example would be: "Evidence that radon is a significant health hazard comes from..." Good practice in drafting multiple-choice questions places the blank ALWAYS at the end of the stem, never within it.

(2) Stems should be brief and convey the essential idea of the question. Stems are used for testing, not for teaching; two sentence stems that convey information first and then ask for responses violate good practice.

(3) In some formats, the examinee is required to pick an incorrect response from several correct responses. These are called "EXCEPT, NOT formats." When used, the writer should always write the word NOT or EXCEPT in capital letters to emphasize the true nature of the question. An example would be: "Which of the following is NOT an example of the passive voice?..." "LEAST, BEST or MOST formats" also require all caps of LEAST; i.e., "The LEAST likely of the following materials to occur on the Moon is...."

(4) Options should be brief, of similar length, presented in a logical order, and no choice should be so absurd as to render an option useless for the testing of thinking or of content.

(5) All options should flow grammatically from the stem. If an item reads poorly, students' confusion will yield results that are not measures of actual knowledge.

References: Clegg, V. L., and Cashin, W. E., 1986, Improving multiple choice tests: IDEA Paper No. 16, KS State University. Educational Testing Service, 1994, Developing good multiple-choice test questions: Princeton, ETS. Jacobs, L. C., and Chase, C. I., 1992, Developing and Using Tests Effectively: San Francisco, Jossey-Bass.