Constructing Multiple-Choice Items
The following is quoted from Wilbert McKeachie, McKeachie’s Teaching Tips, an older edition from the 1990-s.
-
Teachers’ manuals that are provided for many textbooks contain multiple-choice items. You will not be able to rely on a manual as the source of all your questions, because it often will not contain many good questions and may cover only textbook material. You need to assess what students have learned in class as well as what they have read.
-
A second source of such items is the students themselves. This is not a particularly satisfactory source of test questions, because only about 10 percent of the items thus received will be usable. However, this technique is a useful pedagogical device because it gets the students to read their assignments more analytically. It also gives the instructor a good index of what the students are getting out of the various sections of their reading and gives you a chance to remind them of the goals of the course going beyond memory of details.
-
There are statistical methods for evaluating questions, but the best suggestions for improvement come from students themselves in their discussion of the test. It seems almost criminal to waste this experience with items; therefore I recommend a permanent file.
-
If you have a problem, but no good distractor (incorrect alternative), give the item in short-answer or essay form and use the students’ own responses for alternatives for a later use of the item in multiple-choice form.
-
Multiple-choice questions typically have four or five alternatives. Rather than wasting your and your students’ time with extra alternatives that don’t test a discrimination that is important, use only as many alternatives as make meaningful discriminations. Costin (1972) has shown that three-choice items are about as effective as four-choice.
-
For measuring understanding, I like questions that require the student to predict the outcome of a situation rather than those that simply ask the student to label the phenomenon.
-
Multiple-choice items need not stand alone. You can use a sequence of related items to measure more complex thinking.
-
Rules for stating the problem. a. The problem should be stated briefly but completely; the problem should not test the student’s ability to understand complex sentence structure except when the teacher is deliberately measuring that ability. b. The problem should be stated in a positive, not a negative, form. Somehow, even intelligent adults often fail to see a “not” in reading a sentence. If you must use “not,” under-line it. c. It should be possible to understand the problem without reading the alternatives. d. The test is more interesting if the questions are worded in concrete rather than abstract terms. Such items are particularly worthwhile if you wish to measure the student’s ability to apply concepts to concrete situations. In math and science, problems arising in the application of the math or theory are not only more interesting but also more likely to encourage students to generalize the concepts or algorithms to situations in which they will use them.
-
Grouping items under headings will improve student performance (Marcinkiewicz & Clariana, 1997).
Rules for developing the suggested solutions.
- The suggested wrong answers should represent errors commonly made by the students being tested.
- The right answer should be unquestionably right, checked by two or three colleagues.
- The suggested answers should be as brief as possible.
- The position of the right answers should be scattered.
- Numerical answers should be placed in numerical order.
- Even wrong alternatives should not contain words unfamiliar to students. Use “all of the above” and “none of the above” rarely. Usually they are tossed in when you can’t think of another good distractor.
- The right answer should not be given away by irrelevant clues.
A few examples of commonly occurring irrelevant clues are:
- Alternatives that include absolute terms such as “always” and “never” are rarely right answers.
- Alternatives that are longer and more elaborate than the others are frequently right answers.
- If the lead of the item is an incomplete statement, then alternatives that do not complete it grammatically are obviously wrong.*
*Many of the foregoing rules are derived directly or indirectly from notes taken in the “Test Construction” class of Dr. R. M. W. Travers. For a more detailed exposition, see his book, How to Make Achievement Tests (New York: Odyssey Press, 1950), which is still an excellent source.