This is a back-to-basics article about the undervalued and little-discussed multiple choice question. It’s not as exciting as discussing 3D virtual learning environments, but it might be just as important. If you need to use tests, then you want to reduce the errors that occur from poorly written items.
The rules covered here make tests more accurate, so the questions are interpreted as intended and the answer options are clear and without hints. Just in case you’re not familiar with multiple choice terminology, it’s explained in the visual below.
Here are the ten rules. If you have any others, please add them through the Comments form below.
Rule #1: Test comprehension and critical thinking, not just recall
Multiple choice questions are criticized for testing the superficial recall of knowledge. You can go beyond this by asking learners to interpret facts, evaluate situations, explain cause and effect, make inferences, and predict results.
Rule #2: Use simple sentence structure and precise wording
Write test questions in a simple structure that is easy to understand. And try to be as accurate as possible in your word choices. Words can have many meanings depending on colloquial usage and context.
Rule #3: Place most of the words in the question stem
If you’re using a question stem, rather than an entire question, ensure that most of the words are in the stem. This way, the answer options can be short, making them less confusing and more legible.
Rule #4: Make all distractors plausible
All of the wrong answer choices should be completely reasonable. This can be very hard to accomplish, but avoid throwing in those give-away distractors as it detracts from the test’s validity. If you’re really stuck, get help from your friendly SME. (BTW, this word can also be spelled as “distracter.”)
Rule #5: Keep all answer choices the same length
This can be difficult to achieve, but expert test-takers can use answer length as a hint to the correct answer. Often the longest answer is the correct one. When I can’t get all four answers to the same length, I use two short and two long.
Rule #6: Avoid double negatives
No big news here, right? Don’t use combinations of these words in the same question: not, no, nor, the -un prefix, etc. For example, this type of question could confuse test-takers: ‘Which of the following comments would NOT be unwelcome in a work situation?’ Flip it around and write it in the positive form: ‘Which of the following comments are acceptable in a work situation?’
Rule #7: Mix up the order of the correct answers
Make sure that most of your correct answers aren’t in the “b” and “c” positions, which can often happen. Keep correct answers in random positions and don’t let them fall into a pattern that can be detected. When your test is written, go through and reorder where the correct answers are placed, if necessary.
Rule #8: Keep the number of options consistent
Did you ever have to convince a SME that he or she can’t have answer choices that go to ‘h’ in one question and ‘c’ in the next? It’s something of a user interface issue. Making the number of options consistent from question to question helps learners know what to expect. Research doesn’t seem to agree on whether 3 or 4 or 5 options is best. Personally, I like to use 4 options. It feels fair.
Rule #9: Avoid tricking test-takers
As faulty as they are, tests exist to measure knowledge. Never use questions or answer options that could trick a learner. If a question or its options can be interpreted in two ways or if the difference between options is too subtle, then find a way to rewrite it.
Rule #10: Use ‘All of the Above’ and ‘None of the Above’ with caution
I hate this rule because when you run out of distracters, All of the Above and None of the Above can come in handy. But they may not promote good instruction. Here’s why. All of the Above can be an obvious give-away answer when it’s not used consistently. Also, the All of the Above option can encourage guessing if the learner thinks one or two answers are correct. In addition, the downside to None of the Above is that you can’t tell if the learner really knew the correct answer.
Related Articles:
Writing Multiple Choice Questions for Higher Order Thinking
Are Your Online Tests Reliable?
Are Your Online Tests Valid?
Tips for Writing Matching Format Test Items
Thank you for sharing such a great information with us
This is a great question and it’s good that you are aware of the nuances here. I have a book recommendation, which I haven’t read yet, but suspect this will answer your questions. “Practice and Feedback for Deeper Learning by Patti Shank.”
Best,
Connie
What tips do you have for writing test item feedback? We are struggling to find a balance between very specific or very general feedback.
On one hand, we don’t want learners to use the feedback to cheat, but it would help them remember which questions they missed and why. However, if they failed and take a different form of the test, they may not be able to transfer that knowledge well. The general feedback might provide a nice study guide of sorts, but could be too broad to be useful.
Hi Santosh,
I get what you are looking for but I don’t know of a product like that. There could be one or it could be something like a survey tool. I wonder if Survey Monkey has that capability. You might be able to accomplish this with some light programming in one of Google’s free tools. Try asking in one of the eLearning Development groups on LinkedIn. Let us know what you find!
Connie
Hello Mam,
I am making a empathy and communication skill questionnaire for my organisation. A question for example is as under:
Answer each statement honestly, thinking about your likely behaviour. There are no wrong or right answers.
Q1. I try to have the last word on a subject.
(a) Always or Nearly Always
(b) Often
(c) Sometimes
(d) Never or Hardly ever
Now can you suggest me a quiz maker app which can assign marks for each choice. Here (a) would have 0 points and (d) will have the highest scoring that is 3 points. So on that scale (b) will have 1 point, and (c) will have 2 points. Do you know any app /software that has the feature of assigning marks to individual choices. I know of some that assigns points scored only to one choice, that is the correct one. Regards
Julie – I answered this a few days ago but I guess it didn’t save. It would be interesting to ask these questions to a professional editor. I keep things consistent in punctuation and within the options for each test, course, organization. Like you, I place a period at the end of sentences. I also place a period at the end of the answer options for options that are intended to complete a sentence. As to your #2, I typically use lower case for the answer choices. I see that in SAT sample questions, they also use lower case. But I work for many different clients and if their standard is upper case, I do that. I hope this helps. Good question!
1. Do you put a “full stop” at the end of each option? I have been putting one at the end of options which are complete sentences, but not when they are single words/phrases. Is that correct?
2. Do you use upper or lower case used for single word/phrase options, e.g.
a. Seventy
b. Seventy-five etc or
a. seventy
b. seventy-five
That sounds awesome. I can see why students would like that. Since our goal is to help learners gain mastery and competence, I don’t see why this wouldn’t be a good strategy.
Best,
Connie
I am looking at a “quiz qualifier” for some multiple choice tests where students can explain a few of the answers they selected. This way, I can better assess their knowledge. Any input on that? I have tried on a few quizzes and the students really appreciate the opportunity.
Hi Karen,
In the real world, it does seem that a multiple choice question for “creating” is a bad fit so I do not think the assignment is particularly meaningful. That said, the best I can suggest is to simulate creation via a multiple choice question. For example, if the LO is to create an innovative title for an article, you could say, “You must create a title for an online magazine article. Select the title that stands out as one that will grab attention.” Or if the LO is to create a design with effective use of white space, you could show three designs, each with a rationale for why it is effective. Then the learner must select one.
Also, see if this article about writing multiple choice questions for higher order thinking gives you any ideas.
Good luck!
Connie
May I ask you a question?
We have this task to create question under creating level of the revised Bloom’s taxonomy. Now my question is, is it possible to create a multiple choice questions in creating level? I’m really having a hard time trying to look for any sample multiple choice questions under creating level.
You’re right, Mohammad. I’m starting to write an eBook on this topic that will have lots of examples. Thanks for your input.
Connie
Connie, your article is good, but could be great if you would provide at least one couple of examples for each rule. One poor and one better examples. Think about it.
Hi Madam,
No offence meant. My only opinion was that this article needs to be validated. Wishing you a good day and a happy new year.
Ranjini.
Hi Ranjini,
Yes, there are many other guidelines too. These were, in my estimation, the top ten. And they were researched 🙂
Best,
Connie
hi mam,
good article, but mam lot of another guide lines related to multiple choice questions. i prepared a document about it. pls research your document.
Of course, Jeremy. Thanks for asking and good luck on your project! It sounds very important. Will send as an email too.
Connie
Good afternoon! We are seeking your permission to include your 10 Rules For Writing Multiple Choice Questions in a project we are undertaking.
The project is to develop and support implementation of a new qualification for the Ministry of Education, Science and Technology in Kosovo. The qualification is Level 5 Certificate for Teaching Assistants for Special Needs Children.
We would acknowledge the resource materials, with web references for further detail and information.
It would be really helpful to have your e-mail confirmation that we can use and reference materials from your website if that is acceptable to you.
We look forward to hearing from you!
Thanks for sharing this information, Sarah.
Best,
Connie
Kahoot or socrative are good for creating multiple choice. I’m not keen on them as a learning tool but AQA use multiple choice in A level business and economics papers. So tend to use to prepare learners.
Good points, Bryan.
Both reliability and validity are put into question if you don’t have the same amount of options/distractors for each one. More distractors make it more difficult, fewer make it easier. There can be no argument about that-there must be consistency, otherwise what is the point of testing?
Hi Alice,
Thanks, for reiterating an important point! Please come back and continue to add to the conversation.
Connie
Great pointers! Here’s my favorite. When considering which questions will be on the test, refer back to your class objectives. If they were written according to Bloom’s Taxonomy, then precise wording, appropriate skill level, and critical thinking will be that much easier to build into your questions.
In my opinion, tt does seem that if you use All of the above, you’d need to use it pretty often. Otherwise, it’s obviously the correct answer in many cases. Better to use a multiple selection multiple choice question, if your audience can handle that type of interaction.
Agree with #10. Lack of credible distractors leads you to AOTA. I feel you need some questions with AOTA as a distractor. But one study showed that actually caused more correct answers!
Hi Ann,
Great advice! It’s readers like you that add so much to this site. Thanks.
Best,
Connie
An additional rule: Ensure that your question is at the correct level of difficulty. A SME should be able to answer it with ease; naive learners should not be able to guess it consistently. (I am currently reviewing an e-learning module done by a reputable company and have been horrified at the exercise questions. Some are so obtuse no SME could produce the answer, while others are at a level my nine-year-old could figure out!)
Hi Sue,
Of course you can use the list. It’s there to help people. I hope your class goes well!
Connie
Hi Connie,
I love your 10 rules! I will shortly be teaching colleagues enrolled on our staff development course how to improve their question writing skills. I wonder if you would mind if I use your ideas as a resource (with reference to you, of course) as a basis for discussion in one of my face to face sessions? This would involve printing off copies to use as a paper and pen exercise. Colleagues will then use the rules to help them design their own computer based quizzes.
Many thanks for a great resource.
Hi Joe,
I do agree, it would be better to have an inconsistent number of distractors rather than an implausible one. You know how rules are … Thanks for sharing your insight!
Connie
This is a helpful list, thanks. Many educators disagree with rule #8 though. Rule #4 takes precedence: sometimes it’s best to throw in a question with two or three distractors rather than come up with implausible distractors in the name of consistency.
I haven’t seen anyone split the difference here, but I will be bold enough to come up with my own rule: Don’t have any questions with EXTRA distractors, but an occasional question with FEWER distractors is better than forcing implausible distractors into a question for the sake of uniformity. Studies show that having fewer distractors does not, oddly enough, improve performance based on chance. You can create excellent questions with two distractors.
Excellent list, thanks for sharing this with us.
Regards, David.
Thanks for the link Michael. I like your article.
Connie
I’ve written some additional tips for ordering distrators and the key that do not contribute unnecessarily to the wrong things.
Check out http://www.viral-notebook.com/wordpress/2009/05/06/increasing-cognitive-load-part-2/