Writing Multiple Choice Questions For Higher Order Thinking

One of the biggest criticisms of multiple choice questions is that they only test factual knowledge. But it doesn’t have to be that way. We can also use multiple choice questions to assess higher-order thinking.

Higher Order Thinking in a Nutshell

Higher order thinking goes beyond memorizing and recalling facts and data. It even goes beyond comprehension. Higher-order thinking refers to cognitive processes that involve analytical, critical or creative thinking.

The concept is based on various learning taxonomies. In Bloom’s taxonomy, for example, higher order thinking includes application, analysis, synthesis and evaluation. In Anderson’s updated version of Bloom’s taxonomy, it involves  application, analysis, evaluation and creating.

Because test items must be aligned with learning objectives, you’ll need to include higher-order thinking skills from the start. And yes, these may be better measured through open ended questions, essays and discussions. But if you find yourself needing to use multiple choice tests as a necessity, you can make the best of this situation with these three approaches.

1. Real-world Scenarios

One of the best ways to promote and assess higher-order thinking is to use scenario-based questions, particularly ones that simulate real work experiences. The sample question pairs below demonstrate how to transform a question from the simple recall to a higher-order thinking skill using the scenarios approach. Note: Only the question stems are shown. Assume there are multiple choice options beneath each question.

Before: What symbol does a formula always start with in Excel?

After: If you want to total the first ten data cells in column B, which one of the following formulas should you use?


Before: What is the first concern of an emergency worker?

After: You arrive at the scene of an accident where people are panicked and yelling. Three people appear to be injured. What action will you take first?


Before: When writing a CSS style sheet, what property is used to position an element to the left or right?

After: How will the text and photo be aligned using the code below?
CSS: .photo {float: right;}   HTML: My Cat <img src=”cat.jpg” alt=”photo of stuff” class=”photo” />


2. Analysis of Visuals

You can also assess critical thinking skills by asking learners to analyze or interpret information from visuals, which are provided as part of the question stem or the answer choices. In many cases, visuals such as job aids, diagrams and graphs simulate workplace tasks. See a few examples below. Note: Assume there are multiple choice options beneath each question and that visuals are displayed.

Before: What are the components of a computer’s video system? (Select all that apply.)

After: Using the repair flowchart shown here, what should you check if the monitor stops working?


Before: Choose the best definition of rate card.

After: Using the rate card, what is the best time for a customer in the United States to make an international call to Brazil?


Before: Which country has the largest population?

After: What does this graph predict about world population in the year 2020?


Before: Select the best definition of active listening.

After: Which video best demonstrates active listening during a call with an irate customer?


3. The Answer Plus The Reason Why

A third approach to measuring critical or creative thinking is to ask learners to synthesize what they’ve learned into an explanation. Although these test items are more difficult to write, they can be quite satisfactory for assessments. The possible responses include the answer and then a variety of reasons that support the answer. Of course, only one reason is logical and correct, based on the knowledge and skills being assessed.

Before: What are three signs of edema?

After: A patient entered the hospital with edema of both lower extremities. What action should the nurse take and why?


Before: Select the most effective tone for writing technical documentation.

After: Select the paragraph and reasoning that best demonstrate how to start a technical document.


Before: Name three qualities of effective entrepreneurs.

After: Review the financials of these three startup companies. Which one would you invest in and why?

If you have additional ways to measure higher-order thinking in multiple choice questions, please share them below. Also see 10 Rules for Writing Multiple Choice Questions for ways to make your tests more accurate and error-free.


Get The eLearning Coach delivered to your Inbox twice a month, with articles, tips and resources. Sign up below.


Related Posts Plugin for WordPress, Blogger...

Comments

  1. Robin Phares says

    Writing a good multiple choice question is a difficult task starting at grade K and continuing all the way through adult learning. Just for fun, I goggled “How to cheat at Multiple Choice Questions” and found 3,590,000 hits. The top ones included: “when in doubt choose c” and “all of the above” is usually the correct answer.
    I found your post very helpful when thinking about writing multiple choice questions. The scenario option is perfect for the instructional designer. It seems as if it will almost write itself.

    I took a stab at a before and after.
    Before: What is the best app to use to find the definition of a word?
    a) dictionary.com
    b) webster word of the day
    c) color dictionary
    After: Your students are reading “The Whirligig” by Paul Fleischman and come across the word “inadvertently”. What is the best way for them to find the definition on their tablet?
    a) look at dictionary.com
    b) open the app “Webster word of the day”
    c) use the “color dictionary” app

    I believe that as an Instructional Designer assessment will be the most difficult part of our job. It will be easy to design a course and sit back and marvel at our work. The real “test” will be in the “tests” and the more authentic we can design them the better we will be.

Trackbacks

  1. [...] Writing Multiple Choice Questions For Higher Order Thinking: Instructional Design and eLearning From theelearningcoach.com – Today, 4:47 AM Use multiple-choice questions to measure critical thinking, such as analysis, evaluation and interpretation. Before and After examples.One of the biggest criticisms of multiple choice questions is that they only test factual knowledge. But it doesn’t have to be that way. We can also use multiple choice questions to assess higher-order thinking. [...]

  2. [...] Writing Multiple Choice Questions For Higher Order Thinking … From theelearningcoach.com – Today, 1:38 PM You can also assess critical thinking skills by asking learners to analyze or interpret information from visuals, which are provided as part of the question stem or the answer choices. In many cases, visuals such as job aids, diagrams and graphs … [...]

  3. [...] Planning < Education and Teaching < steben45 Get flash to fully experience Pearltrees Writing Multiple Choice Questions For Higher Order Thinking: Instructional Design and eLearning One of the biggest criticisms of multiple choice questions is that they only test factual [...]

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>