• Home
  • Free Resources
    • Storyboard Depot
    • Instructional Design Programs
    • Best Stock Photo Sites
    • PowerPoint Grids
    • eLearning Examples
    • Glossary of Online Learning Terms
    • Graphic Editing Tools
    • Icon Collections
  • My Books
  • About
  • Book Store
  • ON-DEMAND COURSES

The eLearning Coach

For designing effective learning experiences

  • Podcasts
  • LXD
  • eLearning Design
    • Audience
    • Instructional Design
    • Learning Objectives
    • Storyboarding
    • Tests
    • User Interface
  • eLearning 2.0
  • Cognition
  • Multimedia
    • Audio
    • Graphics
    • Video
  • Reviews
    • Book Reviews
    • Software Reviews
  • At Work
You are here: Home / eLearning Design / Writing Multiple Choice Questions For Higher Order Thinking

Writing Multiple Choice Questions For Higher Order Thinking

by Connie Malamed

Tweet181
Pin59
Share247
487 Shares

Writing Multiple Choice Questions for Higher Order Thinking

One of the biggest criticisms of multiple choice questions is that they only test factual knowledge. But it doesn’t have to be that way. We can also use multiple choice questions to assess higher-order thinking.

Higher Order Thinking in a Nutshell

Higher order thinking goes beyond memorizing and recalling facts and data. It even goes beyond comprehension. Higher-order thinking refers to cognitive processes that involve analytical, critical and creative thinking. The concept is based on various learning taxonomies, such as application, analysis, evaluation, creation, problem-solving, connecting ideas and making decisions. Keep in mind that many of these cognitive tasks, including the recall of information, appear to occur simultaneously. See Alternatives to Bloom’s Taxonomy for criticisms of the hierarchical classifications.

Because test items must be aligned with performance objectives, you’ll need to include higher-order thinking skills from the start. And yes, these may be better measured through open ended questions, essays and discussions. But if you find yourself needing to use multiple choice tests, you can make the best of this situation with these three approaches.

1. Real-world Scenarios

One of the most effective ways to assess higher-order thinking is to use scenario-based questions, particularly ones that simulate real work experiences. The sample question pairs below demonstrate how to transform a question from the simple recall to a higher-order thinking skill using the scenarios approach. Note: Only the question stems are shown. Assume there are multiple choice options beneath each question.

Example 1:

  • Before: What symbol does a formula always start with in Excel?
  • After: To total the first ten data cells in column B, which one of the following formulas should you use?

Example 2:

  • Before: What is the first concern of an emergency worker?
  • After: An emergency technician (ET) arrived at the scene of an accident. Three people were injured. One person had scrapes and bruises, a second was moaning in pain and a third was quickly losing blood from a skull injury. Why did the ET triage the skull injury as the highest priority?

Example 3:

  • Before: When writing a CSS style sheet, what property is used to position an element to the left or right?
  • After: How will the text and photo be aligned using the code below?
    CSS: .photo {float: right;}   HTML: My Cat <img src=”cat.jpg” alt=”photo of stuff” class=”photo” />

2. Analysis of Visuals

You can also assess critical thinking skills by asking learners to analyze or interpret information from visuals, which are provided as part of the question stem or the answer choices. In many cases, visuals such as job aids, diagrams and graphs simulate workplace tasks. See a few examples below. Note: Assume there are multiple choice options beneath each question and that visuals are displayed.

Example 1:

  • Before: What is/are the function(s) of a computer’s projecting mechanism? (Select all that apply.)
  • After: Using the repair flowchart shown here, what should you check if the monitor stops working?

Example 2:

  • Before: Choose the best definition of rate card.
  • After: Using the rate card, what is the best time for a customer in the United States to make an international call to Brazil?

Example 3:

  • Before: Which country will have the largest population in the year 2030?
  • After: What does this graph predict about world population in the year 2030?

Example 4:

  • Before: Select the best definition of active listening.
  • After: Which video best demonstrates active listening during a call with an irate customer?

3. The Answer Plus The Reason Why

A third approach to measuring critical or creative thinking is to ask learners to synthesize what they’ve learned into an explanation. Although these test items are more difficult to write, they can be quite satisfactory for assessments. The possible responses include the answer and then a variety of reasons that support the answer. Of course, only one reason is logical and correct, based on the knowledge and skills being assessed.

Example 1:

  • Before: What are three signs of edema?
  • After: A patient entered the hospital with edema of both lower extremities. What action should the nurse take and why?

Example 2:

  • Before: Select the most effective tone for writing technical documentation.
  • After: Select the paragraph and reasoning that best demonstrate how to start a technical document.

Example 3:

  • Before: Name three qualities of effective entrepreneurs.
  • After: Review the financials of these three startup companies. Which one would you invest in and why?

Also see 10 Rules for Writing Multiple Choice Questions for ways to make your tests more accurate and error-free. Plus see Patti Shank’s book: Write Better Multiple-Choice Questions to Assess Learning for more great tips.

 


Get the latest articles, resources and freebies once a month plus my free eBook, Writing for Instructional Design.

Comments

  1. Jess says

    June 7, 2022 at 2:03 pm

    Hi Connie,

    Thank you so much for your article. I’m currently a MSc student doing developmental and educational psychology research and have been suggested a dissertation topic on visual learning strategies and higher-order thinking. The scope of study was narrowed to using the argumentation vie-diagram (AVD) (a diagram which is similar to mind-map but focuses on helping one to creative critical solutions for both arguments and counterargument as well as to create compromises/solutions to solve a case scenario/problem based question/statement etc. by integrating the ideas from both sides) as the learning strategy and the HOT skill would be creative thinking — the highest level of HOT skills according to the Bloom’s Taxonomy. My supervisor suggested me to use MCQs to test creative thinking/creative solutions using MCQs as the test format with the use of AVD as the learning strategy. However, since the nature of MCQ consists of close-ended questions so it provides little to no space for creativity. I’ve been struggling to find related rating scale and MCQs for assessing creativity based on this matter. May I ask if you have any ideas in creating MCQs to suit this case scenario?

  2. Connie Malamed says

    January 11, 2022 at 10:29 pm

    Glad it was helpful, Carlton.

  3. Carlton Johnson says

    January 8, 2022 at 3:50 am

    Greate article Connie. I’m not always a fan of quizzes etc as the way they are written often means they are just testing short term memory. However, multiple-choice questions written in the way you have described can be brilliant for testing so much more. Thanks for sharing.

  4. Connie Malamed says

    November 5, 2021 at 9:30 pm

    Hi Amy,
    I do have three examples there, such as “A patient entered the hospital with edema of both lower extremities. What action should the nurse take and why?”
    Maybe you’d like to see an answer choice? Even though I’m not a medical person, I can imagine an answer like this:
    a. The nurse calls a resident heart specialist to examine the patient because she wants to rule out congestive heart failure, which can be an underlying condition of edema.

    Is that what you were looking for?

  5. Amy P. says

    November 4, 2021 at 9:59 am

    Can you give an example of an “answer plus the reason why” question?

  6. Connie Malamed says

    September 25, 2016 at 7:24 am

    Hi Dick,
    I think your advice is sound as long as we avoid the appearance of “tricking” the student and that the test questions are tied to the original learning objectives. Thanks for your thoughtful comment.
    Connie

  7. Dick Johnson says

    September 24, 2016 at 2:10 am

    The questions should be written very generally (“Which alternative(s) is/are correct regarding [subject]-“) Then all the logic should be in the alternatives. Yes, it’s a pain then to come up with 4-5 alternatives that not all are correct, BUT all make sense to a student – IF the student hasn’t studied the subject well enough.

    And of course one should always use “multiple answers” of the quiz making tool has this implemented in a smart way.

  8. Robin Phares says

    January 19, 2013 at 7:48 pm

    Writing a good multiple choice question is a difficult task starting at grade K and continuing all the way through adult learning. Just for fun, I goggled “How to cheat at Multiple Choice Questions” and found 3,590,000 hits. The top ones included: “when in doubt choose c” and “all of the above” is usually the correct answer.
    I found your post very helpful when thinking about writing multiple choice questions. The scenario option is perfect for the instructional designer. It seems as if it will almost write itself.

    I took a stab at a before and after.
    Before: What is the best app to use to find the definition of a word?
    a) dictionary.com
    b) webster word of the day
    c) color dictionary
    After: Your students are reading “The Whirligig” by Paul Fleischman and come across the word “inadvertently”. What is the best way for them to find the definition on their tablet?
    a) look at dictionary.com
    b) open the app “Webster word of the day”
    c) use the “color dictionary” app

    I believe that as an Instructional Designer assessment will be the most difficult part of our job. It will be easy to design a course and sit back and marvel at our work. The real “test” will be in the “tests” and the more authentic we can design them the better we will be.

Trackbacks

  1. Assessment & Practice | Pearltrees says:
    October 31, 2012 at 6:49 pm

    […] Planning < Education and Teaching < steben45 Get flash to fully experience Pearltrees Writing Multiple Choice Questions For Higher Order Thinking: Instructional Design and eLearning One of the biggest criticisms of multiple choice questions is that they only test factual […]

  2. Writing multiple choice questions for higher order thinking [Malamed] says:
    October 19, 2012 at 9:49 am

    […] thinking [Malamed] On 10/19/2012, in elearning, instructional design, by Daniel Christian Writing multiple choice questions for higher order thinking — from theelearningcoach.com by Connie […]

  3. gen2oh.net » Learning in Two Dimensions says:
    October 17, 2012 at 5:42 pm

    […] e-Learning Coach offers three tricks for asking good multiple choice questions. Read ‘em and use ‘em. She knows what she’s talking […]

  4. Writing Multiple Choice Questions For Higher Order Thinking: Instructional Design and eLearning | 21st Century Learning in Today's Classrooms | Scoop.it says:
    October 11, 2012 at 4:06 pm

    […] Writing Multiple Choice Questions For Higher Order Thinking: Instructional Design and eLearning From theelearningcoach.com – Today, 3:06 PM […]

  5. Writing Multiple Choice Questions For Higher Order Thinking: Instructional Design and eLearning | Zentrum für multimediales Lehren und Lernen | Scoop.it says:
    October 9, 2012 at 2:59 pm

    […] Writing Multiple Choice Questions For Higher Order Thinking: Instructional Design and eLearning From theelearningcoach.com – Today, 1:59 PM […]

  6. Writing Multiple Choice Questions For Higher Order Thinking ... | Assessment for Learning - Ontario | Scoop.it says:
    October 9, 2012 at 2:38 pm

    […] Writing Multiple Choice Questions For Higher Order Thinking … From theelearningcoach.com – Today, 1:38 PM You can also assess critical thinking skills by asking learners to analyze or interpret information from visuals, which are provided as part of the question stem or the answer choices. In many cases, visuals such as job aids, diagrams and graphs … […]

  7. ¿Qué se dice en la web? (Links 08/10/2012) | Bean-Thinking.com says:
    October 8, 2012 at 4:00 pm

    […] de enseñanza/aprendizaje. Finalmente, de the learning coach, les dejamos este post sobre las actividades de selección múltiple y su uso en actividades de pensamiento elevado, tema relacionado con nuestra última videoconferencia sobre […]

  8. Writing Multiple Choice Questions For Higher Order Thinking: Instructional Design and eLearning | 21st Century Learning in Today's Classrooms | Scoop.it says:
    October 7, 2012 at 6:11 pm

    […] Writing Multiple Choice Questions For Higher Order Thinking: Instructional Design and eLearning From theelearningcoach.com – Today, 5:11 PM […]

  9. Writing Multiple Choice Questions For Higher Order Thinking: Instructional Design and eLearning | C21 learning: ideas and tools for teachers | Scoop.it says:
    October 7, 2012 at 4:13 pm

    […] Writing Multiple Choice Questions For Higher Order Thinking: Instructional Design and eLearning From theelearningcoach.com – Today, 3:12 PM Use multiple-choice questions to measure critical thinking, such as analysis, evaluation and interpretation. Before and After examples. […]

  10. Writing Multiple Choice Questions For Higher Order Thinking: Instructional Design and eLearning | Creating Better Online Assessments | Scoop.it says:
    October 7, 2012 at 12:59 pm

    […] Writing Multiple Choice Questions For Higher Order Thinking: Instructional Design and eLearning From theelearningcoach.com – Today, 11:58 AM […]

  11. Writing Multiple Choice Questions For Higher Order Thinking: Instructional Design and eLearning | ict - tics | Scoop.it says:
    October 7, 2012 at 9:49 am

    […] Writing Multiple Choice Questions For Higher Order Thinking: Instructional Design and eLearning From theelearningcoach.com – Today, 8:49 AM […]

  12. Writing Multiple Choice Questions For Higher Order Thinking: Instructional Design and eLearning | Utbildning på nätet | Scoop.it says:
    October 7, 2012 at 2:34 am

    […] Writing Multiple Choice Questions For Higher Order Thinking: Instructional Design and eLearning From theelearningcoach.com – Today, 1:34 AM […]

  13. Writing Multiple Choice Questions For Higher Order Thinking: Instructional Design and eLearning | Flexibility Enables Learning says:
    October 6, 2012 at 9:33 pm

    […] on theelearningcoach.com Like this:LikeBe the first to like this. PermalinkLeave a […]

  14. Writing Multiple Choice Questions For Higher Order Thinking: Instructional Design and eLearning | eLearning and Blended Learning in Higher Education | Scoop.it says:
    October 6, 2012 at 9:32 pm

    […] Writing Multiple Choice Questions For Higher Order Thinking: Instructional Design and eLearning From theelearningcoach.com – Today, 8:32 PM […]

  15. Writing Multiple Choice Questions For Higher Order Thinking: Instructional Design and eLearning | :: The 4th Era :: | Scoop.it says:
    October 6, 2012 at 8:26 pm

    […] Writing Multiple Choice Questions For Higher Order Thinking: Instructional Design and eLearning From theelearningcoach.com – Today, 7:26 PM […]

  16. Writing Multiple Choice Questions For Higher Order Thinking ... | elearning stuff | Scoop.it says:
    October 6, 2012 at 3:53 am

    […] Writing Multiple Choice Questions For Higher Order Thinking … From theelearningcoach.com – Today, 2:53 AM Use multiple-choice questions to measure critical thinking, such as analysis, evaluation and interpretation. Before and After examples. […]

  17. Writing Multiple Choice Questions For Higher Order Thinking: Instructional Design and eLearning | Gamification of VET | Scoop.it says:
    October 5, 2012 at 5:47 am

    […] Writing Multiple Choice Questions For Higher Order Thinking: Instructional Design and eLearning From theelearningcoach.com – Today, 4:47 AM Use multiple-choice questions to measure critical thinking, such as analysis, evaluation and interpretation. Before and After examples.One of the biggest criticisms of multiple choice questions is that they only test factual knowledge. But it doesn’t have to be that way. We can also use multiple choice questions to assess higher-order thinking. […]

  18. Writing Multiple Choice Questions For Higher Order Thinking: Instructional Design and eLearning | elearning&knowledge_management | Scoop.it says:
    October 4, 2012 at 4:43 pm

    […] Writing Multiple Choice Questions For Higher Order Thinking: Instructional Design and eLearning From theelearningcoach.com – Today, 3:43 PM Use multiple-choice questions to measure critical thinking, such as analysis, evaluation and interpretation. Before and After examples. […]

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Writing for Instructional Design

Subscribe to my newsletter AND get my eBook, Writing for Instructional Design.

Get the latest articles, resources and freebies once a month plus my free eBook, Writing for Instructional Design.

The Learning Guild

CSUN Instructional Design Program

CSUN Instructional Design Program

Fullerton Instructional Design Program

Fullerton Instructional Design Program

Learn ID at University of San Diego

u of san diego id program

ANSWERS TO QUESTIONS ABOUT A CAREER IN ID

answers to 5 key questions about a career in instructional design

NOW AVAILABLE AS AN EBOOK

You can now buy the email course as a PDF to get all the lessons at once. Price: $15.00 US.

eBook Cover
Click the image to buy the book.

SITE MENU

Home
Resources
My Books
About
Book Store
Membership Community

TOPIC MENU

Podcasts
LXD
eLearning Design
eLearning 2.0
Cognition
Multimedia (Graphics & Audio)
Reviews
At Work

RESOURCES

Storyboard Depot
Instructional Design Programs
Best Stock Images
PowerPoint Grids
eLearning Examples
Glossary of Online Learning
Graphic Editing Tools
Icon Collections

© Copyright 2021 Connie Malamed · All Rights Reserved