• Home
  • Free Resources
    • Storyboard Depot
    • Instructional Design Programs
    • Best Stock Photo Sites
    • PowerPoint Grids
    • eLearning Examples
    • Glossary of Online Learning Terms
    • Graphic Editing Tools
    • Icon Collections
  • My Books
  • About
  • Book Store
  • ON-DEMAND COURSES

The eLearning Coach

For designing effective learning experiences

  • Podcasts
  • LXD
  • eLearning Design
    • Audience
    • Instructional Design
    • Learning Objectives
    • Storyboarding
    • Tests
    • User Interface
  • eLearning 2.0
  • Cognition
  • Multimedia
    • Audio
    • Graphics
    • Video
  • Reviews
    • Book Reviews
    • Software Reviews
  • At Work
You are here: Home / eLearning Design / Instructional Design / Are Your Online Tests Valid?

Are Your Online Tests Valid?

How to Improve Test Validity

by Connie Malamed

Tweet12
Pin10
Share3
25 Shares

Are your online tests valid?

You can’t make it through a year of designing eLearning without having to write test or assessment questions. If you don’t have formal training in this area you may not be familiar with test validity. Or perhaps you slept through that Educational Statistics class. (Personally, I found my Statistics textbook to be an excellent sleeping pill.)

It’s A Measuring Instrument

When you develop a test, you’re supposed to be measuring the knowledge and skills that the learner constructed, retained and can apply to new situations. However, the assessment may fail to result in a meaningful measurement for a few reasons.

  • First, your test can’t represent all the content in the course.
  • Second, perhaps you tested some of the least significant content.
  • And third, the most important aspect of workplace learning is usually the application of knowledge in the real world. Testing knowledge alone in a vacuum has its limitations. Listen to How to Plan, Design and Write Tests or download the transcript. 

Test Validity

In an effort to make your tests better measures of meaningful learning, consider the two criteria of effective tests: validity and reliability. This article explains test validity. Next, I recommend reading Are your online tests reliable? Test validity answers the question:

“Are you measuring what you think you’re measuring?”

In the world of workplace learning, focus on whether your test best represents the performance that needs improvement. Because you want the test to accurately assess knowledge and skills, the test questions must correspond to the most important performance objectives.

How To Weight Learning Objectives/Actions

One way to achieve greater validity is to weight the objectives. This helps ensure you are testing the most important content. Here is a reasonably quick way to go about it.

  1. Review the course performance goal so you can keep it in mind.
  2. Create a matrix in order to determine the value of each learning objective or action.
  3. Score each objective in terms of its importance to the learner’s job, its difficulty and the frequency in which it occurs in the course.
  4. Then total the scores.
  5. Objectives with a low score are low priority and will not be tested; objectives with the highest scores will be tested. (Note: if the training has not yet been developed, consider eliminating low scoring items from the course altogether.) See the matrix below for an example of learning objectives for a course on Executive Communication Skills.

Example Of Weighting Objectives

PERFORMANCE OBJECTIVE/ACTION  IMPORTANCE
TO JOB
 TASK DIFFICULTY  FREQUENCY  TOTAL
Use effective techniques to overcome barriers that prevent accurate communication between supervisors and frontline workers. 5 4 4 13
Determine the factors that keep executives isolated from staff. 3 3 1 7
Apply safety practices to the tasks that your warehouse employees perform. 5 4 2 11

Other Ways To Improve Test Validity

  1. Keep the test items aligned with the high priority learning objectives.
  2. Develop test items that measure the application of knowledge, as in scenario-based items. See Writing Multiple Choice Test Questions for Higher Order Thinking.
  3. Have test items reviewed by an expert who understands the skills that the workplace requires.
  4. Ask a few target members of the audience to read through the test questions to ensure they are clearly worded.
  5. Allow a test to evolve by monitoring learner performance. Revise items that cause difficulty or revise that aspect of the course.

Resources:

  • Are Your Online Tests Reliable?
  • Writing Better Tests for Job Training
  • Learn more about making tests valid in Wikipedia.

Get the latest articles, resources and freebies once a month plus my free eBook, Writing for Instructional Design.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Writing for Instructional Design

Subscribe to my newsletter AND get my eBook, Writing for Instructional Design.

Get the latest articles, resources and freebies once a month plus my free eBook, Writing for Instructional Design.

The Learning Guild

CSUN Instructional Design Program

CSUN Instructional Design Program

Fullerton Instructional Design Program

Fullerton Instructional Design Program

Learn ID at University of San Diego

u of san diego id program

ANSWERS TO QUESTIONS ABOUT A CAREER IN ID

answers to 5 key questions about a career in instructional design

NOW AVAILABLE AS AN EBOOK

You can now buy the email course as a PDF to get all the lessons at once. Price: $15.00 US.

eBook Cover
Click the image to buy the book.

SITE MENU

Home
Resources
My Books
About
Book Store
Membership Community

TOPIC MENU

Podcasts
LXD
eLearning Design
eLearning 2.0
Cognition
Multimedia (Graphics & Audio)
Reviews
At Work

RESOURCES

Storyboard Depot
Instructional Design Programs
Best Stock Images
PowerPoint Grids
eLearning Examples
Glossary of Online Learning
Graphic Editing Tools
Icon Collections

© Copyright 2021 Connie Malamed · All Rights Reserved