Are Your Online Tests Valid?

• Tweet
• Share
• Sharebar

• Tweet
• Share

You can’t make it through a year of designing eLearning without having to write test questions at some point. If you don’t have formal training in this area or if you forgot what you learned in that sleep-inducing Educational Statistics class, this article is for you. (Personally, I found my Educational Statistics textbook to be an excellent sleeping pill.)

It’s A Measuring Instrument

When you’re asked to develop a test, you’re supposed to be measuring the knowledge that the learner constructed and encoded into long-term memory. Often, this is not a meaningful measurement for a few reasons.

First, your test can’t represent all the content in the course. Second, perhaps you tested some of the least significant content. And third, the most important aspect of workplace learning is usually the application of knowledge in the real world. Testing knowledge alone in a vacuum has its limitations. There are probably other reasons why a typical test may not be a meaningful measure, but they escape me now.

Validity

In an effort to make your tests better measures of meaningful learning, consider the two criteria of effective tests: validity and reliability. This article will explain validity, which answers the question:

“ARE YOU MEASURING WHAT YOU THINK YOU’RE MEASURING?”

In the world of workplace eLearning, the important thing to focus on is whether your test best represents the content of interest. Because you want the test to accurately assess knowledge and skills (as much as possible), the questions must correspond to the most important learning objectives or actions.

How To Weight Learning Objectives/Actions

One way to achieve greater validity is to weight the objectives to be sure you are testing the most important content. Here’s a reasonably quick way to go about it.

1. Review the course goal so you can keep it in mind.
2. Create a matrix in order to determine the value of each learning objective or action.
3. Score each objective in terms of its importance to the learner’s job, its difficulty and the frequency in which it occurs in the course.
4. Then total the scores.
5. Objectives with a low score are low priority and will not be tested; objectives with the highest scores will be tested. (Note: if the training has not yet been developed, consider eliminating low scoring items from the course altogether.) See the matrix below for an example of learning objectives for a course on Executive Communication Skills.

Example Of Weighting Objectives

LEARNING OBJECTIVE/ACTION	IMPORTANCE TO JOB	TASK DIFFICULTY	FREQUENCY	TOTAL
Describe the communication barriers that can occur between executives and employees in the hierarchical workplace.	5	4	4	14
Determine the factors that keep executives isolated from staff.	3	3	1	7
Describe the day-to-day tasks that your warehouse employees perform.	5	4	2	11

Other Ways To Improve Test Validity

1. Again, keep the test items aligned with the high priority learning objectives.
2. Develop test items that measure the application of knowledge, as in scenario-based items.
3. Have test items reviewed by an expert who understands the skills that the workplace requires.
4. Ask a few target members of the audience to read through the test questions to ensure they are clearly worded.
5. Allow a test to evolve by monitoring learner performance. Revise items that cause difficulty or revise that aspect of the course.

Related Article:
Are Your Online Tests Reliable?

What issues do you have with developing online tests? What tips can you share?