Evaluating Your Online Courses

All courses need evaluation. How else can we know what is needed by the audience and whether we have achieved our goals? Evaluations can be conducted during the initial planning stages, when storyboards or scripts are ready, and after participants have completed the course.

Evaluation Formula

Evaluations can be conducted using both formal and informal methods. Either way, a straightforward equation underlies the process.

COLLECTING EVALUATION DATA + MAKING REVISIONS = COURSE IMPROVEMENTS

Needs Analysis

You can conduct an evaluation during the planning stages of a project. Some might call it an analysis, but it is also an evaluation—a way to assess your audience and the content to find out what is missing. From the start, you can evaluate where your organization’s knowledge gaps exist for a particular domain.

For example, perhaps you have been asked to develop an online course for employees regarding diversity training. Through an online survey, you can find out what supervisors think their employees need to know and you can discover what the sticky points are from the employees’ perspective. You can also identify the top diversity issues through discussions with HR and use your legal department to identify what employees need to know from a legal perspective. This front-end work is essentially your needs analysis.

Formative Evaluations

Getting an evaluation from a few sample audience members while you are still in the storyboard or prototype stage can result in huge improvements in a course. The goal of formative evaluation is to get data and feedback before the heavy production is started, so that you can revise accordingly. Revisions are a lot easier to make at this stage of the process.There are numerous approaches to formative feedback, I’ll describe one that is easy to implement here.

It’s Just You and the Learner

One-on-one evaluations are powerful ways to gain insight into the course content and its usability. With this approach, you sit with the evaluator as he or she reviews the storyboards, scripts and draft graphics in either paper-based or electronic form. During this time, have the evaluator point out instances when the content is confusing or difficult to understand.

To find usability issues, have the evaluator review a prototype of the course or review the interactions that are specified in the storyboard. Find out whether the screen instructions are clear, whether it is easy to perform the next required action and whether the screen design is well-organized. The user interface should never interfere with the learning process.

Take meticulous notes from this evaluation and consider making an audio recording. When evaluations from all sample audience members are complete, collate the data and revise accordingly. Formative evaluation can catch many errors, awkward sentences, unclear explanations and graphics before you even get started.

Summative Evaluations

Summative evaluations are conducted after learners complete a fully operational course. These types of evaluations can assess the impact of a particular course or of an entire curriculum.

Evaluating Test Performance

During the planning stages of the course, you set goals and wrote objectives to meet these goals. If the course was designed systematically, then the test questions should be aligned with the objectives. Collecting test performance data, therefore, is an effective way to evaluate the course. By examining the collective test scores, you can determine if most learners are mastering the content. If so, this is a good indication that the instruction is sound. If content mastery is somewhat elusive, it is critical that you find out why. Look for patterns in the data. If a specific question or two are consistently answered incorrectly, either the test question itself is unclear or the instruction for that content needs improvement. Modify, adapt and revise as needed.

No Test Available

If the online course does not include an assessment, you can collect data through an online survey upon completion of the course. In the survey, ask specific questions related to the course content and its relevance, instructional approach, graphics and usability. You can also discover the effectiveness of a course by discussing its impact with supervisors, management and human resources. They should have a sense as to how transferable the course’s content is to the workplace.

Conclusion

Organizations and clients are often hesitant to spend money on course evaluation, not realizing that this is the only way to fully close the course development loop. If you can find a way to integrate evaluation procedures into your training process, you will see that the benefits are huge.

What types of evaluations do you find effective?
Use the comment form to let us know.