• Home
  • Free Resources
    • Storyboard Depot
    • Instructional Design Programs
    • Best Stock Photo Sites
    • PowerPoint Grids
    • eLearning Examples
    • Glossary of Online Learning Terms
    • Graphic Editing Tools
    • Icon Collections
  • My Books
  • About
  • Book Store
  • ON-DEMAND COURSES

The eLearning Coach

For designing effective learning experiences

  • Podcasts
  • LXD
  • eLearning Design
    • Audience
    • Instructional Design
    • Learning Objectives
    • Storyboarding
    • Tests
    • User Interface
  • eLearning 2.0
  • Cognition
  • Multimedia
    • Audio
    • Graphics
    • Video
  • Reviews
    • Book Reviews
    • Software Reviews
  • At Work
You are here: Home / eLearning Design / Evaluating Your Online Courses

Evaluating Your Online Courses

by Connie Malamed

Tweet
Pin10
Share6
16 Shares

Evaluating Your Online Courses

Evaluating eLearning courses creates the feedback loop that designers need to continually improve learning solutions. How else can we know what that audience needs and whether they improve their performance? You can conduct evaluations during the design phase, after creating prototypes, when draft versions of storyboards are ready, after participants have completed a training program, and by following trends in data over time.

Evaluation Formula

You can use both formal and informal methods to evaluate eLearning courses. Either way, a straightforward equation underlies the process.

COLLECT EVALUATION DATA + MAKE REVISIONS = COURSE IMPROVEMENTS

Formative Evaluations of eLearning Courses

Conduct formative evaluations of eLearning while a course is in the early stages of development, such as while prototyping or storyboarding. The goal of formative evaluation is to get data and feedback before you go into full production, so that you can revise accordingly. Revisions are easier to make in a storyboard than in finished media.

There are numerous approaches to formative evaluations of eLearning, I’ll describe one that is easy to implement here.

It’s Just You and the Learner

One-on-one evaluations are powerful ways to gain insight into an eLearning course regarding its content and usability. With this approach, you sit with the evaluator as he or she reviews the storyboards, scripts and draft graphics in either paper-based or electronic form. During this time, have the evaluator point out instances when the content is difficult to understand.

To find usability issues, have the evaluator review a prototype of the course or review the interactions that are specified in the storyboard. Find out whether the screen instructions are clear, whether it is easy to perform the next required action and whether the screen design is well-organized. The user interface should never interfere with the learning process.

Take meticulous notes from this evaluation and consider recording the feedback. When evaluations from all sample audience members are complete, collate the data and revise your work accordingly. Formative evaluation can catch many errors, awkward sentences, unclear explanations, and graphics before you even get started. For more on quality processes, see Set Yourself Up to Create Quality eLearning.

Summative Evaluations of eLearning Courses

Conduct summative evaluations of eLearning courses after learners complete a fully operational course. Well-written questions that test higher order thinking may assess how much a participant learned and remembers immediately at the end of a course. But it does not measure the impact of training on an organization, which is the ultimate goal. Therefore, it’s important to measure other metrics over time as well.

Evaluating Test Performance

During the planning stages of the course, you set goals and wrote performance objectives to meet these goals. If the course was designed systematically, then the test questions will be aligned with the priority objectives. Collecting test performance data, therefore, is an effective way to evaluate immediate understanding and retention. But in workplace learning, this is often not a sufficient evaluation.

At the lowest level, you can evaluate whether learners are acquiring the skills taught in a course by analyzing  the collective test scores. If the results of the assessment are not good, look for patterns in the data. When answers to a specific question or two are consistently incorrect, either the test question itself is unclear or the instruction for that content needs improvement. Modify, adapt and revise as needed.

Evaluating Metrics

During the analysis and design phases of a project, identify the metrics that can measure the effectiveness of a learning solution over time. This will be part of your evaluation plan. All organizations collect metrics of some kind. Examples include:

  • New annual memberships to a professional association
  • Annual turnover rate at a company
  • Number of customer complaints in a month
  • Quarterly sales figures
  • Number of graduates from low income households

Access to this type of data will allow you to measure the effectiveness of training in the long run. You can also determine whether the cost of training is worth the time, effort and funds. See Will Thalheimer’s LTEM model for a sound research-based approach.

Conclusion

Organizations and clients are often hesitant to spend money on evaluating eLearning courses, not realizing that this is the only way to fully close the course design and development loop. If you can find a way to integrate evaluation procedures into your training process, you will see that the benefits are huge.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Writing for Instructional Design

Subscribe to my newsletter AND get my eBook, Writing for Instructional Design.

Get the latest articles, resources and freebies once a month plus my free eBook, Writing for Instructional Design.

The Learning Guild

CSUN Instructional Design Program

CSUN Instructional Design Program

Fullerton Instructional Design Program

Fullerton Instructional Design Program

Learn ID at University of San Diego

u of san diego id program

ANSWERS TO QUESTIONS ABOUT A CAREER IN ID

answers to 5 key questions about a career in instructional design

NOW AVAILABLE AS AN EBOOK

You can now buy the email course as a PDF to get all the lessons at once. Price: $15.00 US.

eBook Cover
Click the image to buy the book.

SITE MENU

Home
Resources
My Books
About
Book Store
Membership Community

TOPIC MENU

Podcasts
LXD
eLearning Design
eLearning 2.0
Cognition
Multimedia (Graphics & Audio)
Reviews
At Work

RESOURCES

Storyboard Depot
Instructional Design Programs
Best Stock Images
PowerPoint Grids
eLearning Examples
Glossary of Online Learning
Graphic Editing Tools
Icon Collections

© Copyright 2021 Connie Malamed · All Rights Reserved