Also, I think in academia, the stakes are often higher. In the workplace, there are many reasons for testing and organizations will continue to ask eLearning designers to develop tests. So from a practical perspective, improving the validity and reliability of the tests is the best we can do for now. Thanks for your opinion!

To assume that any test can be consistent and/or reliable is to be too presumptive about human nature.

The results of a test are really what’s being referred to in this post, for the whole reason for creating a test is to gather results from those participants who sit it.

There is a fallacy in believing that any test, however well designed, can possibly be truly consistent. Even the best tests vary in their ‘consistency’ because the way people interpret tests will (also) vary considerably. This variation is not necessarily directly connected with the actual knowledge or skill abilities of participants.

For this reason, academic assessment methods, in countries throughout the world, are forever being reviewed. It is for this reason also that honours degree candidates, having sat their final examinations, may still be required to undergo an oral examination. Even the consistency of the results of these tests can be put in dispute simply because of the test environment itself, and how it is viewed and accepted by the participants.

Many studies have been done, especially on survey questionnaires and the like, which indicate clearly that interpretation of even an apparently unequivocal question can vary considerably, and among participants of similar background, ability and intelligence.

My own personal feeling is that to be confident about the consistency of any test is folly. Such an assumption will ultimately lead to inconsistent gathered information, as has been found with qualifying examination results collected and analysed throughout the world.

Catchya later
from Middle-earth