Podcast: Play in new window | Download
Subscribe: Spotify | Podchaser | RSS | iTunes
How much do you think about writing test items? Do you have a strategy or do you throw together a test at the end? Don’t miss out on the valuable data you can derive from a well-designed test. In this episode, you’ll find guidance and advice from professor and psychometrician, Michael Rodriguez, PhD. He is also the co-author of Developing and Validating Test Items.
Michael is the Campbell Leadership Chair in Education and Human Development and associate professor in the University of Minnesota Department of Educational Psychology. He teaches courses in educational measurement and assessment. His research is on understanding the psychometric properties of tests, and developing ways to maximize test accessibility to diverse test takers, including those with cognitive impairments or limited English proficiency.
WHAT WE DISCUSS:
- Definition of psychometrics
- When it’s best to write test items in the instructional design process
- The role of validity in test development
- Important guidelines for making sure that test items contribute to validity
- Things to avoid when writing test items
- A new type of test format: Multiple True/False
- Whether true/talse questions provide a good measure of student learning
- Some ways to write effective true/false questions
- How to plan for test design
- Whether Bloom’s Taxonomy is based on research
- Ways to test higher-order thinking
- How to choose an item format
- Common errors that novice test developers make
- Benefits of instructional design
- The focus of new academic research on assessment
TIME: 45 minutes
RATE: Rate this podcast in iTunes
TRANSCRIPT: Download the ELC 039 Transcript .
RESOURCES:
- More about Dr. Rodriguez
- Developing and Validating Test Items: Textbook written by Thomas Haladyna and Michael Rodriguez
Get smarter about online learning with articles, tips and resources delivered to your Inbox once a month. Plus get my eBook on writing for instructional design.
Thank you Connie for this informative interview! So insightful and helpful for me as a teacher.
You’re welcome. This was one of my favs. He was so informative!
Connie
Great stuff! I was able to immediately apply several takeaways to my job.
I Just created a competency assessment that is entirely based on fill in the blanks (text entry boxes) geared toward making learners think more deeply than what multiple choice questions require.
As an example…the course I just finished is Workplace Harassment, and one question on the assessment has to do with Ageism (harassment based on a person’s age passing a certain threshold). Originally, this is what I had:
Original Question: “At what age do employees become protected against ageism harassment in the workplace?”
Choices: A. 20; B. 30; C. 40; D. 50.
Correct answer: C. 40
After listening to this episode, I re-wrote this question to the following:
Re-written Question: “Enter an age in the box below that depicts an employee who is protected from ageism harassment.”
Choice: Text Entry Box (allows only numbers, up to two digits in length.)
Correct entry: any 2-digit number between 40 and 99, inclusive
Incredible episode!
Adam
Really enjoyed the podcast. Looking forward to reading the transcript when it is available.
Thanks, Sylvester. I’ll be uploading soon.
This was excellent and helpful. Thanks.
Ok, thank you for the response!
At 10:20 you ask about guidelines. Dr. Rodriguez talks about it in general, then at 11:30 says most practitioners follow a common set of guidelines. I’d love to know what he’s referring to.
Thank you,
Conzolo
Oh, thanks for clarifying. I actually do not think he was referring to one specific guideline but more to all of the content that’s out there. For example, there are books on the topic (including his) and many guidelines if you search for them.
Best,
Connie
Great Interview!
Just discovered this podcast and love it!
Any chance you could link to the guidelines on validity that he mentions?
Which guidelines is that? I didn’t think he mentioned a specific set of guidelines.
Thanks for the interview, Connie. I learned–and/or had my awareness heightened–in lots of areas, including some dangers of the wrong “distractor” options, multiple choice alternatives, and the purpose of Bloom’s Taxonomy. As a test taker, I’ve seen so many questionable (in the bad sense!) items on tests. I applaud your and Michael’s efforts to improve this important field.
Jay
Thanks, Jay. I learned some new things too and I have to say, I really enjoy talking to academics because they live and breathe this stuff 🙂
Best,
Connie