We’ve all participated in eLearning modules that have had a quiz as part of their end of section assessment. Now depending on that experience you are either quite ok with questions or you just let out a resounding groan. Unfortunately I think in most situations it was the groan. I’m right there with you.
Over the years I’ve taken a lot of different eLearning modules or reviewed them for clients and stakeholders. One of the key things that always elicits a negative response in me is badly written questions. Questions that have nothing to do with the content you have just invested your time in learning, or questions that are written in an attempt to trick you into selecting the wrong answer, or they seem to be written because the author needed to have some interactive element and couldn’t think of anything more exciting than a multiple choice question.
My personal pet peeve is the ‘stupid joke’ throw away answer. Let me give you an example:
Which of the following are examples of bullying and/or harassment?
- A colleague commenting lewdly on the clothing of one of their team mates.
- Your boss berating you loudly in front of your colleagues calling you ‘stupid’.
- An assertive request for work that is overdue to be completed by 5pm.
- Aliens abducting you for experimentation.
Did you see it? What a waste of time for the learner! It doesn’t really show much respect for them or the content that is being reviewed. Answer D is simply ‘filler’ and a time waster for the learner. We want them to be using their learning power on the learning objectives not distractions.
Bad assessment questions are one of the key killers of any interest or love for e-learning modules, and are likely one of the main reasons a lot of people are not fans of online learning.
So how do we do better? Here’s some tips for effective questions.
1. Do you really need a quiz?
This should always be the number one question you should ask yourself as a developer, or a good one to pose to your client. Is a quiz really necessary? Are there other ways you can assess learning?
For example could you use a short answer interaction, an essay response, a branched scenarios with points awarded depending on the path chosen, or even a reflective learning journal instead? The main difference is that there will be a little more work involved to get the assessment right, and in some instances, the interaction with a mentor or coach to discuss answers. Why not? Having different types of assessment gives a better opportunity for any learning to be embedded, not to mentions further support from a said coach or mentor to better develop the learner? This is a great way to look at introducing a blended learning program or even help support a Learning & Development model such as 70:20:10.
Now you may ask that question and determine that your assessment does need to be in the form of a quiz due to resources or client requirements. That’s still ok, and if it’s the case, move onto tip number 2!
2. Build questions before content
Avoid creating assessments that do not directly measure skills described in one or more of the learning objectives. If the questions doesn’t address one of your learning objectives then you shouldn’t be asking it. Help the learner focus on what is important. Make sure each question maps back to the outcomes and then focus the learning content on the assessment.
In other words write your question to address the learning outcome, and then ensure the relevant content is in your module to allow the learner to answer it.
This is why some developers will always build their questions first and work backwards.
3. Keep it fair
Nothing motivates a learner to ‘rage quit’ more than questions that are unfair, impossible to decipher, irrelevant or are designed to trick them outright into giving the wrong answer. The purpose of a learning event is to inform, assist and support a learner to do something right. Make sure your questions respect the learner.
Give them clear directions about how to answer the question. If it’s a multiple choice with only one selection, tell them so. If it is a multiple answer question (tick all that apply) tell them to ‘tick all that apply’.
Here’s a great tip to avoid learner anger with multiple answer questions, if only one of the answers is correct, than use a multiple choice question where they can only select one answer, don’t use a multiple answer question where they can tick more than one answer. Why? Most people have been inherently ‘trained’ by taking e-learning courses over time that if they get a multiple answer question there is a least two correct answers. If you set it up so there is only one, chances are they will get it wrong because the question allows them to tick multiple answers, they will generally always tick more.
Don’t make the question harder by for them by forcing them to also interpret the question. For example:
This question forces the learner to have to decipher multiple answers, and what the question is actually trying to ascertain. If you think about it, this question is testing the user on whether they can answer a question incorrectly, rather than the actual actions we want them to learn. Think about it.
We want to support our learner in making the changes they need to close the performance gap. In this case you are better off to get rid of the ‘NOT’ and change it to a multiple answer question asking them to ‘select all of the answers that are correct”, this question is much easier to decipher, and it reflects what you are actually attempting to test.
4. Make it challenging
When writing questions, make them challenging. Avoid obvious distractors like the following:
What is the capital of Australia?
- Sydney
- Canberra
- New York
- London
C and D are obvious distractors. They are throw away answers (like the alien abduction joke above). They don’t challenge, and again we are disrespecting the learner’s time.
Here are some other examples where questions aren’t challenging
The problem with ‘None of the above’ or ‘All of the above’ is that generally when they are used, they’re the correct answer.
There’s no challenge to these types of questions, because usually the SME or developer has run out of inspiration and thrown them in as an ‘easy win’ question.
I’ve once seen a quiz successfully completed by a learner who didn’t read a single question just ticked ‘All’ or ‘None’. Guess who successfully passed the quiz?
Here’s another not so challenging question that causes learners to groan.
If the right answer is 3 times longer than the wrong answers, or is clearly more plausible than the others, there is no point in asking the question. There’s no challenge in answering it, the learner just selects the longest answer and contemplates how they are going to have their coffee as soon as this dead simple assessment is finished! (Flat white, skim milk please)
5. Quality Vs Quantity
Try to avoid lengthy assessments. It is better to have 5 excellent questions that all map really well back to your learning outcomes, than to have 15 substandard ones that meander about and link to your learning objectives in a tangential way.
Remember, as discussed earlier you can always supplement your assessment with other methods then questions.
If you have lots of good quality questions, than use quiz banks to control the amount of questions your learner’s access as opposed to all of them.
6. Let your learners learn from mistakes
Try to avoid using the default
Correct/Incorrect or Yes/No as feedback.
Provide learners with constructive feedback when they get the wrong answers. Tell them why the answer was incorrect, don’t leave them hanging none the wiser as to what they did wrong. Reading ‘why’ they were incorrect can be just as useful a learning experience as any other.
If they get it right reinforce their correct answer with the reasons why they are correct, this allows additional embedding of the skills they are learning.
7. Test and Tell
Avoid giving them the content upfront. Instead identify opportunities to ask them to think with a question before correcting them with the right answer. Use questions to support them in ‘pulling’ content to assist them in answering the questions. You can also provide hints and direct them to further support as required.
8. Avoid Yes or No Questions
Finally avoid using too many Yes/No or True/False questions if you can. This gives learners a 50/50 chance of getting it right. No real risk involved. Giving learners a 50% chance of success is probably not the best way for learning. Including a ‘maybe’ option encourages learners to justify their answer, via short answer or essay entry. Another way to assess learning.
My keeping these tips in mind and spending a little more time in development creating some really effective questions, you’re going to successfully check that learning has taken place, and have a happy learner as well.
Maybe next time they have to do an online course, they won’t groan as much!