Evidence-Based Instructional Design: Should We Encourage Learners to Make Mistakes?

Big Mountain Amusement Park has hired us to train new employees to operate their flagship roller coaster, the Brain Buster. As usual, the key to this project is training transfer--it's crucial that our learners apply the knowledge and skills they acquire in training back on the job. After all, the lives of park guests may depend on it.

We're fortunate to have access to a simulator of the Brain Buster ride. This will give our learners an opportunity to practice their skills in a realistic environment without putting actual guests at risk. In addition, because the simulator has good fidelity (i.e., it looks, feels, and responds like the real thing), we should be able to maximize training transfer.

So, we'll train learners on the correct procedure for running the Brain Buster and then let them practice on the simulator. Obviously, we should encourage learners to take the simulation as seriously as possible. They should follow the procedure exactly and avoid making mistakes. Perfect practice makes perfect!

This approach, called error avoidance training (EAT), seems to be the default approach to training, in everything from software skills to medical procedures.

On the other hand...wouldn't the learners learn more by playing around with the simulator and finding out what works and what doesn't? Wouldn't that give them a deeper understanding of how the coaster works than simply forcing them to follow step-by-step instructions? Won't they be better prepared to deal with unexpected situations that aren't included in the procedure? Should we actually encourage the learners to explore and make mistakes?

This approach, called error management training (EMT), is far less common.

There seem to be good arguments for both error avoidance and error management. Let's look at what the research has to say.

Research on EAT and EMT

• Radiology technology students completing simulation-based training were randomly assigned to either an EAT- or an EMT-based program. Students in the EMT-based program scored higher in task performance by external observers who were not aware of the purpose of the research. (Error Management Training and Simulation Education, The Clinical Teacher)
• Medical students completing training with an ultrasound simulator were randomly assigned to either an EAT- or an EMT-based program. Students in the EMT-based program scored higher in the post-course test and performed better back on the job. (Imperfect Practice Makes Perfect: Error Management Training Improves Transfer of Learning, Medical Education)
• Older workers completing a word-processing training program were randomly assigned to either an EAT- or an EMT-based program. Workers in the EMT-based program scored higher on tests and were more likely to ask for assistance when necessary. (Learning from Our Mistakes: Error Management Training for Mature Learners, Journal of Business and Psychology)
• A meta-analysis of 24 studies shows a moderate positive effect of EMT compared to EAT approaches. Importantly, the effects were more pronounced for post-training performance than for in-class testing. In addition, the effect was more pronounced when the performance task differed somewhat from the training task. Both active exploration and encouragement of errors were beneficial aspects of EMT. (Effectiveness of Error Management Training: A Meta-Analysis, Journal of Applied Psychology)

Conclusions

I summarized only a few of the dozens of peer-review articles available on EMT and EAT. In general, all of this research suggests that EMT is superior to EAT, but the size of the effect varies substantially from study to study. It's also important to note that most of these studies are based on simulation-based training in the medical or aviation industries. It's less clear how this would work in a traditional classroom or linear eLearning course.

On the whole, however, we can conclude the following with a high degree of confidence:

• Error management training leads to modest improvements in on-the-job performance compared to error avoidance training.
• Error management training more effectively prepares learners to address real-world problems than error training. In other words, learners who complete EMT-based approaches are better able to apply what they've learned to problems that differ somewhat from the exact problems presented in training. Students who are encouraged to explore and make mistakes can generalize their knowledge to real-world problems more effectively than those who are encouraged to minimize errors.
• For EMT to be effective, learners should be encouraged to both (a) explore and (b) make mistakes and learn from them.

Practical Implications

It seems clear that we should adopt an error management approach to training, especially when we're using simulations. Let's go back to the example from the beginning of this article. Based on the evidence above, it's reasonable to believe that learners who complete an error-management approach with the Brain Buster sim will be better prepared to deal with the real coaster than those who follow the error avoidance approach.

The strong, intuitive preference for error avoidance approaches may create a practical obstacle, however. How comfortable do you think Big Mountain Amusement Park management will be when we encourage learners to explore and make mistakes on the Brain Buster sim? As with so many aspect of ID, it's not enough to know the right solution, you also have to be good with people so you can convince them to use it.