progressively less wrong

Learning Hack #3: Humans are terrible judges of their learning. (This means you!)

This post is part of a series about how to apply principles of cognitive science to teaching and learning. It’s grounded in a story about a student (here called Kyra) who inspired me to change my teaching. Read more about Kyra in My Three Biggest Teaching Mistakes. The other posts in this series can be found at Learning Hack #1: Learning is NOT data entry; it’s more like knot-tying and Learning Hack #2: Learning is an active process–not a passive one and Learning Hack #3: Humans are terrible judges of their learning. (This means you). And if you’d like, you can also skip right to my suggestions for how to implement this principle.

So far, I’ve suggest that if you want to learn better, you should 1.) think about how to make more connections between what you want to learn and what you already know and 2.) increase the amount of cognitive work you’re doing to understand it. If you’re doing that, then you’re learning. But … sorry … your learning is still probably all messed up.

It’s nothing personal. It’s not just you. Richard Feynman described it well when he said, “The first rule is not to fool yourself, and you are the easiest person to fool.” What that means is that you are susceptible to all kinds of learning mistakes: you’ll think that you’ve learned something when you haven’t, you’ll think that your understanding is accurate when it isn’t, and you’ll think that you’re competent at something when you’re not.

What gives?

Learning Glitches

Our minds evolved to draw conclusions quickly. That meant we usually drew those conclusions on incomplete information. Good thing too. Imagine if, instead of running, we tended to stop and check out that rustling in the grass, just to make sure it really was a lion. Instead our mind makes assumptions. It fills in the blanks when information is missing. But as useful as this quick, intuitive approach to thinking is, it makes us susceptible to all kinds of mistakes. 

The list of these thinking glitches is long, but let’s consider two really common mistakes through a simple example. Imagine watching a YouTube video to help you learn about the Coriolis effect. (The Coriolis effect explains, among other things, how the apparent movements of winds is affected by the rotation of the Earth.) The speaker on your YouTube video is smart, so she knows what she’s talking about. She’s also plainspoken, so she’s easy to understand. She’s clear. She’s funny. She has illustrations and diagrams that help to make her point.

A lot of people would consider watching this video a great learning experience, because it’s so fun to watch. And they’d stop there.

But not you! You know about the pitfalls of passive learning so you’ve taken steps to make your learning more active. Specifically, after you watch the video, you’ve decided to teach the idea to your friend by playing the video with the sound off and becoming the narrator yourself, describing all the images on the screen in your own words.

Two Learning Mistakes

Nice job! That’s a great active learning strategy!

The bad news is that even with all of this, you make several mistakes when you write an essay explaining the Coriolis effect on a science test the next day. Why?

No matter how careful you’ve been, you’ll habitually and unavoidably make two kinds of errors when you weave new threads into your memory net.

Mistakes are inevitable.
  1. First, you think you’ve learned more than you actually have. Lucid explanations, like the one in this video make you feel like you get the idea. Your mind identifies with and adopts the explainer’s viewpoint in the same way that you identify with and adopt the viewpoints of characters in a movie. You slip into a story trance. It doesn’t help that you also get a little dopamine hit when the explainer tells a good joke or makes a witty remark. All of this combines to give you the feeling of understanding. But what do you actually remember about the video? Mostly, you remember the feeling of getting it, rather than the information itself. You think you’ve tied new threads securely to your memory net, but the knots are loose.
  2. Second, you fill in the blanks with your own mistakes. No explanation is ever totally complete, so … you fill in the blanks. You have to! The person on the video says something. Then she says something else. You make assumptions about how the speaker got (logically) from point A to point B. It makes sense in your mind. It may even feel like she said it, but it’s actually just your own ideas. It’s like you’ve tied in the new learning onto the wrong thread in your net. All of this conspires to make your learning feel more complete, better understood, and more certain that it actually is.

This is inevitable. You won’t know that you’ve done it. It just happens. C’est la vie. The good news is that while you can’t prevent these learning mistakes, you can find them and fix them. But you’ll need help.

Feedback: The Antidote to Learning Mistakes

The fastest way to dispel your own illusions is to perform before an audience.

I don’t have to tell you how hard this is, putting your potential ignorance or short-comings or clumsiness or limitations out there for others to see. It takes courage. But there is no way around it. If you want to protect yourself from your own illusions, someone has to watch you mess up. Until you do, your mistakes are invisible, even to you–especially to you. “You are the easiest person to fool.”

For example, I’m learning to speak French, and it’s a complicated gig. Sure, I employ active, connection-making strategies: storytelling flash cards with complete sentences and lots of pictures that force me to work at remembering while also giving me a rich story to connect to, listening to French podcasts without a transcript, writing in French to pen-pals in Paris, reading French comics before I go to sleep. But despite all of this, I am making lots of invisible mistakes—mistakes woven right into my memory net: vocabulary, grammar, idiom, pronunciation. Remember, it’s inevitable.

If I really want to learn French, I have to perform it in front of an audience. This is why language immersion is so powerful. We need someone who can see our mistakes to make them visible and reflect them back to us.

We need to go public with our understanding in order to see our mistakes.

This is what teachers were invented for, and it is why even in the age of information, they will always be worth having. The greatest value teachers bring us is probably the motivation they bring us by giving us a challenging task and a deadline. But the second greatest gift they bring is not their lectures, not their explanations, not even their assignments (however finely-crafted they may be). The second greatest gifts our teachers give us is to point out our mistakes, to show us where we’ve gone wrong and how we could do better next time.

When I was working with Kyra, I thought I was giving this kind of feedback. And I bet she thought she was getting some every time I passed back a quiz. But real feedback would require a fundamental shift for both of us.

Kyra and I weren’t engaged in real feedback; what we were actually doing was playing a game of guess-what-the-teacher-wants: Kyra tried to prove what she knew, and my roles was to confirm or deny whether she’d succeeded. In this game the student loses points for mistakes and get points for only those things that match the “right” answer. Kyra played this game rationally. She avoided taking risks. And I perpetuated the game by focusing on “rightness” or “wrongness” rather than trying to uncover the parts of Kyra’s memory net that we could build on to become progressively less wrong.

How could we do this differently?

Leveraging Feedback

Of the three principles that I’ve drawn from cognitive science, this is the one that I find hardest to implement. The culture of guess-what-the-teacher-wants runs deep. It’s what students expect. It’s what families expect. It’s presumed by the traditions and policies of schools. The following list represents some ways I’m trying to change this, but it is a list in its early states. I welcome your thoughts and suggestions in the comments.

  1. Seek out multiple sources of feedback. The best feedback comes from a teacher, but teachers are not the only source.
    • When you check your work against an answer key, you’re looking for feedback on whether you’re on the right track. It may not be very rich feedback, (You might still be making invisible errors that a simple key won’t uncover.) but it’s better than no feedback at all.
    • Comparing your work to an example or model is even better. Emulating a well-structured essay, checking your work against a similar math problem, listening to a native speaker pronounce their French word, or even studying a working lawn mower when you’re trying to fix your broken one. There is some evidence that having such models available is more useful to learners than an explanation of how to do something well.
    • If you have a group of peers that is studying with you, they could provide a rich and interactive source of feedback. One of my former students tells me that this is how he survived his first year at one of the world’s most competitive colleges. When he wasn’t in class, he was working with fellow students on problem sets. Another friend set up Skype conversations with other students while taking an online course.
  2. Take risks. As students, instead of trying to prove to what we know, we would do better by trying to break what we know. (Check out this video for a striking example of this.) It’s not that we should try to mess up on purpose. But we should attempt things that are just beyond our capabilities where we are bound to mess up. A student’s goal should be to put it all out there. This can only work, of course, if teachers reduce the stakes. Otherwise students will avoid seeking feedback altogether.
  3. Cut to the chase. Try to simulate, as closely as you can, the type of thing that you’re trying to learn to do. Doing so can make the world itself a source of feedback. Are you learning to survive in the wilderness? Spend a weekend out there with your pocketknife. Computer programming? Write a program to make dancing cats appear on your computer screen. Not everything can lend itself to real-world experience. (I don’t think we should train novice doctors by just having them try stuff out on patients.) But even in high-stakes fields like medicine or flying spacecraft, people use realistic simulations to find and address the invisible mistakes so that they can tie and re-tie the knots in their memory nets.
  4. Reflect. This isn’t about getting feedback, so much as paying attention to it. Ask yourself questions. What did you learn? What mistakes did you make? How might you do it better next time? How close are you to reaching your goals? Reflection allows you to simultaneously make new connections (knots), make your learning active, and make use of feedback.
  5. If you’re a teacher, leverage these experiences for your students. Make feedback part of the lesson plan, even if it doesn’t come from you. Give exemplars of excellent work. Have students solve problems together. Give them time to summarize. Let them struggle with problems that they can’t do yet, so that their thinking becomes visible to you and, more importantly to them.
  6. If you’re a teacher, give feedback in three parts. 1. First, let them know where they stand by telling them where they are in relation to the goal. “This is a good start, Kyra, but it’s missing a critical piece.” 2. Next, expose their illusions. Reveal the mistakes they probably didn’t even know they’ve made. “This doesn’t explain WHY your body needs oxygen. Just saying you’d die without it isn’t an explanation. It’s just a restatement of that need. A good explanation would give a REASON for WHY you’d die without oxygen.” 3. Give the student a direction for actively respond to the feedback. “I think that page 187 does a really good job of explaining this. Take five minutes to read it. Then close the book and try to explain it to me.”
  7. Create an culture of risk-taking. This is hard. Exposing your shortcomings is never easy. I don’t have a magic bullet for making this easier. But I do know that mixing up evaluation (i.e. grades) with feedback shuts down student risk-taking. I now give a lot more assignments that aren’t graded or aren’t weighted heavily so that they are willing to push their own boundaries.