Friday, February 1, 2013

The Problem with Instructional Videos

With the second offering of my MOOC Introduction to Mathematical Thinking about to go live on March 2, I am once again asking myself if the current MOOC structure is the best way to make effective, quality higher education available in a cost-effective way on global scale, making use of the existing technology.

The two words that inhibit my confidence that we’ll ever achieve what I and my fellow first-generation MOOC instructors are trying to do, are “effective” and “quality.”

The task gets a whole lot easier if you set your sights really, really low. Say, “Pass the standardized course test that comes at the end.” But that’s equivalent to the goal of engineers who set out to build something
routine, like a software package or a bridge. Does the software do what was intended? Does the bridge meet the specifications? It’s also a meaningful goal of human training, where people want to acquire a new skill.

But no one, surely, would make passing a standardized test the goal of higher education, or even a significant metric thereof. The purpose, after all, is to build more capable thinkers. No, the thought that anyone would make that kind of mistake seems so unlikely, I’ll move on without giving it any more attention, and get back to my main theme: the videotaped lecture.

I’ve commented on a number of occasions (for example, in my blog) that I think the videotaped lecture is, from a learning perspective, the least important constituent of a MOOC, and that, for me at least, MOOCs seemed to offer the possibility of scaling (at least some elements of) higher education because they can draw on our experience with Facebook, rather than YouTube.

One huge problem with a videotaped lecture is that we know that instructional videos about science (and other disciplines where the learner starts with some beliefs, including mathematics) simply do not work.

In true MOOC fashion, we are now far enough in to my column that I should give you a multiple choice quiz. Here it is:

QUIZ: Where do trees get most of their mass?
1. From nutrients in the soil.
2. From the water
3. From the energy coming from the sun
4. From the air

When you have made your selection, take a look at this video to get the answer.

Done that? Did you notice the way the video was put together. Most of the video was devoted to the presenter (Derek Muller, who got his Ph.D. at the University of Sydney, Australia, a few years ago on the effectiveness of science videos) discovering people’s misconceptions. That certainly makes for “good television,” but does it have a place in an educational video? You bet it does.

The reason is, what is arguably the main finding of Muller’s research: that the principal effects of a well made, clear, instructional, science video are (1) to reinforce the viewer’s existing belief, whatever it is, and (2) to make that viewer even more confident in that belief. Nothwithstanding the fact that the video might present information that flatly contradicts the belief.

Muller summarized those findings in a critique of Khan Academy a couple of years ago, which is how I first came across his work. Anyone thinking of giving a MOOC should spend the eight minutes it takes to watch that video.

Since completing his doctorate and critiquing Khan, Muller has gone on to make a number of science videos. He is, clearly, still experimenting with the format (and I for one hope he continues to do so), and as a result, the videos are of varying quality. But a consistent theme is to begin with common misconceptions and force people to confront those erroneous beliefs.

Sure, this means getting people to say wrong things on camera, which can make some viewers feel uneasy. This has led to some criticism – though anyone you see on the final video has agreed to be shown, of course. He addresses this issue in an amusing fashion in another video. But the real point is that learning does involve confronting – and then correcting – our misconceptions. One of the most crucial abilities of a good teacher is to tell people they are wrong, and help them correct the error, without making them feel small or stupid.

The fact is, the experts make mistakes all the time. Indeed, an expert only achieves that status by having learned how to capitalize from being proved wrong, over and over again. In a sequel to the tree-mass video, Muller made another film about the mechanism trees use to acquire that mass, and in that video (which is truly amazing) you see three experts give the wrong answer.

So if videotaped instruction doesn’t work, how can we achieve learning in a MOOC? Well, there are not many things available. Other than the lecture videos, some screen-readable or downloadable course readings, and a few online quizzes, the only other possible source of learning within a MOOC is the body of other students. (In a physical class, the professor herself can play a role, but for a MOOC class of 60,000 or more, that’s clearly out of the question.)

That’s why I think MOOCs are more Facebook than YouTube, and why I think the key to making them anything more than just textbooks-on-steroids – an approach we know won’t work – is to learn how to structure them to encourage and support group collaborative work.


David said...

I wonder how successful a video which is less of one person talking and more of a conversation between an expert (yourself) and a novice (a student with some misconceptions).

You could model the use of the language and mathematical thinking in your conversation with the student, check for and challenge the misconceptions of the student.

Basically, it would look like a one on one tutoring session filmed.

Keith Devlin said...

David, I think that could be very effective, but I fear that in general such an approach is probably not practical for a professor developing a MOOC under current circumstances.

To cover the range of common misconceptions would require doing a lot of interviews, each of some length, and then spending a lot of time editing to select interviews that work well pedagogically, and in those to focus on the crucial misunderstandings.

Derek Muller clearly devotes a lot of time to this. A typical professor simply does not have that time, and I suspect few would have the interest to do it, as it takes them well away from their main professional activities. Put simply, making good videos is extremely time consuming! (Influenced by the flavor of Khan Academy videos, in my MOOC, I deliberately set out to make my videos look “amateurish” and not staged, complete with mistakes I correct on camera, but that took a significant amount of time to set up, given that the aggregate content had to be clear and correct.)

On the other hand, with recorded educational materials, they only need to be made once, and can then be re-used many times with each new class of students, so it would make sense if professors who wanted to prepare such videos could obtain funding to buy time and pay for someone to work with them on the recording and editing. Maybe that is a development we will see. I think it's definitely worth a try, and given the millions of dollars that educational philanthropists and investors are cuurrently pouring into initiatives that we *know* cannot work, it would be genuinely helpful if they were to support such an experiment.

Unknown said...

Teaching science is frequently about correcting misconceptions since students have a lifetime of experience with things like forces and growing plants.

The challenge with mathematics instruction, in my experience, is that students frequently do not have any misconceptions to correct. Frequently, students are a "blank slate" since they do not typically consider questions like "Is 1 = 0.999...?" or "What is the average value of a continuous function over an interval?" There is nothing in their daily lives to compel them to confront such ideas. At first blush one might be tempted to think that mathematics is easier to teach since students have fewer misconceptions to overcome; however, if you have ever tried to teach mathematics you know this is not the case.

Abstract mathematical thinking is sufficiently different from the empirical thinking of science (broad brush strokes) that the effective instructional techniques of one do not necessarily inform the other.

So to me it seems we are left with two takeaways...

(i) Who, if anyone, is investigating the essential elements of an effective mathematics video? I think is on the right track. With my students, I have seen her videos create mathematical curiosity, which is an important first step in mathematics teaching.

(ii) The key to learning is cognitive effort on the part of the student under the guidance of an invested teacher/professor. If the technology employed does not force the student to mentally engage then very little learning will be achieved. It is for this reason that pencil and paper may just be the best educational technology ever conceived.

My two pence, for what it's worth.