Classroom Clickers Are Good; Except When They Are Not

Many math instructors use clickers in their larger lecture classes, and can cite numerous studies to show that they lead to more student attention and better learning. A recent research paper on clicker use devotes a page-long introductory section to a review of some of that literature. (Shapiro et al, Computers & Education 111 (2017), 44–59) But the paper—by clicker aficionadas, I should stress—is not all good news. In fact, its main new finding is that when clickers are used in what may be the most common way, they actually have a negative effect on student learning. This finding was sufficiently startling that EdSurge put out a feature article on the paper on May 25, which is how I learned of the result.

The most common (I believe) use of clickers is to provide students with frequent quiz questions to check that they are retaining important facts. (The early MOOCs, including my own, used simple, machine-graded quizzes embedded in the video lectures to achieve the same result.) And a lot of that research I just alluded to showed that the clickers achieve that goal.

So too does the latest study. All of which is fine and dandy if the main goal of the course is retention of facts. Where things get messy is when it comes to conceptual understanding of the material—a goal that almost all mathematicians agree is crucial.

In the new study, the researchers looked at two versions of a course (physics, not mathematics), one fact-focused, the other more conceptual and problem solving. In each course, they gave one group fact-based clicker questions and a second group clicker questions that concentrated on conceptual understanding in addition to retention of basic facts.

As the researchers expected, both kinds of questions resulted in improved performance in fact- based questions on a test administered at the end.

Neither kind of question led to improved performance in a problem-based test questions that required conceptual understanding.

The researchers expressed surprise that the students who were given the conceptual clicker questions did not show improvement in conceptual questions performance. But that was not the big surprise. That was, wait for it: students who were given only fact-based clicker questions actually performed worse on conceptual, problem solving questions.

To those of us who are by nature heavy on the conceptual understanding, not showing improvement as a result of enforced fact-retention comes as no big surprise. But a negative effect! That’s news.

By way of explanation, the researchers suggest that the fact-based clicker questions focus the student’s attention on retention of what are, of course, surface features, and do so to the detriment of acquiring the deeper understanding required to solve problems.

If this conclusion is correct—and is certainly seems eminently reasonable—the message is clear. Use clickers, but do so with questions that focus on conceptual understanding, not retention of basic facts.

The authors also recommend class discussions of the concepts being tested by the clicker questions, again something that comes natural to we concepts matter folks.

I would expect the new finding to have implications for game-based math learning, which regular readers will know is something I have been working on for some years now. The games I have been developing are entirely problem-solving challenges that require deep understanding, and university studies have shown they achieve the goal of better problem-solving skills. (See the December 4, 2015 Devlin’s Angle post.) The majority of math learning games, in contrast, focus on retention of basic facts. Based on the new clickers study, I would hypothesize that, even if a game were built on math concepts (many are not), unless the gameplay involves active, problem-solving engagement with those concepts, the result could be, not just no conceptual learning, but a drop in performance on a problem solving test.

Both clickers and video games set up a feedback cycle that can quickly become addictive. With both technologies, regular positive feedback leads to improvement in what the clicker- questions or game-challenges ask for. Potentially more pernicious, however, that positive feedback will result in the students thinking they are doing just fine overall—and hence have no need to wrestle more deeply with the material. And that sets them up for failure once they have to go beneath the surface fact they have retained. Thinking you are winning all the time seduces you to ease off, and as a result is the path to eventual failure. If you want success, the best diet is a series of challenges— that is to say, challenges in coming to grips with the essence of the material to be learned—where you experience some successes, some failures from which you can recover, and the occasional crash-and- burn to prevent over-confidence.

That’s not just the secret to learning math. It’s the secret to success in almost any walk of life.